CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PETfold_pre2.0(20) - scored higher in this pairwise comparison

  4. Performance of PPfold(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for PETfold_pre2.0(20) & PPfold(seed) [.zip] - may take several seconds...


Overview

Metric PETfold_pre2.0(20) PPfold(seed)
MCC 0.616 > 0.344
Average MCC ± 95% Confidence Intervals 0.629 ± 0.142 > 0.265 ± 0.172
Sensitivity 0.400 > 0.128
Positive Predictive Value 0.952 > 0.927
Total TP 237 > 76
Total TN 86961 < 87128
Total FP 24 > 12
Total FP CONTRA 0 = 0
Total FP INCONS 12 > 6
Total FP COMP 12 > 6
Total FN 355 < 516
P-value 5.19332990918e-08

^top




Performance plots


  1. Comparison of performance of PETfold_pre2.0(20) and PPfold(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(20) and PPfold(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(20) and PPfold(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(20) and PPfold(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(20) and PPfold(seed)).

^top





Performance of PETfold_pre2.0(20) - scored higher in this pairwise comparison

1. Total counts & total scores for PETfold_pre2.0(20)

Total Base Pair Counts
Total TP 237
Total TN 86961
Total FP 24
Total FP CONTRA 0
Total FP INCONS 12
Total FP COMP 12
Total FN 355
Total Scores
MCC 0.616
Average MCC ± 95% Confidence Intervals 0.629 ± 0.142
Sensitivity 0.400
Positive Predictive Value 0.952
Nr of predictions 23

^top



2. Individual counts for PETfold_pre2.0(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
RFA_00390 0.97 0.93 1.00 14 1417 2 0 0 2 1
RFA_00416 0.97 0.93 1.00 14 1471 2 0 0 2 1
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00658 0.00 0.00 0.00 0 1126 3 0 2 1 14
RFA_00664 -0.01 0.00 0.00 0 986 4 0 4 0 14
RFA_00749 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00764 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
SPR_00277 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_00721 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_01157 1.00 1.00 1.00 21 2754 1 0 0 1 0
SRP_00020 0.58 0.33 1.00 11 5554 0 0 0 0 22
SRP_00058 0.54 0.29 1.00 10 5768 0 0 0 0 24
SRP_00084 0.45 0.21 1.00 7 5453 0 0 0 0 27
SRP_00105 0.45 0.21 1.00 7 5558 0 0 0 0 27
SRP_00134 0.41 0.28 0.62 8 5982 6 0 5 1 21
SRP_00141 0.38 0.15 1.00 6 6322 1 0 0 1 35
SRP_00231 0.42 0.21 0.88 7 5452 1 0 1 0 27
SRP_00273 0.52 0.28 1.00 11 6775 0 0 0 0 29
SRP_00274 0.43 0.18 1.00 7 6896 0 0 0 0 31
SRP_00338 0.44 0.19 1.00 7 5453 0 0 0 0 29
SRP_00341 0.47 0.23 1.00 7 4746 0 0 0 0 24
SRP_00357 0.44 0.19 1.00 7 5349 0 0 0 0 29

^top



Performance of PPfold(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for PPfold(seed)

Total Base Pair Counts
Total TP 76
Total TN 87128
Total FP 12
Total FP CONTRA 0
Total FP INCONS 6
Total FP COMP 6
Total FN 516
Total Scores
MCC 0.344
Average MCC ± 95% Confidence Intervals 0.265 ± 0.172
Sensitivity 0.128
Positive Predictive Value 0.927
Nr of predictions 23

^top



2. Individual counts for PPfold(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
RFA_00390 1.00 1.00 1.00 15 1416 1 0 0 1 0
RFA_00416 1.00 1.00 1.00 15 1470 1 0 0 1 0
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00658 0.38 0.14 1.00 2 1126 1 0 0 1 12
RFA_00664 0.38 0.14 1.00 2 988 1 0 0 1 12
RFA_00749 0.78 0.75 0.82 9 892 2 0 2 0 3
RFA_00764 0.78 0.75 0.82 9 892 2 0 2 0 3
RFA_00765 0.78 0.75 0.82 9 892 2 0 2 0 3
SPR_00277 0.00 0.00 0.00 0 2926 0 0 0 0 21
SPR_00721 0.00 0.00 0.00 0 2926 0 0 0 0 21
SPR_01157 0.00 0.00 0.00 0 2775 0 0 0 0 21
SRP_00020 0.00 0.00 0.00 0 5565 0 0 0 0 33
SRP_00058 0.00 0.00 0.00 0 5778 0 0 0 0 34
SRP_00084 0.00 0.00 0.00 0 5460 0 0 0 0 34
SRP_00105 0.00 0.00 0.00 0 5565 0 0 0 0 34
SRP_00134 0.00 0.00 0.00 0 5995 0 0 0 0 29
SRP_00141 0.00 0.00 0.00 0 6328 0 0 0 0 41
SRP_00231 0.00 0.00 0.00 0 5460 0 0 0 0 34
SRP_00273 0.00 0.00 0.00 0 6786 0 0 0 0 40
SRP_00274 0.00 0.00 0.00 0 6903 0 0 0 0 38
SRP_00338 0.00 0.00 0.00 0 5460 0 0 0 0 36
SRP_00341 0.00 0.00 0.00 0 4753 0 0 0 0 31
SRP_00357 0.00 0.00 0.00 0 5356 0 0 0 0 36

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.