CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MCFold - scored higher in this pairwise comparison

  4. Performance of PPfold(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for MCFold & PPfold(seed) [.zip] - may take several seconds...


Overview

Metric MCFold PPfold(seed)
MCC 0.489 > 0.175
Average MCC ± 95% Confidence Intervals 0.481 ± 0.116 > 0.077 ± 0.084
Sensitivity 0.538 > 0.050
Positive Predictive Value 0.456 < 0.634
Total TP 282 > 26
Total TN 44750 < 45327
Total FP 423 > 80
Total FP CONTRA 86 > 1
Total FP INCONS 250 > 14
Total FP COMP 87 > 65
Total FN 242 < 498
P-value 5.1503931209e-08

^top




Performance plots


  1. Comparison of performance of MCFold and PPfold(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MCFold and PPfold(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MCFold and PPfold(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for MCFold and PPfold(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MCFold and PPfold(seed)).

^top





Performance of MCFold - scored higher in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 282
Total TN 44750
Total FP 423
Total FP CONTRA 86
Total FP INCONS 250
Total FP COMP 87
Total FN 242
Total Scores
MCC 0.489
Average MCC ± 95% Confidence Intervals 0.481 ± 0.116
Sensitivity 0.538
Positive Predictive Value 0.456
Nr of predictions 22

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A 0.44 0.44 0.47 8 511 10 0 9 1 10
3AMU_B 0.50 0.58 0.44 11 1132 15 4 10 1 8
3J16_L 0.54 0.57 0.52 12 1136 12 4 7 1 9
3J20_0 0.66 0.71 0.63 15 1195 11 3 6 2 6
3J2L_3 0.77 0.79 0.75 27 2984 15 1 8 6 7
3J3D_C 0.54 0.63 0.48 12 943 13 4 9 0 7
3J3E_7 0.55 0.56 0.54 19 2706 23 1 15 7 15
3J3E_8 0.15 0.20 0.12 3 2716 36 11 12 13 12
3J3F_8 0.14 0.21 0.10 4 4720 51 17 20 14 15
3J3F_7 0.88 0.89 0.86 32 2897 9 1 4 4 4
3RKF_A 0.89 0.88 0.91 21 843 3 0 2 1 3
3SD1_A 0.43 0.45 0.43 13 1503 17 1 16 0 16
3ZEX_C 0.30 0.34 0.27 10 5337 28 6 21 1 19
3ZEX_D 0.23 0.26 0.23 9 2756 33 6 25 2 26
3ZND_W 0.23 0.38 0.15 3 1171 26 9 8 9 5
4A1C_3 0.86 0.86 0.86 32 2726 9 0 5 4 5
4A1C_2 0.18 0.25 0.14 5 4480 45 12 19 14 15
4ENB_A 0.78 0.73 0.85 11 459 4 0 2 2 4
4ENC_A 0.34 0.33 0.38 5 483 11 0 8 3 10
4FRG_B 0.35 0.38 0.35 9 1176 17 3 14 0 15
4FRN_A 0.09 0.11 0.10 3 1818 28 2 25 1 25
4JF2_A 0.74 0.75 0.75 18 1058 7 1 5 1 6

^top



Performance of PPfold(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for PPfold(seed)

Total Base Pair Counts
Total TP 26
Total TN 45327
Total FP 80
Total FP CONTRA 1
Total FP INCONS 14
Total FP COMP 65
Total FN 498
Total Scores
MCC 0.175
Average MCC ± 95% Confidence Intervals 0.077 ± 0.084
Sensitivity 0.050
Positive Predictive Value 0.634
Nr of predictions 22

^top



2. Individual counts for PPfold(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A 0.49 0.33 0.75 6 520 2 0 2 0 12
3AMU_B 0.00 0.00 0.00 0 1157 0 0 0 0 19
3J16_L 0.00 0.00 0.00 0 1159 0 0 0 0 21
3J20_0 0.00 0.00 0.00 0 1219 0 0 0 0 21
3J2L_3 0.00 0.00 0.00 0 3020 0 0 0 0 34
3J3D_C 0.00 0.00 0.00 0 968 0 0 0 0 19
3J3E_7 0.00 0.00 0.00 0 2741 0 0 0 0 34
3J3E_8 0.00 0.00 0.00 0 2742 6 0 0 6 15
3J3F_8 0.19 0.11 0.33 2 4755 24 0 4 20 17
3J3F_7 0.00 0.00 0.00 0 2934 0 0 0 0 36
3RKF_A 0.00 0.00 0.00 0 866 0 0 0 0 24
3SD1_A 0.00 0.00 0.00 0 1533 0 0 0 0 29
3ZEX_C 0.15 0.07 0.33 2 5368 24 0 4 20 27
3ZEX_D 0.00 0.00 0.00 0 2796 0 0 0 0 35
3ZND_W 0.00 0.00 0.00 0 1191 0 0 0 0 8
4A1C_3 0.00 0.00 0.00 0 2763 0 0 0 0 37
4A1C_2 0.10 0.05 0.20 1 4511 23 0 4 19 19
4ENB_A 0.00 0.00 0.00 0 472 0 0 0 0 15
4ENC_A 0.00 0.00 0.00 0 496 0 0 0 0 15
4FRG_B 0.00 0.00 0.00 0 1202 0 0 0 0 24
4FRN_A 0.00 0.00 0.00 0 1848 0 0 0 0 28
4JF2_A 0.76 0.63 0.94 15 1066 1 1 0 0 9

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.