CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MaxExpect - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for MaxExpect & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric MaxExpect Murlet(seed)
MCC 0.542 > 0.441
Average MCC ± 95% Confidence Intervals 0.526 ± 0.122 > 0.436 ± 0.069
Sensitivity 0.457 > 0.241
Positive Predictive Value 0.647 < 0.814
Total TP 365 > 192
Total TN 133979 < 134307
Total FP 274 > 51
Total FP CONTRA 27 > 6
Total FP INCONS 172 > 38
Total FP COMP 75 > 7
Total FN 433 < 606
P-value 5.1503931209e-08

^top




Performance plots


  1. Comparison of performance of MaxExpect and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MaxExpect and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MaxExpect and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for MaxExpect and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MaxExpect and Murlet(seed)).

^top





Performance of MaxExpect - scored higher in this pairwise comparison

1. Total counts & total scores for MaxExpect

Total Base Pair Counts
Total TP 365
Total TN 133979
Total FP 274
Total FP CONTRA 27
Total FP INCONS 172
Total FP COMP 75
Total FN 433
Total Scores
MCC 0.542
Average MCC ± 95% Confidence Intervals 0.526 ± 0.122
Sensitivity 0.457
Positive Predictive Value 0.647
Nr of predictions 22

^top



2. Individual counts for MaxExpect [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A -0.01 0.00 0.00 0 1525 15 2 13 0 20
3GX2_A 0.77 0.63 0.96 25 4345 2 0 1 1 15
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3J3E_8 0.07 0.06 0.08 2 7478 32 2 21 9 31
3J3F_8 0.32 0.31 0.33 11 12213 36 3 19 14 25
3JYX_4 0.31 0.30 0.31 10 12214 34 4 18 12 23
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NPB_A 0.71 0.61 0.82 28 6987 9 0 6 3 18
3O58_3 0.35 0.34 0.35 12 12369 35 2 20 13 23
3PDR_A 0.76 0.63 0.94 45 12832 5 0 3 2 27
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.58 0.48 0.71 20 3888 8 1 7 0 22
3W1K_J 0.87 0.79 0.97 30 4155 1 1 0 0 8
3W3S_B 0.88 0.78 1.00 31 4722 1 0 0 1 9
3ZEX_C 0.41 0.27 0.64 14 14174 16 2 6 8 38
4A1C_2 0.14 0.15 0.14 5 11746 41 4 26 11 28
4AOB_A 0.50 0.40 0.63 17 4344 11 2 8 1 25
4ENB_A 0.76 0.58 1.00 11 1264 0 0 0 0 8
4ENC_A 0.31 0.26 0.38 5 1313 8 1 7 0 14
4FRG_B 0.30 0.22 0.41 7 3469 10 1 9 0 25
4FRN_A 0.45 0.36 0.57 13 5128 10 2 8 0 23
4JF2_A 0.78 0.61 1.00 19 2831 0 0 0 0 12

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 192
Total TN 134307
Total FP 51
Total FP CONTRA 6
Total FP INCONS 38
Total FP COMP 7
Total FN 606
Total Scores
MCC 0.441
Average MCC ± 95% Confidence Intervals 0.436 ± 0.069
Sensitivity 0.241
Positive Predictive Value 0.814
Nr of predictions 22

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A -0.01 0.00 0.00 0 1528 12 0 12 0 20
3GX2_A 0.59 0.35 1.00 14 4357 1 0 0 1 26
3IVN_B 0.53 0.39 0.75 12 2330 4 2 2 0 19
3J3E_8 0.25 0.06 1.00 2 7501 2 0 0 2 31
3J3F_8 0.44 0.19 1.00 7 12239 0 0 0 0 29
3JYX_4 0.23 0.09 0.60 3 12241 4 0 2 2 30
3LA5_A 0.58 0.41 0.82 14 2468 3 1 2 0 20
3NPB_A 0.49 0.28 0.87 13 7006 2 1 1 0 33
3O58_3 0.45 0.20 1.00 7 12396 0 0 0 0 28
3PDR_A 0.44 0.19 1.00 14 12866 0 0 0 0 58
3RKF_A 0.53 0.35 0.80 12 2196 3 1 2 0 22
3SD1_A 0.47 0.26 0.85 11 3903 2 0 2 0 31
3W1K_J 0.43 0.21 0.89 8 4177 1 0 1 0 30
3W3S_B 0.41 0.23 0.75 9 4741 3 0 3 0 31
3ZEX_C 0.37 0.13 1.00 7 14189 0 0 0 0 45
4A1C_2 0.46 0.21 1.00 7 11774 0 0 0 0 26
4AOB_A 0.58 0.33 1.00 14 4357 1 0 0 1 28
4ENB_A 0.56 0.32 1.00 6 1269 0 0 0 0 13
4ENC_A 0.56 0.32 1.00 6 1320 0 0 0 0 13
4FRG_B 0.15 0.09 0.25 3 3474 9 0 9 0 29
4FRN_A 0.58 0.39 0.88 14 5135 2 1 1 0 22
4JF2_A 0.51 0.29 0.90 9 2840 2 0 1 1 22

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.