CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of RNAalifold(seed) - scored higher in this pairwise comparison

  4. Performance of Contrafold - scored lower in this pairwise comparison

  5. Compile and download dataset for RNAalifold(seed) & Contrafold [.zip] - may take several seconds...


Overview

Metric RNAalifold(seed) Contrafold
MCC 0.728 > 0.670
Average MCC ± 95% Confidence Intervals 0.632 ± 0.097 < 0.674 ± 0.111
Sensitivity 0.607 < 0.687
Positive Predictive Value 0.872 > 0.654
Total TP 684 < 774
Total TN 516772 > 516372
Total FP 175 < 562
Total FP CONTRA 38 < 152
Total FP INCONS 62 < 258
Total FP COMP 75 < 152
Total FN 442 > 352
P-value 6.38150542963e-08

^top




Performance plots


  1. Comparison of performance of RNAalifold(seed) and Contrafold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNAalifold(seed) and Contrafold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNAalifold(seed) and Contrafold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for RNAalifold(seed) and Contrafold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNAalifold(seed) and Contrafold).

^top





Performance of RNAalifold(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for RNAalifold(seed)

Total Base Pair Counts
Total TP 684
Total TN 516772
Total FP 175
Total FP CONTRA 38
Total FP INCONS 62
Total FP COMP 75
Total FN 442
Total Scores
MCC 0.728
Average MCC ± 95% Confidence Intervals 0.632 ± 0.097
Sensitivity 0.607
Positive Predictive Value 0.872
Nr of predictions 25

^top



2. Individual counts for RNAalifold(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A -0.03 0.00 0.00 0 516 12 0 12 0 18
3A3A_A 0.79 0.63 1.00 19 1481 0 0 0 0 11
3GX2_A 0.88 0.79 1.00 22 1427 1 0 0 1 6
3IVN_B 0.71 0.65 0.79 15 884 4 2 2 0 8
3IYQ_A 0.36 0.27 0.48 14 22411 19 13 2 4 37
3IZ4_A 0.46 0.26 0.81 25 25505 8 6 0 2 70
3J20_2 0.85 0.83 0.87 343 421974 104 11 40 53 69
3J3E_8 0.00 0.00 0.00 0 2742 0 0 0 0 15
3J3F_8 0.61 0.37 1.00 7 4754 2 0 0 2 12
3LA5_A 0.73 0.64 0.84 16 935 3 1 2 0 9
3NPB_A 0.77 0.59 1.00 22 2256 3 0 0 3 15
3O58_3 0.60 0.36 1.00 8 4756 1 0 0 1 14
3PDR_A 0.81 0.66 1.00 33 4807 1 0 0 1 17
3RKF_A 0.74 0.67 0.84 16 847 3 1 2 0 8
3SD1_A 0.70 0.59 0.85 17 1513 3 2 1 0 12
3W1K_J 0.78 0.61 1.00 19 1659 0 0 0 0 12
3W3S_B 0.71 0.52 1.00 17 1972 2 0 0 2 16
3ZEX_C 0.49 0.24 1.00 7 5367 2 0 0 2 22
4A1C_2 0.50 0.25 1.00 5 4511 2 0 0 2 15
4AOB_A 0.85 0.72 1.00 21 1416 2 0 0 2 8
4ENB_A 0.63 0.40 1.00 6 466 0 0 0 0 9
4ENC_A 0.63 0.40 1.00 6 490 0 0 0 0 9
4FRG_B 0.84 0.71 1.00 17 1185 0 0 0 0 7
4FRN_A 0.78 0.68 0.90 19 1827 2 1 1 0 9
4JF2_A 0.61 0.42 0.91 10 1071 1 1 0 0 14

^top



Performance of Contrafold - scored lower in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 774
Total TN 516372
Total FP 562
Total FP CONTRA 152
Total FP INCONS 258
Total FP COMP 152
Total FN 352
Total Scores
MCC 0.670
Average MCC ± 95% Confidence Intervals 0.674 ± 0.111
Sensitivity 0.687
Positive Predictive Value 0.654
Nr of predictions 25

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A 0.46 0.39 0.58 7 516 5 3 2 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3GX2_A 0.93 0.89 0.96 25 1423 2 1 0 1 3
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IYQ_A 0.28 0.39 0.20 20 22341 95 44 35 16 31
3IZ4_A 0.57 0.58 0.57 55 25440 49 18 23 8 40
3J20_2 0.70 0.72 0.69 296 421939 197 27 106 64 116
3J3E_8 0.12 0.13 0.11 2 2724 32 5 11 16 13
3J3F_8 0.35 0.47 0.26 9 4726 44 13 13 18 10
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NPB_A 0.86 0.84 0.89 31 2243 9 1 3 5 6
3O58_3 0.32 0.36 0.29 8 4736 21 7 13 1 14
3PDR_A 0.83 0.86 0.80 43 4786 13 5 6 2 7
3RKF_A 0.87 0.83 0.91 20 844 2 2 0 0 4
3SD1_A 0.68 0.69 0.69 20 1504 9 5 4 0 9
3W1K_J 0.97 0.97 0.97 30 1647 1 1 0 0 1
3W3S_B 0.98 0.97 1.00 32 1957 1 0 0 1 1
3ZEX_C 0.35 0.34 0.37 10 5347 23 4 13 6 19
4A1C_2 0.21 0.25 0.19 5 4489 33 9 13 11 15
4AOB_A 0.53 0.52 0.56 15 1410 13 3 9 1 14
4ENB_A 0.85 0.73 1.00 11 461 1 0 0 1 4
4ENC_A 0.85 0.73 1.00 11 485 1 0 0 1 4
4FRG_B 0.73 0.71 0.77 17 1180 5 3 2 0 7
4FRN_A 0.74 0.71 0.77 20 1822 6 1 5 0 8
4JF2_A 0.89 0.79 1.00 19 1063 0 0 0 0 5

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.