CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Contrafold - scored higher in this pairwise comparison

  4. Performance of RNAfold - scored lower in this pairwise comparison

  5. Compile and download dataset for Contrafold & RNAfold [.zip] - may take several seconds...


Overview

Metric Contrafold RNAfold
MCC 0.699 > 0.604
Average MCC ± 95% Confidence Intervals 0.803 ± 0.049 > 0.787 ± 0.054
Sensitivity 0.722 > 0.641
Positive Predictive Value 0.677 > 0.570
Total TP 2953 > 2621
Total TN 5047877 > 5047640
Total FP 2109 < 2593
Total FP CONTRA 499 < 592
Total FP INCONS 910 < 1386
Total FP COMP 700 > 615
Total FN 1138 < 1470
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Contrafold and RNAfold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Contrafold and RNAfold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Contrafold and RNAfold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Contrafold and RNAfold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Contrafold and RNAfold).

^top





Performance of Contrafold - scored higher in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 2953
Total TN 5047877
Total FP 2109
Total FP CONTRA 499
Total FP INCONS 910
Total FP COMP 700
Total FN 1138
Total Scores
MCC 0.699
Average MCC ± 95% Confidence Intervals 0.803 ± 0.049
Sensitivity 0.722
Positive Predictive Value 0.677
Nr of predictions 121

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 2 0 0 2 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.71 0.63 0.83 5 230 1 1 0 0 3
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 1.00 1.00 1.00 13 200 0 0 0 0 0
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.63 0.58 0.70 7 282 3 2 1 0 5
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 5 0 0 5 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 0.94 0.94 0.94 17 339 2 0 1 1 1
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.46 0.39 0.58 7 516 5 3 2 0 11
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.88 0.90 0.87 26 2410 11 0 4 7 3
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 0 0 0 0 0
2WDL_A - 0.69 0.71 0.68 556 1444510 320 75 181 64 232
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_E - 0.00 0.00 0.00 0 172 0 0 0 0 5
2WW9_D - 0.63 0.40 1.00 4 737 5 0 0 5 6
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WWQ_V 1.00 1.00 1.00 19 1185 4 0 0 4 0
2XKV_B 0.64 0.73 0.57 8 1821 27 0 6 21 3
2XQD_Y 0.85 0.86 0.86 18 1108 4 0 3 1 3
2XXA_G 0.10 0.11 0.12 4 2012 30 2 27 1 31
2Y8W_B - 1.00 1.00 1.00 6 82 3 0 0 3 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 1.00 1.00 1.00 7 537 4 0 0 4 0
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
2ZY6_A - 0.69 0.78 0.64 7 265 5 2 2 1 2
2ZZM_B 0.80 0.80 0.80 12 1343 13 0 3 10 3
2ZZN_D 0.93 0.95 0.91 21 961 3 2 0 1 1
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.98 0.97 1.00 32 1787 0 0 0 0 1
3AKZ_H 0.44 0.50 0.40 10 1102 18 6 9 3 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.75 0.79 0.71 15 1136 9 0 6 3 4
3DW4_A - -0.03 0.00 0.00 0 136 6 1 3 2 6
3G4S_9 0.45 0.50 0.42 13 2705 24 6 12 6 13
3GCA_A - 0.76 0.71 0.83 5 152 1 1 0 0 2
3GX2_A 0.93 0.89 0.96 25 1423 2 1 0 1 3
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3ID5_D - 0.66 0.60 0.75 3 225 6 0 1 5 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.86 0.79 0.96 22 1449 1 0 1 0 6
3IYQ_A 0.28 0.39 0.20 20 22341 95 44 35 16 31
3IZ4_A 0.57 0.58 0.57 55 25440 49 18 23 8 40
3IZF_C 0.89 0.91 0.86 32 2603 11 0 5 6 3
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.28 0.31 0.28 8 2221 26 2 19 5 18
3J0L_g - 0.00 0.00 0.00 0 176 1 0 0 1 2
3J0L_7 - -0.02 0.00 0.00 0 512 7 1 6 0 10
3J0L_h - 0.87 0.88 0.88 28 2108 6 0 4 2 4
3J0L_1 - 0.83 0.77 0.91 10 473 3 0 1 2 3
3J0L_a - 0.53 0.55 0.55 6 400 6 3 2 1 5
3J16_L 0.55 0.57 0.55 12 1137 10 3 7 0 9
3JYV_7 -0.02 0.00 0.00 0 1091 20 4 16 0 20
3JYX_3 0.33 0.47 0.24 7 2349 24 16 6 2 8
3JYX_5 - 0.43 0.50 0.37 240 1968358 697 172 232 293 241
3JYX_4 0.39 0.58 0.27 7 4730 35 13 6 16 5
3KIY_A - 0.70 0.71 0.69 561 1485072 329 73 174 82 225
3KTW_C - 0.56 0.60 0.54 15 1752 20 2 11 7 10
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.98 0.98 0.98 43 3670 6 0 1 5 1
3NKB_B - 0.63 0.68 0.59 13 713 9 0 9 0 6
3NMU_E - 1.00 1.00 1.00 3 213 3 0 0 3 0
3NPB_A 0.86 0.84 0.89 31 2243 9 1 3 5 6
3O58_2 0.92 0.94 0.91 29 2722 10 0 3 7 2
3O58_3 0.32 0.36 0.29 8 4736 21 7 13 1 14
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.83 0.86 0.80 43 4786 13 5 6 2 7
3R4F_A - 0.81 0.82 0.82 18 884 5 2 2 1 4
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.87 0.83 0.91 20 844 2 2 0 0 4
3SD1_A 0.68 0.69 0.69 20 1504 9 5 4 0 9
3SIU_F - 0.87 0.88 0.88 7 137 1 0 1 0 1
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.81 0.77 0.85 17 1256 5 0 3 2 5
3UZL_B 0.90 0.88 0.93 14 1278 8 0 1 7 2
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_3 0.81 0.81 0.81 30 2726 10 0 7 3 7
4A1C_2 0.21 0.25 0.19 5 4489 33 9 13 11 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.53 0.52 0.56 15 1410 13 3 9 1 14
4ENB_A 0.85 0.73 1.00 11 461 1 0 0 1 4
4ENC_A 0.85 0.73 1.00 11 485 1 0 0 1 4

^top



Performance of RNAfold - scored lower in this pairwise comparison

1. Total counts & total scores for RNAfold

Total Base Pair Counts
Total TP 2621
Total TN 5047640
Total FP 2593
Total FP CONTRA 592
Total FP INCONS 1386
Total FP COMP 615
Total FN 1470
Total Scores
MCC 0.604
Average MCC ± 95% Confidence Intervals 0.787 ± 0.054
Sensitivity 0.641
Positive Predictive Value 0.570
Nr of predictions 121

^top



2. Individual counts for RNAfold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - -0.03 0.00 0.00 0 229 7 1 6 0 8
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 1.00 1.00 1.00 12 280 0 0 0 0 0
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.64 0.61 0.69 11 512 5 0 5 0 7
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 8 0 0 8 0
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 0 0 0 0 0
2WDL_A - 0.49 0.52 0.47 408 1444460 493 86 368 39 380
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WWQ_V 1.00 1.00 1.00 19 1185 5 0 0 5 0
2XKV_B 0.64 0.73 0.57 8 1821 25 0 6 19 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.33 0.43 0.27 3 533 11 3 5 3 4
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 3 2 0 1 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.35 0.36 0.36 12 1786 22 1 20 1 21
3AKZ_H 0.48 0.50 0.48 10 1106 13 4 7 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - -0.03 0.00 0.00 0 136 6 1 3 2 6
3G4S_9 0.78 0.77 0.80 20 2711 12 2 3 7 6
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.81 0.79 0.85 22 1423 5 2 2 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 5 0 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.69 0.68 0.70 19 1445 9 1 7 1 9
3IYQ_A 0.24 0.33 0.17 17 22340 95 45 38 12 34
3IZ4_A 0.56 0.60 0.52 57 25426 57 26 27 4 38
3IZF_C 0.75 0.77 0.73 27 2603 15 0 10 5 8
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.30 0.31 0.31 8 2224 25 2 16 7 18
3J0L_g - -0.01 0.00 0.00 0 173 5 1 2 2 2
3J0L_7 - -0.02 0.00 0.00 0 507 12 2 10 0 10
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_a - 0.18 0.18 0.22 2 402 8 3 4 1 9
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3JYV_7 -0.02 0.00 0.00 0 1089 22 4 18 0 20
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3JYX_5 - 0.36 0.46 0.29 220 1968243 783 251 288 244 261
3JYX_4 0.61 0.83 0.45 10 4734 33 10 2 21 2
3KIY_A - 0.50 0.53 0.48 414 1485013 501 90 363 48 372
3KTW_C - 0.96 0.96 0.96 24 1755 9 0 1 8 1
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.93 0.93 0.93 41 3670 9 0 3 6 3
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.91 0.86 0.97 32 2245 5 1 0 4 5
3O58_2 0.72 0.74 0.70 23 2721 19 0 10 9 8
3O58_3 0.41 0.50 0.34 11 4732 35 6 15 14 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.90 0.90 0.90 45 4790 7 2 3 2 5
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.77 0.76 0.79 22 1505 6 4 2 0 7
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_3 0.88 0.86 0.89 32 2727 7 0 4 3 5
4A1C_2 0.19 0.25 0.15 5 4482 43 11 18 14 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 0.85 0.73 1.00 11 461 2 0 0 2 4
4ENC_A 0.37 0.33 0.45 5 485 7 0 6 1 10

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.