CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of RNAfold - scored higher in this pairwise comparison

  4. Performance of McQFold - scored lower in this pairwise comparison

  5. Compile and download dataset for RNAfold & McQFold [.zip] - may take several seconds...


Overview

Metric RNAfold McQFold
MCC 0.603 > 0.534
Average MCC ± 95% Confidence Intervals 0.785 ± 0.054 > 0.748 ± 0.056
Sensitivity 0.639 > 0.503
Positive Predictive Value 0.569 < 0.569
Total TP 2607 > 2049
Total TN 5047465 < 5048449
Total FP 2593 > 1836
Total FP CONTRA 592 > 366
Total FP INCONS 1386 > 1186
Total FP COMP 615 > 284
Total FN 1470 < 2028
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of RNAfold and McQFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNAfold and McQFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNAfold and McQFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for RNAfold and McQFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNAfold and McQFold).

^top





Performance of RNAfold - scored higher in this pairwise comparison

1. Total counts & total scores for RNAfold

Total Base Pair Counts
Total TP 2607
Total TN 5047465
Total FP 2593
Total FP CONTRA 592
Total FP INCONS 1386
Total FP COMP 615
Total FN 1470
Total Scores
MCC 0.603
Average MCC ± 95% Confidence Intervals 0.785 ± 0.054
Sensitivity 0.639
Positive Predictive Value 0.569
Nr of predictions 120

^top



2. Individual counts for RNAfold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - -0.03 0.00 0.00 0 229 7 1 6 0 8
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 1.00 1.00 1.00 12 280 0 0 0 0 0
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.64 0.61 0.69 11 512 5 0 5 0 7
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 8 0 0 8 0
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 0 0 0 0 0
2WDL_A - 0.49 0.52 0.47 408 1444460 493 86 368 39 380
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WWQ_V 1.00 1.00 1.00 19 1185 5 0 0 5 0
2XKV_B 0.64 0.73 0.57 8 1821 25 0 6 19 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.33 0.43 0.27 3 533 11 3 5 3 4
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 3 2 0 1 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.35 0.36 0.36 12 1786 22 1 20 1 21
3AKZ_H 0.48 0.50 0.48 10 1106 13 4 7 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - -0.03 0.00 0.00 0 136 6 1 3 2 6
3G4S_9 0.78 0.77 0.80 20 2711 12 2 3 7 6
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.81 0.79 0.85 22 1423 5 2 2 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 5 0 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.69 0.68 0.70 19 1445 9 1 7 1 9
3IYQ_A 0.24 0.33 0.17 17 22340 95 45 38 12 34
3IZ4_A 0.56 0.60 0.52 57 25426 57 26 27 4 38
3IZF_C 0.75 0.77 0.73 27 2603 15 0 10 5 8
3J0L_2 - 0.30 0.31 0.31 8 2224 25 2 16 7 18
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.18 0.18 0.22 2 402 8 3 4 1 9
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_7 - -0.02 0.00 0.00 0 507 12 2 10 0 10
3J0L_g - -0.01 0.00 0.00 0 173 5 1 2 2 2
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3JYV_7 -0.02 0.00 0.00 0 1089 22 4 18 0 20
3JYX_5 - 0.36 0.46 0.29 220 1968243 783 251 288 244 261
3JYX_4 0.61 0.83 0.45 10 4734 33 10 2 21 2
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3KIY_A - 0.50 0.53 0.48 414 1485013 501 90 363 48 372
3KTW_C - 0.96 0.96 0.96 24 1755 9 0 1 8 1
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.93 0.93 0.93 41 3670 9 0 3 6 3
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.91 0.86 0.97 32 2245 5 1 0 4 5
3O58_3 0.41 0.50 0.34 11 4732 35 6 15 14 11
3O58_2 0.72 0.74 0.70 23 2721 19 0 10 9 8
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.90 0.90 0.90 45 4790 7 2 3 2 5
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.77 0.76 0.79 22 1505 6 4 2 0 7
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_2 0.19 0.25 0.15 5 4482 43 11 18 14 15
4A1C_3 0.88 0.86 0.89 32 2727 7 0 4 3 5
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 0.85 0.73 1.00 11 461 2 0 0 2 4
4ENC_A 0.37 0.33 0.45 5 485 7 0 6 1 10

^top



Performance of McQFold - scored lower in this pairwise comparison

1. Total counts & total scores for McQFold

Total Base Pair Counts
Total TP 2049
Total TN 5048449
Total FP 1836
Total FP CONTRA 366
Total FP INCONS 1186
Total FP COMP 284
Total FN 2028
Total Scores
MCC 0.534
Average MCC ± 95% Confidence Intervals 0.748 ± 0.056
Sensitivity 0.503
Positive Predictive Value 0.569
Nr of predictions 120

^top



2. Individual counts for McQFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.59 0.36 1.00 4 182 0 0 0 0 7
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 1 0 0 1 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 0.91 0.83 1.00 15 452 1 0 0 1 3
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.79 0.63 1.00 5 231 0 0 0 0 3
2KGP_A - 0.81 0.67 1.00 6 115 1 0 0 1 3
2KMJ_A - 1.00 1.00 1.00 11 157 1 0 0 1 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.75 0.83 0.68 19 1996 15 7 2 6 4
2KU0_A - 0.86 0.75 1.00 9 283 0 0 0 0 3
2KUR_A 0.91 0.84 1.00 16 451 0 0 0 0 3
2KUU_A 0.91 0.83 1.00 15 432 1 0 0 1 3
2KUV_A 0.91 0.84 1.00 16 423 0 0 0 0 3
2KUW_A 0.94 0.89 1.00 16 454 0 0 0 0 2
2KX8_A 0.93 0.94 0.94 15 355 1 0 1 0 1
2KXM_A - 0.78 0.63 1.00 5 144 0 0 0 0 3
2KZL_A - 1.00 1.00 1.00 13 507 2 0 0 2 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 1 0 0 1 0
2L3C_B - 1.00 1.00 1.00 14 225 1 0 0 1 0
2L3E_A - 1.00 1.00 1.00 12 228 2 0 0 2 0
2L3J_B 0.95 0.90 1.00 27 958 0 0 0 0 3
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 1 0 1 0 2
2LBS_A - 1.00 1.00 1.00 14 193 1 0 0 1 0
2LC8_A 0.64 0.61 0.69 11 512 5 1 4 0 7
2LDL_A - 0.81 0.67 1.00 6 134 0 0 0 0 3
2LDT_A - 0.79 0.64 1.00 7 155 0 0 0 0 4
2LHP_A - 1.00 1.00 1.00 15 246 1 0 0 1 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.55 0.55 0.55 16 2411 15 5 8 2 13
2LQZ_A - 1.00 1.00 1.00 8 124 3 0 0 3 0
2LWK_A - 0.95 0.91 1.00 10 197 2 0 0 2 1
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WDL_A - 0.37 0.37 0.36 294 1444515 545 84 429 32 494
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_F - 0.86 0.75 1.00 6 106 1 0 0 1 2
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WW9_D - 0.44 0.40 0.50 4 733 9 4 0 5 6
2WWQ_V 0.00 0.00 0.00 0 1204 0 0 0 0 19
2XKV_B 0.33 0.36 0.31 4 1822 27 2 7 18 7
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.27 0.26 0.30 9 2015 22 0 21 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 2 0 0 2 0
2Y9C_V - 0.72 0.72 0.72 13 888 8 0 5 3 5
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.18 0.20 0.19 3 1342 22 4 9 9 12
2ZZN_D 0.93 0.95 0.91 21 961 2 2 0 0 1
3A2K_C 0.98 0.95 1.00 21 1087 0 0 0 0 1
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.98 0.97 1.00 32 1787 1 0 0 1 1
3AKZ_H 0.78 0.80 0.76 16 1106 7 2 3 2 4
3AM1_B - 1.00 1.00 1.00 29 1437 1 0 0 1 0
3AMU_B 0.86 0.79 0.94 15 1141 4 0 1 3 4
3DW4_A - -0.03 0.00 0.00 0 137 3 0 3 0 6
3G4S_9 0.39 0.42 0.37 11 2706 22 6 13 3 15
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.56 0.57 0.57 16 1421 13 4 8 1 12
3HAY_E - 1.00 1.00 1.00 14 910 4 0 0 4 0
3HJW_D - 0.94 0.94 0.94 15 593 3 0 1 2 1
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.66 0.60 0.75 3 225 6 0 1 5 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.21 0.21 0.24 6 1447 19 2 17 0 22
3IYQ_A 0.22 0.31 0.15 16 22335 100 45 44 11 35
3IZ4_A 0.46 0.46 0.47 44 25442 56 14 36 6 51
3IZF_C 0.85 0.83 0.88 29 2607 7 0 4 3 6
3J0L_2 - 0.29 0.31 0.30 8 2223 21 1 18 2 18
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.21 0.18 0.29 2 404 6 3 2 1 9
3J0L_1 - 0.73 0.62 0.89 8 475 4 0 1 3 5
3J0L_h - 0.75 0.66 0.88 21 2116 3 0 3 0 11
3J0L_7 - -0.02 0.00 0.00 0 509 10 2 8 0 10
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3JYV_7 -0.02 0.00 0.00 0 1091 20 4 16 0 20
3JYX_5 - 0.00 0.00 0.00 0 1969002 0 0 0 0 481
3JYX_4 0.35 0.58 0.22 7 4724 33 17 8 8 5
3JYX_3 0.28 0.40 0.20 6 2348 24 17 7 0 9
3KIY_A - 0.38 0.38 0.38 302 1485090 544 83 405 56 484
3KTW_C - 0.52 0.52 0.54 13 1756 17 2 9 6 12
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.97 0.95 0.98 42 3671 5 0 1 4 2
3NKB_B - 0.42 0.42 0.44 8 717 10 2 8 0 11
3NMU_E - 0.57 0.33 1.00 1 215 3 0 0 3 2
3NPB_A 0.90 0.81 1.00 30 2248 5 0 0 5 7
3O58_3 0.30 0.36 0.25 8 4732 27 15 9 3 14
3O58_2 0.93 0.87 1.00 27 2727 2 0 0 2 4
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.83 0.80 0.87 40 4794 9 2 4 3 10
3R4F_A - 1.00 1.00 1.00 22 884 2 0 0 2 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.82 0.83 0.83 24 1504 5 1 4 0 5
3SIU_F - 0.86 0.75 1.00 6 139 0 0 0 0 2
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 2 0 0 2 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.95 0.91 1.00 20 1256 2 0 0 2 2
3UZL_B 0.62 0.75 0.52 12 1270 17 8 3 6 4
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_2 0.19 0.25 0.16 5 4484 33 11 16 6 15
4A1C_3 0.28 0.30 0.28 11 2724 30 4 24 2 26
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.50 0.48 0.54 14 1411 13 3 9 1 15
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.