CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MaxExpect - scored higher in this pairwise comparison

  4. Performance of McQFold - scored lower in this pairwise comparison

  5. Compile and download dataset for MaxExpect & McQFold [.zip] - may take several seconds...


Overview

Metric MaxExpect McQFold
MCC 0.662 > 0.542
Average MCC ± 95% Confidence Intervals 0.738 ± 0.059 > 0.708 ± 0.060
Sensitivity 0.678 > 0.550
Positive Predictive Value 0.647 > 0.536
Total TP 2585 > 2098
Total TN 2549493 < 2549570
Total FP 1881 < 2156
Total FP CONTRA 399 < 474
Total FP INCONS 1010 < 1345
Total FP COMP 472 > 337
Total FN 1230 < 1717
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of MaxExpect and McQFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MaxExpect and McQFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MaxExpect and McQFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for MaxExpect and McQFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MaxExpect and McQFold).

^top





Performance of MaxExpect - scored higher in this pairwise comparison

1. Total counts & total scores for MaxExpect

Total Base Pair Counts
Total TP 2585
Total TN 2549493
Total FP 1881
Total FP CONTRA 399
Total FP INCONS 1010
Total FP COMP 472
Total FN 1230
Total Scores
MCC 0.662
Average MCC ± 95% Confidence Intervals 0.738 ± 0.059
Sensitivity 0.678
Positive Predictive Value 0.647
Nr of predictions 117

^top



2. Individual counts for MaxExpect [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KE6_A 0.94 0.94 0.94 17 449 2 0 1 1 1
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - -0.04 0.00 0.00 0 283 9 1 8 0 12
2KUR_A 0.95 0.95 0.95 18 448 1 0 1 0 1
2KUU_A 0.94 0.94 0.94 17 429 2 0 1 1 1
2KUV_A 0.94 0.95 0.95 18 420 1 0 1 0 1
2KUW_A 0.94 0.94 0.94 17 452 2 0 1 1 1
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 513 15 3 12 0 18
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.84 0.83 0.86 24 2412 11 0 4 7 5
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 0.94 0.89 1.00 8 1144 14 0 0 14 1
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WW9_F - 0.64 0.75 0.60 6 102 4 1 3 0 2
2WWQ_V 0.97 0.95 1.00 18 1186 3 0 0 3 1
2XKV_B 0.64 0.73 0.57 8 1821 26 0 6 20 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.24 0.26 0.25 9 2009 28 1 26 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3ADB_C - 0.86 0.85 0.88 28 1787 4 0 4 0 5
3AKZ_H 1.00 1.00 1.00 20 1107 2 0 0 2 0
3AM1_B - 0.86 0.86 0.86 25 1437 4 0 4 0 4
3AMU_B 0.73 0.79 0.68 15 1135 10 0 7 3 4
3GX2_A 0.93 0.89 0.96 25 1423 2 1 0 1 3
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.80 0.79 0.81 22 1445 6 1 4 1 6
3IYQ_A 0.24 0.33 0.17 17 22342 96 41 40 15 34
3IZ4_A 0.61 0.61 0.62 58 25442 42 14 22 6 37
3IZF_C 0.89 0.91 0.86 32 2603 8 0 5 3 3
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.30 0.31 0.31 8 2224 25 2 16 7 18
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_7 - -0.02 0.00 0.00 0 507 12 3 9 0 10
3J0L_h - 0.93 0.88 1.00 28 2112 2 0 0 2 4
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_a - 0.39 0.36 0.44 4 402 6 4 1 1 7
3J16_L 0.34 0.33 0.37 7 1140 12 0 12 0 14
3J20_2 0.60 0.63 0.58 258 421926 239 28 156 55 154
3J20_0 0.53 0.57 0.50 12 1195 13 2 10 1 9
3J20_1 0.75 0.75 0.75 15 1092 6 0 5 1 5
3J2C_O - 0.74 0.71 0.78 29 3950 13 0 8 5 12
3J2C_M - 0.48 0.54 0.43 52 39804 88 29 40 19 44
3J2L_3 0.80 0.82 0.78 28 2984 12 1 7 4 6
3JYV_7 0.48 0.45 0.53 9 1094 9 3 5 1 11
3JYX_3 0.77 0.80 0.75 12 2362 24 0 4 20 3
3JYX_4 0.55 0.83 0.37 10 4729 34 14 3 17 2
3KIY_A - 0.67 0.68 0.66 531 1485073 336 81 195 60 255
3KTW_C - 0.72 0.72 0.72 18 1755 14 2 5 7 7
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.94 0.93 0.95 41 3671 7 0 2 5 3
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - -0.02 0.00 0.00 0 208 10 5 3 2 3
3NPB_A 0.77 0.73 0.82 27 2245 10 0 6 4 10
3O58_2 0.91 0.94 0.88 29 2721 11 0 4 7 2
3O58_3 0.40 0.50 0.33 11 4731 36 8 14 14 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.92 0.90 0.94 45 4792 5 1 2 2 5
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.70 0.69 0.71 20 1505 8 3 5 0 9
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 1.00 1.00 1.00 12 142 0 0 0 0 0
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.58 0.59 0.59 13 1254 9 2 7 0 9
3UZL_B 1.00 1.00 1.00 16 1277 8 0 0 8 0
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W3S_B 0.97 0.94 1.00 31 1958 1 0 0 1 2
3ZEX_F - -0.01 0.00 0.00 0 908 11 2 4 5 4
3ZEX_G - 0.98 0.96 1.00 43 6464 17 0 0 17 2
3ZEX_E - -0.01 0.00 0.00 0 8252 54 21 31 2 34
3ZEX_D 0.82 0.77 0.87 27 2765 11 0 4 7 8
3ZEX_B - 0.37 0.39 0.35 134 420964 302 66 188 48 212
3ZEX_H - 0.26 0.37 0.19 7 3589 29 18 11 0 12
3ZEX_C 0.54 0.45 0.65 13 5354 17 2 5 10 16
4A1C_2 0.19 0.25 0.16 5 4484 41 10 17 14 15
4A1C_3 0.83 0.81 0.86 30 2728 7 0 5 2 7
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ATO_G - 0.45 0.43 0.50 3 214 3 3 0 0 4
4ENB_A 0.85 0.73 1.00 11 461 0 0 0 0 4
4ENC_A 0.36 0.33 0.42 5 484 8 0 7 1 10
4FNJ_A - 0.91 0.83 1.00 10 240 0 0 0 0 2
4FRG_B 0.29 0.25 0.35 6 1185 11 4 7 0 18
4FRN_A 0.51 0.46 0.57 13 1825 10 5 5 0 15
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0

^top



Performance of McQFold - scored lower in this pairwise comparison

1. Total counts & total scores for McQFold

Total Base Pair Counts
Total TP 2098
Total TN 2549570
Total FP 2156
Total FP CONTRA 474
Total FP INCONS 1345
Total FP COMP 337
Total FN 1717
Total Scores
MCC 0.542
Average MCC ± 95% Confidence Intervals 0.708 ± 0.060
Sensitivity 0.550
Positive Predictive Value 0.536
Nr of predictions 117

^top



2. Individual counts for McQFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 1 0 0 1 0
2KE6_A 0.91 0.83 1.00 15 452 1 0 0 1 3
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.75 0.83 0.68 19 1996 15 7 2 6 4
2KU0_A - 0.86 0.75 1.00 9 283 0 0 0 0 3
2KUR_A 0.91 0.84 1.00 16 451 0 0 0 0 3
2KUU_A 0.91 0.83 1.00 15 432 1 0 0 1 3
2KUV_A 0.91 0.84 1.00 16 423 0 0 0 0 3
2KUW_A 0.94 0.89 1.00 16 454 0 0 0 0 2
2KX8_A 0.93 0.94 0.94 15 355 1 0 1 0 1
2KXM_A - 0.78 0.63 1.00 5 144 0 0 0 0 3
2KZL_A - 1.00 1.00 1.00 13 507 2 0 0 2 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 1 0 0 1 0
2L3C_B - 1.00 1.00 1.00 14 225 1 0 0 1 0
2L3E_A - 1.00 1.00 1.00 12 228 2 0 0 2 0
2L3J_B 0.95 0.90 1.00 27 958 0 0 0 0 3
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 1 0 1 0 2
2LBS_A - 1.00 1.00 1.00 14 193 1 0 0 1 0
2LC8_A 0.64 0.61 0.69 11 512 5 1 4 0 7
2LDL_A - 0.81 0.67 1.00 6 134 0 0 0 0 3
2LDT_A - 0.79 0.64 1.00 7 155 0 0 0 0 4
2LHP_A - 1.00 1.00 1.00 15 246 1 0 0 1 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.55 0.55 0.55 16 2411 15 5 8 2 13
2LQZ_A - 1.00 1.00 1.00 8 124 3 0 0 3 0
2LWK_A - 0.95 0.91 1.00 10 197 2 0 0 2 1
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WW9_D - 0.44 0.40 0.50 4 733 9 4 0 5 6
2WW9_F - 0.86 0.75 1.00 6 106 1 0 0 1 2
2WWQ_V 0.00 0.00 0.00 0 1204 0 0 0 0 19
2XKV_B 0.33 0.36 0.31 4 1822 27 2 7 18 7
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.27 0.26 0.30 9 2015 22 0 21 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 2 0 0 2 0
2Y9C_V - 0.72 0.72 0.72 13 888 8 0 5 3 5
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
3A2K_C 0.98 0.95 1.00 21 1087 0 0 0 0 1
3ADB_C - 0.98 0.97 1.00 32 1787 1 0 0 1 1
3AKZ_H 0.78 0.80 0.76 16 1106 7 2 3 2 4
3AM1_B - 1.00 1.00 1.00 29 1437 1 0 0 1 0
3AMU_B 0.86 0.79 0.94 15 1141 4 0 1 3 4
3GX2_A 0.56 0.57 0.57 16 1421 13 4 8 1 12
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.21 0.21 0.24 6 1447 19 2 17 0 22
3IYQ_A 0.22 0.31 0.15 16 22335 100 45 44 11 35
3IZ4_A 0.46 0.46 0.47 44 25442 56 14 36 6 51
3IZF_C 0.85 0.83 0.88 29 2607 7 0 4 3 6
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.29 0.31 0.30 8 2223 21 1 18 2 18
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_7 - -0.02 0.00 0.00 0 509 10 2 8 0 10
3J0L_h - 0.75 0.66 0.88 21 2116 3 0 3 0 11
3J0L_1 - 0.73 0.62 0.89 8 475 4 0 1 3 5
3J0L_a - 0.21 0.18 0.29 2 404 6 3 2 1 9
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3J20_2 0.55 0.56 0.55 231 421948 234 41 148 45 181
3J20_0 0.74 0.76 0.73 16 1197 7 3 3 1 5
3J20_1 1.00 1.00 1.00 20 1092 1 0 0 1 0
3J2C_O - 0.62 0.61 0.64 25 3948 17 3 11 3 16
3J2C_M - 0.38 0.44 0.34 42 39801 95 33 49 13 54
3J2L_3 0.71 0.68 0.74 23 2989 11 0 8 3 11
3JYV_7 -0.02 0.00 0.00 0 1091 20 4 16 0 20
3JYX_3 0.28 0.40 0.20 6 2348 24 17 7 0 9
3JYX_4 0.35 0.58 0.22 7 4724 33 17 8 8 5
3KIY_A - 0.38 0.38 0.38 302 1485090 544 83 405 56 484
3KTW_C - 0.52 0.52 0.54 13 1756 17 2 9 6 12
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.97 0.95 0.98 42 3671 5 0 1 4 2
3NKB_B - 0.42 0.42 0.44 8 717 10 2 8 0 11
3NMU_E - 0.57 0.33 1.00 1 215 3 0 0 3 2
3NPB_A 0.90 0.81 1.00 30 2248 5 0 0 5 7
3O58_2 0.93 0.87 1.00 27 2727 2 0 0 2 4
3O58_3 0.30 0.36 0.25 8 4732 27 15 9 3 14
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.83 0.80 0.87 40 4794 9 2 4 3 10
3R4F_A - 1.00 1.00 1.00 22 884 2 0 0 2 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.82 0.83 0.83 24 1504 5 1 4 0 5
3SIU_F - 0.86 0.75 1.00 6 139 0 0 0 0 2
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 2 0 0 2 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.95 0.91 1.00 20 1256 2 0 0 2 2
3UZL_B 0.62 0.75 0.52 12 1270 17 8 3 6 4
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W3S_B 0.49 0.45 0.54 15 1961 14 1 12 1 18
3ZEX_F - 0.00 0.00 0.00 0 910 9 0 4 5 4
3ZEX_G - 0.36 0.40 0.34 18 6454 40 11 24 5 27
3ZEX_E - 0.00 0.00 0.00 0 8256 53 17 31 5 34
3ZEX_D 0.76 0.66 0.88 23 2770 6 1 2 3 12
3ZEX_B - 0.12 0.14 0.12 47 420944 379 78 283 18 299
3ZEX_H - 0.27 0.37 0.21 7 3591 30 14 13 3 12
3ZEX_C 0.33 0.34 0.32 10 5343 25 4 17 4 19
4A1C_2 0.19 0.25 0.16 5 4484 33 11 16 6 15
4A1C_3 0.28 0.30 0.28 11 2724 30 4 24 2 26
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.50 0.48 0.54 14 1411 13 3 9 1 15
4ATO_G - -0.03 0.00 0.00 0 215 5 0 5 0 7
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0
4FNJ_A - -0.04 0.00 0.00 0 243 7 0 7 0 12
4FRG_B 0.93 0.88 1.00 21 1181 0 0 0 0 3
4FRN_A 0.82 0.71 0.95 20 1827 1 1 0 0 8
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.