CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PknotsRG - scored higher in this pairwise comparison

  4. Performance of McQFold - scored lower in this pairwise comparison

  5. Compile and download dataset for PknotsRG & McQFold [.zip] - may take several seconds...


Overview

Metric PknotsRG McQFold
MCC 0.591 > 0.534
Average MCC ± 95% Confidence Intervals 0.786 ± 0.051 > 0.748 ± 0.056
Sensitivity 0.629 > 0.503
Positive Predictive Value 0.556 < 0.569
Total TP 2563 > 2049
Total TN 5047437 < 5048449
Total FP 2643 > 1836
Total FP CONTRA 598 > 366
Total FP INCONS 1452 > 1186
Total FP COMP 593 > 284
Total FN 1514 < 2028
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of PknotsRG and McQFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PknotsRG and McQFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PknotsRG and McQFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PknotsRG and McQFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PknotsRG and McQFold).

^top





Performance of PknotsRG - scored higher in this pairwise comparison

1. Total counts & total scores for PknotsRG

Total Base Pair Counts
Total TP 2563
Total TN 5047437
Total FP 2643
Total FP CONTRA 598
Total FP INCONS 1452
Total FP COMP 593
Total FN 1514
Total Scores
MCC 0.591
Average MCC ± 95% Confidence Intervals 0.786 ± 0.051
Sensitivity 0.629
Positive Predictive Value 0.556
Nr of predictions 120

^top



2. Individual counts for PknotsRG [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 0.88 0.89 0.89 16 449 3 0 2 1 2
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.77 0.88 0.70 7 226 3 2 1 0 1
2KGP_A - 0.69 0.67 0.75 6 113 2 0 2 0 3
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.85 0.87 0.83 20 2000 12 4 0 8 3
2KU0_A - 0.96 0.92 1.00 11 281 0 0 0 0 1
2KUR_A 0.89 0.89 0.89 17 448 2 0 2 0 2
2KUU_A 0.88 0.89 0.89 16 429 3 0 2 1 2
2KUV_A 0.89 0.89 0.89 17 420 2 0 2 0 2
2KUW_A 0.94 0.94 0.94 17 452 2 0 1 1 1
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.66 0.61 0.73 11 513 4 2 2 0 7
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.81 0.86 0.76 25 2407 10 6 2 2 4
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.87 1.00 0.78 7 107 2 2 0 0 0
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WDL_A - 0.47 0.49 0.45 389 1444457 514 87 389 38 399
2WRQ_Y 0.94 0.89 1.00 8 1144 14 0 0 14 1
2WW9_F - 0.64 0.75 0.60 6 102 4 1 3 0 2
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WWQ_V 1.00 1.00 1.00 19 1185 5 0 0 5 0
2XKV_B 0.64 0.73 0.57 8 1821 24 0 6 18 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.43 0.57 0.33 4 532 10 2 6 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 2 2 0 0 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.35 0.36 0.36 12 1786 21 1 20 0 21
3AKZ_H 0.46 0.50 0.43 10 1104 15 6 7 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - -0.03 0.00 0.00 0 137 3 0 3 0 6
3G4S_9 0.83 0.85 0.81 22 2709 12 2 3 7 4
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.80 0.79 0.81 22 1422 6 2 3 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.54 0.60 0.50 3 223 6 1 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.21 0.21 0.24 6 1447 19 1 18 0 22
3IYQ_A 0.24 0.33 0.17 17 22340 94 45 38 11 34
3IZ4_A 0.56 0.60 0.52 57 25426 57 25 28 4 38
3IZF_C 0.75 0.77 0.73 27 2603 15 0 10 5 8
3J0L_2 - 0.29 0.31 0.29 8 2222 27 2 18 7 18
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.18 0.18 0.22 2 402 8 3 4 1 9
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_7 - 0.38 0.50 0.31 5 503 12 3 8 1 5
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3JYV_7 -0.02 0.00 0.00 0 1089 22 4 18 0 20
3JYX_5 - 0.37 0.46 0.29 220 1968248 778 241 293 244 261
3JYX_4 0.61 0.83 0.45 10 4734 33 10 2 21 2
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3KIY_A - 0.47 0.50 0.45 393 1485010 525 91 386 48 393
3KTW_C - 1.00 1.00 1.00 25 1755 7 0 0 7 0
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.93 0.93 0.93 41 3670 9 0 3 6 3
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.93 0.86 1.00 32 2246 4 0 0 4 5
3O58_3 0.41 0.50 0.34 11 4732 35 6 15 14 11
3O58_2 0.72 0.74 0.70 23 2721 19 0 10 9 8
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.90 0.90 0.90 45 4790 7 2 3 2 5
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.73 0.72 0.75 21 1505 7 4 3 0 8
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_2 0.18 0.25 0.14 5 4480 43 11 20 12 15
4A1C_3 0.85 0.84 0.86 31 2727 8 0 5 3 6
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0

^top



Performance of McQFold - scored lower in this pairwise comparison

1. Total counts & total scores for McQFold

Total Base Pair Counts
Total TP 2049
Total TN 5048449
Total FP 1836
Total FP CONTRA 366
Total FP INCONS 1186
Total FP COMP 284
Total FN 2028
Total Scores
MCC 0.534
Average MCC ± 95% Confidence Intervals 0.748 ± 0.056
Sensitivity 0.503
Positive Predictive Value 0.569
Nr of predictions 120

^top



2. Individual counts for McQFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.59 0.36 1.00 4 182 0 0 0 0 7
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 1 0 0 1 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 0.91 0.83 1.00 15 452 1 0 0 1 3
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.79 0.63 1.00 5 231 0 0 0 0 3
2KGP_A - 0.81 0.67 1.00 6 115 1 0 0 1 3
2KMJ_A - 1.00 1.00 1.00 11 157 1 0 0 1 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.75 0.83 0.68 19 1996 15 7 2 6 4
2KU0_A - 0.86 0.75 1.00 9 283 0 0 0 0 3
2KUR_A 0.91 0.84 1.00 16 451 0 0 0 0 3
2KUU_A 0.91 0.83 1.00 15 432 1 0 0 1 3
2KUV_A 0.91 0.84 1.00 16 423 0 0 0 0 3
2KUW_A 0.94 0.89 1.00 16 454 0 0 0 0 2
2KX8_A 0.93 0.94 0.94 15 355 1 0 1 0 1
2KXM_A - 0.78 0.63 1.00 5 144 0 0 0 0 3
2KZL_A - 1.00 1.00 1.00 13 507 2 0 0 2 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 1 0 0 1 0
2L3C_B - 1.00 1.00 1.00 14 225 1 0 0 1 0
2L3E_A - 1.00 1.00 1.00 12 228 2 0 0 2 0
2L3J_B 0.95 0.90 1.00 27 958 0 0 0 0 3
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 1 0 1 0 2
2LBS_A - 1.00 1.00 1.00 14 193 1 0 0 1 0
2LC8_A 0.64 0.61 0.69 11 512 5 1 4 0 7
2LDL_A - 0.81 0.67 1.00 6 134 0 0 0 0 3
2LDT_A - 0.79 0.64 1.00 7 155 0 0 0 0 4
2LHP_A - 1.00 1.00 1.00 15 246 1 0 0 1 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.55 0.55 0.55 16 2411 15 5 8 2 13
2LQZ_A - 1.00 1.00 1.00 8 124 3 0 0 3 0
2LWK_A - 0.95 0.91 1.00 10 197 2 0 0 2 1
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WDL_A - 0.37 0.37 0.36 294 1444515 545 84 429 32 494
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_F - 0.86 0.75 1.00 6 106 1 0 0 1 2
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WW9_D - 0.44 0.40 0.50 4 733 9 4 0 5 6
2WWQ_V 0.00 0.00 0.00 0 1204 0 0 0 0 19
2XKV_B 0.33 0.36 0.31 4 1822 27 2 7 18 7
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.27 0.26 0.30 9 2015 22 0 21 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 2 0 0 2 0
2Y9C_V - 0.72 0.72 0.72 13 888 8 0 5 3 5
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.18 0.20 0.19 3 1342 22 4 9 9 12
2ZZN_D 0.93 0.95 0.91 21 961 2 2 0 0 1
3A2K_C 0.98 0.95 1.00 21 1087 0 0 0 0 1
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.98 0.97 1.00 32 1787 1 0 0 1 1
3AKZ_H 0.78 0.80 0.76 16 1106 7 2 3 2 4
3AM1_B - 1.00 1.00 1.00 29 1437 1 0 0 1 0
3AMU_B 0.86 0.79 0.94 15 1141 4 0 1 3 4
3DW4_A - -0.03 0.00 0.00 0 137 3 0 3 0 6
3G4S_9 0.39 0.42 0.37 11 2706 22 6 13 3 15
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.56 0.57 0.57 16 1421 13 4 8 1 12
3HAY_E - 1.00 1.00 1.00 14 910 4 0 0 4 0
3HJW_D - 0.94 0.94 0.94 15 593 3 0 1 2 1
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.66 0.60 0.75 3 225 6 0 1 5 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.21 0.21 0.24 6 1447 19 2 17 0 22
3IYQ_A 0.22 0.31 0.15 16 22335 100 45 44 11 35
3IZ4_A 0.46 0.46 0.47 44 25442 56 14 36 6 51
3IZF_C 0.85 0.83 0.88 29 2607 7 0 4 3 6
3J0L_2 - 0.29 0.31 0.30 8 2223 21 1 18 2 18
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.21 0.18 0.29 2 404 6 3 2 1 9
3J0L_1 - 0.73 0.62 0.89 8 475 4 0 1 3 5
3J0L_h - 0.75 0.66 0.88 21 2116 3 0 3 0 11
3J0L_7 - -0.02 0.00 0.00 0 509 10 2 8 0 10
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3JYV_7 -0.02 0.00 0.00 0 1091 20 4 16 0 20
3JYX_5 - 0.00 0.00 0.00 0 1969002 0 0 0 0 481
3JYX_4 0.35 0.58 0.22 7 4724 33 17 8 8 5
3JYX_3 0.28 0.40 0.20 6 2348 24 17 7 0 9
3KIY_A - 0.38 0.38 0.38 302 1485090 544 83 405 56 484
3KTW_C - 0.52 0.52 0.54 13 1756 17 2 9 6 12
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.97 0.95 0.98 42 3671 5 0 1 4 2
3NKB_B - 0.42 0.42 0.44 8 717 10 2 8 0 11
3NMU_E - 0.57 0.33 1.00 1 215 3 0 0 3 2
3NPB_A 0.90 0.81 1.00 30 2248 5 0 0 5 7
3O58_3 0.30 0.36 0.25 8 4732 27 15 9 3 14
3O58_2 0.93 0.87 1.00 27 2727 2 0 0 2 4
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.83 0.80 0.87 40 4794 9 2 4 3 10
3R4F_A - 1.00 1.00 1.00 22 884 2 0 0 2 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.82 0.83 0.83 24 1504 5 1 4 0 5
3SIU_F - 0.86 0.75 1.00 6 139 0 0 0 0 2
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 2 0 0 2 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.95 0.91 1.00 20 1256 2 0 0 2 2
3UZL_B 0.62 0.75 0.52 12 1270 17 8 3 6 4
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_2 0.19 0.25 0.16 5 4484 33 11 16 6 15
4A1C_3 0.28 0.30 0.28 11 2724 30 4 24 2 26
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.50 0.48 0.54 14 1411 13 3 9 1 15
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.