CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of RNAsubopt - scored higher in this pairwise comparison

  4. Performance of MCFold - scored lower in this pairwise comparison

  5. Compile and download dataset for RNAsubopt & MCFold [.zip] - may take several seconds...


Overview

Metric RNAsubopt MCFold
MCC 0.671 > 0.577
Average MCC ± 95% Confidence Intervals 0.705 ± 0.056 > 0.643 ± 0.066
Sensitivity 0.605 > 0.597
Positive Predictive Value 0.750 > 0.566
Total TP 1385 > 1368
Total TN 233472 > 232902
Total FP 562 < 1236
Total FP CONTRA 58 < 142
Total FP INCONS 404 < 907
Total FP COMP 100 < 187
Total FN 905 < 922
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of RNAsubopt and MCFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNAsubopt and MCFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNAsubopt and MCFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for RNAsubopt and MCFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNAsubopt and MCFold).

^top





Performance of RNAsubopt - scored higher in this pairwise comparison

1. Total counts & total scores for RNAsubopt

Total Base Pair Counts
Total TP 1385
Total TN 233472
Total FP 562
Total FP CONTRA 58
Total FP INCONS 404
Total FP COMP 100
Total FN 905
Total Scores
MCC 0.671
Average MCC ± 95% Confidence Intervals 0.705 ± 0.056
Sensitivity 0.605
Positive Predictive Value 0.750
Nr of predictions 109

^top



2. Individual counts for RNAsubopt [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.95 0.91 1.00 10 396 0 0 0 0 1
2K63_A - 0.89 0.80 1.00 8 398 0 0 0 0 2
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 0.88 0.78 1.00 7 224 0 0 0 0 2
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.86 0.84 0.89 16 1110 3 0 2 1 3
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - -0.02 0.00 0.00 0 623 7 0 7 0 20
2KGP_A - 1.00 1.00 1.00 9 291 0 0 0 0 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KU0_A - 0.83 0.69 1.00 11 692 0 0 0 0 5
2KUR_A 0.85 0.81 0.89 17 1109 2 0 2 0 4
2KUU_A 0.82 0.76 0.89 16 1110 3 0 2 1 5
2KUV_A 0.83 0.77 0.89 17 1109 2 0 2 0 5
2KUW_A 0.90 0.86 0.95 18 1109 1 0 1 0 3
2KX8_A 0.91 0.83 1.00 15 846 0 0 0 0 3
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_B 0.94 0.88 1.00 22 2123 0 0 0 0 3
2L1F_A 0.93 0.88 1.00 21 2059 0 0 0 0 3
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.87 0.76 1.00 13 548 0 0 0 0 4
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A -0.01 0.00 0.00 0 1525 15 2 13 0 20
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.81 0.77 0.86 30 6070 9 0 5 4 9
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.83 0.70 1.00 7 224 0 0 0 0 3
2WRQ_Y 0.57 0.59 0.56 10 2832 12 5 3 4 7
2WW9_E - -0.01 0.00 0.00 0 556 5 0 5 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_D - 0.15 0.12 0.21 3 1939 11 4 7 0 23
2WWQ_V 0.76 0.68 0.86 19 2904 5 0 3 2 9
2XKV_B 0.51 0.50 0.53 10 4541 23 0 9 14 10
2XQD_Y 0.75 0.67 0.86 18 2829 3 0 3 0 9
2XXA_G 0.47 0.43 0.53 18 5117 16 1 15 0 24
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.66 0.63 0.71 15 2257 8 1 5 2 9
2YIE_X - 0.82 0.67 1.00 8 1370 3 0 0 3 4
2YIE_Z - -0.01 0.00 0.00 0 1525 15 3 12 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 1 4 0 5
2ZZN_D 0.48 0.44 0.52 12 2462 11 1 10 0 15
3A2K_C 0.42 0.39 0.46 11 2902 13 2 11 0 17
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3AKZ_H 0.39 0.36 0.43 10 2678 13 2 11 0 18
3AM1_B - 0.91 0.83 1.00 29 3211 0 0 0 0 6
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3DW4_A - -0.02 0.00 0.00 0 345 6 0 6 0 10
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.44 0.38 0.54 15 4343 14 1 12 1 25
3HAY_E - 0.91 0.83 1.00 15 2263 3 0 0 3 3
3HJW_D - 0.91 0.83 1.00 15 1638 2 0 0 2 3
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.40 0.33 0.50 3 490 5 0 3 2 6
3IVN_B 0.76 0.58 1.00 18 2328 0 0 0 0 13
3IWN_A 0.63 0.58 0.70 19 4251 9 0 8 1 14
3IZF_C 0.70 0.61 0.80 33 6862 8 1 7 0 21
3J0L_1 - 0.76 0.63 0.92 12 1212 2 0 1 1 7
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.35 0.31 0.42 5 1116 7 1 6 0 11
3J0L_h - 0.75 0.60 0.93 26 6077 4 0 2 2 17
3J0L_7 - -0.01 0.00 0.00 0 1213 12 0 12 0 17
3J0L_g - -0.01 0.00 0.00 0 460 5 1 4 0 4
3J0L_2 - 0.25 0.24 0.28 8 6187 25 0 21 4 25
3J16_L 0.41 0.37 0.48 11 2752 12 1 11 0 19
3JYV_7 -0.01 0.00 0.00 0 2829 21 0 21 0 32
3JYX_4 0.19 0.21 0.17 7 12204 38 11 24 3 26
3JYX_3 0.63 0.63 0.63 17 6301 21 1 9 11 10
3KTW_C - 0.75 0.65 0.88 28 4528 5 2 2 1 15
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.77 0.69 0.86 42 9131 8 0 7 1 19
3NKB_B - 0.57 0.50 0.65 13 1996 7 0 7 0 13
3NMU_E - 0.54 0.50 0.60 3 556 5 0 2 3 3
3O58_2 0.72 0.74 0.70 28 7220 13 4 8 1 10
3O58_3 0.34 0.34 0.34 12 12368 34 2 21 11 23
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.75 0.63 0.90 45 12830 7 1 4 2 27
3R4F_A - 0.96 0.92 1.00 23 2122 0 0 0 0 2
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.61 0.50 0.75 21 3888 7 1 6 0 21
3SIU_F - 0.85 0.73 1.00 8 370 0 0 0 0 3
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_3 0.70 0.59 0.82 32 7101 7 1 6 0 22
4A1C_2 0.14 0.15 0.13 5 11742 43 5 29 9 28
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ENB_A 0.70 0.58 0.85 11 1262 2 1 1 0 8
4ENC_A 0.32 0.26 0.42 5 1314 7 0 7 0 14

^top



Performance of MCFold - scored lower in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1368
Total TN 232902
Total FP 1236
Total FP CONTRA 142
Total FP INCONS 907
Total FP COMP 187
Total FN 922
Total Scores
MCC 0.577
Average MCC ± 95% Confidence Intervals 0.643 ± 0.066
Sensitivity 0.597
Positive Predictive Value 0.566
Nr of predictions 109

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 1 0 0 1 0
2K63_A - 0.90 0.90 0.90 9 396 4 0 1 3 1
2K66_A - 1.00 1.00 1.00 10 221 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 222 1 0 0 1 0
2KDQ_B 0.95 0.91 1.00 10 396 1 0 0 1 1
2KE6_A 1.00 1.00 1.00 19 1109 2 0 0 2 0
2KEZ_A - 0.90 0.90 0.90 9 266 1 0 1 0 1
2KFC_A - 0.28 0.25 0.36 5 616 9 2 7 0 15
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 548 2 0 0 2 0
2KU0_A - 0.97 0.94 1.00 15 688 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 21 1107 0 0 0 0 0
2KUU_A 0.98 0.95 1.00 20 1108 1 0 0 1 1
2KUV_A 0.98 0.95 1.00 21 1107 0 0 0 0 1
2KUW_A 0.95 0.95 0.95 20 1107 1 0 1 0 1
2KX8_A 0.91 0.89 0.94 16 844 2 0 1 1 2
2KXM_A - 1.00 1.00 1.00 10 341 1 0 0 1 0
2KZL_A - 0.80 0.84 0.76 16 1464 9 1 4 4 3
2L1F_B 0.73 0.76 0.70 19 2118 9 0 8 1 6
2L1F_A 0.72 0.75 0.69 18 2054 9 0 8 1 6
2L2K_A - 1.00 1.00 1.00 19 842 1 0 0 1 0
2L3C_B - 0.97 0.94 1.00 16 545 0 0 0 0 1
2L3E_A - 1.00 1.00 1.00 13 582 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 34 2451 0 0 0 0 0
2L5Z_A - 0.96 0.92 1.00 11 314 0 0 0 0 1
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - -0.03 0.00 0.00 0 615 15 0 15 0 19
2LBS_A - 1.00 1.00 1.00 14 482 1 0 0 1 0
2LC8_A 0.41 0.45 0.39 9 1517 16 0 14 2 11
2LDL_A - 0.95 0.91 1.00 10 341 2 0 0 2 1
2LDT_A - 0.75 0.73 0.79 11 451 3 0 3 0 4
2LHP_A - 1.00 1.00 1.00 16 650 1 0 0 1 0
2LI4_A - 0.97 0.94 1.00 15 481 0 0 0 0 1
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 1.00 1.00 1.00 10 266 0 0 0 0 0
2LKR_A - 0.93 0.92 0.95 36 6067 14 0 2 12 3
2LQZ_A - 0.91 0.91 0.91 10 340 1 1 0 0 1
2LWK_A - 0.92 0.92 0.92 12 483 2 0 1 1 1
2RP0_A - 0.44 0.42 0.50 5 341 5 1 4 0 7
2RPT_A - 1.00 1.00 1.00 7 183 2 0 0 2 0
2RRC_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2WRQ_Y 0.27 0.35 0.22 6 2823 25 8 13 4 11
2WW9_E - -0.02 0.00 0.00 0 547 14 0 14 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_D - -0.01 0.00 0.00 0 1925 28 5 23 0 26
2WWQ_V 0.15 0.18 0.15 5 2892 30 4 25 1 23
2XKV_B 0.20 0.25 0.17 5 4530 37 2 23 12 15
2XQD_Y 0.34 0.37 0.32 10 2819 22 1 20 1 17
2XXA_G 0.23 0.24 0.24 10 5110 32 1 30 1 32
2Y8W_B - 1.00 1.00 1.00 6 184 2 0 0 2 0
2Y9C_V - 0.62 0.67 0.59 16 2251 13 2 9 2 8
2YIE_X - 0.06 0.08 0.06 1 1361 21 1 15 5 11
2YIE_Z - 0.36 0.50 0.27 6 1518 18 4 12 2 6
2ZY6_A - 0.51 0.58 0.47 7 688 8 3 5 0 5
2ZZN_D 0.61 0.63 0.61 17 2457 13 1 10 2 10
3A2K_C 0.44 0.46 0.42 13 2895 18 2 16 0 15
3A3A_A 0.89 0.86 0.91 32 3620 3 1 2 0 5
3AKZ_H 0.40 0.43 0.39 12 2670 19 3 16 0 16
3AM1_B - 0.93 0.89 0.97 31 3208 2 0 1 1 4
3AMU_B 0.44 0.48 0.42 13 2972 18 2 16 0 14
3DW4_A - 0.90 0.90 0.90 9 341 3 0 1 2 1
3GCA_A - 0.04 0.06 0.07 1 514 13 1 12 0 16
3GX2_A 0.47 0.48 0.48 19 4331 22 0 21 1 21
3HAY_E - 1.00 1.00 1.00 18 2260 11 0 0 11 0
3HJW_D - 0.94 0.94 0.94 17 1635 8 0 1 7 1
3IAB_R - 0.84 0.93 0.76 13 1018 8 2 2 4 1
3ID5_D - 0.34 0.33 0.38 3 488 9 0 5 4 6
3IVN_B 0.39 0.39 0.40 12 2316 18 0 18 0 19
3IWN_A 0.19 0.21 0.18 7 4238 33 8 25 0 26
3IZF_C 0.71 0.69 0.74 37 6853 14 0 13 1 17
3J0L_1 - 0.79 0.79 0.79 15 1206 6 0 4 2 4
3J0L_8 - 1.00 1.00 1.00 8 182 0 0 0 0 0
3J0L_a - 0.16 0.19 0.17 3 1110 15 1 14 0 13
3J0L_h - 0.91 0.91 0.91 39 6062 13 0 4 9 4
3J0L_7 - -0.01 0.00 0.00 0 1206 19 4 15 0 17
3J0L_g - 0.13 0.25 0.08 1 452 12 8 4 0 3
3J0L_2 - 0.21 0.24 0.18 8 6172 39 7 29 3 25
3J16_L 0.45 0.47 0.44 14 2743 19 1 17 1 16
3JYV_7 0.21 0.22 0.21 7 2817 26 0 26 0 25
3JYX_4 0.20 0.24 0.17 8 12199 43 16 23 4 25
3JYX_3 0.41 0.52 0.33 14 6285 34 12 17 5 13
3KTW_C - 0.40 0.40 0.43 17 4520 24 4 19 1 26
3LA5_A 0.32 0.32 0.34 11 2453 21 1 20 0 23
3NDB_M - 0.23 0.23 0.24 14 9121 46 1 44 1 47
3NKB_B - 0.56 0.58 0.56 15 1989 13 0 12 1 11
3NMU_E - 0.28 0.33 0.25 2 553 12 0 6 6 4
3O58_2 0.20 0.24 0.17 9 7208 44 5 38 1 29
3O58_3 0.22 0.26 0.19 9 12355 45 9 30 6 26
3OVB_D - 1.00 1.00 1.00 11 584 2 0 0 2 0
3OVS_D - 1.00 1.00 1.00 13 548 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 12 729 5 0 0 5 0
3PDR_A 0.66 0.61 0.72 44 12819 19 0 17 2 28
3R4F_A - 0.92 0.92 0.92 23 2120 5 0 2 3 2
3R9X_C - 0.69 0.70 0.70 7 585 7 0 3 4 3
3RKF_A 0.70 0.65 0.76 22 2182 7 1 6 0 12
3SD1_A 0.33 0.33 0.35 14 3876 26 0 26 0 28
3SIU_F - 0.42 0.45 0.42 5 366 7 1 6 0 6
3SN2_B 0.55 0.58 0.54 7 393 6 0 6 0 5
3TRZ_Z - -0.04 0.00 0.00 0 201 9 0 9 0 6
3TS0_U - 1.00 1.00 1.00 6 247 2 0 0 2 0
3TS2_V - -0.02 0.00 0.00 0 268 9 0 8 1 5
3U4M_B - 0.59 0.59 0.59 22 3123 15 0 15 0 15
3VJR_D - 1.00 1.00 1.00 13 617 1 0 0 1 0
4A1C_3 0.68 0.67 0.71 36 7089 17 1 14 2 18
4A1C_2 0.13 0.15 0.11 5 11735 56 10 31 15 28
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ENB_A 0.61 0.63 0.60 12 1255 8 2 6 0 7
4ENC_A 0.28 0.32 0.27 6 1304 17 2 14 1 13

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.