CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Contrafold - scored higher in this pairwise comparison

  4. Performance of MCFold - scored lower in this pairwise comparison

  5. Compile and download dataset for Contrafold & MCFold [.zip] - may take several seconds...


Overview

Metric Contrafold MCFold
MCC 0.689 > 0.577
Average MCC ± 95% Confidence Intervals 0.725 ± 0.052 > 0.643 ± 0.066
Sensitivity 0.618 > 0.597
Positive Predictive Value 0.773 > 0.566
Total TP 1415 > 1368
Total TN 233489 > 232902
Total FP 512 < 1236
Total FP CONTRA 65 < 142
Total FP INCONS 350 < 907
Total FP COMP 97 < 187
Total FN 875 < 922
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Contrafold and MCFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Contrafold and MCFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Contrafold and MCFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Contrafold and MCFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Contrafold and MCFold).

^top





Performance of Contrafold - scored higher in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 1415
Total TN 233489
Total FP 512
Total FP CONTRA 65
Total FP INCONS 350
Total FP COMP 97
Total FN 875
Total Scores
MCC 0.689
Average MCC ± 95% Confidence Intervals 0.725 ± 0.052
Sensitivity 0.618
Positive Predictive Value 0.773
Nr of predictions 109

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 0 0 0 0 0
2K63_A - 0.90 0.90 0.90 9 396 2 0 1 1 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - 0.45 0.25 0.83 5 624 1 0 1 0 15
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 1.00 1.00 1.00 13 548 0 0 0 0 0
2KU0_A - 0.55 0.44 0.70 7 693 3 0 3 0 9
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.75 0.74 0.78 14 1467 4 2 2 0 5
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.92 0.90 0.95 18 971 1 0 1 0 2
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.45 0.35 0.58 7 1528 5 2 3 0 13
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.84 0.79 0.89 31 6070 6 0 4 2 8
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.88 0.85 0.92 11 484 1 0 1 0 2
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.83 0.70 1.00 7 224 0 0 0 0 3
2WRQ_Y 0.57 0.59 0.56 10 2832 13 5 3 5 7
2WW9_E - 0.00 0.00 0.00 0 561 0 0 0 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_D - 0.39 0.23 0.67 6 1944 3 2 1 0 20
2WWQ_V 0.78 0.68 0.90 19 2905 4 0 2 2 9
2XKV_B 0.51 0.50 0.53 10 4541 25 0 9 16 10
2XQD_Y 0.78 0.70 0.86 19 2828 3 0 3 0 8
2XXA_G 0.13 0.12 0.15 5 5117 29 1 28 0 37
2Y8W_B - 1.00 1.00 1.00 6 184 3 0 0 3 0
2Y9C_V - 0.66 0.63 0.71 15 2257 8 1 5 2 9
2YIE_X - 0.73 0.67 0.80 8 1368 3 1 1 1 4
2YIE_Z - 0.43 0.42 0.45 5 1529 6 3 3 0 7
2ZY6_A - 0.58 0.58 0.58 7 691 5 0 5 0 5
2ZZN_D 0.82 0.78 0.88 21 2461 3 0 3 0 6
3A2K_C 0.44 0.43 0.46 12 2900 14 2 12 0 16
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3AKZ_H 0.39 0.39 0.41 11 2674 17 4 12 1 17
3AM1_B - 0.68 0.63 0.73 22 3210 8 1 7 0 13
3AMU_B 0.65 0.59 0.73 16 2981 8 0 6 2 11
3DW4_A - -0.02 0.00 0.00 0 345 6 0 6 0 10
3GCA_A - 0.49 0.29 0.83 5 522 1 0 1 0 12
3GX2_A 0.77 0.63 0.96 25 4345 2 0 1 1 15
3HAY_E - 0.91 0.83 1.00 15 2263 6 0 0 6 3
3HJW_D - 0.94 0.89 1.00 16 1637 2 0 0 2 2
3IAB_R - 0.86 0.86 0.86 12 1021 4 1 1 2 2
3ID5_D - 0.49 0.33 0.75 3 492 6 0 1 5 6
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.80 0.67 0.96 22 4255 1 0 1 0 11
3IZF_C 0.68 0.61 0.77 33 6860 10 1 9 0 21
3J0L_1 - 0.73 0.58 0.92 11 1213 2 0 1 1 8
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.50 0.44 0.58 7 1116 5 2 3 0 9
3J0L_h - 0.75 0.65 0.88 28 6073 6 0 4 2 15
3J0L_7 - -0.01 0.00 0.00 0 1218 7 0 7 0 17
3J0L_g - 0.00 0.00 0.00 0 464 1 1 0 0 4
3J0L_2 - 0.25 0.24 0.28 8 6187 26 0 21 5 25
3J16_L 0.46 0.40 0.55 12 2753 10 1 9 0 18
3JYV_7 -0.01 0.00 0.00 0 2830 20 0 20 0 32
3JYX_4 0.20 0.21 0.20 7 12211 35 6 22 7 26
3JYX_3 0.28 0.30 0.27 8 6298 23 8 14 1 19
3KTW_C - 0.44 0.40 0.50 17 4526 18 2 15 1 26
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.81 0.72 0.92 44 9132 5 0 4 1 17
3NKB_B - 0.54 0.50 0.59 13 1994 9 0 9 0 13
3NMU_E - 0.71 0.50 1.00 3 558 3 0 0 3 3
3O58_2 0.78 0.76 0.81 29 7224 10 2 5 3 9
3O58_3 0.28 0.26 0.31 9 12374 20 3 17 0 26
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 1 0 0 1 1
3PDR_A 0.69 0.60 0.80 43 12826 13 0 11 2 29
3R4F_A - 0.79 0.76 0.83 19 2122 4 1 3 0 6
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.73 0.59 0.91 20 2189 2 1 1 0 14
3SD1_A 0.57 0.48 0.69 20 3887 9 2 7 0 22
3SIU_F - 0.74 0.64 0.88 7 370 1 0 1 0 4
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.59 0.46 0.77 17 3138 5 0 5 0 20
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_3 0.66 0.57 0.78 31 7100 9 1 8 0 23
4A1C_2 0.16 0.15 0.17 5 11751 33 5 20 8 28
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ENB_A 0.73 0.58 0.92 11 1263 1 1 0 0 8
4ENC_A 0.73 0.58 0.92 11 1314 1 1 0 0 8

^top



Performance of MCFold - scored lower in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1368
Total TN 232902
Total FP 1236
Total FP CONTRA 142
Total FP INCONS 907
Total FP COMP 187
Total FN 922
Total Scores
MCC 0.577
Average MCC ± 95% Confidence Intervals 0.643 ± 0.066
Sensitivity 0.597
Positive Predictive Value 0.566
Nr of predictions 109

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 1 0 0 1 0
2K63_A - 0.90 0.90 0.90 9 396 4 0 1 3 1
2K66_A - 1.00 1.00 1.00 10 221 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 222 1 0 0 1 0
2KDQ_B 0.95 0.91 1.00 10 396 1 0 0 1 1
2KE6_A 1.00 1.00 1.00 19 1109 2 0 0 2 0
2KEZ_A - 0.90 0.90 0.90 9 266 1 0 1 0 1
2KFC_A - 0.28 0.25 0.36 5 616 9 2 7 0 15
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 548 2 0 0 2 0
2KU0_A - 0.97 0.94 1.00 15 688 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 21 1107 0 0 0 0 0
2KUU_A 0.98 0.95 1.00 20 1108 1 0 0 1 1
2KUV_A 0.98 0.95 1.00 21 1107 0 0 0 0 1
2KUW_A 0.95 0.95 0.95 20 1107 1 0 1 0 1
2KX8_A 0.91 0.89 0.94 16 844 2 0 1 1 2
2KXM_A - 1.00 1.00 1.00 10 341 1 0 0 1 0
2KZL_A - 0.80 0.84 0.76 16 1464 9 1 4 4 3
2L1F_B 0.73 0.76 0.70 19 2118 9 0 8 1 6
2L1F_A 0.72 0.75 0.69 18 2054 9 0 8 1 6
2L2K_A - 1.00 1.00 1.00 19 842 1 0 0 1 0
2L3C_B - 0.97 0.94 1.00 16 545 0 0 0 0 1
2L3E_A - 1.00 1.00 1.00 13 582 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 34 2451 0 0 0 0 0
2L5Z_A - 0.96 0.92 1.00 11 314 0 0 0 0 1
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - -0.03 0.00 0.00 0 615 15 0 15 0 19
2LBS_A - 1.00 1.00 1.00 14 482 1 0 0 1 0
2LC8_A 0.41 0.45 0.39 9 1517 16 0 14 2 11
2LDL_A - 0.95 0.91 1.00 10 341 2 0 0 2 1
2LDT_A - 0.75 0.73 0.79 11 451 3 0 3 0 4
2LHP_A - 1.00 1.00 1.00 16 650 1 0 0 1 0
2LI4_A - 0.97 0.94 1.00 15 481 0 0 0 0 1
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 1.00 1.00 1.00 10 266 0 0 0 0 0
2LKR_A - 0.93 0.92 0.95 36 6067 14 0 2 12 3
2LQZ_A - 0.91 0.91 0.91 10 340 1 1 0 0 1
2LWK_A - 0.92 0.92 0.92 12 483 2 0 1 1 1
2RP0_A - 0.44 0.42 0.50 5 341 5 1 4 0 7
2RPT_A - 1.00 1.00 1.00 7 183 2 0 0 2 0
2RRC_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2WRQ_Y 0.27 0.35 0.22 6 2823 25 8 13 4 11
2WW9_E - -0.02 0.00 0.00 0 547 14 0 14 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_D - -0.01 0.00 0.00 0 1925 28 5 23 0 26
2WWQ_V 0.15 0.18 0.15 5 2892 30 4 25 1 23
2XKV_B 0.20 0.25 0.17 5 4530 37 2 23 12 15
2XQD_Y 0.34 0.37 0.32 10 2819 22 1 20 1 17
2XXA_G 0.23 0.24 0.24 10 5110 32 1 30 1 32
2Y8W_B - 1.00 1.00 1.00 6 184 2 0 0 2 0
2Y9C_V - 0.62 0.67 0.59 16 2251 13 2 9 2 8
2YIE_X - 0.06 0.08 0.06 1 1361 21 1 15 5 11
2YIE_Z - 0.36 0.50 0.27 6 1518 18 4 12 2 6
2ZY6_A - 0.51 0.58 0.47 7 688 8 3 5 0 5
2ZZN_D 0.61 0.63 0.61 17 2457 13 1 10 2 10
3A2K_C 0.44 0.46 0.42 13 2895 18 2 16 0 15
3A3A_A 0.89 0.86 0.91 32 3620 3 1 2 0 5
3AKZ_H 0.40 0.43 0.39 12 2670 19 3 16 0 16
3AM1_B - 0.93 0.89 0.97 31 3208 2 0 1 1 4
3AMU_B 0.44 0.48 0.42 13 2972 18 2 16 0 14
3DW4_A - 0.90 0.90 0.90 9 341 3 0 1 2 1
3GCA_A - 0.04 0.06 0.07 1 514 13 1 12 0 16
3GX2_A 0.47 0.48 0.48 19 4331 22 0 21 1 21
3HAY_E - 1.00 1.00 1.00 18 2260 11 0 0 11 0
3HJW_D - 0.94 0.94 0.94 17 1635 8 0 1 7 1
3IAB_R - 0.84 0.93 0.76 13 1018 8 2 2 4 1
3ID5_D - 0.34 0.33 0.38 3 488 9 0 5 4 6
3IVN_B 0.39 0.39 0.40 12 2316 18 0 18 0 19
3IWN_A 0.19 0.21 0.18 7 4238 33 8 25 0 26
3IZF_C 0.71 0.69 0.74 37 6853 14 0 13 1 17
3J0L_1 - 0.79 0.79 0.79 15 1206 6 0 4 2 4
3J0L_8 - 1.00 1.00 1.00 8 182 0 0 0 0 0
3J0L_a - 0.16 0.19 0.17 3 1110 15 1 14 0 13
3J0L_h - 0.91 0.91 0.91 39 6062 13 0 4 9 4
3J0L_7 - -0.01 0.00 0.00 0 1206 19 4 15 0 17
3J0L_g - 0.13 0.25 0.08 1 452 12 8 4 0 3
3J0L_2 - 0.21 0.24 0.18 8 6172 39 7 29 3 25
3J16_L 0.45 0.47 0.44 14 2743 19 1 17 1 16
3JYV_7 0.21 0.22 0.21 7 2817 26 0 26 0 25
3JYX_4 0.20 0.24 0.17 8 12199 43 16 23 4 25
3JYX_3 0.41 0.52 0.33 14 6285 34 12 17 5 13
3KTW_C - 0.40 0.40 0.43 17 4520 24 4 19 1 26
3LA5_A 0.32 0.32 0.34 11 2453 21 1 20 0 23
3NDB_M - 0.23 0.23 0.24 14 9121 46 1 44 1 47
3NKB_B - 0.56 0.58 0.56 15 1989 13 0 12 1 11
3NMU_E - 0.28 0.33 0.25 2 553 12 0 6 6 4
3O58_2 0.20 0.24 0.17 9 7208 44 5 38 1 29
3O58_3 0.22 0.26 0.19 9 12355 45 9 30 6 26
3OVB_D - 1.00 1.00 1.00 11 584 2 0 0 2 0
3OVS_D - 1.00 1.00 1.00 13 548 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 12 729 5 0 0 5 0
3PDR_A 0.66 0.61 0.72 44 12819 19 0 17 2 28
3R4F_A - 0.92 0.92 0.92 23 2120 5 0 2 3 2
3R9X_C - 0.69 0.70 0.70 7 585 7 0 3 4 3
3RKF_A 0.70 0.65 0.76 22 2182 7 1 6 0 12
3SD1_A 0.33 0.33 0.35 14 3876 26 0 26 0 28
3SIU_F - 0.42 0.45 0.42 5 366 7 1 6 0 6
3SN2_B 0.55 0.58 0.54 7 393 6 0 6 0 5
3TRZ_Z - -0.04 0.00 0.00 0 201 9 0 9 0 6
3TS0_U - 1.00 1.00 1.00 6 247 2 0 0 2 0
3TS2_V - -0.02 0.00 0.00 0 268 9 0 8 1 5
3U4M_B - 0.59 0.59 0.59 22 3123 15 0 15 0 15
3VJR_D - 1.00 1.00 1.00 13 617 1 0 0 1 0
4A1C_3 0.68 0.67 0.71 36 7089 17 1 14 2 18
4A1C_2 0.13 0.15 0.11 5 11735 56 10 31 15 28
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ENB_A 0.61 0.63 0.60 12 1255 8 2 6 0 7
4ENC_A 0.28 0.32 0.27 6 1304 17 2 14 1 13

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.