CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CentroidFold - scored higher in this pairwise comparison

  4. Performance of MCFold - scored lower in this pairwise comparison

  5. Compile and download dataset for CentroidFold & MCFold [.zip] - may take several seconds...


Overview

Metric CentroidFold MCFold
MCC 0.630 > 0.510
Average MCC ± 95% Confidence Intervals 0.669 ± 0.054 > 0.582 ± 0.065
Sensitivity 0.546 > 0.525
Positive Predictive Value 0.734 > 0.503
Total TP 1550 > 1491
Total TN 342004 > 341155
Total FP 655 < 1702
Total FP CONTRA 73 < 187
Total FP INCONS 490 < 1284
Total FP COMP 92 < 231
Total FN 1291 < 1350
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CentroidFold and MCFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CentroidFold and MCFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CentroidFold and MCFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CentroidFold and MCFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CentroidFold and MCFold).

^top





Performance of CentroidFold - scored higher in this pairwise comparison

1. Total counts & total scores for CentroidFold

Total Base Pair Counts
Total TP 1550
Total TN 342004
Total FP 655
Total FP CONTRA 73
Total FP INCONS 490
Total FP COMP 92
Total FN 1291
Total Scores
MCC 0.630
Average MCC ± 95% Confidence Intervals 0.669 ± 0.054
Sensitivity 0.546
Positive Predictive Value 0.734
Nr of predictions 118

^top



2. Individual counts for CentroidFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KU0_A - 0.47 0.31 0.71 5 696 2 0 2 0 11
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.92 0.90 0.95 18 971 1 0 1 0 2
2LA5_A - 0.42 0.26 0.71 5 623 2 0 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.49 0.35 0.70 7 1530 3 1 2 0 13
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.82 0.74 0.91 29 6073 4 0 3 1 10
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LU0_A - 0.93 0.88 1.00 14 1162 0 0 0 0 2
2LWK_A - 0.92 0.92 0.92 12 483 1 0 1 0 1
2M58_A - 0.09 0.06 0.14 1 1646 6 0 6 0 16
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.57 0.59 0.56 10 2832 12 5 3 4 7
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_E - 0.00 0.00 0.00 0 561 0 0 0 0 14
2WW9_D - 0.29 0.12 0.75 3 1949 1 0 1 0 23
2WWQ_V 0.80 0.68 0.95 19 2906 3 0 1 2 9
2XKV_B 0.51 0.50 0.53 10 4541 25 0 9 16 10
2XQD_Y 0.75 0.67 0.86 18 2829 3 0 3 0 9
2XXA_G 0.15 0.12 0.20 5 5126 20 1 19 0 37
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.66 0.63 0.71 15 2257 8 1 5 2 9
2YIE_Z - 0.43 0.42 0.45 5 1529 6 3 3 0 7
2YIE_X - 0.67 0.58 0.78 7 1369 3 1 1 1 5
2ZZN_D 0.82 0.78 0.88 21 2461 3 0 3 0 6
3A2K_C 0.44 0.43 0.46 12 2900 14 2 12 0 16
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3AKZ_H 0.43 0.39 0.48 11 2678 12 2 10 0 17
3AM1_B - 0.68 0.63 0.73 22 3210 8 1 7 0 13
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3GX2_A 0.79 0.63 1.00 25 4346 1 0 0 1 15
3IAB_R - 0.86 0.86 0.86 12 1021 4 1 1 2 2
3ID5_D - -0.01 0.00 0.00 0 490 6 3 3 0 9
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.80 0.67 0.96 22 4255 1 0 1 0 11
3IZF_C 0.68 0.61 0.77 33 6860 10 1 9 0 21
3J0L_1 - 0.65 0.47 0.90 9 1215 2 0 1 1 10
3J0L_2 - 0.25 0.24 0.28 8 6187 24 0 21 3 25
3J0L_g - 0.00 0.00 0.00 0 465 0 0 0 0 4
3J0L_7 - -0.01 0.00 0.00 0 1220 5 0 5 0 17
3J0L_h - 0.77 0.65 0.90 28 6074 5 0 3 2 15
3J0L_a - 0.41 0.31 0.56 5 1119 4 2 2 0 11
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J16_L 0.50 0.40 0.63 12 2756 7 0 7 0 18
3J20_0 0.44 0.40 0.50 12 2826 13 0 12 1 18
3J2L_3 0.53 0.49 0.58 26 7830 21 0 19 2 27
3J3D_C 0.49 0.43 0.57 12 2754 9 0 9 0 16
3J3E_7 0.46 0.37 0.59 20 7106 14 1 13 0 34
3J3E_8 0.00 0.00 0.00 0 7485 21 2 16 3 33
3J3F_7 0.20 0.18 0.23 9 7221 30 1 29 0 41
3J3F_8 0.34 0.33 0.34 12 12211 34 4 19 11 24
3JYV_7 0.77 0.59 1.00 19 2831 0 0 0 0 13
3JYX_4 0.23 0.21 0.25 7 12218 25 5 16 4 26
3JYX_3 0.29 0.30 0.30 8 6301 20 8 11 1 19
3KTW_C - 0.45 0.40 0.53 17 4528 16 2 13 1 26
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.80 0.70 0.91 43 9133 5 0 4 1 18
3NKB_B - 0.57 0.50 0.65 13 1996 7 0 7 0 13
3NMU_E - 0.71 0.50 1.00 3 558 3 0 0 3 3
3O58_3 0.38 0.29 0.50 10 12383 10 0 10 0 25
3O58_2 0.82 0.76 0.88 29 7227 7 1 3 3 9
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 1 0 0 1 1
3PDR_A 0.73 0.60 0.90 43 12832 7 0 5 2 29
3R4F_A - 0.81 0.76 0.86 19 2123 3 1 2 0 6
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.58 0.48 0.71 20 3888 8 1 7 0 22
3SIU_F - 0.79 0.64 1.00 7 371 0 0 0 0 4
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.57 0.46 0.71 17 3136 7 0 7 0 20
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
3ZEX_F - 0.00 0.00 0.00 0 2619 9 2 7 0 12
3ZEX_C 0.47 0.27 0.82 14 14179 3 1 2 0 38
3ZEX_G - 0.36 0.28 0.46 21 16425 29 2 23 4 53
3ZEX_D 0.73 0.59 0.91 29 6989 3 0 3 0 20
3ZEX_H - 0.19 0.18 0.21 7 9011 27 4 23 0 31
3ZND_W 0.20 0.22 0.19 5 2977 23 1 20 2 18
4A1C_2 0.16 0.15 0.17 5 11752 29 5 19 5 28
4A1C_3 0.67 0.56 0.81 30 7103 7 1 6 0 24
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ATO_G - 0.31 0.20 0.50 2 524 2 0 2 0 8
4ENB_A 0.76 0.58 1.00 11 1264 0 0 0 0 8
4ENC_A 0.73 0.58 0.92 11 1314 1 1 0 0 8
4FNJ_A - 0.15 0.13 0.22 2 586 7 0 7 0 14
4FRG_B 0.24 0.22 0.28 7 3461 18 3 15 0 25
4FRN_A 0.40 0.28 0.59 10 5134 7 2 5 0 26
4HXH_A - 1.00 1.00 1.00 6 319 0 0 0 0 0
4JF2_A 0.78 0.61 1.00 19 2831 0 0 0 0 12
4JRC_A - 0.81 0.70 0.94 16 1523 1 0 1 0 7

^top



Performance of MCFold - scored lower in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1491
Total TN 341155
Total FP 1702
Total FP CONTRA 187
Total FP INCONS 1284
Total FP COMP 231
Total FN 1350
Total Scores
MCC 0.510
Average MCC ± 95% Confidence Intervals 0.582 ± 0.065
Sensitivity 0.525
Positive Predictive Value 0.503
Nr of predictions 118

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 222 1 0 0 1 0
2KE6_A 1.00 1.00 1.00 19 1109 2 0 0 2 0
2KMJ_A - 1.00 1.00 1.00 11 367 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 548 2 0 0 2 0
2KU0_A - 0.97 0.94 1.00 15 688 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 21 1107 0 0 0 0 0
2KUU_A 0.98 0.95 1.00 20 1108 1 0 0 1 1
2KUV_A 0.98 0.95 1.00 21 1107 0 0 0 0 1
2KUW_A 0.95 0.95 0.95 20 1107 1 0 1 0 1
2KX8_A 0.91 0.89 0.94 16 844 2 0 1 1 2
2KXM_A - 1.00 1.00 1.00 10 341 1 0 0 1 0
2KZL_A - 0.80 0.84 0.76 16 1464 9 1 4 4 3
2L1F_A 0.72 0.75 0.69 18 2054 9 0 8 1 6
2L1F_B 0.73 0.76 0.70 19 2118 9 0 8 1 6
2L2K_A - 1.00 1.00 1.00 19 842 1 0 0 1 0
2L3C_B - 0.97 0.94 1.00 16 545 0 0 0 0 1
2L3E_A - 1.00 1.00 1.00 13 582 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 34 2451 0 0 0 0 0
2L5Z_A - 0.96 0.92 1.00 11 314 0 0 0 0 1
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - -0.03 0.00 0.00 0 615 15 0 15 0 19
2LBS_A - 1.00 1.00 1.00 14 482 1 0 0 1 0
2LC8_A 0.41 0.45 0.39 9 1517 16 0 14 2 11
2LDL_A - 0.95 0.91 1.00 10 341 2 0 0 2 1
2LDT_A - 0.75 0.73 0.79 11 451 3 0 3 0 4
2LHP_A - 1.00 1.00 1.00 16 650 1 0 0 1 0
2LI4_A - 0.97 0.94 1.00 15 481 0 0 0 0 1
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 1.00 1.00 1.00 10 266 0 0 0 0 0
2LKR_A - 0.93 0.92 0.95 36 6067 14 0 2 12 3
2LQZ_A - 0.91 0.91 0.91 10 340 1 1 0 0 1
2LU0_A - 1.00 1.00 1.00 16 1160 5 0 0 5 0
2LWK_A - 0.92 0.92 0.92 12 483 2 0 1 1 1
2M58_A - 0.20 0.24 0.18 4 1631 19 2 16 1 13
2RPT_A - 1.00 1.00 1.00 7 183 2 0 0 2 0
2RRC_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2WRQ_Y 0.27 0.35 0.22 6 2823 25 8 13 4 11
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_E - -0.02 0.00 0.00 0 547 14 0 14 0 14
2WW9_D - -0.01 0.00 0.00 0 1925 28 5 23 0 26
2WWQ_V 0.15 0.18 0.15 5 2892 30 4 25 1 23
2XKV_B 0.20 0.25 0.17 5 4530 37 2 23 12 15
2XQD_Y 0.34 0.37 0.32 10 2819 22 1 20 1 17
2XXA_G 0.23 0.24 0.24 10 5110 32 1 30 1 32
2Y8W_B - 1.00 1.00 1.00 6 184 2 0 0 2 0
2Y9C_V - 0.62 0.67 0.59 16 2251 13 2 9 2 8
2YIE_Z - 0.36 0.50 0.27 6 1518 18 4 12 2 6
2YIE_X - 0.06 0.08 0.06 1 1361 21 1 15 5 11
2ZZN_D 0.61 0.63 0.61 17 2457 13 1 10 2 10
3A2K_C 0.44 0.46 0.42 13 2895 18 2 16 0 15
3A3A_A 0.89 0.86 0.91 32 3620 3 1 2 0 5
3AKZ_H 0.40 0.43 0.39 12 2670 19 3 16 0 16
3AM1_B - 0.93 0.89 0.97 31 3208 2 0 1 1 4
3AMU_B 0.44 0.48 0.42 13 2972 18 2 16 0 14
3GX2_A 0.47 0.48 0.48 19 4331 22 0 21 1 21
3IAB_R - 0.84 0.93 0.76 13 1018 8 2 2 4 1
3ID5_D - 0.34 0.33 0.38 3 488 9 0 5 4 6
3IVN_B 0.39 0.39 0.40 12 2316 18 0 18 0 19
3IWN_A 0.19 0.21 0.18 7 4238 33 8 25 0 26
3IZF_C 0.71 0.69 0.74 37 6853 14 0 13 1 17
3J0L_1 - 0.79 0.79 0.79 15 1206 6 0 4 2 4
3J0L_2 - 0.21 0.24 0.18 8 6172 39 7 29 3 25
3J0L_g - 0.13 0.25 0.08 1 452 12 8 4 0 3
3J0L_7 - -0.01 0.00 0.00 0 1206 19 4 15 0 17
3J0L_h - 0.91 0.91 0.91 39 6062 13 0 4 9 4
3J0L_a - 0.16 0.19 0.17 3 1110 15 1 14 0 13
3J0L_8 - 1.00 1.00 1.00 8 182 0 0 0 0 0
3J16_L 0.45 0.47 0.44 14 2743 19 1 17 1 16
3J20_0 0.59 0.57 0.63 17 2823 12 1 9 2 13
3J2L_3 0.58 0.57 0.60 30 7825 23 2 18 3 23
3J3D_C 0.42 0.43 0.43 12 2747 18 2 14 2 16
3J3E_7 0.40 0.39 0.42 21 7090 29 3 26 0 33
3J3E_8 0.12 0.12 0.12 4 7470 46 3 26 17 29
3J3F_7 0.74 0.74 0.74 37 7210 17 2 11 4 13
3J3F_8 0.13 0.17 0.11 6 12191 61 9 40 12 30
3JYV_7 0.21 0.22 0.21 7 2817 26 0 26 0 25
3JYX_4 0.20 0.24 0.17 8 12199 43 16 23 4 25
3JYX_3 0.41 0.52 0.33 14 6285 34 12 17 5 13
3KTW_C - 0.40 0.40 0.43 17 4520 24 4 19 1 26
3LA5_A 0.32 0.32 0.34 11 2453 21 1 20 0 23
3NDB_M - 0.23 0.23 0.24 14 9121 46 1 44 1 47
3NKB_B - 0.56 0.58 0.56 15 1989 13 0 12 1 11
3NMU_E - 0.28 0.33 0.25 2 553 12 0 6 6 4
3O58_3 0.22 0.26 0.19 9 12355 45 9 30 6 26
3O58_2 0.20 0.24 0.17 9 7208 44 5 38 1 29
3OVB_D - 1.00 1.00 1.00 11 584 2 0 0 2 0
3OVS_D - 1.00 1.00 1.00 13 548 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 12 729 5 0 0 5 0
3PDR_A 0.66 0.61 0.72 44 12819 19 0 17 2 28
3R4F_A - 0.92 0.92 0.92 23 2120 5 0 2 3 2
3R9X_C - 0.69 0.70 0.70 7 585 7 0 3 4 3
3RKF_A 0.70 0.65 0.76 22 2182 7 1 6 0 12
3SD1_A 0.33 0.33 0.35 14 3876 26 0 26 0 28
3SIU_F - 0.42 0.45 0.42 5 366 7 1 6 0 6
3SN2_B 0.55 0.58 0.54 7 393 6 0 6 0 5
3TRZ_Z - -0.04 0.00 0.00 0 201 9 0 9 0 6
3TS0_U - 1.00 1.00 1.00 6 247 2 0 0 2 0
3TS2_V - -0.02 0.00 0.00 0 268 9 0 8 1 5
3U4M_B - 0.59 0.59 0.59 22 3123 15 0 15 0 15
3VJR_D - 1.00 1.00 1.00 13 617 1 0 0 1 0
3ZEX_F - -0.01 0.00 0.00 0 2610 29 4 14 11 12
3ZEX_C 0.24 0.21 0.28 11 14156 29 3 26 0 41
3ZEX_G - 0.00 0.00 0.00 0 16456 15 1 14 0 74
3ZEX_D 0.17 0.18 0.18 9 6970 42 4 38 0 40
3ZEX_H - 0.14 0.18 0.12 7 8987 52 14 37 1 31
3ZND_W 0.19 0.22 0.18 5 2975 26 1 22 3 18
4A1C_2 0.13 0.15 0.11 5 11735 56 10 31 15 28
4A1C_3 0.68 0.67 0.71 36 7089 17 1 14 2 18
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ATO_G - 0.30 0.30 0.33 3 519 8 0 6 2 7
4ENB_A 0.61 0.63 0.60 12 1255 8 2 6 0 7
4ENC_A 0.28 0.32 0.27 6 1304 17 2 14 1 13
4FNJ_A - 0.83 0.81 0.87 13 580 3 0 2 1 3
4FRG_B 0.32 0.34 0.31 11 3450 25 0 25 0 21
4FRN_A 0.12 0.14 0.12 5 5110 38 1 35 2 31
4HXH_A - 1.00 1.00 1.00 6 319 4 0 0 4 0
4JF2_A 0.66 0.68 0.66 21 2818 11 0 11 0 10
4JRC_A - 0.20 0.22 0.21 5 1516 19 0 19 0 18

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.