CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CentroidFold - scored higher in this pairwise comparison

  4. Performance of MCFold - scored lower in this pairwise comparison

  5. Compile and download dataset for CentroidFold & MCFold [.zip] - may take several seconds...


Overview

Metric CentroidFold MCFold
MCC 0.765 > 0.608
Average MCC ± 95% Confidence Intervals 0.768 ± 0.054 > 0.637 ± 0.066
Sensitivity 0.759 > 0.649
Positive Predictive Value 0.779 > 0.582
Total TP 1557 > 1332
Total TN 130909 > 130619
Total FP 654 < 1265
Total FP CONTRA 136 < 254
Total FP INCONS 306 < 703
Total FP COMP 212 < 308
Total FN 495 < 720
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CentroidFold and MCFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CentroidFold and MCFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CentroidFold and MCFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CentroidFold and MCFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CentroidFold and MCFold).

^top





Performance of CentroidFold - scored higher in this pairwise comparison

1. Total counts & total scores for CentroidFold

Total Base Pair Counts
Total TP 1557
Total TN 130909
Total FP 654
Total FP CONTRA 136
Total FP INCONS 306
Total FP COMP 212
Total FN 495
Total Scores
MCC 0.765
Average MCC ± 95% Confidence Intervals 0.768 ± 0.054
Sensitivity 0.759
Positive Predictive Value 0.779
Nr of predictions 118

^top



2. Individual counts for CentroidFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KU0_A - 0.53 0.42 0.71 5 285 2 2 0 0 7
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 0.94 0.94 0.94 17 339 2 0 1 1 1
2LA5_A - 0.77 0.71 0.83 5 269 2 0 1 1 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.51 0.39 0.70 7 518 3 1 2 0 11
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.88 0.86 0.89 25 2412 8 0 3 5 4
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LU0_A - 0.93 0.88 1.00 14 424 0 0 0 0 2
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2M58_A - -0.02 0.00 0.00 0 538 6 0 6 0 12
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 1.00 1.00 1.00 9 1143 13 0 0 13 0
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - 0.00 0.00 0.00 0 172 0 0 0 0 5
2WW9_D - 0.31 0.10 1.00 1 740 3 0 0 3 9
2WWQ_V 1.00 1.00 1.00 19 1185 3 0 0 3 0
2XKV_B 0.64 0.73 0.57 8 1821 27 0 6 21 3
2XQD_Y 0.83 0.81 0.85 17 1109 4 0 3 1 4
2XXA_G 0.13 0.11 0.17 4 2021 21 2 18 1 31
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
2YIE_X - 1.00 1.00 1.00 7 537 3 0 0 3 0
2ZZN_D 0.93 0.95 0.91 21 961 3 2 0 1 1
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3AKZ_H 0.48 0.50 0.48 10 1106 13 3 8 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3GX2_A 0.94 0.89 1.00 25 1424 1 0 0 1 3
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3ID5_D - -0.02 0.00 0.00 0 223 6 3 3 0 5
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.86 0.79 0.96 22 1449 1 0 1 0 6
3IZF_C 0.89 0.91 0.86 32 2603 11 0 5 6 3
3J0L_1 - 0.73 0.62 0.89 8 475 3 0 1 2 5
3J0L_2 - 0.28 0.31 0.28 8 2221 24 2 19 3 18
3J0L_g - 0.00 0.00 0.00 0 176 0 0 0 0 2
3J0L_7 - -0.01 0.00 0.00 0 514 5 1 4 0 10
3J0L_h - 0.89 0.88 0.90 28 2109 5 0 3 2 4
3J0L_a - 0.41 0.36 0.50 4 403 5 3 1 1 7
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J16_L 0.59 0.57 0.63 12 1140 7 0 7 0 9
3J20_0 0.54 0.57 0.52 12 1196 12 3 8 1 9
3J2L_3 0.74 0.76 0.72 26 2984 15 1 9 5 8
3J3D_C 0.73 0.79 0.68 15 946 7 3 4 0 4
3J3E_7 0.61 0.59 0.65 20 2710 13 2 9 2 14
3J3E_8 0.12 0.13 0.13 2 2726 22 4 10 8 13
3J3F_7 0.84 0.86 0.82 31 2896 9 1 6 2 5
3J3F_8 0.36 0.47 0.28 9 4729 37 12 11 14 10
3JYV_7 0.92 0.85 1.00 17 1094 2 0 0 2 3
3JYX_4 0.41 0.58 0.29 7 4732 25 11 6 8 5
3JYX_3 0.35 0.47 0.27 7 2352 21 14 5 2 8
3KTW_C - 0.58 0.60 0.58 15 1754 18 1 10 7 10
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.97 0.95 0.98 42 3671 6 0 1 5 2
3NKB_B - 0.66 0.68 0.65 13 715 7 0 7 0 6
3NMU_E - 1.00 1.00 1.00 3 213 3 0 0 3 0
3O58_3 0.44 0.41 0.47 9 4745 11 1 9 1 13
3O58_2 0.95 0.94 0.97 29 2724 7 0 1 6 2
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.88 0.86 0.90 43 4792 7 2 3 2 7
3R4F_A - 0.83 0.82 0.86 18 885 4 2 1 1 4
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.70 0.69 0.71 20 1505 8 4 4 0 9
3SIU_F - 0.93 0.88 1.00 7 138 0 0 0 0 1
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.81 0.77 0.85 17 1256 5 0 3 2 5
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3ZEX_F - 0.00 0.00 0.00 0 910 9 0 4 5 4
3ZEX_C 0.48 0.34 0.67 10 5359 6 1 4 1 19
3ZEX_G - 0.45 0.44 0.47 20 6464 32 6 17 9 25
3ZEX_D 0.88 0.86 0.91 30 2763 8 0 3 5 5
3ZEX_H - 0.27 0.37 0.21 7 3591 27 16 11 0 12
3ZND_W 0.24 0.38 0.16 3 1172 25 9 7 9 5
4A1C_2 0.22 0.25 0.19 5 4490 29 9 12 8 15
4A1C_3 0.80 0.78 0.83 29 2728 8 0 6 2 8
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4ATO_G - 0.61 0.57 0.67 4 214 2 0 2 0 3
4ENB_A 0.85 0.73 1.00 11 461 1 0 0 1 4
4ENC_A 0.85 0.73 1.00 11 485 1 0 0 1 4
4FNJ_A - 0.91 0.83 1.00 10 240 0 0 0 0 2
4FRG_B 0.75 0.71 0.81 17 1181 4 3 1 0 7
4FRN_A 0.80 0.71 0.91 20 1826 2 1 1 0 8
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.89 0.79 1.00 19 1063 0 0 0 0 5
4JRC_A - 0.38 0.41 0.39 7 604 11 0 11 0 10

^top



Performance of MCFold - scored lower in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1332
Total TN 130619
Total FP 1265
Total FP CONTRA 254
Total FP INCONS 703
Total FP COMP 308
Total FN 720
Total Scores
MCC 0.608
Average MCC ± 95% Confidence Intervals 0.637 ± 0.066
Sensitivity 0.649
Positive Predictive Value 0.582
Nr of predictions 118

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 1 0 0 1 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 200 0 0 0 0 0
2KU0_A - 1.00 1.00 1.00 12 280 1 0 0 1 0
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 1 0 0 1 0
2KZL_A - 1.00 1.00 1.00 13 507 5 0 0 5 0
2L1F_A 0.79 0.78 0.82 18 741 4 0 4 0 5
2L1F_B 0.80 0.79 0.83 19 768 4 0 4 0 5
2L2K_A - 1.00 1.00 1.00 17 332 1 0 0 1 0
2L3C_B - 1.00 1.00 1.00 14 225 1 0 0 1 0
2L3E_A - 1.00 1.00 1.00 12 228 2 0 0 2 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - -0.03 0.00 0.00 0 265 10 0 10 0 7
2LBS_A - 1.00 1.00 1.00 14 193 1 0 0 1 0
2LC8_A 0.44 0.44 0.47 8 511 10 0 9 1 10
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 0.95 0.91 1.00 10 152 0 0 0 0 1
2LHP_A - 1.00 1.00 1.00 15 246 1 0 0 1 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 11 0 0 11 0
2LQZ_A - 1.00 1.00 1.00 8 124 3 0 0 3 0
2LU0_A - 1.00 1.00 1.00 16 422 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 2 0 0 2 0
2M58_A - 0.22 0.25 0.23 3 531 13 1 9 3 9
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 1 0 0 1 0
2WRQ_Y 0.38 0.56 0.26 5 1133 21 8 6 7 4
2WW9_D - -0.02 0.00 0.00 0 723 21 6 12 3 10
2WW9_E - -0.03 0.00 0.00 0 165 8 1 6 1 5
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WWQ_V 0.19 0.21 0.19 4 1183 20 3 14 3 15
2XKV_B 0.20 0.27 0.16 3 1816 30 4 12 14 8
2XQD_Y 0.39 0.43 0.38 9 1105 18 2 13 3 12
2XXA_G 0.23 0.26 0.24 9 2007 30 1 28 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 2 0 0 2 0
2Y9C_V - 0.65 0.72 0.59 13 884 12 1 8 3 5
2YIE_X - -0.01 0.00 0.00 0 535 15 1 8 6 7
2YIE_Z - 0.53 0.63 0.45 5 591 8 2 4 2 3
2ZZN_D 0.69 0.73 0.67 16 960 11 1 7 3 6
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3A3A_A 0.98 0.97 1.00 29 1471 0 0 0 0 1
3AKZ_H 0.46 0.50 0.43 10 1104 16 5 8 3 10
3AM1_B - 1.00 1.00 1.00 29 1437 1 0 0 1 0
3AMU_B 0.50 0.58 0.44 11 1132 15 4 10 1 8
3GX2_A 0.58 0.61 0.57 17 1419 15 1 12 2 11
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3ID5_D - 0.54 0.60 0.50 3 223 7 1 2 4 2
3IVN_B 0.45 0.48 0.46 11 879 14 5 8 1 12
3IWN_A 0.19 0.21 0.20 6 1442 24 5 19 0 22
3IZF_C 0.92 0.94 0.89 33 2603 10 0 4 6 2
3J0L_g - -0.02 0.00 0.00 0 170 7 4 2 1 2
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_1 - 0.88 0.85 0.92 11 472 5 0 1 4 2
3J0L_h - 0.98 0.97 1.00 31 2109 2 0 0 2 1
3J0L_7 - -0.02 0.00 0.00 0 504 15 6 9 0 10
3J0L_a - 0.14 0.18 0.15 2 398 13 1 10 2 9
3J0L_2 - 0.25 0.27 0.25 7 2222 29 4 17 8 19
3J16_L 0.54 0.57 0.52 12 1136 12 4 7 1 9
3J20_0 0.66 0.71 0.63 15 1195 11 3 6 2 6
3J2L_3 0.77 0.79 0.75 27 2984 15 1 8 6 7
3J3D_C 0.54 0.63 0.48 12 943 13 4 9 0 7
3J3E_7 0.55 0.56 0.54 19 2706 23 1 15 7 15
3J3E_8 0.15 0.20 0.12 3 2716 36 11 12 13 12
3J3F_7 0.88 0.89 0.86 32 2897 9 1 4 4 4
3J3F_8 0.14 0.21 0.10 4 4720 51 17 20 14 15
3JYV_7 0.25 0.30 0.24 6 1086 20 8 11 1 14
3JYX_4 0.36 0.58 0.23 7 4725 37 19 5 13 5
3JYX_3 0.46 0.67 0.32 10 2347 28 15 6 7 5
3KTW_C - 0.55 0.60 0.52 15 1751 20 4 10 6 10
3LA5_A 0.44 0.44 0.48 11 931 12 2 10 0 14
3NDB_M - 0.29 0.30 0.30 13 3671 35 1 29 5 31
3NKB_B - 0.68 0.74 0.64 14 713 8 2 6 0 5
3NMU_E - 0.51 0.67 0.40 2 211 8 1 2 5 1
3O58_3 0.28 0.36 0.22 8 4727 39 14 15 10 14
3O58_2 0.25 0.29 0.23 9 2715 31 6 24 1 22
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.78 0.80 0.77 40 4788 14 5 7 2 10
3R4F_A - 0.95 0.95 0.95 21 884 4 0 1 3 1
3R9X_C - 0.87 0.88 0.88 7 221 5 0 1 4 1
3RKF_A 0.89 0.88 0.91 21 843 3 0 2 1 3
3SD1_A 0.43 0.45 0.43 13 1503 17 1 16 0 16
3SIU_F - 0.53 0.63 0.50 5 135 5 2 3 0 3
3SN2_B 0.58 0.58 0.64 7 143 4 0 4 0 5
3TRZ_Z - -0.05 0.00 0.00 0 88 4 0 4 0 5
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - -0.05 0.00 0.00 0 102 7 0 6 1 5
3U4M_B - 0.74 0.77 0.71 17 1252 10 1 6 3 5
3VJR_D - 1.00 1.00 1.00 12 239 1 0 0 1 0
3ZEX_D 0.23 0.26 0.23 9 2756 33 6 25 2 26
3ZEX_G - 0.00 0.00 0.00 0 6493 14 6 8 0 45
3ZEX_C 0.30 0.34 0.27 10 5337 28 6 21 1 19
3ZEX_H - 0.22 0.32 0.16 6 3588 36 15 16 5 13
3ZEX_F - 0.00 0.00 0.00 0 909 22 0 5 17 4
3ZND_W 0.23 0.38 0.15 3 1171 26 9 8 9 5
4A1C_3 0.86 0.86 0.86 32 2726 9 0 5 4 5
4A1C_2 0.18 0.25 0.14 5 4480 45 12 19 14 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4ATO_G - 0.36 0.43 0.33 3 211 8 0 6 2 4
4ENB_A 0.78 0.73 0.85 11 459 4 0 2 2 4
4ENC_A 0.34 0.33 0.38 5 483 11 0 8 3 10
4FNJ_A - 0.91 0.92 0.92 11 238 1 0 1 0 1
4FRG_B 0.35 0.38 0.35 9 1176 17 3 14 0 15
4FRN_A 0.09 0.11 0.10 3 1818 28 2 25 1 25
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.74 0.75 0.75 18 1058 7 1 5 1 6
4JRC_A - 0.24 0.29 0.23 5 600 17 0 17 0 12

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.