CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CentroidFold - scored higher in this pairwise comparison

  4. Performance of Fold - scored lower in this pairwise comparison

  5. Compile and download dataset for CentroidFold & Fold [.zip] - may take several seconds...


Overview

Metric CentroidFold Fold
MCC 0.683 > 0.633
Average MCC ± 95% Confidence Intervals 0.767 ± 0.054 > 0.731 ± 0.063
Sensitivity 0.673 > 0.666
Positive Predictive Value 0.694 > 0.604
Total TP 2041 > 2018
Total TN 1068166 > 1067766
Total FP 1221 < 1717
Total FP CONTRA 257 < 380
Total FP INCONS 641 < 941
Total FP COMP 323 < 396
Total FN 991 < 1014
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CentroidFold and Fold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CentroidFold and Fold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CentroidFold and Fold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CentroidFold and Fold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CentroidFold and Fold).

^top





Performance of CentroidFold - scored higher in this pairwise comparison

1. Total counts & total scores for CentroidFold

Total Base Pair Counts
Total TP 2041
Total TN 1068166
Total FP 1221
Total FP CONTRA 257
Total FP INCONS 641
Total FP COMP 323
Total FN 991
Total Scores
MCC 0.683
Average MCC ± 95% Confidence Intervals 0.767 ± 0.054
Sensitivity 0.673
Positive Predictive Value 0.694
Nr of predictions 113

^top



2. Individual counts for CentroidFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.53 0.42 0.71 5 285 2 2 0 0 7
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 0.94 0.94 0.94 17 339 2 0 1 1 1
2LA5_A - 0.77 0.71 0.83 5 269 2 0 1 1 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.51 0.39 0.70 7 518 3 1 2 0 11
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.88 0.86 0.89 25 2412 8 0 3 5 4
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LU0_A - 0.93 0.88 1.00 14 424 0 0 0 0 2
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2M58_A - -0.02 0.00 0.00 0 538 6 0 6 0 12
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WWQ_V 1.00 1.00 1.00 19 1185 3 0 0 3 0
2XKV_B 0.64 0.73 0.57 8 1821 27 0 6 21 3
2XQD_Y 0.83 0.81 0.85 17 1109 4 0 3 1 4
2XXA_G 0.13 0.11 0.17 4 2021 21 2 18 1 31
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
2YIE_X - 1.00 1.00 1.00 7 537 3 0 0 3 0
3ADB_C - 0.98 0.97 1.00 32 1787 0 0 0 0 1
3AKZ_H 0.48 0.50 0.48 10 1106 13 3 8 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3IYQ_A 0.29 0.39 0.22 20 22347 86 39 34 13 31
3IZ4_A 0.60 0.57 0.64 54 25451 38 16 15 7 41
3IZF_C 0.89 0.91 0.86 32 2603 11 0 5 6 3
3J0L_h - 0.89 0.88 0.90 28 2109 5 0 3 2 4
3J0L_1 - 0.73 0.62 0.89 8 475 3 0 1 2 5
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.28 0.31 0.28 8 2221 24 2 19 3 18
3J0L_a - 0.41 0.36 0.50 4 403 5 3 1 1 7
3J0L_g - 0.00 0.00 0.00 0 176 0 0 0 0 2
3J0L_7 - -0.01 0.00 0.00 0 514 5 1 4 0 10
3J16_L 0.59 0.57 0.63 12 1140 7 0 7 0 9
3J20_1 1.00 1.00 1.00 20 1092 2 0 0 2 0
3J20_2 0.70 0.69 0.70 286 421962 177 23 97 57 126
3J20_0 0.54 0.57 0.52 12 1196 12 3 8 1 9
3J2C_M - 0.60 0.60 0.60 58 39829 58 14 24 20 38
3J2C_O - 0.95 0.90 1.00 37 3950 6 0 0 6 4
3J2L_3 0.74 0.76 0.72 26 2984 15 1 9 5 8
3J3D_C 0.73 0.79 0.68 15 946 7 3 4 0 4
3J3E_8 0.12 0.13 0.13 2 2726 22 4 10 8 13
3J3E_7 0.61 0.59 0.65 20 2710 13 2 9 2 14
3J3F_8 0.36 0.47 0.28 9 4729 37 12 11 14 10
3J3F_7 0.84 0.86 0.82 31 2896 9 1 6 2 5
3J3V_B 0.72 0.74 0.71 20 2628 16 1 7 8 7
3NDB_M - 0.97 0.95 0.98 42 3671 6 0 1 5 2
3NKB_B - 0.66 0.68 0.65 13 715 7 0 7 0 6
3NMU_E - 1.00 1.00 1.00 3 213 3 0 0 3 0
3NPB_A 0.87 0.84 0.91 31 2244 8 1 2 5 6
3O58_2 0.95 0.94 0.97 29 2724 7 0 1 6 2
3O58_3 0.44 0.41 0.47 9 4745 11 1 9 1 13
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.88 0.86 0.90 43 4792 7 2 3 2 7
3R4F_A - 0.83 0.82 0.86 18 885 4 2 1 1 4
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.70 0.69 0.71 20 1505 8 4 4 0 9
3SIU_F - 0.93 0.88 1.00 7 138 0 0 0 0 1
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.81 0.77 0.85 17 1256 5 0 3 2 5
3UZL_B 0.93 0.88 1.00 14 1279 7 0 0 7 2
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W1K_J 0.97 0.97 0.97 30 1647 1 1 0 0 1
3W3S_B 0.95 0.91 1.00 30 1959 1 0 0 1 3
3ZEX_G - 0.45 0.44 0.47 20 6464 32 6 17 9 25
3ZEX_F - 0.00 0.00 0.00 0 910 9 0 4 5 4
3ZEX_C 0.48 0.34 0.67 10 5359 6 1 4 1 19
3ZEX_H - 0.27 0.37 0.21 7 3591 27 16 11 0 12
3ZEX_B - 0.27 0.25 0.31 85 421074 210 38 155 17 261
3ZEX_D 0.88 0.86 0.91 30 2763 8 0 3 5 5
3ZEX_E - 0.00 0.00 0.00 0 8255 51 19 30 2 34
3ZND_W 0.24 0.38 0.16 3 1172 25 9 7 9 5
4A1C_2 0.22 0.25 0.19 5 4490 29 9 12 8 15
4A1C_3 0.80 0.78 0.83 29 2728 8 0 6 2 8
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.50 0.48 0.54 14 1411 13 3 9 1 15
4ATO_G - 0.61 0.57 0.67 4 214 2 0 2 0 3
4ENB_A 0.85 0.73 1.00 11 461 1 0 0 1 4
4ENC_A 0.85 0.73 1.00 11 485 1 0 0 1 4
4FNJ_A - 0.91 0.83 1.00 10 240 0 0 0 0 2
4FRG_B 0.75 0.71 0.81 17 1181 4 3 1 0 7
4FRN_A 0.80 0.71 0.91 20 1826 2 1 1 0 8
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.89 0.79 1.00 19 1063 0 0 0 0 5
4JRC_A - 0.38 0.41 0.39 7 604 11 0 11 0 10

^top



Performance of Fold - scored lower in this pairwise comparison

1. Total counts & total scores for Fold

Total Base Pair Counts
Total TP 2018
Total TN 1067766
Total FP 1717
Total FP CONTRA 380
Total FP INCONS 941
Total FP COMP 396
Total FN 1014
Total Scores
MCC 0.633
Average MCC ± 95% Confidence Intervals 0.731 ± 0.063
Sensitivity 0.666
Positive Predictive Value 0.604
Nr of predictions 113

^top



2. Individual counts for Fold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.96 0.92 1.00 11 281 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 0.91 0.91 0.91 21 740 2 0 2 0 2
2L1F_B 0.91 0.92 0.92 22 767 2 0 2 0 2
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 513 15 3 12 0 18
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.90 0.90 0.90 26 2411 10 0 3 7 3
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LU0_A - 0.93 0.88 1.00 14 424 0 0 0 0 2
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2M58_A - -0.02 0.00 0.00 0 529 15 0 15 0 12
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WWQ_V 0.97 0.95 1.00 18 1186 3 0 0 3 1
2XKV_B 0.64 0.73 0.57 8 1821 26 0 6 20 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 1.00 1.00 1.00 35 2010 1 0 0 1 0
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2YIE_X - 0.39 0.43 0.38 3 536 8 1 4 3 4
3ADB_C - 0.86 0.85 0.88 28 1787 4 0 4 0 5
3AKZ_H 1.00 1.00 1.00 20 1107 2 0 0 2 0
3AM1_B - 0.86 0.86 0.86 25 1437 4 0 4 0 4
3AMU_B 0.73 0.79 0.68 15 1135 10 0 7 3 4
3IYQ_A 0.24 0.33 0.18 17 22345 95 40 38 17 34
3IZ4_A 0.60 0.61 0.59 58 25437 47 16 25 6 37
3IZF_C 0.87 0.89 0.86 31 2604 8 0 5 3 4
3J0L_h - 0.93 0.88 1.00 28 2112 2 0 0 2 4
3J0L_2 - 0.30 0.31 0.31 8 2224 25 2 16 7 18
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_7 - -0.02 0.00 0.00 0 507 12 2 10 0 10
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_a - 0.15 0.18 0.17 2 399 11 4 6 1 9
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J16_L 0.76 0.76 0.76 16 1138 5 0 5 0 5
3J20_2 0.55 0.58 0.53 239 421918 270 31 180 59 173
3J20_1 1.00 1.00 1.00 20 1092 1 0 0 1 0
3J20_0 0.54 0.57 0.52 12 1196 12 3 8 1 9
3J2C_O - 0.71 0.76 0.67 31 3941 21 5 10 6 10
3J2C_M - 0.23 0.27 0.20 26 39796 116 31 72 13 70
3J2L_3 0.80 0.82 0.78 28 2984 12 1 7 4 6
3J3D_C 0.47 0.53 0.43 10 945 13 5 8 0 9
3J3E_7 0.59 0.59 0.61 20 2708 15 2 11 2 14
3J3E_8 -0.01 0.00 0.00 0 2719 34 5 18 11 15
3J3F_8 0.32 0.42 0.25 8 4729 41 12 12 17 11
3J3F_7 0.94 0.94 0.94 34 2898 4 1 1 2 2
3J3V_B 0.74 0.78 0.70 21 2626 17 2 7 8 6
3NDB_M - 1.00 1.00 1.00 44 3670 4 0 0 4 0
3NKB_B - 0.41 0.42 0.42 8 716 11 4 7 0 11
3NMU_E - -0.02 0.00 0.00 0 207 11 6 3 2 3
3NPB_A 0.77 0.73 0.82 27 2245 11 0 6 5 10
3O58_2 0.86 0.87 0.84 27 2722 12 0 5 7 4
3O58_3 0.39 0.50 0.31 11 4728 41 9 16 16 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.93 0.92 0.94 46 4791 5 1 2 2 4
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.68 0.69 0.69 20 1504 9 4 5 0 9
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 1.00 1.00 1.00 12 142 0 0 0 0 0
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.58 0.59 0.59 13 1254 9 2 7 0 9
3UZL_B 1.00 1.00 1.00 16 1277 8 0 0 8 0
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W1K_J 0.97 0.97 0.97 30 1647 1 1 0 0 1
3W3S_B 0.94 0.91 0.97 30 1958 2 0 1 1 3
3ZEX_G - 0.98 0.96 1.00 43 6464 17 0 0 17 2
3ZEX_F - -0.01 0.00 0.00 0 908 11 2 4 5 4
3ZEX_C 0.28 0.34 0.23 10 5330 46 9 25 12 19
3ZEX_H - 0.26 0.37 0.19 7 3588 30 19 11 0 12
3ZEX_B - 0.32 0.36 0.29 123 420926 346 87 216 43 223
3ZEX_D 0.90 0.86 0.94 30 2764 8 0 2 6 5
3ZEX_E - -0.01 0.00 0.00 0 8249 57 20 35 2 34
3ZND_W 0.24 0.38 0.16 3 1172 25 9 7 9 5
4A1C_2 0.19 0.25 0.15 5 4482 43 11 18 14 15
4A1C_3 0.86 0.84 0.89 31 2728 7 0 4 3 6
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ATO_G - 0.38 0.43 0.38 3 212 5 5 0 0 4
4ENB_A 0.37 0.33 0.45 5 461 7 0 6 1 10
4ENC_A 0.36 0.33 0.42 5 484 8 0 7 1 10
4FNJ_A - 0.91 0.83 1.00 10 240 0 0 0 0 2
4FRG_B 0.22 0.25 0.23 6 1176 20 7 13 0 18
4FRN_A 0.46 0.46 0.46 13 1820 15 7 8 0 15
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.89 0.79 1.00 19 1063 0 0 0 0 5
4JRC_A - 0.34 0.35 0.35 6 605 11 0 11 0 11

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.