CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MCFold - scored higher in this pairwise comparison

  4. Performance of Vsfold5 - scored lower in this pairwise comparison

  5. Compile and download dataset for MCFold & Vsfold5 [.zip] - may take several seconds...


Overview

Metric MCFold Vsfold5
MCC 0.638 > 0.629
Average MCC ± 95% Confidence Intervals 0.667 ± 0.063 < 0.677 ± 0.062
Sensitivity 0.682 > 0.628
Positive Predictive Value 0.610 < 0.642
Total TP 1400 > 1288
Total TN 117014 < 117304
Total FP 1206 > 890
Total FP CONTRA 235 > 194
Total FP INCONS 661 > 524
Total FP COMP 310 > 172
Total FN 652 < 764
P-value 2.29562105844e-05

^top




Performance plots


  1. Comparison of performance of MCFold and Vsfold5. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MCFold and Vsfold5).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MCFold and Vsfold5).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for MCFold and Vsfold5. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MCFold and Vsfold5).

^top





Performance of MCFold - scored higher in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1400
Total TN 117014
Total FP 1206
Total FP CONTRA 235
Total FP INCONS 661
Total FP COMP 310
Total FN 652
Total Scores
MCC 0.638
Average MCC ± 95% Confidence Intervals 0.667 ± 0.063
Sensitivity 0.682
Positive Predictive Value 0.610
Nr of predictions 124

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 4 0 0 4 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 1 0 0 1 0
2KDQ_B 1.00 1.00 1.00 10 164 1 0 0 1 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.54 0.63 0.50 5 226 5 2 3 0 3
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 200 0 0 0 0 0
2KU0_A - 1.00 1.00 1.00 12 280 1 0 0 1 0
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 1 0 0 1 0
2KZL_A - 1.00 1.00 1.00 13 507 5 0 0 5 0
2L1F_A 0.79 0.78 0.82 18 741 4 0 4 0 5
2L1F_B 0.80 0.79 0.83 19 768 4 0 4 0 5
2L2K_A - 1.00 1.00 1.00 17 332 1 0 0 1 0
2L3C_B - 1.00 1.00 1.00 14 225 1 0 0 1 0
2L3E_A - 1.00 1.00 1.00 12 228 2 0 0 2 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - -0.03 0.00 0.00 0 265 10 0 10 0 7
2LBS_A - 1.00 1.00 1.00 14 193 1 0 0 1 0
2LC8_A 0.44 0.44 0.47 8 511 10 0 9 1 10
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 0.95 0.91 1.00 10 152 0 0 0 0 1
2LHP_A - 1.00 1.00 1.00 15 246 1 0 0 1 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 11 0 0 11 0
2LU0_A - 1.00 1.00 1.00 16 422 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 2 0 0 2 0
2M58_A - 0.22 0.25 0.23 3 531 13 1 9 3 9
2RP0_A - 0.70 0.71 0.71 5 109 2 1 1 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 1 0 0 1 0
2WRQ_Y 0.38 0.56 0.26 5 1133 21 8 6 7 4
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - -0.03 0.00 0.00 0 165 8 1 6 1 5
2WW9_D - -0.02 0.00 0.00 0 723 21 6 12 3 10
2WWQ_V 0.19 0.21 0.19 4 1183 20 3 14 3 15
2XKV_B 0.20 0.27 0.16 3 1816 30 4 12 14 8
2XQD_Y 0.39 0.43 0.38 9 1105 18 2 13 3 12
2XXA_G 0.23 0.26 0.24 9 2007 30 1 28 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 2 0 0 2 0
2Y9C_V - 0.65 0.72 0.59 13 884 12 1 8 3 5
2YIE_X - -0.01 0.00 0.00 0 535 15 1 8 6 7
2YIE_Z - 0.53 0.63 0.45 5 591 8 2 4 2 3
2ZY6_A - 0.77 0.78 0.78 7 267 6 1 1 4 2
2ZZN_D 0.69 0.73 0.67 16 960 11 1 7 3 6
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3A3A_A 0.98 0.97 1.00 29 1471 0 0 0 0 1
3AKZ_H 0.46 0.50 0.43 10 1104 16 5 8 3 10
3AM1_B - 1.00 1.00 1.00 29 1437 1 0 0 1 0
3AMU_B 0.50 0.58 0.44 11 1132 15 4 10 1 8
3DW4_A - 1.00 1.00 1.00 6 134 3 0 0 3 0
3GCA_A - -0.05 0.00 0.00 0 150 8 4 4 0 7
3GX2_A 0.58 0.61 0.57 17 1419 15 1 12 2 11
3HAY_E - 1.00 1.00 1.00 14 910 10 0 0 10 0
3HJW_D - 1.00 1.00 1.00 16 593 5 0 0 5 0
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3ID5_D - 0.54 0.60 0.50 3 223 7 1 2 4 2
3IVN_B 0.45 0.48 0.46 11 879 14 5 8 1 12
3IWN_A 0.19 0.21 0.20 6 1442 24 5 19 0 22
3IZF_C 0.92 0.94 0.89 33 2603 10 0 4 6 2
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.14 0.18 0.15 2 398 13 1 10 2 9
3J0L_1 - 0.88 0.85 0.92 11 472 5 0 1 4 2
3J0L_h - 0.98 0.97 1.00 31 2109 2 0 0 2 1
3J0L_7 - -0.02 0.00 0.00 0 504 15 6 9 0 10
3J0L_g - -0.02 0.00 0.00 0 170 7 4 2 1 2
3J0L_2 - 0.25 0.27 0.25 7 2222 29 4 17 8 19
3J16_L 0.54 0.57 0.52 12 1136 12 4 7 1 9
3J20_0 0.66 0.71 0.63 15 1195 11 3 6 2 6
3J2L_3 0.77 0.79 0.75 27 2984 15 1 8 6 7
3J3D_C 0.54 0.63 0.48 12 943 13 4 9 0 7
3J3E_8 0.15 0.20 0.12 3 2716 36 11 12 13 12
3J3E_7 0.55 0.56 0.54 19 2706 23 1 15 7 15
3J3F_8 0.14 0.21 0.10 4 4720 51 17 20 14 15
3J3F_7 0.88 0.89 0.86 32 2897 9 1 4 4 4
3JYV_7 0.25 0.30 0.24 6 1086 20 8 11 1 14
3JYX_3 0.46 0.67 0.32 10 2347 28 15 6 7 5
3JYX_4 0.36 0.58 0.23 7 4725 37 19 5 13 5
3KTW_C - 0.55 0.60 0.52 15 1751 20 4 10 6 10
3LA5_A 0.44 0.44 0.48 11 931 12 2 10 0 14
3NDB_M - 0.29 0.30 0.30 13 3671 35 1 29 5 31
3NKB_B - 0.68 0.74 0.64 14 713 8 2 6 0 5
3NMU_E - 0.51 0.67 0.40 2 211 8 1 2 5 1
3O58_3 0.28 0.36 0.22 8 4727 39 14 15 10 14
3O58_2 0.25 0.29 0.23 9 2715 31 6 24 1 22
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.78 0.80 0.77 40 4788 14 5 7 2 10
3R4F_A - 0.95 0.95 0.95 21 884 4 0 1 3 1
3R9X_C - 0.87 0.88 0.88 7 221 5 0 1 4 1
3RKF_A 0.89 0.88 0.91 21 843 3 0 2 1 3
3SD1_A 0.43 0.45 0.43 13 1503 17 1 16 0 16
3SIU_F - 0.53 0.63 0.50 5 135 5 2 3 0 3
3SN2_B 0.58 0.58 0.64 7 143 4 0 4 0 5
3TRZ_Z - -0.05 0.00 0.00 0 88 4 0 4 0 5
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - -0.05 0.00 0.00 0 102 7 0 6 1 5
3U4M_B - 0.74 0.77 0.71 17 1252 10 1 6 3 5
3VJR_D - 1.00 1.00 1.00 12 239 1 0 0 1 0
3ZEX_D 0.23 0.26 0.23 9 2756 33 6 25 2 26
3ZND_W 0.23 0.38 0.15 3 1171 26 9 8 9 5
4A1C_3 0.86 0.86 0.86 32 2726 9 0 5 4 5
4A1C_2 0.18 0.25 0.14 5 4480 45 12 19 14 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4ATO_G - 0.36 0.43 0.33 3 211 8 0 6 2 4
4ENB_A 0.78 0.73 0.85 11 459 4 0 2 2 4
4ENC_A 0.34 0.33 0.38 5 483 11 0 8 3 10
4FRG_B 0.35 0.38 0.35 9 1176 17 3 14 0 15
4FRN_A 0.09 0.11 0.10 3 1818 28 2 25 1 25
4JF2_A 0.74 0.75 0.75 18 1058 7 1 5 1 6
4JRC_A - 0.24 0.29 0.23 5 600 17 0 17 0 12

^top



Performance of Vsfold5 - scored lower in this pairwise comparison

1. Total counts & total scores for Vsfold5

Total Base Pair Counts
Total TP 1288
Total TN 117304
Total FP 890
Total FP CONTRA 194
Total FP INCONS 524
Total FP COMP 172
Total FN 764
Total Scores
MCC 0.629
Average MCC ± 95% Confidence Intervals 0.677 ± 0.062
Sensitivity 0.628
Positive Predictive Value 0.642
Nr of predictions 124

^top



2. Individual counts for Vsfold5 [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 0.95 0.90 1.00 9 165 0 0 0 0 1
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 0.78 0.63 1.00 5 97 0 0 0 0 3
2KFC_A - 0.77 0.88 0.70 7 226 3 2 1 0 1
2KGP_A - 0.81 0.67 1.00 6 115 1 0 0 1 3
2KMJ_A - 0.95 0.91 1.00 10 158 0 0 0 0 1
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KU0_A - -0.04 0.00 0.00 0 281 11 0 11 0 12
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 0.97 0.94 1.00 15 356 0 0 0 0 1
2KXM_A - 0.78 0.63 1.00 5 144 0 0 0 0 3
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 0.70 0.61 0.82 14 746 3 0 3 0 9
2L1F_B 0.71 0.63 0.83 15 773 3 0 3 0 9
2L2K_A - 0.94 0.88 1.00 15 334 0 0 0 0 2
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 0.93 0.87 1.00 26 959 0 0 0 0 4
2L5Z_A - 0.68 0.67 0.75 6 103 2 0 2 0 3
2L94_A 0.97 0.94 1.00 17 340 1 0 0 1 1
2LA5_A - -0.03 0.00 0.00 0 268 7 0 7 0 7
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 515 13 0 13 0 18
2LDL_A - 0.94 0.89 1.00 8 132 1 0 0 1 1
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 0.94 0.89 1.00 8 92 0 0 0 0 1
2LKR_A - 0.31 0.31 0.32 9 2412 19 5 14 0 20
2LU0_A - 0.68 0.63 0.77 10 425 3 0 3 0 6
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2M58_A - 0.37 0.33 0.44 4 535 5 4 1 0 8
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 0.68 0.71 0.71 5 65 2 0 2 0 2
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 0.94 0.89 1.00 8 1144 13 0 0 13 1
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WW9_D - 0.08 0.10 0.09 1 730 13 3 7 3 9
2WWQ_V 0.78 0.74 0.82 14 1187 4 1 2 1 5
2XKV_B -0.01 0.00 0.00 0 1803 32 17 15 0 11
2XQD_Y 0.95 0.90 1.00 19 1110 1 0 0 1 2
2XXA_G -0.02 0.00 0.00 0 2014 31 2 29 0 35
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.74 0.72 0.76 13 889 7 0 4 3 5
2YIE_X - 0.43 0.57 0.33 4 532 10 2 6 2 3
2YIE_Z - -0.02 0.00 0.00 0 591 11 5 6 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZN_D 0.95 0.95 0.95 21 962 1 1 0 0 1
3A2K_C 0.95 0.91 1.00 20 1088 0 0 0 0 2
3A3A_A 0.95 0.90 1.00 27 1473 0 0 0 0 3
3AKZ_H 0.97 0.95 1.00 19 1108 1 0 0 1 1
3AM1_B - 0.86 0.86 0.86 25 1437 4 3 1 0 4
3AMU_B 0.77 0.79 0.75 15 1137 8 0 5 3 4
3DW4_A - 0.91 0.83 1.00 5 135 0 0 0 0 1
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.61 0.57 0.67 16 1425 9 0 8 1 12
3HAY_E - 1.00 1.00 1.00 14 910 4 0 0 4 0
3HJW_D - 0.50 0.56 0.47 9 590 10 2 8 0 7
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 4 0 2 2 2
3IVN_B 0.88 0.78 1.00 18 885 0 0 0 0 5
3IWN_A 0.26 0.25 0.30 7 1449 16 2 14 0 21
3IZF_C 0.85 0.83 0.88 29 2607 5 0 4 1 6
3J0L_8 - 0.74 0.57 1.00 4 72 0 0 0 0 3
3J0L_a - 0.39 0.36 0.44 4 402 6 4 1 1 7
3J0L_1 - 0.25 0.23 0.30 3 474 8 3 4 1 10
3J0L_h - 0.20 0.19 0.24 6 2115 19 3 16 0 26
3J0L_7 - -0.02 0.00 0.00 0 506 13 3 10 0 10
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_2 - 0.42 0.42 0.44 11 2225 21 1 13 7 15
3J16_L 0.98 0.95 1.00 20 1139 0 0 0 0 1
3J20_0 0.76 0.76 0.76 16 1198 6 3 2 1 5
3J2L_3 0.70 0.71 0.71 24 2986 13 3 7 3 10
3J3D_C 0.63 0.68 0.59 13 946 9 3 6 0 6
3J3E_8 0.10 0.13 0.08 2 2718 31 10 12 9 13
3J3E_7 0.60 0.59 0.63 20 2709 13 1 11 1 14
3J3F_8 0.36 0.47 0.27 9 4728 38 13 11 14 10
3J3F_7 0.25 0.25 0.26 9 2900 25 4 21 0 27
3JYV_7 -0.02 0.00 0.00 0 1092 19 4 15 0 20
3JYX_3 0.12 0.13 0.11 2 2360 20 6 10 4 13
3JYX_4 0.63 0.83 0.48 10 4735 32 9 2 21 2
3KTW_C - 0.46 0.44 0.50 11 1758 15 2 9 4 14
3LA5_A -0.02 0.00 0.00 0 938 16 0 16 0 25
3NDB_M - 0.07 0.07 0.08 3 3678 33 2 31 0 41
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 0.28 0.33 0.25 1 212 5 2 1 2 2
3O58_3 0.51 0.59 0.45 13 4735 28 11 5 12 9
3O58_2 0.50 0.52 0.50 16 2722 19 3 13 3 15
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.69 0.64 0.74 32 4797 13 3 8 2 18
3R4F_A - 0.98 0.95 1.00 21 885 1 0 0 1 1
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.90 0.92 0.88 22 841 3 3 0 0 2
3SD1_A 0.13 0.14 0.15 4 1507 22 5 17 0 25
3SIU_F - 0.86 0.75 1.00 6 139 0 0 0 0 2
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3ZEX_D 0.09 0.09 0.12 3 2771 24 5 17 2 32
3ZND_W 0.58 0.75 0.46 6 1178 17 4 3 10 2
4A1C_3 0.29 0.30 0.31 11 2727 26 4 21 1 26
4A1C_2 0.33 0.40 0.28 8 4487 37 8 13 16 12
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4ATO_G - 0.67 0.86 0.55 6 209 5 5 0 0 1
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0
4FRG_B 0.60 0.58 0.64 14 1180 8 3 5 0 10
4FRN_A 0.15 0.14 0.17 4 1825 19 3 16 0 24
4JF2_A 0.96 1.00 0.92 24 1056 2 2 0 0 0
4JRC_A - 0.28 0.29 0.31 5 606 11 2 9 0 12

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.