CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Vsfold4 - scored higher in this pairwise comparison

  4. Performance of MCFold - scored lower in this pairwise comparison

  5. Compile and download dataset for Vsfold4 & MCFold [.zip] - may take several seconds...


Overview

Metric Vsfold4 MCFold
MCC 0.652 > 0.639
Average MCC ± 95% Confidence Intervals 0.687 ± 0.059 > 0.671 ± 0.063
Sensitivity 0.636 < 0.677
Positive Predictive Value 0.680 > 0.615
Total TP 1324 < 1411
Total TN 122249 > 121902
Total FP 799 < 1195
Total FP CONTRA 160 < 239
Total FP INCONS 463 < 644
Total FP COMP 176 < 312
Total FN 759 > 672
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Vsfold4 and MCFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Vsfold4 and MCFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Vsfold4 and MCFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Vsfold4 and MCFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Vsfold4 and MCFold).

^top





Performance of Vsfold4 - scored higher in this pairwise comparison

1. Total counts & total scores for Vsfold4

Total Base Pair Counts
Total TP 1324
Total TN 122249
Total FP 799
Total FP CONTRA 160
Total FP INCONS 463
Total FP COMP 176
Total FN 759
Total Scores
MCC 0.652
Average MCC ± 95% Confidence Intervals 0.687 ± 0.059
Sensitivity 0.636
Positive Predictive Value 0.680
Nr of predictions 126

^top



2. Individual counts for Vsfold4 [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.35 0.36 0.40 4 176 6 0 6 0 7
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 0.78 0.63 1.00 5 97 0 0 0 0 3
2KFC_A - 0.79 0.63 1.00 5 231 0 0 0 0 3
2KGP_A - -0.05 0.00 0.00 0 117 4 0 4 0 9
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KU0_A - -0.04 0.00 0.00 0 283 9 2 7 0 12
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.78 0.63 1.00 5 144 0 0 0 0 3
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 0.67 0.57 0.81 13 747 3 0 3 0 10
2L1F_B 0.69 0.58 0.82 14 774 3 0 3 0 10
2L2K_A - 0.94 0.88 1.00 15 334 0 0 0 0 2
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 0.93 0.87 1.00 26 959 0 0 0 0 4
2L5Z_A - 0.68 0.67 0.75 6 103 2 0 2 0 3
2L94_A 0.94 0.89 1.00 16 341 1 0 0 1 2
2LA5_A - 0.77 0.71 0.83 5 269 1 0 1 0 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 515 13 0 13 0 18
2LDL_A - 0.94 0.89 1.00 8 132 1 0 0 1 1
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.35 0.38 0.34 11 2408 21 7 14 0 18
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LU0_A - 0.68 0.63 0.77 10 425 3 0 3 0 6
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2M58_A - -0.02 0.00 0.00 0 538 6 0 6 0 12
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - -0.07 0.00 0.00 0 68 4 0 4 0 7
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 0.94 0.89 1.00 8 1144 13 0 0 13 1
2WW9_D - 0.12 0.10 0.17 1 735 8 1 4 3 9
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WWQ_V 0.78 0.74 0.82 14 1187 4 1 2 1 5
2XKV_B -0.01 0.00 0.00 0 1808 27 13 14 0 11
2XQD_Y 0.95 0.90 1.00 19 1110 1 0 0 1 2
2XXA_G -0.02 0.00 0.00 0 2016 29 1 28 0 35
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.75 0.67 0.86 12 892 4 0 2 2 6
2YIE_Z - -0.02 0.00 0.00 0 591 11 5 6 0 8
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZN_D 0.95 0.95 0.95 21 962 1 1 0 0 1
3A2K_C 0.95 0.91 1.00 20 1088 0 0 0 0 2
3A3A_A 0.95 0.90 1.00 27 1473 0 0 0 0 3
3AKZ_H 0.77 0.75 0.79 15 1108 6 2 2 2 5
3AM1_B - 0.93 0.86 1.00 25 1441 0 0 0 0 4
3AMU_B 0.73 0.74 0.74 14 1138 8 0 5 3 5
3DW4_A - 0.91 0.83 1.00 5 135 0 0 0 0 1
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.60 0.57 0.64 16 1424 10 0 9 1 12
3HAY_E - 1.00 1.00 1.00 14 910 4 0 0 4 0
3HJW_D - 0.53 0.56 0.53 9 592 8 2 6 0 7
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 4 0 2 2 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.28 0.25 0.35 7 1452 13 2 11 0 21
3IZF_C 0.85 0.83 0.88 29 2607 6 0 4 2 6
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_a - 0.39 0.36 0.44 4 402 6 4 1 1 7
3J0L_8 - 0.74 0.57 1.00 4 72 0 0 0 0 3
3J0L_2 - 0.47 0.46 0.50 12 2226 18 1 11 6 14
3J0L_1 - 0.26 0.23 0.33 3 475 7 2 4 1 10
3J0L_h - 0.44 0.38 0.52 12 2117 11 2 9 0 20
3J0L_7 - 0.69 0.70 0.70 7 509 4 1 2 1 3
3J16_L 0.98 0.95 1.00 20 1139 0 0 0 0 1
3J20_0 0.95 0.95 0.95 20 1198 2 1 0 1 1
3J2L_3 0.71 0.71 0.73 24 2987 12 3 6 3 10
3J3D_C 0.48 0.53 0.45 10 946 12 4 8 0 9
3J3E_8 0.10 0.13 0.09 2 2720 32 9 11 12 13
3J3E_7 0.61 0.56 0.68 19 2713 10 1 8 1 15
3J3F_8 0.35 0.42 0.30 8 4734 30 11 8 11 11
3J3F_7 0.25 0.25 0.26 9 2900 25 6 19 0 27
3JYV_7 -0.02 0.00 0.00 0 1092 19 4 15 0 20
3JYX_3 0.64 0.67 0.63 10 2362 16 5 1 10 5
3JYX_4 0.19 0.25 0.15 3 4736 33 9 8 16 9
3KTW_C - 0.43 0.40 0.48 10 1759 15 2 9 4 15
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.33 0.32 0.35 14 3674 26 2 24 0 30
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 0.57 0.33 1.00 1 215 3 0 0 3 2
3O58_2 0.53 0.52 0.55 16 2725 16 3 10 3 15
3O58_3 0.12 0.14 0.11 3 4737 35 8 16 11 19
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.69 0.64 0.74 32 4797 13 3 8 2 18
3R4F_A - 0.98 0.95 1.00 21 885 1 0 0 1 1
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.23 0.24 0.24 7 1504 22 5 17 0 22
3SIU_F - 0.86 0.75 1.00 6 139 0 0 0 0 2
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3ZEX_G - 0.31 0.29 0.34 13 6469 31 5 20 6 32
3ZEX_D 0.66 0.63 0.71 22 2765 10 4 5 1 13
3ZND_W 0.58 0.75 0.46 6 1178 17 4 3 10 2
4A1C_3 0.29 0.30 0.31 11 2727 26 4 21 1 26
4A1C_2 0.36 0.40 0.33 8 4492 28 4 12 12 12
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4ATO_G - 0.38 0.43 0.38 3 212 5 5 0 0 4
4ENB_A 0.85 0.73 1.00 11 461 0 0 0 0 4
4ENC_A 0.77 0.60 1.00 9 487 0 0 0 0 6
4FRG_B 0.60 0.58 0.64 14 1180 8 3 5 0 10
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.74 0.67 0.84 16 1063 3 0 3 0 8
4JRC_A - 0.34 0.29 0.42 5 610 7 0 7 0 12

^top



Performance of MCFold - scored lower in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1411
Total TN 121902
Total FP 1195
Total FP CONTRA 239
Total FP INCONS 644
Total FP COMP 312
Total FN 672
Total Scores
MCC 0.639
Average MCC ± 95% Confidence Intervals 0.671 ± 0.063
Sensitivity 0.677
Positive Predictive Value 0.615
Nr of predictions 126

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 4 0 0 4 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 1 0 0 1 0
2KDQ_B 1.00 1.00 1.00 10 164 1 0 0 1 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.54 0.63 0.50 5 226 5 2 3 0 3
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 200 0 0 0 0 0
2KU0_A - 1.00 1.00 1.00 12 280 1 0 0 1 0
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 1 0 0 1 0
2KZL_A - 1.00 1.00 1.00 13 507 5 0 0 5 0
2L1F_A 0.79 0.78 0.82 18 741 4 0 4 0 5
2L1F_B 0.80 0.79 0.83 19 768 4 0 4 0 5
2L2K_A - 1.00 1.00 1.00 17 332 1 0 0 1 0
2L3C_B - 1.00 1.00 1.00 14 225 1 0 0 1 0
2L3E_A - 1.00 1.00 1.00 12 228 2 0 0 2 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - -0.03 0.00 0.00 0 265 10 0 10 0 7
2LBS_A - 1.00 1.00 1.00 14 193 1 0 0 1 0
2LC8_A 0.44 0.44 0.47 8 511 10 0 9 1 10
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 0.95 0.91 1.00 10 152 0 0 0 0 1
2LHP_A - 1.00 1.00 1.00 15 246 1 0 0 1 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 11 0 0 11 0
2LQZ_A - 1.00 1.00 1.00 8 124 3 0 0 3 0
2LU0_A - 1.00 1.00 1.00 16 422 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 2 0 0 2 0
2M58_A - 0.22 0.25 0.23 3 531 13 1 9 3 9
2RP0_A - 0.70 0.71 0.71 5 109 2 1 1 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 1 0 0 1 0
2WRQ_Y 0.38 0.56 0.26 5 1133 21 8 6 7 4
2WW9_D - -0.02 0.00 0.00 0 723 21 6 12 3 10
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - -0.03 0.00 0.00 0 165 8 1 6 1 5
2WWQ_V 0.19 0.21 0.19 4 1183 20 3 14 3 15
2XKV_B 0.20 0.27 0.16 3 1816 30 4 12 14 8
2XQD_Y 0.39 0.43 0.38 9 1105 18 2 13 3 12
2XXA_G 0.23 0.26 0.24 9 2007 30 1 28 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 2 0 0 2 0
2Y9C_V - 0.65 0.72 0.59 13 884 12 1 8 3 5
2YIE_Z - 0.53 0.63 0.45 5 591 8 2 4 2 3
2YIE_X - -0.01 0.00 0.00 0 535 15 1 8 6 7
2ZY6_A - 0.77 0.78 0.78 7 267 6 1 1 4 2
2ZZN_D 0.69 0.73 0.67 16 960 11 1 7 3 6
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3A3A_A 0.98 0.97 1.00 29 1471 0 0 0 0 1
3AKZ_H 0.46 0.50 0.43 10 1104 16 5 8 3 10
3AM1_B - 1.00 1.00 1.00 29 1437 1 0 0 1 0
3AMU_B 0.50 0.58 0.44 11 1132 15 4 10 1 8
3DW4_A - 1.00 1.00 1.00 6 134 3 0 0 3 0
3GCA_A - -0.05 0.00 0.00 0 150 8 4 4 0 7
3GX2_A 0.58 0.61 0.57 17 1419 15 1 12 2 11
3HAY_E - 1.00 1.00 1.00 14 910 10 0 0 10 0
3HJW_D - 1.00 1.00 1.00 16 593 5 0 0 5 0
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3ID5_D - 0.54 0.60 0.50 3 223 7 1 2 4 2
3IVN_B 0.45 0.48 0.46 11 879 14 5 8 1 12
3IWN_A 0.19 0.21 0.20 6 1442 24 5 19 0 22
3IZF_C 0.92 0.94 0.89 33 2603 10 0 4 6 2
3J0L_g - -0.02 0.00 0.00 0 170 7 4 2 1 2
3J0L_a - 0.14 0.18 0.15 2 398 13 1 10 2 9
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.25 0.27 0.25 7 2222 29 4 17 8 19
3J0L_1 - 0.88 0.85 0.92 11 472 5 0 1 4 2
3J0L_h - 0.98 0.97 1.00 31 2109 2 0 0 2 1
3J0L_7 - -0.02 0.00 0.00 0 504 15 6 9 0 10
3J16_L 0.54 0.57 0.52 12 1136 12 4 7 1 9
3J20_0 0.66 0.71 0.63 15 1195 11 3 6 2 6
3J2L_3 0.77 0.79 0.75 27 2984 15 1 8 6 7
3J3D_C 0.54 0.63 0.48 12 943 13 4 9 0 7
3J3E_8 0.15 0.20 0.12 3 2716 36 11 12 13 12
3J3E_7 0.55 0.56 0.54 19 2706 23 1 15 7 15
3J3F_8 0.14 0.21 0.10 4 4720 51 17 20 14 15
3J3F_7 0.88 0.89 0.86 32 2897 9 1 4 4 4
3JYV_7 0.25 0.30 0.24 6 1086 20 8 11 1 14
3JYX_3 0.46 0.67 0.32 10 2347 28 15 6 7 5
3JYX_4 0.36 0.58 0.23 7 4725 37 19 5 13 5
3KTW_C - 0.55 0.60 0.52 15 1751 20 4 10 6 10
3LA5_A 0.44 0.44 0.48 11 931 12 2 10 0 14
3NDB_M - 0.29 0.30 0.30 13 3671 35 1 29 5 31
3NKB_B - 0.68 0.74 0.64 14 713 8 2 6 0 5
3NMU_E - 0.51 0.67 0.40 2 211 8 1 2 5 1
3O58_2 0.25 0.29 0.23 9 2715 31 6 24 1 22
3O58_3 0.28 0.36 0.22 8 4727 39 14 15 10 14
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.78 0.80 0.77 40 4788 14 5 7 2 10
3R4F_A - 0.95 0.95 0.95 21 884 4 0 1 3 1
3R9X_C - 0.87 0.88 0.88 7 221 5 0 1 4 1
3RKF_A 0.89 0.88 0.91 21 843 3 0 2 1 3
3SD1_A 0.43 0.45 0.43 13 1503 17 1 16 0 16
3SIU_F - 0.53 0.63 0.50 5 135 5 2 3 0 3
3SN2_B 0.58 0.58 0.64 7 143 4 0 4 0 5
3TRZ_Z - -0.05 0.00 0.00 0 88 4 0 4 0 5
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - -0.05 0.00 0.00 0 102 7 0 6 1 5
3U4M_B - 0.74 0.77 0.71 17 1252 10 1 6 3 5
3VJR_D - 1.00 1.00 1.00 12 239 1 0 0 1 0
3ZEX_G - 0.00 0.00 0.00 0 6493 14 6 8 0 45
3ZEX_D 0.23 0.26 0.23 9 2756 33 6 25 2 26
3ZND_W 0.23 0.38 0.15 3 1171 26 9 8 9 5
4A1C_3 0.86 0.86 0.86 32 2726 9 0 5 4 5
4A1C_2 0.18 0.25 0.14 5 4480 45 12 19 14 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4ATO_G - 0.36 0.43 0.33 3 211 8 0 6 2 4
4ENB_A 0.78 0.73 0.85 11 459 4 0 2 2 4
4ENC_A 0.34 0.33 0.38 5 483 11 0 8 3 10
4FRG_B 0.35 0.38 0.35 9 1176 17 3 14 0 15
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.74 0.75 0.75 18 1058 7 1 5 1 6
4JRC_A - 0.24 0.29 0.23 5 600 17 0 17 0 12

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.