CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Vsfold5 - scored higher in this pairwise comparison

  4. Performance of MCFold - scored lower in this pairwise comparison

  5. Compile and download dataset for Vsfold5 & MCFold [.zip] - may take several seconds...


Overview

Metric Vsfold5 MCFold
MCC 0.542 > 0.539
Average MCC ± 95% Confidence Intervals 0.610 ± 0.059 > 0.608 ± 0.062
Sensitivity 0.471 < 0.559
Positive Predictive Value 0.632 > 0.529
Total TP 1318 < 1564
Total TN 306938 > 306067
Total FP 860 < 1634
Total FP CONTRA 87 < 171
Total FP INCONS 682 < 1223
Total FP COMP 91 < 240
Total FN 1482 > 1236
P-value 0.657358697881

^top




Performance plots


  1. Comparison of performance of Vsfold5 and MCFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Vsfold5 and MCFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Vsfold5 and MCFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Vsfold5 and MCFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Vsfold5 and MCFold).

^top





Performance of Vsfold5 - scored higher in this pairwise comparison

1. Total counts & total scores for Vsfold5

Total Base Pair Counts
Total TP 1318
Total TN 306938
Total FP 860
Total FP CONTRA 87
Total FP INCONS 682
Total FP COMP 91
Total FN 1482
Total Scores
MCC 0.542
Average MCC ± 95% Confidence Intervals 0.610 ± 0.059
Sensitivity 0.471
Positive Predictive Value 0.632
Nr of predictions 124

^top



2. Individual counts for Vsfold5 [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 0 0 0 0 0
2K63_A - 0.95 0.90 1.00 9 397 0 0 0 0 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.90 0.82 1.00 9 397 0 0 0 0 2
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KEZ_A - 0.70 0.50 1.00 5 271 0 0 0 0 5
2KFC_A - 0.48 0.35 0.70 7 620 3 0 3 0 13
2KGP_A - 0.81 0.67 1.00 6 294 1 0 0 1 3
2KMJ_A - 0.95 0.91 1.00 10 368 0 0 0 0 1
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KU0_A - -0.02 0.00 0.00 0 692 11 0 11 0 16
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.91 0.83 1.00 15 846 0 0 0 0 3
2KXM_A - 0.70 0.50 1.00 5 346 0 0 0 0 5
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_A 0.69 0.58 0.82 14 2063 3 0 3 0 10
2L1F_B 0.70 0.60 0.83 15 2127 3 0 3 0 10
2L2K_A - 0.89 0.79 1.00 15 846 0 0 0 0 4
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.87 0.76 1.00 26 2459 0 0 0 0 8
2L5Z_A - 0.60 0.50 0.75 6 317 2 0 2 0 6
2L94_A 0.95 0.90 1.00 18 972 0 0 0 0 2
2LA5_A - -0.02 0.00 0.00 0 623 7 0 7 0 19
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A -0.01 0.00 0.00 0 1527 13 0 13 0 20
2LDL_A - 0.85 0.73 1.00 8 343 1 0 0 1 3
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2LKR_A - 0.27 0.23 0.32 9 6077 19 2 17 0 30
2LU0_A - 0.69 0.63 0.77 10 1163 3 0 3 0 6
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2M58_A - 0.32 0.24 0.44 4 1644 5 1 4 0 13
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 0.70 0.71 0.71 5 183 2 0 2 0 2
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.53 0.53 0.53 9 2833 12 5 3 4 8
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_E - -0.01 0.00 0.00 0 556 5 0 5 0 14
2WW9_D - 0.15 0.12 0.21 3 1939 11 1 10 0 23
2WWQ_V 0.64 0.50 0.82 14 2909 4 0 3 1 14
2XKV_B -0.01 0.00 0.00 0 4528 32 12 20 0 20
2XQD_Y 0.86 0.74 1.00 20 2830 0 0 0 0 7
2XXA_G -0.01 0.00 0.00 0 5120 31 2 29 0 42
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.61 0.54 0.68 13 2259 7 1 5 1 11
2YIE_X - 0.30 0.33 0.29 4 1364 10 2 8 0 8
2YIE_Z - -0.01 0.00 0.00 0 1529 11 3 8 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 1 4 0 5
2ZZN_D 0.86 0.78 0.95 21 2463 1 0 1 0 6
3A2K_C 0.84 0.71 1.00 20 2906 0 0 0 0 8
3A3A_A 0.85 0.73 1.00 27 3628 0 0 0 0 10
3AKZ_H 0.84 0.71 1.00 20 2681 0 0 0 0 8
3AM1_B - 0.78 0.71 0.86 25 3211 4 1 3 0 10
3AMU_B 0.67 0.59 0.76 16 2982 7 0 5 2 11
3DW4_A - 0.70 0.50 1.00 5 346 0 0 0 0 5
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.51 0.40 0.67 16 4347 9 0 8 1 24
3HAY_E - 0.91 0.83 1.00 15 2263 3 0 0 3 3
3HJW_D - 0.48 0.50 0.47 9 1634 10 0 10 0 9
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.44 0.33 0.60 3 491 4 0 2 2 6
3IVN_B 0.76 0.58 1.00 18 2328 0 0 0 0 13
3IWN_A 0.25 0.21 0.30 7 4255 16 1 15 0 26
3IZF_C 0.70 0.56 0.88 30 6869 4 0 4 0 24
3J0L_8 - 0.70 0.50 1.00 4 186 0 0 0 0 4
3J0L_a - 0.39 0.31 0.50 5 1118 5 2 3 0 11
3J0L_1 - 0.27 0.21 0.36 4 1214 7 1 6 0 15
3J0L_h - 0.18 0.14 0.24 6 6080 19 1 18 0 37
3J0L_7 - -0.01 0.00 0.00 0 1212 13 0 13 0 17
3J0L_g - -0.01 0.00 0.00 0 461 4 1 3 0 4
3J0L_2 - 0.35 0.33 0.37 11 6186 21 3 16 2 22
3J16_L 0.82 0.67 1.00 20 2755 0 0 0 0 10
3J20_0 0.67 0.57 0.81 17 2829 5 1 3 1 13
3J2L_3 0.56 0.45 0.71 24 7841 13 1 9 3 29
3J3D_C 0.60 0.54 0.68 15 2753 7 0 7 0 13
3J3E_8 0.06 0.06 0.08 2 7477 31 5 19 7 31
3J3E_7 0.47 0.37 0.61 20 7107 13 0 13 0 34
3J3F_8 0.33 0.33 0.33 12 12210 35 5 19 11 24
3J3F_7 0.21 0.18 0.26 9 7226 25 1 24 0 41
3JYV_7 -0.01 0.00 0.00 0 2831 19 0 19 0 32
3JYX_3 0.17 0.15 0.20 4 6308 18 4 12 2 23
3JYX_4 0.31 0.30 0.31 10 12214 32 3 19 10 23
3KTW_C - 0.38 0.30 0.50 13 4534 13 3 10 0 30
3LA5_A -0.01 0.00 0.00 0 2469 16 0 16 0 34
3NDB_M - 0.06 0.05 0.08 3 9144 33 1 32 0 58
3NKB_B - 0.59 0.54 0.67 14 1995 7 0 7 0 12
3NMU_E - 0.20 0.17 0.25 1 557 5 0 3 2 5
3O58_3 0.39 0.37 0.41 13 12371 28 7 12 9 22
3O58_2 0.45 0.42 0.48 16 7227 19 3 14 2 22
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.57 0.44 0.74 32 12837 13 0 11 2 40
3R4F_A - 0.94 0.88 1.00 22 2123 0 0 0 0 3
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.75 0.65 0.88 22 2186 3 0 3 0 12
3SD1_A 0.11 0.10 0.15 4 3890 22 0 22 0 38
3SIU_F - 0.73 0.55 1.00 6 372 0 0 0 0 5
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
3ZEX_D 0.08 0.06 0.11 3 6994 24 0 24 0 46
3ZND_W 0.41 0.39 0.43 9 2982 14 2 10 2 14
4A1C_3 0.26 0.22 0.32 12 7103 25 2 23 0 42
4A1C_2 0.24 0.24 0.24 8 11748 37 3 22 12 25
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ATO_G - 0.56 0.60 0.55 6 517 5 1 4 0 4
4ENB_A 0.89 0.79 1.00 15 1260 0 0 0 0 4
4ENC_A 0.86 0.79 0.94 15 1310 1 1 0 0 4
4FRG_B 0.56 0.47 0.68 15 3464 7 1 6 0 17
4FRN_A 0.13 0.11 0.17 4 5128 19 2 17 0 32
4JF2_A 0.84 0.77 0.92 24 2824 2 1 1 0 7
4JRC_A - 0.25 0.22 0.31 5 1524 11 0 11 0 18

^top



Performance of MCFold - scored lower in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1564
Total TN 306067
Total FP 1634
Total FP CONTRA 171
Total FP INCONS 1223
Total FP COMP 240
Total FN 1236
Total Scores
MCC 0.539
Average MCC ± 95% Confidence Intervals 0.608 ± 0.062
Sensitivity 0.559
Positive Predictive Value 0.529
Nr of predictions 124

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 1 0 0 1 0
2K63_A - 0.90 0.90 0.90 9 396 4 0 1 3 1
2K66_A - 1.00 1.00 1.00 10 221 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 222 1 0 0 1 0
2KDQ_B 0.95 0.91 1.00 10 396 1 0 0 1 1
2KE6_A 1.00 1.00 1.00 19 1109 2 0 0 2 0
2KEZ_A - 0.90 0.90 0.90 9 266 1 0 1 0 1
2KFC_A - 0.28 0.25 0.36 5 616 9 2 7 0 15
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 548 2 0 0 2 0
2KU0_A - 0.97 0.94 1.00 15 688 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 21 1107 0 0 0 0 0
2KUU_A 0.98 0.95 1.00 20 1108 1 0 0 1 1
2KUV_A 0.98 0.95 1.00 21 1107 0 0 0 0 1
2KUW_A 0.95 0.95 0.95 20 1107 1 0 1 0 1
2KX8_A 0.91 0.89 0.94 16 844 2 0 1 1 2
2KXM_A - 1.00 1.00 1.00 10 341 1 0 0 1 0
2KZL_A - 0.80 0.84 0.76 16 1464 9 1 4 4 3
2L1F_A 0.72 0.75 0.69 18 2054 9 0 8 1 6
2L1F_B 0.73 0.76 0.70 19 2118 9 0 8 1 6
2L2K_A - 1.00 1.00 1.00 19 842 1 0 0 1 0
2L3C_B - 0.97 0.94 1.00 16 545 0 0 0 0 1
2L3E_A - 1.00 1.00 1.00 13 582 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 34 2451 0 0 0 0 0
2L5Z_A - 0.96 0.92 1.00 11 314 0 0 0 0 1
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - -0.03 0.00 0.00 0 615 15 0 15 0 19
2LBS_A - 1.00 1.00 1.00 14 482 1 0 0 1 0
2LC8_A 0.41 0.45 0.39 9 1517 16 0 14 2 11
2LDL_A - 0.95 0.91 1.00 10 341 2 0 0 2 1
2LDT_A - 0.75 0.73 0.79 11 451 3 0 3 0 4
2LHP_A - 1.00 1.00 1.00 16 650 1 0 0 1 0
2LI4_A - 0.97 0.94 1.00 15 481 0 0 0 0 1
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 1.00 1.00 1.00 10 266 0 0 0 0 0
2LKR_A - 0.93 0.92 0.95 36 6067 14 0 2 12 3
2LU0_A - 1.00 1.00 1.00 16 1160 5 0 0 5 0
2LWK_A - 0.92 0.92 0.92 12 483 2 0 1 1 1
2M58_A - 0.20 0.24 0.18 4 1631 19 2 16 1 13
2RP0_A - 0.44 0.42 0.50 5 341 5 1 4 0 7
2RPT_A - 1.00 1.00 1.00 7 183 2 0 0 2 0
2RRC_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2WRQ_Y 0.27 0.35 0.22 6 2823 25 8 13 4 11
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_E - -0.02 0.00 0.00 0 547 14 0 14 0 14
2WW9_D - -0.01 0.00 0.00 0 1925 28 5 23 0 26
2WWQ_V 0.15 0.18 0.15 5 2892 30 4 25 1 23
2XKV_B 0.20 0.25 0.17 5 4530 37 2 23 12 15
2XQD_Y 0.34 0.37 0.32 10 2819 22 1 20 1 17
2XXA_G 0.23 0.24 0.24 10 5110 32 1 30 1 32
2Y8W_B - 1.00 1.00 1.00 6 184 2 0 0 2 0
2Y9C_V - 0.62 0.67 0.59 16 2251 13 2 9 2 8
2YIE_X - 0.06 0.08 0.06 1 1361 21 1 15 5 11
2YIE_Z - 0.36 0.50 0.27 6 1518 18 4 12 2 6
2ZY6_A - 0.51 0.58 0.47 7 688 8 3 5 0 5
2ZZN_D 0.61 0.63 0.61 17 2457 13 1 10 2 10
3A2K_C 0.44 0.46 0.42 13 2895 18 2 16 0 15
3A3A_A 0.89 0.86 0.91 32 3620 3 1 2 0 5
3AKZ_H 0.40 0.43 0.39 12 2670 19 3 16 0 16
3AM1_B - 0.93 0.89 0.97 31 3208 2 0 1 1 4
3AMU_B 0.44 0.48 0.42 13 2972 18 2 16 0 14
3DW4_A - 0.90 0.90 0.90 9 341 3 0 1 2 1
3GCA_A - 0.04 0.06 0.07 1 514 13 1 12 0 16
3GX2_A 0.47 0.48 0.48 19 4331 22 0 21 1 21
3HAY_E - 1.00 1.00 1.00 18 2260 11 0 0 11 0
3HJW_D - 0.94 0.94 0.94 17 1635 8 0 1 7 1
3IAB_R - 0.84 0.93 0.76 13 1018 8 2 2 4 1
3ID5_D - 0.34 0.33 0.38 3 488 9 0 5 4 6
3IVN_B 0.39 0.39 0.40 12 2316 18 0 18 0 19
3IWN_A 0.19 0.21 0.18 7 4238 33 8 25 0 26
3IZF_C 0.71 0.69 0.74 37 6853 14 0 13 1 17
3J0L_8 - 1.00 1.00 1.00 8 182 0 0 0 0 0
3J0L_a - 0.16 0.19 0.17 3 1110 15 1 14 0 13
3J0L_1 - 0.79 0.79 0.79 15 1206 6 0 4 2 4
3J0L_h - 0.91 0.91 0.91 39 6062 13 0 4 9 4
3J0L_7 - -0.01 0.00 0.00 0 1206 19 4 15 0 17
3J0L_g - 0.13 0.25 0.08 1 452 12 8 4 0 3
3J0L_2 - 0.21 0.24 0.18 8 6172 39 7 29 3 25
3J16_L 0.45 0.47 0.44 14 2743 19 1 17 1 16
3J20_0 0.59 0.57 0.63 17 2823 12 1 9 2 13
3J2L_3 0.58 0.57 0.60 30 7825 23 2 18 3 23
3J3D_C 0.42 0.43 0.43 12 2747 18 2 14 2 16
3J3E_8 0.12 0.12 0.12 4 7470 46 3 26 17 29
3J3E_7 0.40 0.39 0.42 21 7090 29 3 26 0 33
3J3F_8 0.13 0.17 0.11 6 12191 61 9 40 12 30
3J3F_7 0.74 0.74 0.74 37 7210 17 2 11 4 13
3JYV_7 0.21 0.22 0.21 7 2817 26 0 26 0 25
3JYX_3 0.41 0.52 0.33 14 6285 34 12 17 5 13
3JYX_4 0.20 0.24 0.17 8 12199 43 16 23 4 25
3KTW_C - 0.40 0.40 0.43 17 4520 24 4 19 1 26
3LA5_A 0.32 0.32 0.34 11 2453 21 1 20 0 23
3NDB_M - 0.23 0.23 0.24 14 9121 46 1 44 1 47
3NKB_B - 0.56 0.58 0.56 15 1989 13 0 12 1 11
3NMU_E - 0.28 0.33 0.25 2 553 12 0 6 6 4
3O58_3 0.22 0.26 0.19 9 12355 45 9 30 6 26
3O58_2 0.20 0.24 0.17 9 7208 44 5 38 1 29
3OVB_D - 1.00 1.00 1.00 11 584 2 0 0 2 0
3OVS_D - 1.00 1.00 1.00 13 548 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 12 729 5 0 0 5 0
3PDR_A 0.66 0.61 0.72 44 12819 19 0 17 2 28
3R4F_A - 0.92 0.92 0.92 23 2120 5 0 2 3 2
3R9X_C - 0.69 0.70 0.70 7 585 7 0 3 4 3
3RKF_A 0.70 0.65 0.76 22 2182 7 1 6 0 12
3SD1_A 0.33 0.33 0.35 14 3876 26 0 26 0 28
3SIU_F - 0.42 0.45 0.42 5 366 7 1 6 0 6
3SN2_B 0.55 0.58 0.54 7 393 6 0 6 0 5
3TRZ_Z - -0.04 0.00 0.00 0 201 9 0 9 0 6
3TS0_U - 1.00 1.00 1.00 6 247 2 0 0 2 0
3TS2_V - -0.02 0.00 0.00 0 268 9 0 8 1 5
3U4M_B - 0.59 0.59 0.59 22 3123 15 0 15 0 15
3VJR_D - 1.00 1.00 1.00 13 617 1 0 0 1 0
3ZEX_D 0.17 0.18 0.18 9 6970 42 4 38 0 40
3ZND_W 0.19 0.22 0.18 5 2975 26 1 22 3 18
4A1C_3 0.68 0.67 0.71 36 7089 17 1 14 2 18
4A1C_2 0.13 0.15 0.11 5 11735 56 10 31 15 28
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ATO_G - 0.30 0.30 0.33 3 519 8 0 6 2 7
4ENB_A 0.61 0.63 0.60 12 1255 8 2 6 0 7
4ENC_A 0.28 0.32 0.27 6 1304 17 2 14 1 13
4FRG_B 0.32 0.34 0.31 11 3450 25 0 25 0 21
4FRN_A 0.12 0.14 0.12 5 5110 38 1 35 2 31
4JF2_A 0.66 0.68 0.66 21 2818 11 0 11 0 10
4JRC_A - 0.20 0.22 0.21 5 1516 19 0 19 0 18

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.