CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Vsfold4 - scored higher in this pairwise comparison

  4. Performance of MCFold - scored lower in this pairwise comparison

  5. Compile and download dataset for Vsfold4 & MCFold [.zip] - may take several seconds...


Overview

Metric Vsfold4 MCFold
MCC 0.593 > 0.577
Average MCC ± 95% Confidence Intervals 0.642 ± 0.060 < 0.643 ± 0.066
Sensitivity 0.507 < 0.597
Positive Predictive Value 0.701 > 0.566
Total TP 1161 < 1368
Total TN 233663 > 232902
Total FP 562 < 1236
Total FP CONTRA 53 < 142
Total FP INCONS 442 < 907
Total FP COMP 67 < 187
Total FN 1129 > 922
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Vsfold4 and MCFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Vsfold4 and MCFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Vsfold4 and MCFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Vsfold4 and MCFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Vsfold4 and MCFold).

^top





Performance of Vsfold4 - scored higher in this pairwise comparison

1. Total counts & total scores for Vsfold4

Total Base Pair Counts
Total TP 1161
Total TN 233663
Total FP 562
Total FP CONTRA 53
Total FP INCONS 442
Total FP COMP 67
Total FN 1129
Total Scores
MCC 0.593
Average MCC ± 95% Confidence Intervals 0.642 ± 0.060
Sensitivity 0.507
Positive Predictive Value 0.701
Nr of predictions 109

^top



2. Individual counts for Vsfold4 [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.37 0.36 0.40 4 396 6 0 6 0 7
2K63_A - 0.95 0.90 1.00 9 397 0 0 0 0 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KEZ_A - 0.70 0.50 1.00 5 271 0 0 0 0 5
2KFC_A - 0.49 0.25 1.00 5 625 0 0 0 0 15
2KGP_A - -0.02 0.00 0.00 0 296 4 0 4 0 9
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KU0_A - -0.02 0.00 0.00 0 694 9 0 9 0 16
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.70 0.50 1.00 5 346 0 0 0 0 5
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_B 0.68 0.56 0.82 14 2128 3 0 3 0 11
2L1F_A 0.66 0.54 0.81 13 2064 3 0 3 0 11
2L2K_A - 0.89 0.79 1.00 15 846 0 0 0 0 4
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.87 0.76 1.00 26 2459 0 0 0 0 8
2L5Z_A - 0.60 0.50 0.75 6 317 2 0 2 0 6
2L94_A 0.92 0.85 1.00 17 973 0 0 0 0 3
2LA5_A - 0.46 0.26 0.83 5 624 1 0 1 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A -0.01 0.00 0.00 0 1527 13 0 13 0 20
2LDL_A - 0.85 0.73 1.00 8 343 1 0 0 1 3
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.31 0.28 0.34 11 6073 21 3 18 0 28
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - -0.03 0.00 0.00 0 186 4 0 4 0 7
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.53 0.53 0.53 9 2833 12 5 3 4 8
2WW9_E - -0.01 0.00 0.00 0 556 5 0 5 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_D - 0.19 0.12 0.33 3 1944 6 0 6 0 23
2WWQ_V 0.64 0.50 0.82 14 2909 4 0 3 1 14
2XKV_B -0.01 0.00 0.00 0 4533 27 8 19 0 20
2XQD_Y 0.86 0.74 1.00 20 2830 0 0 0 0 7
2XXA_G -0.01 0.00 0.00 0 5122 29 1 28 0 42
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.63 0.50 0.80 12 2263 4 0 3 1 12
2YIE_X - 0.36 0.33 0.40 4 1368 6 2 4 0 8
2YIE_Z - -0.01 0.00 0.00 0 1529 11 3 8 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 1 4 0 5
2ZZN_D 0.86 0.78 0.95 21 2463 1 0 1 0 6
3A2K_C 0.84 0.71 1.00 20 2906 0 0 0 0 8
3A3A_A 0.85 0.73 1.00 27 3628 0 0 0 0 10
3AKZ_H 0.67 0.57 0.80 16 2681 5 1 3 1 12
3AM1_B - 0.84 0.71 1.00 25 3215 0 0 0 0 10
3AMU_B 0.64 0.56 0.75 15 2983 7 0 5 2 12
3DW4_A - 0.70 0.50 1.00 5 346 0 0 0 0 5
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.50 0.40 0.64 16 4346 10 0 9 1 24
3HAY_E - 0.91 0.83 1.00 15 2263 3 0 0 3 3
3HJW_D - 0.51 0.50 0.53 9 1636 8 0 8 0 9
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.44 0.33 0.60 3 491 4 0 2 2 6
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.27 0.21 0.35 7 4258 13 1 12 0 26
3IZF_C 0.69 0.56 0.86 30 6868 5 0 5 0 24
3J0L_1 - 0.28 0.21 0.40 4 1215 6 1 5 0 15
3J0L_8 - 0.70 0.50 1.00 4 186 0 0 0 0 4
3J0L_a - 0.39 0.31 0.50 5 1118 5 2 3 0 11
3J0L_h - 0.38 0.28 0.52 12 6082 11 0 11 0 31
3J0L_7 - 0.51 0.41 0.64 7 1214 4 0 4 0 10
3J0L_g - -0.01 0.00 0.00 0 461 4 1 3 0 4
3J0L_2 - 0.38 0.36 0.41 12 6187 18 3 14 1 21
3J16_L 0.82 0.67 1.00 20 2755 0 0 0 0 10
3JYV_7 -0.01 0.00 0.00 0 2831 19 0 19 0 32
3JYX_4 0.10 0.09 0.12 3 12220 33 2 21 10 30
3JYX_3 0.60 0.56 0.65 15 6305 11 4 4 3 12
3KTW_C - 0.36 0.28 0.48 12 4535 13 3 10 0 31
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.28 0.23 0.35 14 9140 26 0 26 0 47
3NKB_B - 0.59 0.54 0.67 14 1995 7 0 7 0 12
3NMU_E - 0.41 0.17 1.00 1 560 3 0 0 3 5
3O58_2 0.47 0.42 0.53 16 7230 16 3 11 2 22
3O58_3 0.09 0.09 0.10 3 12373 35 5 22 8 32
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.57 0.44 0.74 32 12837 13 0 11 2 40
3R4F_A - 0.94 0.88 1.00 22 2123 0 0 0 0 3
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.19 0.17 0.24 7 3887 22 1 21 0 35
3SIU_F - 0.73 0.55 1.00 6 372 0 0 0 0 5
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_3 0.26 0.22 0.32 12 7103 25 2 23 0 42
4A1C_2 0.28 0.24 0.33 8 11757 28 0 16 12 25
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ENB_A 0.76 0.58 1.00 11 1264 0 0 0 0 8
4ENC_A 0.69 0.47 1.00 9 1317 0 0 0 0 10

^top



Performance of MCFold - scored lower in this pairwise comparison

1. Total counts & total scores for MCFold

Total Base Pair Counts
Total TP 1368
Total TN 232902
Total FP 1236
Total FP CONTRA 142
Total FP INCONS 907
Total FP COMP 187
Total FN 922
Total Scores
MCC 0.577
Average MCC ± 95% Confidence Intervals 0.643 ± 0.066
Sensitivity 0.597
Positive Predictive Value 0.566
Nr of predictions 109

^top



2. Individual counts for MCFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 1 0 0 1 0
2K63_A - 0.90 0.90 0.90 9 396 4 0 1 3 1
2K66_A - 1.00 1.00 1.00 10 221 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 222 1 0 0 1 0
2KDQ_B 0.95 0.91 1.00 10 396 1 0 0 1 1
2KE6_A 1.00 1.00 1.00 19 1109 2 0 0 2 0
2KEZ_A - 0.90 0.90 0.90 9 266 1 0 1 0 1
2KFC_A - 0.28 0.25 0.36 5 616 9 2 7 0 15
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 548 2 0 0 2 0
2KU0_A - 0.97 0.94 1.00 15 688 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 21 1107 0 0 0 0 0
2KUU_A 0.98 0.95 1.00 20 1108 1 0 0 1 1
2KUV_A 0.98 0.95 1.00 21 1107 0 0 0 0 1
2KUW_A 0.95 0.95 0.95 20 1107 1 0 1 0 1
2KX8_A 0.91 0.89 0.94 16 844 2 0 1 1 2
2KXM_A - 1.00 1.00 1.00 10 341 1 0 0 1 0
2KZL_A - 0.80 0.84 0.76 16 1464 9 1 4 4 3
2L1F_B 0.73 0.76 0.70 19 2118 9 0 8 1 6
2L1F_A 0.72 0.75 0.69 18 2054 9 0 8 1 6
2L2K_A - 1.00 1.00 1.00 19 842 1 0 0 1 0
2L3C_B - 0.97 0.94 1.00 16 545 0 0 0 0 1
2L3E_A - 1.00 1.00 1.00 13 582 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 34 2451 0 0 0 0 0
2L5Z_A - 0.96 0.92 1.00 11 314 0 0 0 0 1
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - -0.03 0.00 0.00 0 615 15 0 15 0 19
2LBS_A - 1.00 1.00 1.00 14 482 1 0 0 1 0
2LC8_A 0.41 0.45 0.39 9 1517 16 0 14 2 11
2LDL_A - 0.95 0.91 1.00 10 341 2 0 0 2 1
2LDT_A - 0.75 0.73 0.79 11 451 3 0 3 0 4
2LHP_A - 1.00 1.00 1.00 16 650 1 0 0 1 0
2LI4_A - 0.97 0.94 1.00 15 481 0 0 0 0 1
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 1.00 1.00 1.00 10 266 0 0 0 0 0
2LKR_A - 0.93 0.92 0.95 36 6067 14 0 2 12 3
2LQZ_A - 0.91 0.91 0.91 10 340 1 1 0 0 1
2LWK_A - 0.92 0.92 0.92 12 483 2 0 1 1 1
2RP0_A - 0.44 0.42 0.50 5 341 5 1 4 0 7
2RPT_A - 1.00 1.00 1.00 7 183 2 0 0 2 0
2RRC_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2WRQ_Y 0.27 0.35 0.22 6 2823 25 8 13 4 11
2WW9_E - -0.02 0.00 0.00 0 547 14 0 14 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_D - -0.01 0.00 0.00 0 1925 28 5 23 0 26
2WWQ_V 0.15 0.18 0.15 5 2892 30 4 25 1 23
2XKV_B 0.20 0.25 0.17 5 4530 37 2 23 12 15
2XQD_Y 0.34 0.37 0.32 10 2819 22 1 20 1 17
2XXA_G 0.23 0.24 0.24 10 5110 32 1 30 1 32
2Y8W_B - 1.00 1.00 1.00 6 184 2 0 0 2 0
2Y9C_V - 0.62 0.67 0.59 16 2251 13 2 9 2 8
2YIE_X - 0.06 0.08 0.06 1 1361 21 1 15 5 11
2YIE_Z - 0.36 0.50 0.27 6 1518 18 4 12 2 6
2ZY6_A - 0.51 0.58 0.47 7 688 8 3 5 0 5
2ZZN_D 0.61 0.63 0.61 17 2457 13 1 10 2 10
3A2K_C 0.44 0.46 0.42 13 2895 18 2 16 0 15
3A3A_A 0.89 0.86 0.91 32 3620 3 1 2 0 5
3AKZ_H 0.40 0.43 0.39 12 2670 19 3 16 0 16
3AM1_B - 0.93 0.89 0.97 31 3208 2 0 1 1 4
3AMU_B 0.44 0.48 0.42 13 2972 18 2 16 0 14
3DW4_A - 0.90 0.90 0.90 9 341 3 0 1 2 1
3GCA_A - 0.04 0.06 0.07 1 514 13 1 12 0 16
3GX2_A 0.47 0.48 0.48 19 4331 22 0 21 1 21
3HAY_E - 1.00 1.00 1.00 18 2260 11 0 0 11 0
3HJW_D - 0.94 0.94 0.94 17 1635 8 0 1 7 1
3IAB_R - 0.84 0.93 0.76 13 1018 8 2 2 4 1
3ID5_D - 0.34 0.33 0.38 3 488 9 0 5 4 6
3IVN_B 0.39 0.39 0.40 12 2316 18 0 18 0 19
3IWN_A 0.19 0.21 0.18 7 4238 33 8 25 0 26
3IZF_C 0.71 0.69 0.74 37 6853 14 0 13 1 17
3J0L_1 - 0.79 0.79 0.79 15 1206 6 0 4 2 4
3J0L_8 - 1.00 1.00 1.00 8 182 0 0 0 0 0
3J0L_a - 0.16 0.19 0.17 3 1110 15 1 14 0 13
3J0L_h - 0.91 0.91 0.91 39 6062 13 0 4 9 4
3J0L_7 - -0.01 0.00 0.00 0 1206 19 4 15 0 17
3J0L_g - 0.13 0.25 0.08 1 452 12 8 4 0 3
3J0L_2 - 0.21 0.24 0.18 8 6172 39 7 29 3 25
3J16_L 0.45 0.47 0.44 14 2743 19 1 17 1 16
3JYV_7 0.21 0.22 0.21 7 2817 26 0 26 0 25
3JYX_4 0.20 0.24 0.17 8 12199 43 16 23 4 25
3JYX_3 0.41 0.52 0.33 14 6285 34 12 17 5 13
3KTW_C - 0.40 0.40 0.43 17 4520 24 4 19 1 26
3LA5_A 0.32 0.32 0.34 11 2453 21 1 20 0 23
3NDB_M - 0.23 0.23 0.24 14 9121 46 1 44 1 47
3NKB_B - 0.56 0.58 0.56 15 1989 13 0 12 1 11
3NMU_E - 0.28 0.33 0.25 2 553 12 0 6 6 4
3O58_2 0.20 0.24 0.17 9 7208 44 5 38 1 29
3O58_3 0.22 0.26 0.19 9 12355 45 9 30 6 26
3OVB_D - 1.00 1.00 1.00 11 584 2 0 0 2 0
3OVS_D - 1.00 1.00 1.00 13 548 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 12 729 5 0 0 5 0
3PDR_A 0.66 0.61 0.72 44 12819 19 0 17 2 28
3R4F_A - 0.92 0.92 0.92 23 2120 5 0 2 3 2
3R9X_C - 0.69 0.70 0.70 7 585 7 0 3 4 3
3RKF_A 0.70 0.65 0.76 22 2182 7 1 6 0 12
3SD1_A 0.33 0.33 0.35 14 3876 26 0 26 0 28
3SIU_F - 0.42 0.45 0.42 5 366 7 1 6 0 6
3SN2_B 0.55 0.58 0.54 7 393 6 0 6 0 5
3TRZ_Z - -0.04 0.00 0.00 0 201 9 0 9 0 6
3TS0_U - 1.00 1.00 1.00 6 247 2 0 0 2 0
3TS2_V - -0.02 0.00 0.00 0 268 9 0 8 1 5
3U4M_B - 0.59 0.59 0.59 22 3123 15 0 15 0 15
3VJR_D - 1.00 1.00 1.00 13 617 1 0 0 1 0
4A1C_3 0.68 0.67 0.71 36 7089 17 1 14 2 18
4A1C_2 0.13 0.15 0.11 5 11735 56 10 31 15 28
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4ENB_A 0.61 0.63 0.60 12 1255 8 2 6 0 7
4ENC_A 0.28 0.32 0.27 6 1304 17 2 14 1 13

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.