CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CentroidFold - scored higher in this pairwise comparison

  4. Performance of Vsfold5 - scored lower in this pairwise comparison

  5. Compile and download dataset for CentroidFold & Vsfold5 [.zip] - may take several seconds...


Overview

Metric CentroidFold Vsfold5
MCC 0.626 > 0.491
Average MCC ± 95% Confidence Intervals 0.674 ± 0.051 > 0.572 ± 0.060
Sensitivity 0.544 > 0.425
Positive Predictive Value 0.726 > 0.574
Total TP 1825 > 1426
Total TN 504319 < 504348
Total FP 799 < 1162
Total FP CONTRA 93 < 125
Total FP INCONS 597 < 935
Total FP COMP 109 > 102
Total FN 1528 < 1927
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CentroidFold and Vsfold5. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CentroidFold and Vsfold5).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CentroidFold and Vsfold5).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CentroidFold and Vsfold5. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CentroidFold and Vsfold5).

^top





Performance of CentroidFold - scored higher in this pairwise comparison

1. Total counts & total scores for CentroidFold

Total Base Pair Counts
Total TP 1825
Total TN 504319
Total FP 799
Total FP CONTRA 93
Total FP INCONS 597
Total FP COMP 109
Total FN 1528
Total Scores
MCC 0.626
Average MCC ± 95% Confidence Intervals 0.674 ± 0.051
Sensitivity 0.544
Positive Predictive Value 0.726
Nr of predictions 124

^top



2. Individual counts for CentroidFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - 0.47 0.31 0.71 5 696 2 0 2 0 11
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.92 0.90 0.95 18 971 1 0 1 0 2
2LA5_A - 0.42 0.26 0.71 5 623 2 0 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.49 0.35 0.70 7 1530 3 1 2 0 13
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.82 0.74 0.91 29 6073 4 0 3 1 10
2LU0_A - 0.93 0.88 1.00 14 1162 0 0 0 0 2
2LWK_A - 0.92 0.92 0.92 12 483 1 0 1 0 1
2M58_A - 0.09 0.06 0.14 1 1646 6 0 6 0 16
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.57 0.59 0.56 10 2832 12 5 3 4 7
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_E - 0.00 0.00 0.00 0 561 0 0 0 0 14
2WW9_D - 0.29 0.12 0.75 3 1949 1 0 1 0 23
2WWQ_V 0.80 0.68 0.95 19 2906 3 0 1 2 9
2XKV_B 0.51 0.50 0.53 10 4541 25 0 9 16 10
2XQD_Y 0.75 0.67 0.86 18 2829 3 0 3 0 9
2XXA_G 0.15 0.12 0.20 5 5126 20 1 19 0 37
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.66 0.63 0.71 15 2257 8 1 5 2 9
2YIE_X - 0.67 0.58 0.78 7 1369 3 1 1 1 5
2YIE_Z - 0.43 0.42 0.45 5 1529 6 3 3 0 7
2ZZM_B 0.18 0.16 0.23 5 3464 17 3 14 0 27
2ZZN_D 0.82 0.78 0.88 21 2461 3 0 3 0 6
3A2K_C 0.44 0.43 0.46 12 2900 14 2 12 0 16
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3ADB_C - 0.92 0.84 1.00 32 4154 0 0 0 0 6
3AKZ_H 0.43 0.39 0.48 11 2678 12 2 10 0 17
3AM1_B - 0.68 0.63 0.73 22 3210 8 1 7 0 13
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3GX2_A 0.79 0.63 1.00 25 4346 1 0 0 1 15
3IAB_R - 0.86 0.86 0.86 12 1021 4 1 1 2 2
3ID5_D - -0.01 0.00 0.00 0 490 6 3 3 0 9
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.80 0.67 0.96 22 4255 1 0 1 0 11
3IYQ_A 0.36 0.37 0.34 35 60624 71 17 50 4 59
3IZ4_A 0.55 0.45 0.68 59 70789 33 4 24 5 73
3IZF_C 0.68 0.61 0.77 33 6860 10 1 9 0 21
3J0L_h - 0.77 0.65 0.90 28 6074 5 0 3 2 15
3J0L_1 - 0.65 0.47 0.90 9 1215 2 0 1 1 10
3J0L_7 - -0.01 0.00 0.00 0 1220 5 0 5 0 17
3J0L_g - 0.00 0.00 0.00 0 465 0 0 0 0 4
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.41 0.31 0.56 5 1119 4 2 2 0 11
3J0L_2 - 0.25 0.24 0.28 8 6187 24 0 21 3 25
3J16_L 0.50 0.40 0.63 12 2756 7 0 7 0 18
3J20_0 0.44 0.40 0.50 12 2826 13 0 12 1 18
3J20_1 0.91 0.87 0.95 20 2905 2 0 1 1 3
3J2C_O - 0.76 0.65 0.89 41 10250 6 0 5 1 22
3J2L_3 0.53 0.49 0.58 26 7830 21 0 19 2 27
3J3D_C 0.49 0.43 0.57 12 2754 9 0 9 0 16
3J3E_7 0.46 0.37 0.59 20 7106 14 1 13 0 34
3J3E_8 0.00 0.00 0.00 0 7485 21 2 16 3 33
3J3F_8 0.34 0.33 0.34 12 12211 34 4 19 11 24
3J3F_7 0.20 0.18 0.23 9 7221 30 1 29 0 41
3J3V_B 0.47 0.37 0.60 21 6986 14 1 13 0 36
3JYV_7 0.77 0.59 1.00 19 2831 0 0 0 0 13
3JYX_4 0.23 0.21 0.25 7 12218 25 5 16 4 26
3JYX_3 0.29 0.30 0.30 8 6301 20 8 11 1 19
3KTW_C - 0.45 0.40 0.53 17 4528 16 2 13 1 26
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.80 0.70 0.91 43 9133 5 0 4 1 18
3NKB_B - 0.57 0.50 0.65 13 1996 7 0 7 0 13
3NMU_E - 0.71 0.50 1.00 3 558 3 0 0 3 3
3NPB_A 0.77 0.70 0.86 32 6984 7 1 4 2 14
3O58_3 0.38 0.29 0.50 10 12383 10 0 10 0 25
3O58_2 0.82 0.76 0.88 29 7227 7 1 3 3 9
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 1 0 0 1 1
3PDR_A 0.73 0.60 0.90 43 12832 7 0 5 2 29
3R4F_A - 0.81 0.76 0.86 19 2123 3 1 2 0 6
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.58 0.48 0.71 20 3888 8 1 7 0 22
3SIU_F - 0.79 0.64 1.00 7 371 0 0 0 0 4
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.57 0.46 0.71 17 3136 7 0 7 0 20
3UZL_B 0.72 0.54 0.95 20 3549 1 0 1 0 17
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
3W3S_B 0.87 0.78 0.97 31 4721 2 0 1 1 9
3ZEX_E - 0.00 0.00 0.00 0 21895 52 3 47 2 77
3ZEX_D 0.73 0.59 0.91 29 6989 3 0 3 0 20
3ZND_W 0.20 0.22 0.19 5 2977 23 1 20 2 18
4A1C_3 0.67 0.56 0.81 30 7103 7 1 6 0 24
4A1C_2 0.16 0.15 0.17 5 11752 29 5 19 5 28
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.45 0.33 0.61 14 4348 10 1 8 1 28
4ATO_G - 0.31 0.20 0.50 2 524 2 0 2 0 8
4ENB_A 0.76 0.58 1.00 11 1264 0 0 0 0 8
4ENC_A 0.73 0.58 0.92 11 1314 1 1 0 0 8
4FRG_B 0.24 0.22 0.28 7 3461 18 3 15 0 25
4FRN_A 0.40 0.28 0.59 10 5134 7 2 5 0 26
4JF2_A 0.78 0.61 1.00 19 2831 0 0 0 0 12
4JRC_A - 0.81 0.70 0.94 16 1523 1 0 1 0 7

^top



Performance of Vsfold5 - scored lower in this pairwise comparison

1. Total counts & total scores for Vsfold5

Total Base Pair Counts
Total TP 1426
Total TN 504348
Total FP 1162
Total FP CONTRA 125
Total FP INCONS 935
Total FP COMP 102
Total FN 1927
Total Scores
MCC 0.491
Average MCC ± 95% Confidence Intervals 0.572 ± 0.060
Sensitivity 0.425
Positive Predictive Value 0.574
Nr of predictions 124

^top



2. Individual counts for Vsfold5 [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KMJ_A - 0.95 0.91 1.00 10 368 0 0 0 0 1
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.57 0.50 0.65 20 5120 11 5 6 0 20
2KU0_A - -0.02 0.00 0.00 0 692 11 0 11 0 16
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.91 0.83 1.00 15 846 0 0 0 0 3
2KXM_A - 0.70 0.50 1.00 5 346 0 0 0 0 5
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_B 0.70 0.60 0.83 15 2127 3 0 3 0 10
2L1F_A 0.69 0.58 0.82 14 2063 3 0 3 0 10
2L2K_A - 0.89 0.79 1.00 15 846 0 0 0 0 4
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.87 0.76 1.00 26 2459 0 0 0 0 8
2L5Z_A - 0.60 0.50 0.75 6 317 2 0 2 0 6
2L94_A 0.95 0.90 1.00 18 972 0 0 0 0 2
2LA5_A - -0.02 0.00 0.00 0 623 7 0 7 0 19
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A -0.01 0.00 0.00 0 1527 13 0 13 0 20
2LDL_A - 0.85 0.73 1.00 8 343 1 0 0 1 3
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2LKR_A - 0.27 0.23 0.32 9 6077 19 2 17 0 30
2LU0_A - 0.69 0.63 0.77 10 1163 3 0 3 0 6
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2M58_A - 0.32 0.24 0.44 4 1644 5 1 4 0 13
2RPT_A - 0.70 0.71 0.71 5 183 2 0 2 0 2
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.53 0.53 0.53 9 2833 12 5 3 4 8
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_E - -0.01 0.00 0.00 0 556 5 0 5 0 14
2WW9_D - 0.15 0.12 0.21 3 1939 11 1 10 0 23
2WWQ_V 0.64 0.50 0.82 14 2909 4 0 3 1 14
2XKV_B -0.01 0.00 0.00 0 4528 32 12 20 0 20
2XQD_Y 0.86 0.74 1.00 20 2830 0 0 0 0 7
2XXA_G -0.01 0.00 0.00 0 5120 31 2 29 0 42
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.61 0.54 0.68 13 2259 7 1 5 1 11
2YIE_X - 0.30 0.33 0.29 4 1364 10 2 8 0 8
2YIE_Z - -0.01 0.00 0.00 0 1529 11 3 8 0 12
2ZZM_B 0.19 0.19 0.21 6 3458 22 1 21 0 26
2ZZN_D 0.86 0.78 0.95 21 2463 1 0 1 0 6
3A2K_C 0.84 0.71 1.00 20 2906 0 0 0 0 8
3A3A_A 0.85 0.73 1.00 27 3628 0 0 0 0 10
3ADB_C - 0.67 0.53 0.87 20 4163 3 0 3 0 18
3AKZ_H 0.84 0.71 1.00 20 2681 0 0 0 0 8
3AM1_B - 0.78 0.71 0.86 25 3211 4 1 3 0 10
3AMU_B 0.67 0.59 0.76 16 2982 7 0 5 2 11
3GX2_A 0.51 0.40 0.67 16 4347 9 0 8 1 24
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.44 0.33 0.60 3 491 4 0 2 2 6
3IVN_B 0.76 0.58 1.00 18 2328 0 0 0 0 13
3IWN_A 0.25 0.21 0.30 7 4255 16 1 15 0 26
3IYQ_A 0.33 0.31 0.35 29 60642 61 15 40 6 65
3IZ4_A 0.29 0.24 0.36 32 70787 62 8 49 5 100
3IZF_C 0.70 0.56 0.88 30 6869 4 0 4 0 24
3J0L_h - 0.18 0.14 0.24 6 6080 19 1 18 0 37
3J0L_1 - 0.27 0.21 0.36 4 1214 7 1 6 0 15
3J0L_7 - -0.01 0.00 0.00 0 1212 13 0 13 0 17
3J0L_g - -0.01 0.00 0.00 0 461 4 1 3 0 4
3J0L_8 - 0.70 0.50 1.00 4 186 0 0 0 0 4
3J0L_a - 0.39 0.31 0.50 5 1118 5 2 3 0 11
3J0L_2 - 0.35 0.33 0.37 11 6186 21 3 16 2 22
3J16_L 0.82 0.67 1.00 20 2755 0 0 0 0 10
3J20_0 0.67 0.57 0.81 17 2829 5 1 3 1 13
3J20_1 0.71 0.70 0.73 16 2904 6 0 6 0 7
3J2C_O - 0.30 0.24 0.39 15 10258 24 0 23 1 48
3J2L_3 0.56 0.45 0.71 24 7841 13 1 9 3 29
3J3D_C 0.60 0.54 0.68 15 2753 7 0 7 0 13
3J3E_7 0.47 0.37 0.61 20 7107 13 0 13 0 34
3J3E_8 0.06 0.06 0.08 2 7477 31 5 19 7 31
3J3F_8 0.33 0.33 0.33 12 12210 35 5 19 11 24
3J3F_7 0.21 0.18 0.26 9 7226 25 1 24 0 41
3J3V_B 0.16 0.12 0.21 7 6988 26 1 25 0 50
3JYV_7 -0.01 0.00 0.00 0 2831 19 0 19 0 32
3JYX_4 0.31 0.30 0.31 10 12214 32 3 19 10 23
3JYX_3 0.17 0.15 0.20 4 6308 18 4 12 2 23
3KTW_C - 0.38 0.30 0.50 13 4534 13 3 10 0 30
3LA5_A -0.01 0.00 0.00 0 2469 16 0 16 0 34
3NDB_M - 0.06 0.05 0.08 3 9144 33 1 32 0 58
3NKB_B - 0.59 0.54 0.67 14 1995 7 0 7 0 12
3NMU_E - 0.20 0.17 0.25 1 557 5 0 3 2 5
3NPB_A 0.57 0.46 0.72 21 6992 9 1 7 1 25
3O58_3 0.39 0.37 0.41 13 12371 28 7 12 9 22
3O58_2 0.45 0.42 0.48 16 7227 19 3 14 2 22
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.57 0.44 0.74 32 12837 13 0 11 2 40
3R4F_A - 0.94 0.88 1.00 22 2123 0 0 0 0 3
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.75 0.65 0.88 22 2186 3 0 3 0 12
3SD1_A 0.11 0.10 0.15 4 3890 22 0 22 0 38
3SIU_F - 0.73 0.55 1.00 6 372 0 0 0 0 5
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3UZL_B 0.16 0.14 0.20 5 3545 20 1 19 0 32
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
3W3S_B 0.76 0.70 0.82 28 4719 7 1 5 1 12
3ZEX_E - 0.07 0.06 0.08 5 21886 54 5 49 0 72
3ZEX_D 0.08 0.06 0.11 3 6994 24 0 24 0 46
3ZND_W 0.41 0.39 0.43 9 2982 14 2 10 2 14
4A1C_3 0.26 0.22 0.32 12 7103 25 2 23 0 42
4A1C_2 0.24 0.24 0.24 8 11748 37 3 22 12 25
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.18 0.14 0.25 6 4347 19 1 17 1 36
4ATO_G - 0.56 0.60 0.55 6 517 5 1 4 0 4
4ENB_A 0.89 0.79 1.00 15 1260 0 0 0 0 4
4ENC_A 0.86 0.79 0.94 15 1310 1 1 0 0 4
4FRG_B 0.56 0.47 0.68 15 3464 7 1 6 0 17
4FRN_A 0.13 0.11 0.17 4 5128 19 2 17 0 32
4JF2_A 0.84 0.77 0.92 24 2824 2 1 1 0 7
4JRC_A - 0.25 0.22 0.31 5 1524 11 0 11 0 18

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.