CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CentroidFold - scored higher in this pairwise comparison

  4. Performance of Vsfold4 - scored lower in this pairwise comparison

  5. Compile and download dataset for CentroidFold & Vsfold4 [.zip] - may take several seconds...


Overview

Metric CentroidFold Vsfold4
MCC 0.747 > 0.610
Average MCC ± 95% Confidence Intervals 0.770 ± 0.052 > 0.663 ± 0.059
Sensitivity 0.747 > 0.601
Positive Predictive Value 0.753 > 0.628
Total TP 1804 > 1452
Total TN 194734 < 194817
Total FP 867 < 1083
Total FP CONTRA 201 < 242
Total FP INCONS 391 < 619
Total FP COMP 275 > 222
Total FN 610 < 962
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CentroidFold and Vsfold4. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CentroidFold and Vsfold4).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CentroidFold and Vsfold4).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CentroidFold and Vsfold4. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CentroidFold and Vsfold4).

^top





Performance of CentroidFold - scored higher in this pairwise comparison

1. Total counts & total scores for CentroidFold

Total Base Pair Counts
Total TP 1804
Total TN 194734
Total FP 867
Total FP CONTRA 201
Total FP INCONS 391
Total FP COMP 275
Total FN 610
Total Scores
MCC 0.747
Average MCC ± 95% Confidence Intervals 0.770 ± 0.052
Sensitivity 0.747
Positive Predictive Value 0.753
Nr of predictions 126

^top



2. Individual counts for CentroidFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.53 0.42 0.71 5 285 2 2 0 0 7
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 0.94 0.94 0.94 17 339 2 0 1 1 1
2LA5_A - 0.77 0.71 0.83 5 269 2 0 1 1 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.51 0.39 0.70 7 518 3 1 2 0 11
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.88 0.86 0.89 25 2412 8 0 3 5 4
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LU0_A - 0.93 0.88 1.00 14 424 0 0 0 0 2
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2M58_A - -0.02 0.00 0.00 0 538 6 0 6 0 12
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 1.00 1.00 1.00 9 1143 13 0 0 13 0
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - 0.00 0.00 0.00 0 172 0 0 0 0 5
2WW9_D - 0.31 0.10 1.00 1 740 3 0 0 3 9
2WWQ_V 1.00 1.00 1.00 19 1185 3 0 0 3 0
2XKV_B 0.64 0.73 0.57 8 1821 27 0 6 21 3
2XQD_Y 0.83 0.81 0.85 17 1109 4 0 3 1 4
2XXA_G 0.13 0.11 0.17 4 2021 21 2 18 1 31
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
2YIE_X - 1.00 1.00 1.00 7 537 3 0 0 3 0
2ZZM_B 0.13 0.13 0.15 2 1345 20 3 8 9 13
2ZZN_D 0.93 0.95 0.91 21 961 3 2 0 1 1
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.98 0.97 1.00 32 1787 0 0 0 0 1
3AKZ_H 0.48 0.50 0.48 10 1106 13 3 8 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3GX2_A 0.94 0.89 1.00 25 1424 1 0 0 1 3
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3ID5_D - -0.02 0.00 0.00 0 223 6 3 3 0 5
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.86 0.79 0.96 22 1449 1 0 1 0 6
3IYQ_A 0.29 0.39 0.22 20 22347 86 39 34 13 31
3IZ4_A 0.60 0.57 0.64 54 25451 38 16 15 7 41
3IZF_C 0.89 0.91 0.86 32 2603 11 0 5 6 3
3J0L_g - 0.00 0.00 0.00 0 176 0 0 0 0 2
3J0L_2 - 0.28 0.31 0.28 8 2221 24 2 19 3 18
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.41 0.36 0.50 4 403 5 3 1 1 7
3J0L_1 - 0.73 0.62 0.89 8 475 3 0 1 2 5
3J0L_h - 0.89 0.88 0.90 28 2109 5 0 3 2 4
3J0L_7 - -0.01 0.00 0.00 0 514 5 1 4 0 10
3J16_L 0.59 0.57 0.63 12 1140 7 0 7 0 9
3J20_1 1.00 1.00 1.00 20 1092 2 0 0 2 0
3J20_0 0.54 0.57 0.52 12 1196 12 3 8 1 9
3J2C_O - 0.95 0.90 1.00 37 3950 6 0 0 6 4
3J2L_3 0.74 0.76 0.72 26 2984 15 1 9 5 8
3J3D_C 0.73 0.79 0.68 15 946 7 3 4 0 4
3J3E_8 0.12 0.13 0.13 2 2726 22 4 10 8 13
3J3E_7 0.61 0.59 0.65 20 2710 13 2 9 2 14
3J3F_7 0.84 0.86 0.82 31 2896 9 1 6 2 5
3J3F_8 0.36 0.47 0.28 9 4729 37 12 11 14 10
3J3V_B 0.72 0.74 0.71 20 2628 16 1 7 8 7
3JYV_7 0.92 0.85 1.00 17 1094 2 0 0 2 3
3JYX_4 0.41 0.58 0.29 7 4732 25 11 6 8 5
3JYX_3 0.35 0.47 0.27 7 2352 21 14 5 2 8
3KTW_C - 0.58 0.60 0.58 15 1754 18 1 10 7 10
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.97 0.95 0.98 42 3671 6 0 1 5 2
3NKB_B - 0.66 0.68 0.65 13 715 7 0 7 0 6
3NMU_E - 1.00 1.00 1.00 3 213 3 0 0 3 0
3NPB_A 0.87 0.84 0.91 31 2244 8 1 2 5 6
3O58_2 0.95 0.94 0.97 29 2724 7 0 1 6 2
3O58_3 0.44 0.41 0.47 9 4745 11 1 9 1 13
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.88 0.86 0.90 43 4792 7 2 3 2 7
3R4F_A - 0.83 0.82 0.86 18 885 4 2 1 1 4
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.70 0.69 0.71 20 1505 8 4 4 0 9
3SIU_F - 0.93 0.88 1.00 7 138 0 0 0 0 1
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.81 0.77 0.85 17 1256 5 0 3 2 5
3UZL_B 0.93 0.88 1.00 14 1279 7 0 0 7 2
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W3S_B 0.95 0.91 1.00 30 1959 1 0 0 1 3
3ZEX_D 0.88 0.86 0.91 30 2763 8 0 3 5 5
3ZEX_E - 0.00 0.00 0.00 0 8255 51 19 30 2 34
3ZEX_G - 0.45 0.44 0.47 20 6464 32 6 17 9 25
3ZND_W 0.24 0.38 0.16 3 1172 25 9 7 9 5
4A1C_3 0.80 0.78 0.83 29 2728 8 0 6 2 8
4A1C_2 0.22 0.25 0.19 5 4490 29 9 12 8 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.50 0.48 0.54 14 1411 13 3 9 1 15
4ATO_G - 0.61 0.57 0.67 4 214 2 0 2 0 3
4ENB_A 0.85 0.73 1.00 11 461 1 0 0 1 4
4ENC_A 0.85 0.73 1.00 11 485 1 0 0 1 4
4FRG_B 0.75 0.71 0.81 17 1181 4 3 1 0 7
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.89 0.79 1.00 19 1063 0 0 0 0 5
4JRC_A - 0.38 0.41 0.39 7 604 11 0 11 0 10

^top



Performance of Vsfold4 - scored lower in this pairwise comparison

1. Total counts & total scores for Vsfold4

Total Base Pair Counts
Total TP 1452
Total TN 194817
Total FP 1083
Total FP CONTRA 242
Total FP INCONS 619
Total FP COMP 222
Total FN 962
Total Scores
MCC 0.610
Average MCC ± 95% Confidence Intervals 0.663 ± 0.059
Sensitivity 0.601
Positive Predictive Value 0.628
Nr of predictions 126

^top



2. Individual counts for Vsfold4 [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.84 0.83 0.86 19 2002 9 2 1 6 4
2KU0_A - -0.04 0.00 0.00 0 283 9 2 7 0 12
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.78 0.63 1.00 5 144 0 0 0 0 3
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 0.67 0.57 0.81 13 747 3 0 3 0 10
2L1F_B 0.69 0.58 0.82 14 774 3 0 3 0 10
2L2K_A - 0.94 0.88 1.00 15 334 0 0 0 0 2
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 0.93 0.87 1.00 26 959 0 0 0 0 4
2L5Z_A - 0.68 0.67 0.75 6 103 2 0 2 0 3
2L94_A 0.94 0.89 1.00 16 341 1 0 0 1 2
2LA5_A - 0.77 0.71 0.83 5 269 1 0 1 0 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 515 13 0 13 0 18
2LDL_A - 0.94 0.89 1.00 8 132 1 0 0 1 1
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.35 0.38 0.34 11 2408 21 7 14 0 18
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LU0_A - 0.68 0.63 0.77 10 425 3 0 3 0 6
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2M58_A - -0.02 0.00 0.00 0 538 6 0 6 0 12
2RPT_A - -0.07 0.00 0.00 0 68 4 0 4 0 7
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 0.94 0.89 1.00 8 1144 13 0 0 13 1
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_D - 0.12 0.10 0.17 1 735 8 1 4 3 9
2WWQ_V 0.78 0.74 0.82 14 1187 4 1 2 1 5
2XKV_B -0.01 0.00 0.00 0 1808 27 13 14 0 11
2XQD_Y 0.95 0.90 1.00 19 1110 1 0 0 1 2
2XXA_G -0.02 0.00 0.00 0 2016 29 1 28 0 35
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.75 0.67 0.86 12 892 4 0 2 2 6
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - -0.02 0.00 0.00 0 591 11 5 6 0 8
2ZZM_B 0.18 0.20 0.19 3 1342 22 4 9 9 12
2ZZN_D 0.95 0.95 0.95 21 962 1 1 0 0 1
3A2K_C 0.95 0.91 1.00 20 1088 0 0 0 0 2
3A3A_A 0.95 0.90 1.00 27 1473 0 0 0 0 3
3ADB_C - 0.95 0.91 1.00 30 1789 0 0 0 0 3
3AKZ_H 0.77 0.75 0.79 15 1108 6 2 2 2 5
3AM1_B - 0.93 0.86 1.00 25 1441 0 0 0 0 4
3AMU_B 0.73 0.74 0.74 14 1138 8 0 5 3 5
3GX2_A 0.60 0.57 0.64 16 1424 10 0 9 1 12
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 4 0 2 2 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.28 0.25 0.35 7 1452 13 2 11 0 21
3IYQ_A 0.27 0.35 0.21 18 22353 80 40 29 11 33
3IZ4_A 0.39 0.39 0.39 37 25441 63 15 43 5 58
3IZF_C 0.85 0.83 0.88 29 2607 6 0 4 2 6
3J0L_2 - 0.47 0.46 0.50 12 2226 18 1 11 6 14
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_7 - 0.69 0.70 0.70 7 509 4 1 2 1 3
3J0L_a - 0.39 0.36 0.44 4 402 6 4 1 1 7
3J0L_8 - 0.74 0.57 1.00 4 72 0 0 0 0 3
3J0L_h - 0.44 0.38 0.52 12 2117 11 2 9 0 20
3J0L_1 - 0.26 0.23 0.33 3 475 7 2 4 1 10
3J16_L 0.98 0.95 1.00 20 1139 0 0 0 0 1
3J20_1 0.75 0.75 0.75 15 1092 7 0 5 2 5
3J20_0 0.95 0.95 0.95 20 1198 2 1 0 1 1
3J2C_O - 0.33 0.32 0.36 13 3951 26 3 20 3 28
3J2L_3 0.71 0.71 0.73 24 2987 12 3 6 3 10
3J3D_C 0.48 0.53 0.45 10 946 12 4 8 0 9
3J3E_7 0.61 0.56 0.68 19 2713 10 1 8 1 15
3J3E_8 0.10 0.13 0.09 2 2720 32 9 11 12 13
3J3F_7 0.25 0.25 0.26 9 2900 25 6 19 0 27
3J3F_8 0.35 0.42 0.30 8 4734 30 11 8 11 11
3J3V_B 0.19 0.19 0.22 5 2633 24 3 15 6 22
3JYV_7 -0.02 0.00 0.00 0 1092 19 4 15 0 20
3JYX_3 0.64 0.67 0.63 10 2362 16 5 1 10 5
3JYX_4 0.19 0.25 0.15 3 4736 33 9 8 16 9
3KTW_C - 0.43 0.40 0.48 10 1759 15 2 9 4 15
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.33 0.32 0.35 14 3674 26 2 24 0 30
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 0.57 0.33 1.00 1 215 3 0 0 3 2
3NPB_A 0.66 0.59 0.73 22 2248 10 0 8 2 15
3O58_2 0.53 0.52 0.55 16 2725 16 3 10 3 15
3O58_3 0.12 0.14 0.11 3 4737 35 8 16 11 19
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.69 0.64 0.74 32 4797 13 3 8 2 18
3R4F_A - 0.98 0.95 1.00 21 885 1 0 0 1 1
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.23 0.24 0.24 7 1504 22 5 17 0 22
3SIU_F - 0.86 0.75 1.00 6 139 0 0 0 0 2
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.66 0.69 0.65 11 1276 10 3 3 4 5
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W3S_B 0.74 0.70 0.79 23 1960 7 1 5 1 10
3ZEX_E - 0.12 0.15 0.11 5 8259 40 13 27 0 29
3ZEX_D 0.66 0.63 0.71 22 2765 10 4 5 1 13
3ZEX_G - 0.31 0.29 0.34 13 6469 31 5 20 6 32
3ZND_W 0.58 0.75 0.46 6 1178 17 4 3 10 2
4A1C_3 0.29 0.30 0.31 11 2727 26 4 21 1 26
4A1C_2 0.36 0.40 0.33 8 4492 28 4 12 12 12
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.50 0.48 0.54 14 1411 13 3 9 1 15
4ATO_G - 0.38 0.43 0.38 3 212 5 5 0 0 4
4ENB_A 0.85 0.73 1.00 11 461 0 0 0 0 4
4ENC_A 0.77 0.60 1.00 9 487 0 0 0 0 6
4FRG_B 0.60 0.58 0.64 14 1180 8 3 5 0 10
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0
4JF2_A 0.74 0.67 0.84 16 1063 3 0 3 0 8
4JRC_A - 0.34 0.29 0.42 5 610 7 0 7 0 12

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.