CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of UNAFold - scored higher in this pairwise comparison

  4. Performance of HotKnots - scored lower in this pairwise comparison

  5. Compile and download dataset for UNAFold & HotKnots [.zip] - may take several seconds...


Overview

Metric UNAFold HotKnots
MCC 0.775 > 0.772
Average MCC ± 95% Confidence Intervals 0.810 ± 0.050 < 0.811 ± 0.052
Sensitivity 0.786 < 0.793
Positive Predictive Value 0.769 > 0.757
Total TP 1601 < 1614
Total TN 149953 > 149903
Total FP 758 < 783
Total FP CONTRA 154 < 180
Total FP INCONS 327 < 338
Total FP COMP 277 > 265
Total FN 435 > 422
P-value 1.82713256396e-06

^top




Performance plots


  1. Comparison of performance of UNAFold and HotKnots. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for UNAFold and HotKnots).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for UNAFold and HotKnots).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for UNAFold and HotKnots. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for UNAFold and HotKnots).

^top





Performance of UNAFold - scored higher in this pairwise comparison

1. Total counts & total scores for UNAFold

Total Base Pair Counts
Total TP 1601
Total TN 149953
Total FP 758
Total FP CONTRA 154
Total FP INCONS 327
Total FP COMP 277
Total FN 435
Total Scores
MCC 0.775
Average MCC ± 95% Confidence Intervals 0.810 ± 0.050
Sensitivity 0.786
Positive Predictive Value 0.769
Nr of predictions 118

^top



2. Individual counts for UNAFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.95 0.91 1.00 10 176 0 0 0 0 1
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - -0.03 0.00 0.00 0 229 7 1 6 0 8
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.96 0.92 1.00 11 281 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 513 15 3 12 0 18
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 0.95 0.91 1.00 10 152 0 0 0 0 1
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 6 0 0 6 0
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_D - 0.13 0.10 0.20 1 736 9 1 3 5 9
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WWQ_V 1.00 1.00 1.00 19 1185 3 0 0 3 0
2XKV_B 0.58 0.64 0.54 7 1822 23 0 6 17 4
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.75 0.78 0.74 14 887 8 0 5 3 4
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 2 2 0 0 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.86 0.85 0.88 28 1787 4 0 4 0 5
3AKZ_H 0.48 0.50 0.48 10 1106 13 4 7 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - 0.91 0.83 1.00 5 135 2 0 0 2 1
3G4S_9 0.78 0.77 0.80 20 2711 13 2 3 8 6
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.81 0.79 0.85 22 1423 5 2 2 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 0.97 0.94 1.00 15 594 2 0 0 2 1
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 5 0 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.80 0.79 0.81 22 1445 6 1 4 1 6
3IYQ_A 0.23 0.33 0.17 17 22339 96 46 38 12 34
3IZ4_A 0.58 0.60 0.57 57 25436 53 12 31 10 38
3IZF_C 0.89 0.91 0.86 32 2603 9 0 5 4 3
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.55 0.64 0.50 7 397 8 5 2 1 4
3J0L_2 - 0.30 0.31 0.31 8 2224 25 2 16 7 18
3J0L_1 - 0.75 0.69 0.82 9 473 4 0 2 2 4
3J0L_7 - 0.66 0.70 0.64 7 508 5 2 2 1 3
3J16_L 0.31 0.33 0.30 7 1136 16 3 13 0 14
3JYV_7 0.24 0.25 0.25 5 1091 16 7 8 1 15
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3JYX_4 0.61 0.83 0.45 10 4734 31 10 2 19 2
3KTW_C - 1.00 1.00 1.00 25 1755 7 0 0 7 0
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 1.00 1.00 1.00 44 3670 4 0 0 4 0
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.85 0.78 0.94 29 2247 6 0 2 4 8
3O58_2 0.87 0.87 0.87 27 2723 12 0 4 8 4
3O58_3 0.42 0.50 0.35 11 4733 34 5 15 14 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.93 0.92 0.94 46 4791 5 1 2 2 4
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.77 0.76 0.79 22 1505 6 4 2 0 7
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_2 0.19 0.25 0.15 5 4483 42 11 17 14 15
4A1C_3 0.88 0.86 0.89 32 2727 7 0 4 3 5
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 0.85 0.73 1.00 11 461 2 0 0 2 4
4ENC_A 0.37 0.33 0.45 5 485 7 0 6 1 10

^top



Performance of HotKnots - scored lower in this pairwise comparison

1. Total counts & total scores for HotKnots

Total Base Pair Counts
Total TP 1614
Total TN 149903
Total FP 783
Total FP CONTRA 180
Total FP INCONS 338
Total FP COMP 265
Total FN 422
Total Scores
MCC 0.772
Average MCC ± 95% Confidence Intervals 0.811 ± 0.052
Sensitivity 0.793
Positive Predictive Value 0.757
Nr of predictions 118

^top



2. Individual counts for HotKnots [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - -0.03 0.00 0.00 0 229 7 1 6 0 8
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.96 0.92 1.00 11 281 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.64 0.61 0.69 11 512 5 0 5 0 7
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 8 0 0 8 0
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 0 0 0 0 0
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WWQ_V 1.00 1.00 1.00 19 1185 5 0 0 5 0
2XKV_B 0.64 0.73 0.57 8 1821 25 0 6 19 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 3 2 0 1 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.86 0.85 0.88 28 1787 5 0 4 1 5
3AKZ_H 0.48 0.50 0.48 10 1106 13 4 7 2 10
3AM1_B - 0.86 0.86 0.86 25 1437 4 0 4 0 4
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - -0.03 0.00 0.00 0 136 6 1 3 2 6
3G4S_9 0.78 0.77 0.80 20 2711 14 2 3 9 6
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.81 0.79 0.85 22 1423 5 2 2 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 5 0 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.69 0.68 0.70 19 1445 9 1 7 1 9
3IYQ_A 0.24 0.33 0.17 17 22341 96 47 35 14 34
3IZ4_A 0.56 0.60 0.52 57 25426 57 26 27 4 38
3IZF_C 0.89 0.91 0.86 32 2603 9 0 5 4 3
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_g - -0.01 0.00 0.00 0 173 5 1 2 2 2
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.55 0.64 0.50 7 397 8 5 2 1 4
3J0L_2 - 0.34 0.35 0.35 9 2224 23 2 15 6 17
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_7 - 0.38 0.50 0.31 5 503 12 4 7 1 5
3J16_L 0.31 0.33 0.30 7 1136 16 3 13 0 14
3JYV_7 -0.02 0.00 0.00 0 1089 22 4 18 0 20
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3JYX_4 0.61 0.83 0.45 10 4734 31 10 2 19 2
3KTW_C - 0.96 0.96 0.96 24 1755 9 0 1 8 1
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 1.00 1.00 1.00 44 3670 4 0 0 4 0
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.85 0.78 0.94 29 2247 6 0 2 4 8
3O58_2 0.87 0.87 0.87 27 2723 12 0 4 8 4
3O58_3 0.26 0.36 0.19 8 4722 35 16 18 1 14
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.81 0.80 0.82 40 4791 11 3 6 2 10
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.77 0.76 0.79 22 1505 6 4 2 0 7
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_2 0.19 0.25 0.15 5 4483 42 12 16 14 15
4A1C_3 0.88 0.86 0.89 32 2727 7 0 4 3 5
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.