CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of HotKnots - scored higher in this pairwise comparison

  4. Performance of Contrafold - scored lower in this pairwise comparison

  5. Compile and download dataset for HotKnots & Contrafold [.zip] - may take several seconds...


Overview

Metric HotKnots Contrafold
MCC 0.662 > 0.660
Average MCC ± 95% Confidence Intervals 0.720 ± 0.051 > 0.715 ± 0.049
Sensitivity 0.600 > 0.592
Positive Predictive Value 0.736 < 0.740
Total TP 1684 > 1662
Total TN 399800 < 399841
Total FP 713 > 697
Total FP CONTRA 95 = 95
Total FP INCONS 508 > 489
Total FP COMP 110 < 113
Total FN 1124 < 1146
P-value 0.00636070127375

^top




Performance plots


^top





Performance of HotKnots - scored higher in this pairwise comparison

1. Total counts & total scores for HotKnots

Total Base Pair Counts
Total TP 1684
Total TN 399800
Total FP 713
Total FP CONTRA 95
Total FP INCONS 508
Total FP COMP 110
Total FN 1124
Total Scores
MCC 0.662
Average MCC ± 95% Confidence Intervals 0.720 ± 0.051
Sensitivity 0.600
Positive Predictive Value 0.736
Nr of predictions 118

^top



2. Individual counts for HotKnots [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 0 0 0 0 0
2K63_A - 0.95 0.90 1.00 9 397 0 0 0 0 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - -0.02 0.00 0.00 0 623 7 0 7 0 20
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - 0.83 0.69 1.00 11 692 0 0 0 0 5
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.61 0.55 0.69 11 1524 5 0 5 0 9
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.93 0.87 1.00 34 6071 3 0 0 3 5
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.83 0.70 1.00 7 224 0 0 0 0 3
2WRQ_Y 0.57 0.59 0.56 10 2832 13 5 3 5 7
2WW9_D - 0.14 0.12 0.19 3 1937 13 0 13 0 23
2WW9_E - 0.53 0.29 1.00 4 557 0 0 0 0 10
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WWQ_V 0.76 0.68 0.86 19 2904 5 0 3 2 9
2XKV_B 0.51 0.50 0.53 10 4541 23 0 9 14 10
2XQD_Y 0.88 0.78 1.00 21 2829 0 0 0 0 6
2XXA_G 0.34 0.31 0.39 13 5118 20 1 19 0 29
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.65 0.63 0.68 15 2256 8 1 6 1 9
2YIE_X - 0.36 0.33 0.40 4 1368 6 2 4 0 8
2YIE_Z - -0.01 0.00 0.00 0 1525 15 3 12 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 1 4 0 5
2ZZM_B 0.21 0.19 0.25 6 3462 18 0 18 0 26
2ZZN_D 0.82 0.78 0.88 21 2461 3 0 3 0 6
3A2K_C 0.42 0.39 0.46 11 2902 13 2 11 0 17
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3ADB_C - 0.79 0.74 0.85 28 4153 5 0 5 0 10
3AKZ_H 0.43 0.39 0.48 11 2678 12 2 10 0 17
3AM1_B - 0.78 0.71 0.86 25 3211 4 0 4 0 10
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3DW4_A - -0.02 0.00 0.00 0 345 6 0 6 0 10
3G4S_9 0.50 0.39 0.65 22 7347 12 1 11 0 35
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.68 0.55 0.85 22 4345 5 0 4 1 18
3HAY_E - 0.91 0.83 1.00 15 2263 6 0 0 6 3
3HJW_D - 0.94 0.89 1.00 16 1637 2 0 0 2 2
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.44 0.33 0.60 3 491 5 0 2 3 6
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.63 0.58 0.70 19 4251 9 0 8 1 14
3IYQ_A 0.31 0.34 0.29 32 60617 81 22 55 4 62
3IZ4_A 0.51 0.47 0.55 62 70763 52 10 41 1 70
3IZF_C 0.70 0.61 0.80 33 6862 8 1 7 0 21
3J0L_h - 0.75 0.60 0.93 26 6077 4 0 2 2 17
3J0L_g - -0.01 0.00 0.00 0 460 5 1 4 0 4
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.51 0.50 0.53 8 1113 7 3 4 0 8
3J0L_2 - 0.28 0.27 0.29 9 6185 23 3 19 1 24
3J0L_1 - 0.76 0.63 0.92 12 1212 2 0 1 1 7
3J0L_7 - 0.28 0.29 0.29 5 1208 12 0 12 0 12
3J16_L 0.26 0.23 0.30 7 2752 16 1 15 0 23
3JYV_7 -0.01 0.00 0.00 0 2828 22 1 21 0 32
3JYX_3 0.62 0.63 0.61 17 6300 22 1 10 11 10
3JYX_4 0.32 0.30 0.33 10 12216 31 5 15 11 23
3KTW_C - 0.73 0.63 0.84 27 4528 6 2 3 1 16
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.84 0.74 0.96 45 9133 3 0 2 1 16
3NKB_B - 0.59 0.54 0.67 14 1995 7 0 7 0 12
3NMU_E - 0.54 0.50 0.60 3 556 5 0 2 3 3
3NPB_A 0.78 0.65 0.94 30 6989 5 0 2 3 16
3O58_2 0.71 0.71 0.71 27 7222 12 4 7 1 11
3O58_3 0.23 0.26 0.21 9 12360 34 10 24 0 26
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.67 0.56 0.82 40 12831 11 0 9 2 32
3R4F_A - 0.96 0.92 1.00 23 2122 0 0 0 0 2
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.64 0.52 0.79 22 3888 6 1 5 0 20
3SIU_F - 0.85 0.73 1.00 8 370 0 0 0 0 3
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3UZL_B 0.48 0.38 0.61 14 3547 9 0 9 0 23
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_2 0.14 0.15 0.14 5 11745 42 6 25 11 28
4A1C_3 0.70 0.59 0.82 32 7101 7 1 6 0 22
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.50 0.40 0.63 17 4344 11 2 8 1 25
4ENB_A 0.89 0.79 1.00 15 1260 0 0 0 0 4
4ENC_A 0.86 0.79 0.94 15 1310 1 1 0 0 4

^top



Performance of Contrafold - scored lower in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 1662
Total TN 399841
Total FP 697
Total FP CONTRA 95
Total FP INCONS 489
Total FP COMP 113
Total FN 1146
Total Scores
MCC 0.660
Average MCC ± 95% Confidence Intervals 0.715 ± 0.049
Sensitivity 0.592
Positive Predictive Value 0.740
Nr of predictions 118

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 0 0 0 0 0
2K63_A - 0.90 0.90 0.90 9 396 2 0 1 1 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - 0.45 0.25 0.83 5 624 1 0 1 0 15
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 1.00 1.00 1.00 13 548 0 0 0 0 0
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - 0.55 0.44 0.70 7 693 3 0 3 0 9
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.75 0.74 0.78 14 1467 4 2 2 0 5
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.92 0.90 0.95 18 971 1 0 1 0 2
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.45 0.35 0.58 7 1528 5 2 3 0 13
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.84 0.79 0.89 31 6070 6 0 4 2 8
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.88 0.85 0.92 11 484 1 0 1 0 2
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.83 0.70 1.00 7 224 0 0 0 0 3
2WRQ_Y 0.57 0.59 0.56 10 2832 13 5 3 5 7
2WW9_D - 0.39 0.23 0.67 6 1944 3 2 1 0 20
2WW9_E - 0.00 0.00 0.00 0 561 0 0 0 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WWQ_V 0.78 0.68 0.90 19 2905 4 0 2 2 9
2XKV_B 0.51 0.50 0.53 10 4541 25 0 9 16 10
2XQD_Y 0.78 0.70 0.86 19 2828 3 0 3 0 8
2XXA_G 0.13 0.12 0.15 5 5117 29 1 28 0 37
2Y8W_B - 1.00 1.00 1.00 6 184 3 0 0 3 0
2Y9C_V - 0.66 0.63 0.71 15 2257 8 1 5 2 9
2YIE_X - 0.73 0.67 0.80 8 1368 3 1 1 1 4
2YIE_Z - 0.43 0.42 0.45 5 1529 6 3 3 0 7
2ZY6_A - 0.58 0.58 0.58 7 691 5 0 5 0 5
2ZZM_B 0.53 0.47 0.60 15 3461 10 2 8 0 17
2ZZN_D 0.82 0.78 0.88 21 2461 3 0 3 0 6
3A2K_C 0.44 0.43 0.46 12 2900 14 2 12 0 16
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3ADB_C - 0.92 0.84 1.00 32 4154 0 0 0 0 6
3AKZ_H 0.39 0.39 0.41 11 2674 17 4 12 1 17
3AM1_B - 0.68 0.63 0.73 22 3210 8 1 7 0 13
3AMU_B 0.65 0.59 0.73 16 2981 8 0 6 2 11
3DW4_A - -0.02 0.00 0.00 0 345 6 0 6 0 10
3G4S_9 0.30 0.25 0.38 14 7344 23 1 22 0 43
3GCA_A - 0.49 0.29 0.83 5 522 1 0 1 0 12
3GX2_A 0.77 0.63 0.96 25 4345 2 0 1 1 15
3HAY_E - 0.91 0.83 1.00 15 2263 6 0 0 6 3
3HJW_D - 0.94 0.89 1.00 16 1637 2 0 0 2 2
3IAB_R - 0.86 0.86 0.86 12 1021 4 1 1 2 2
3ID5_D - 0.49 0.33 0.75 3 492 6 0 1 5 6
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.80 0.67 0.96 22 4255 1 0 1 0 11
3IYQ_A 0.34 0.37 0.32 35 60615 80 21 55 4 59
3IZ4_A 0.52 0.45 0.61 60 70777 44 4 35 5 72
3IZF_C 0.68 0.61 0.77 33 6860 10 1 9 0 21
3J0L_h - 0.75 0.65 0.88 28 6073 6 0 4 2 15
3J0L_g - 0.00 0.00 0.00 0 464 1 1 0 0 4
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.50 0.44 0.58 7 1116 5 2 3 0 9
3J0L_2 - 0.25 0.24 0.28 8 6187 26 0 21 5 25
3J0L_1 - 0.73 0.58 0.92 11 1213 2 0 1 1 8
3J0L_7 - -0.01 0.00 0.00 0 1218 7 0 7 0 17
3J16_L 0.46 0.40 0.55 12 2753 10 1 9 0 18
3JYV_7 -0.01 0.00 0.00 0 2830 20 0 20 0 32
3JYX_3 0.28 0.30 0.27 8 6298 23 8 14 1 19
3JYX_4 0.20 0.21 0.20 7 12211 35 6 22 7 26
3KTW_C - 0.44 0.40 0.50 17 4526 18 2 15 1 26
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.81 0.72 0.92 44 9132 5 0 4 1 17
3NKB_B - 0.54 0.50 0.59 13 1994 9 0 9 0 13
3NMU_E - 0.71 0.50 1.00 3 558 3 0 0 3 3
3NPB_A 0.76 0.70 0.84 32 6983 8 1 5 2 14
3O58_2 0.78 0.76 0.81 29 7224 10 2 5 3 9
3O58_3 0.28 0.26 0.31 9 12374 20 3 17 0 26
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 1 0 0 1 1
3PDR_A 0.69 0.60 0.80 43 12826 13 0 11 2 29
3R4F_A - 0.79 0.76 0.83 19 2122 4 1 3 0 6
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.73 0.59 0.91 20 2189 2 1 1 0 14
3SD1_A 0.57 0.48 0.69 20 3887 9 2 7 0 22
3SIU_F - 0.74 0.64 0.88 7 370 1 0 1 0 4
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.59 0.46 0.77 17 3138 5 0 5 0 20
3UZL_B 0.70 0.54 0.91 20 3548 2 0 2 0 17
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_2 0.16 0.15 0.17 5 11751 33 5 20 8 28
4A1C_3 0.66 0.57 0.78 31 7100 9 1 8 0 23
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.44 0.36 0.56 15 4344 13 1 11 1 27
4ENB_A 0.73 0.58 0.92 11 1263 1 1 0 0 8
4ENC_A 0.73 0.58 0.92 11 1314 1 1 0 0 8

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.