CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Contrafold - scored higher in this pairwise comparison

  4. Performance of MaxExpect - scored lower in this pairwise comparison

  5. Compile and download dataset for Contrafold & MaxExpect [.zip] - may take several seconds...


Overview

Metric Contrafold MaxExpect
MCC 0.561 > 0.552
Average MCC ± 95% Confidence Intervals 0.679 ± 0.051 > 0.661 ± 0.055
Sensitivity 0.489 > 0.480
Positive Predictive Value 0.644 > 0.635
Total TP 2791 > 2742
Total TN 6831018 < 6831037
Total FP 1693 < 1738
Total FP CONTRA 183 > 156
Total FP INCONS 1360 < 1417
Total FP COMP 150 < 165
Total FN 2919 < 2968
P-value 6.53888633552e-08

^top




Performance plots


  1. Comparison of performance of Contrafold and MaxExpect. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Contrafold and MaxExpect).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Contrafold and MaxExpect).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Contrafold and MaxExpect. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Contrafold and MaxExpect).

^top





Performance of Contrafold - scored higher in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 2791
Total TN 6831018
Total FP 1693
Total FP CONTRA 183
Total FP INCONS 1360
Total FP COMP 150
Total FN 2919
Total Scores
MCC 0.561
Average MCC ± 95% Confidence Intervals 0.679 ± 0.051
Sensitivity 0.489
Positive Predictive Value 0.644
Nr of predictions 118

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KPV_A - 1.00 1.00 1.00 13 548 0 0 0 0 0
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - 0.55 0.44 0.70 7 693 3 0 3 0 9
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.75 0.74 0.78 14 1467 4 2 2 0 5
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.92 0.90 0.95 18 971 1 0 1 0 2
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.45 0.35 0.58 7 1528 5 2 3 0 13
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.84 0.79 0.89 31 6070 6 0 4 2 8
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.88 0.85 0.92 11 484 1 0 1 0 2
2RRC_A - 0.83 0.70 1.00 7 224 0 0 0 0 3
2WRQ_Y 0.57 0.59 0.56 10 2832 13 5 3 5 7
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WW9_E - 0.00 0.00 0.00 0 561 0 0 0 0 14
2WW9_D - 0.39 0.23 0.67 6 1944 3 2 1 0 20
2WWQ_V 0.78 0.68 0.90 19 2905 4 0 2 2 9
2XKV_B 0.51 0.50 0.53 10 4541 25 0 9 16 10
2XQD_Y 0.78 0.70 0.86 19 2828 3 0 3 0 8
2XXA_G 0.13 0.12 0.15 5 5117 29 1 28 0 37
2Y8W_B - 1.00 1.00 1.00 6 184 3 0 0 3 0
2Y9C_V - 0.66 0.63 0.71 15 2257 8 1 5 2 9
2YIE_X - 0.73 0.67 0.80 8 1368 3 1 1 1 4
2YIE_Z - 0.43 0.42 0.45 5 1529 6 3 3 0 7
3A2K_C 0.44 0.43 0.46 12 2900 14 2 12 0 16
3ADB_C - 0.92 0.84 1.00 32 4154 0 0 0 0 6
3AKZ_H 0.39 0.39 0.41 11 2674 17 4 12 1 17
3AM1_B - 0.68 0.63 0.73 22 3210 8 1 7 0 13
3AMU_B 0.65 0.59 0.73 16 2981 8 0 6 2 11
3GX2_A 0.77 0.63 0.96 25 4345 2 0 1 1 15
3IAB_R - 0.86 0.86 0.86 12 1021 4 1 1 2 2
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.80 0.67 0.96 22 4255 1 0 1 0 11
3IYQ_A 0.34 0.37 0.32 35 60615 80 21 55 4 59
3IZ4_A 0.52 0.45 0.61 60 70777 44 4 35 5 72
3IZF_C 0.68 0.61 0.77 33 6860 10 1 9 0 21
3J0L_1 - 0.73 0.58 0.92 11 1213 2 0 1 1 8
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.50 0.44 0.58 7 1116 5 2 3 0 9
3J0L_h - 0.75 0.65 0.88 28 6073 6 0 4 2 15
3J0L_7 - -0.01 0.00 0.00 0 1218 7 0 7 0 17
3J0L_g - 0.00 0.00 0.00 0 464 1 1 0 0 4
3J0L_2 - 0.25 0.24 0.28 8 6187 26 0 21 5 25
3J16_L 0.46 0.40 0.55 12 2753 10 1 9 0 18
3J20_2 0.58 0.51 0.66 321 1116277 172 10 157 5 312
3J20_1 0.96 0.91 1.00 21 2905 4 0 0 4 2
3J20_0 0.44 0.40 0.50 12 2826 13 0 12 1 18
3J2C_M - 0.43 0.34 0.53 71 106357 66 9 54 3 136
3J2C_O - 0.79 0.67 0.93 42 10251 4 0 3 1 21
3J2L_3 0.56 0.49 0.65 26 7835 16 0 14 2 27
3JYV_7 -0.01 0.00 0.00 0 2830 20 0 20 0 32
3JYX_3 0.28 0.30 0.27 8 6298 23 8 14 1 19
3JYX_4 0.20 0.21 0.20 7 12211 35 6 22 7 26
3KIY_A - 0.56 0.47 0.67 585 4053253 305 29 261 15 656
3KTW_C - 0.44 0.40 0.50 17 4526 18 2 15 1 26
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.81 0.72 0.92 44 9132 5 0 4 1 17
3NKB_B - 0.54 0.50 0.59 13 1994 9 0 9 0 13
3NMU_E - 0.71 0.50 1.00 3 558 3 0 0 3 3
3NPB_A 0.76 0.70 0.84 32 6983 8 1 5 2 14
3O58_2 0.78 0.76 0.81 29 7224 10 2 5 3 9
3O58_3 0.28 0.26 0.31 9 12374 20 3 17 0 26
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 1 0 0 1 1
3PDR_A 0.69 0.60 0.80 43 12826 13 0 11 2 29
3R4F_A - 0.79 0.76 0.83 19 2122 4 1 3 0 6
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.73 0.59 0.91 20 2189 2 1 1 0 14
3SD1_A 0.57 0.48 0.69 20 3887 9 2 7 0 22
3SIU_F - 0.74 0.64 0.88 7 370 1 0 1 0 4
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.59 0.46 0.77 17 3138 5 0 5 0 20
3UZL_B 0.70 0.54 0.91 20 3548 2 0 2 0 17
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
3W3S_B 0.89 0.80 1.00 32 4721 1 0 0 1 8
3ZEX_G - 0.28 0.24 0.33 18 16416 41 2 35 4 56
3ZEX_D 0.73 0.67 0.80 33 6980 8 1 7 0 16
3ZEX_B - 0.22 0.18 0.26 100 1071996 294 23 261 10 458
3ZEX_E - 0.00 0.00 0.00 0 21889 58 4 52 2 77
3ZEX_C 0.28 0.21 0.38 11 14167 22 3 15 4 41
3ZEX_F - 0.00 0.00 0.00 0 2619 9 2 7 0 12
3ZEX_H - 0.17 0.18 0.17 7 9003 35 6 29 0 31
4A1C_2 0.16 0.15 0.17 5 11751 33 5 20 8 28
4A1C_3 0.66 0.57 0.78 31 7100 9 1 8 0 23
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.44 0.36 0.56 15 4344 13 1 11 1 27
4ATO_G - 0.44 0.40 0.50 4 520 4 0 4 0 6
4ENB_A 0.73 0.58 0.92 11 1263 1 1 0 0 8
4ENC_A 0.73 0.58 0.92 11 1314 1 1 0 0 8
4FNJ_A - 0.79 0.63 1.00 10 585 0 0 0 0 6
4FRG_B 0.68 0.56 0.82 18 3464 4 1 3 0 14
4FRN_A 0.65 0.56 0.77 20 5125 6 1 5 0 16
4HXH_A - 1.00 1.00 1.00 6 319 0 0 0 0 0

^top



Performance of MaxExpect - scored lower in this pairwise comparison

1. Total counts & total scores for MaxExpect

Total Base Pair Counts
Total TP 2742
Total TN 6831037
Total FP 1738
Total FP CONTRA 156
Total FP INCONS 1417
Total FP COMP 165
Total FN 2968
Total Scores
MCC 0.552
Average MCC ± 95% Confidence Intervals 0.661 ± 0.055
Sensitivity 0.480
Positive Predictive Value 0.635
Nr of predictions 118

^top



2. Individual counts for MaxExpect [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KE6_A 0.92 0.89 0.94 17 1110 2 0 1 1 2
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - -0.02 0.00 0.00 0 694 9 0 9 0 16
2KUR_A 0.90 0.86 0.95 18 1109 1 0 1 0 3
2KUU_A 0.87 0.81 0.94 17 1110 2 0 1 1 4
2KUV_A 0.88 0.82 0.95 18 1109 1 0 1 0 4
2KUW_A 0.90 0.86 0.95 18 1109 1 0 1 0 3
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.83 0.70 1.00 7 344 0 0 0 0 3
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A -0.01 0.00 0.00 0 1525 15 2 13 0 20
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.81 0.74 0.88 29 6072 6 0 4 2 10
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.88 0.85 0.92 11 484 1 0 1 0 2
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.53 0.53 0.53 9 2833 13 5 3 5 8
2WW9_F - 0.69 0.70 0.70 7 290 3 0 3 0 3
2WW9_E - -0.01 0.00 0.00 0 556 5 0 5 0 14
2WW9_D - 0.14 0.12 0.19 3 1937 13 0 13 0 23
2WWQ_V 0.76 0.64 0.90 18 2906 3 0 2 1 10
2XKV_B 0.51 0.50 0.53 10 4541 24 0 9 15 10
2XQD_Y 0.88 0.78 1.00 21 2829 0 0 0 0 6
2XXA_G 0.25 0.24 0.27 10 5114 27 1 26 0 32
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.65 0.63 0.68 15 2256 8 1 6 1 9
2YIE_X - 0.36 0.33 0.40 4 1368 6 2 4 0 8
2YIE_Z - -0.01 0.00 0.00 0 1525 15 3 12 0 12
3A2K_C 0.42 0.39 0.46 11 2902 13 2 11 0 17
3ADB_C - 0.80 0.74 0.88 28 4154 4 0 4 0 10
3AKZ_H 0.84 0.75 0.95 21 2679 1 1 0 0 7
3AM1_B - 0.78 0.71 0.86 25 3211 4 0 4 0 10
3AMU_B 0.64 0.59 0.70 16 2980 9 0 7 2 11
3GX2_A 0.77 0.63 0.96 25 4345 2 0 1 1 15
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.74 0.67 0.81 22 4251 6 0 5 1 11
3IYQ_A 0.29 0.31 0.27 29 60617 84 19 61 4 65
3IZ4_A 0.54 0.46 0.64 61 70781 39 3 31 5 71
3IZF_C 0.71 0.61 0.83 33 6863 7 1 6 0 21
3J0L_1 - 0.76 0.63 0.92 12 1212 2 0 1 1 7
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.39 0.31 0.50 5 1118 5 2 3 0 11
3J0L_h - 0.81 0.65 1.00 28 6077 2 0 0 2 15
3J0L_7 - -0.01 0.00 0.00 0 1213 12 0 12 0 17
3J0L_g - -0.01 0.00 0.00 0 461 4 1 3 0 4
3J0L_2 - 0.25 0.24 0.26 8 6185 25 2 21 2 25
3J16_L 0.29 0.23 0.37 7 2756 12 0 12 0 23
3J20_2 0.49 0.44 0.56 276 1116272 221 10 207 4 357
3J20_1 0.73 0.70 0.76 16 2905 5 0 5 0 7
3J20_0 0.44 0.40 0.50 12 2826 13 0 12 1 18
3J2C_M - 0.38 0.31 0.47 64 106354 76 10 63 3 143
3J2C_O - 0.63 0.51 0.78 32 10255 10 0 9 1 31
3J2L_3 0.62 0.53 0.74 28 7837 12 0 10 2 25
3JYV_7 0.41 0.31 0.56 10 2832 8 0 8 0 22
3JYX_3 0.64 0.63 0.65 17 6302 19 1 8 10 10
3JYX_4 0.31 0.30 0.31 10 12214 34 4 18 12 23
3KIY_A - 0.54 0.45 0.65 553 4053271 314 23 281 10 688
3KTW_C - 0.54 0.47 0.65 20 4529 12 2 9 1 23
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.78 0.69 0.89 42 9133 6 0 5 1 19
3NKB_B - 0.59 0.54 0.67 14 1995 7 0 7 0 12
3NMU_E - -0.01 0.00 0.00 0 553 10 1 7 2 6
3NPB_A 0.71 0.61 0.82 28 6987 9 0 6 3 18
3O58_2 0.75 0.76 0.74 29 7221 11 3 7 1 9
3O58_3 0.35 0.34 0.35 12 12369 35 2 20 13 23
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.76 0.63 0.94 45 12832 5 0 3 2 27
3R4F_A - 0.96 0.92 1.00 23 2122 0 0 0 0 2
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.58 0.48 0.71 20 3888 8 1 7 0 22
3SIU_F - 0.85 0.73 1.00 8 370 0 0 0 0 3
3SN2_B 1.00 1.00 1.00 12 394 0 0 0 0 0
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.45 0.35 0.59 13 3138 9 1 8 0 24
3UZL_B 0.77 0.62 0.96 23 3546 1 0 1 0 14
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
3W3S_B 0.88 0.78 1.00 31 4722 1 0 0 1 9
3ZEX_G - 0.77 0.66 0.91 49 16417 11 0 5 6 25
3ZEX_D 0.69 0.61 0.79 30 6983 8 1 7 0 19
3ZEX_B - 0.30 0.26 0.35 147 1071957 289 29 247 13 411
3ZEX_E - 0.00 0.00 0.00 0 21893 54 2 50 2 77
3ZEX_C 0.41 0.27 0.64 14 14174 16 2 6 8 38
3ZEX_F - 0.00 0.00 0.00 0 2617 11 2 9 0 12
3ZEX_H - 0.19 0.18 0.19 7 9009 29 5 24 0 31
4A1C_2 0.14 0.15 0.14 5 11746 41 4 26 11 28
4A1C_3 0.69 0.57 0.84 31 7103 6 0 6 0 23
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.50 0.40 0.63 17 4344 11 2 8 1 25
4ATO_G - 0.38 0.30 0.50 3 522 3 0 3 0 7
4ENB_A 0.76 0.58 1.00 11 1264 0 0 0 0 8
4ENC_A 0.31 0.26 0.38 5 1313 8 1 7 0 14
4FNJ_A - 0.79 0.63 1.00 10 585 0 0 0 0 6
4FRG_B 0.30 0.22 0.41 7 3469 10 1 9 0 25
4FRN_A 0.45 0.36 0.57 13 5128 10 2 8 0 23
4HXH_A - 1.00 1.00 1.00 6 319 0 0 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.