CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Contrafold - scored higher in this pairwise comparison

  4. Performance of MaxExpect - scored lower in this pairwise comparison

  5. Compile and download dataset for Contrafold & MaxExpect [.zip] - may take several seconds...


Overview

Metric Contrafold MaxExpect
MCC 0.676 > 0.663
Average MCC ± 95% Confidence Intervals 0.769 ± 0.053 > 0.741 ± 0.058
Sensitivity 0.693 > 0.679
Positive Predictive Value 0.662 > 0.648
Total TP 2652 > 2599
Total TN 2549667 < 2549668
Total FP 1832 < 1881
Total FP CONTRA 408 > 399
Total FP INCONS 949 < 1010
Total FP COMP 475 > 472
Total FN 1177 < 1230
P-value 4.1569883686e-08

^top




Performance plots


  1. Comparison of performance of Contrafold and MaxExpect. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Contrafold and MaxExpect).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Contrafold and MaxExpect).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Contrafold and MaxExpect. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Contrafold and MaxExpect).

^top





Performance of Contrafold - scored higher in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 2652
Total TN 2549667
Total FP 1832
Total FP CONTRA 408
Total FP INCONS 949
Total FP COMP 475
Total FN 1177
Total Scores
MCC 0.676
Average MCC ± 95% Confidence Intervals 0.769 ± 0.053
Sensitivity 0.693
Positive Predictive Value 0.662
Nr of predictions 118

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KPV_A - 1.00 1.00 1.00 13 200 0 0 0 0 0
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.63 0.58 0.70 7 282 3 2 1 0 5
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 5 0 0 5 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 0.94 0.94 0.94 17 339 2 0 1 1 1
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.46 0.39 0.58 7 516 5 3 2 0 11
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.88 0.90 0.87 26 2410 11 0 4 7 3
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2RRC_A - 1.00 1.00 1.00 7 68 0 0 0 0 0
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WW9_E - 0.00 0.00 0.00 0 172 0 0 0 0 5
2WW9_D - 0.63 0.40 1.00 4 737 5 0 0 5 6
2WWQ_V 1.00 1.00 1.00 19 1185 4 0 0 4 0
2XKV_B 0.64 0.73 0.57 8 1821 27 0 6 21 3
2XQD_Y 0.85 0.86 0.86 18 1108 4 0 3 1 3
2XXA_G 0.10 0.11 0.12 4 2012 30 2 27 1 31
2Y8W_B - 1.00 1.00 1.00 6 82 3 0 0 3 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 1.00 1.00 1.00 7 537 4 0 0 4 0
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3ADB_C - 0.98 0.97 1.00 32 1787 0 0 0 0 1
3AKZ_H 0.44 0.50 0.40 10 1102 18 6 9 3 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.75 0.79 0.71 15 1136 9 0 6 3 4
3GX2_A 0.93 0.89 0.96 25 1423 2 1 0 1 3
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.86 0.79 0.96 22 1449 1 0 1 0 6
3IYQ_A 0.28 0.39 0.20 20 22341 95 44 35 16 31
3IZ4_A 0.57 0.58 0.57 55 25440 49 18 23 8 40
3IZF_C 0.89 0.91 0.86 32 2603 11 0 5 6 3
3J0L_1 - 0.83 0.77 0.91 10 473 3 0 1 2 3
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.53 0.55 0.55 6 400 6 3 2 1 5
3J0L_h - 0.87 0.88 0.88 28 2108 6 0 4 2 4
3J0L_7 - -0.02 0.00 0.00 0 512 7 1 6 0 10
3J0L_g - 0.00 0.00 0.00 0 176 1 0 0 1 2
3J0L_2 - 0.28 0.31 0.28 8 2221 26 2 19 5 18
3J16_L 0.55 0.57 0.55 12 1137 10 3 7 0 9
3J20_2 0.70 0.72 0.69 296 421939 197 27 106 64 116
3J20_1 1.00 1.00 1.00 20 1092 5 0 0 5 0
3J20_0 0.53 0.57 0.50 12 1195 13 3 9 1 9
3J2C_M - 0.57 0.63 0.52 60 39809 77 21 35 21 36
3J2C_O - 0.96 0.95 0.98 39 3947 7 0 1 6 2
3J2L_3 0.74 0.76 0.72 26 2984 16 1 9 6 8
3JYV_7 -0.02 0.00 0.00 0 1091 20 4 16 0 20
3JYX_3 0.33 0.47 0.24 7 2349 24 16 6 2 8
3JYX_4 0.39 0.58 0.27 7 4730 35 13 6 16 5
3KIY_A - 0.70 0.71 0.69 561 1485072 329 73 174 82 225
3KTW_C - 0.56 0.60 0.54 15 1752 20 2 11 7 10
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.98 0.98 0.98 43 3670 6 0 1 5 1
3NKB_B - 0.63 0.68 0.59 13 713 9 0 9 0 6
3NMU_E - 1.00 1.00 1.00 3 213 3 0 0 3 0
3NPB_A 0.86 0.84 0.89 31 2243 9 1 3 5 6
3O58_2 0.92 0.94 0.91 29 2722 10 0 3 7 2
3O58_3 0.32 0.36 0.29 8 4736 21 7 13 1 14
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.83 0.86 0.80 43 4786 13 5 6 2 7
3R4F_A - 0.81 0.82 0.82 18 884 5 2 2 1 4
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.87 0.83 0.91 20 844 2 2 0 0 4
3SD1_A 0.68 0.69 0.69 20 1504 9 5 4 0 9
3SIU_F - 0.87 0.88 0.88 7 137 1 0 1 0 1
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.81 0.77 0.85 17 1256 5 0 3 2 5
3UZL_B 0.90 0.88 0.93 14 1278 8 0 1 7 2
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W3S_B 0.98 0.97 1.00 32 1957 1 0 0 1 1
3ZEX_G - 0.35 0.38 0.34 17 6457 42 9 24 9 28
3ZEX_D 0.87 0.86 0.88 30 2762 11 0 4 7 5
3ZEX_B - 0.26 0.27 0.25 92 420991 302 58 211 33 254
3ZEX_E - -0.01 0.00 0.00 0 8251 58 20 33 5 34
3ZEX_C 0.35 0.34 0.37 10 5347 23 4 13 6 19
3ZEX_F - 0.00 0.00 0.00 0 910 9 0 4 5 4
3ZEX_H - 0.24 0.37 0.17 7 3583 35 22 13 0 12
4A1C_2 0.21 0.25 0.19 5 4489 33 9 13 11 15
4A1C_3 0.81 0.81 0.81 30 2726 10 0 7 3 7
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.53 0.52 0.56 15 1410 13 3 9 1 14
4ATO_G - 0.52 0.57 0.50 4 212 4 0 4 0 3
4ENB_A 0.85 0.73 1.00 11 461 1 0 0 1 4
4ENC_A 0.85 0.73 1.00 11 485 1 0 0 1 4
4FNJ_A - 0.91 0.83 1.00 10 240 0 0 0 0 2
4FRG_B 0.73 0.71 0.77 17 1180 5 3 2 0 7
4FRN_A 0.74 0.71 0.77 20 1822 6 1 5 0 8
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0

^top



Performance of MaxExpect - scored lower in this pairwise comparison

1. Total counts & total scores for MaxExpect

Total Base Pair Counts
Total TP 2599
Total TN 2549668
Total FP 1881
Total FP CONTRA 399
Total FP INCONS 1010
Total FP COMP 472
Total FN 1230
Total Scores
MCC 0.663
Average MCC ± 95% Confidence Intervals 0.741 ± 0.058
Sensitivity 0.679
Positive Predictive Value 0.648
Nr of predictions 118

^top



2. Individual counts for MaxExpect [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KE6_A 0.94 0.94 0.94 17 449 2 0 1 1 1
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - -0.04 0.00 0.00 0 283 9 1 8 0 12
2KUR_A 0.95 0.95 0.95 18 448 1 0 1 0 1
2KUU_A 0.94 0.94 0.94 17 429 2 0 1 1 1
2KUV_A 0.94 0.95 0.95 18 420 1 0 1 0 1
2KUW_A 0.94 0.94 0.94 17 452 2 0 1 1 1
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 513 15 3 12 0 18
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.84 0.83 0.86 24 2412 11 0 4 7 5
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WRQ_Y 0.94 0.89 1.00 8 1144 14 0 0 14 1
2WW9_F - 0.64 0.75 0.60 6 102 4 1 3 0 2
2WW9_E - -0.03 0.00 0.00 0 168 5 0 4 1 5
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WWQ_V 0.97 0.95 1.00 18 1186 3 0 0 3 1
2XKV_B 0.64 0.73 0.57 8 1821 26 0 6 20 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.24 0.26 0.25 9 2009 28 1 26 1 26
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3ADB_C - 0.86 0.85 0.88 28 1787 4 0 4 0 5
3AKZ_H 1.00 1.00 1.00 20 1107 2 0 0 2 0
3AM1_B - 0.86 0.86 0.86 25 1437 4 0 4 0 4
3AMU_B 0.73 0.79 0.68 15 1135 10 0 7 3 4
3GX2_A 0.93 0.89 0.96 25 1423 2 1 0 1 3
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.80 0.79 0.81 22 1445 6 1 4 1 6
3IYQ_A 0.24 0.33 0.17 17 22342 96 41 40 15 34
3IZ4_A 0.61 0.61 0.62 58 25442 42 14 22 6 37
3IZF_C 0.89 0.91 0.86 32 2603 8 0 5 3 3
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_a - 0.39 0.36 0.44 4 402 6 4 1 1 7
3J0L_h - 0.93 0.88 1.00 28 2112 2 0 0 2 4
3J0L_7 - -0.02 0.00 0.00 0 507 12 3 9 0 10
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_2 - 0.30 0.31 0.31 8 2224 25 2 16 7 18
3J16_L 0.34 0.33 0.37 7 1140 12 0 12 0 14
3J20_2 0.60 0.63 0.58 258 421926 239 28 156 55 154
3J20_1 0.75 0.75 0.75 15 1092 6 0 5 1 5
3J20_0 0.53 0.57 0.50 12 1195 13 2 10 1 9
3J2C_M - 0.48 0.54 0.43 52 39804 88 29 40 19 44
3J2C_O - 0.74 0.71 0.78 29 3950 13 0 8 5 12
3J2L_3 0.80 0.82 0.78 28 2984 12 1 7 4 6
3JYV_7 0.48 0.45 0.53 9 1094 9 3 5 1 11
3JYX_3 0.77 0.80 0.75 12 2362 24 0 4 20 3
3JYX_4 0.55 0.83 0.37 10 4729 34 14 3 17 2
3KIY_A - 0.67 0.68 0.66 531 1485073 336 81 195 60 255
3KTW_C - 0.72 0.72 0.72 18 1755 14 2 5 7 7
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.94 0.93 0.95 41 3671 7 0 2 5 3
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - -0.02 0.00 0.00 0 208 10 5 3 2 3
3NPB_A 0.77 0.73 0.82 27 2245 10 0 6 4 10
3O58_2 0.91 0.94 0.88 29 2721 11 0 4 7 2
3O58_3 0.40 0.50 0.33 11 4731 36 8 14 14 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.92 0.90 0.94 45 4792 5 1 2 2 5
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.70 0.69 0.71 20 1505 8 3 5 0 9
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 1.00 1.00 1.00 12 142 0 0 0 0 0
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.58 0.59 0.59 13 1254 9 2 7 0 9
3UZL_B 1.00 1.00 1.00 16 1277 8 0 0 8 0
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
3W3S_B 0.97 0.94 1.00 31 1958 1 0 0 1 2
3ZEX_G - 0.98 0.96 1.00 43 6464 17 0 0 17 2
3ZEX_D 0.82 0.77 0.87 27 2765 11 0 4 7 8
3ZEX_B - 0.37 0.39 0.35 134 420964 302 66 188 48 212
3ZEX_E - -0.01 0.00 0.00 0 8252 54 21 31 2 34
3ZEX_C 0.54 0.45 0.65 13 5354 17 2 5 10 16
3ZEX_F - -0.01 0.00 0.00 0 908 11 2 4 5 4
3ZEX_H - 0.26 0.37 0.19 7 3589 29 18 11 0 12
4A1C_2 0.19 0.25 0.16 5 4484 41 10 17 14 15
4A1C_3 0.83 0.81 0.86 30 2728 7 0 5 2 7
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ATO_G - 0.45 0.43 0.50 3 214 3 3 0 0 4
4ENB_A 0.85 0.73 1.00 11 461 0 0 0 0 4
4ENC_A 0.36 0.33 0.42 5 484 8 0 7 1 10
4FNJ_A - 0.91 0.83 1.00 10 240 0 0 0 0 2
4FRG_B 0.29 0.25 0.35 6 1185 11 4 7 0 18
4FRN_A 0.51 0.46 0.57 13 1825 10 5 5 0 15
4HXH_A - 1.00 1.00 1.00 6 89 0 0 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.