CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Contrafold - scored higher in this pairwise comparison

  4. Performance of PknotsRG - scored lower in this pairwise comparison

  5. Compile and download dataset for Contrafold & PknotsRG [.zip] - may take several seconds...


Overview

Metric Contrafold PknotsRG
MCC 0.699 > 0.592
Average MCC ± 95% Confidence Intervals 0.803 ± 0.049 > 0.788 ± 0.051
Sensitivity 0.722 > 0.630
Positive Predictive Value 0.677 > 0.557
Total TP 2953 > 2577
Total TN 5047877 > 5047612
Total FP 2109 < 2643
Total FP CONTRA 499 < 598
Total FP INCONS 910 < 1452
Total FP COMP 700 > 593
Total FN 1138 < 1514
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Contrafold and PknotsRG. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Contrafold and PknotsRG).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Contrafold and PknotsRG).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Contrafold and PknotsRG. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Contrafold and PknotsRG).

^top





Performance of Contrafold - scored higher in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 2953
Total TN 5047877
Total FP 2109
Total FP CONTRA 499
Total FP INCONS 910
Total FP COMP 700
Total FN 1138
Total Scores
MCC 0.699
Average MCC ± 95% Confidence Intervals 0.803 ± 0.049
Sensitivity 0.722
Positive Predictive Value 0.677
Nr of predictions 121

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 2 0 0 2 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.71 0.63 0.83 5 230 1 1 0 0 3
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 1.00 1.00 1.00 13 200 0 0 0 0 0
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.63 0.58 0.70 7 282 3 2 1 0 5
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 1.00 1.00 1.00 8 141 0 0 0 0 0
2KZL_A - 1.00 1.00 1.00 13 507 5 0 0 5 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 0.94 0.94 0.94 17 339 2 0 1 1 1
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.46 0.39 0.58 7 516 5 3 2 0 11
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.88 0.90 0.87 26 2410 11 0 4 7 3
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 1.00 1.00 1.00 11 196 1 0 0 1 0
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 1.00 1.00 1.00 7 68 0 0 0 0 0
2WDL_A - 0.69 0.71 0.68 556 1444510 320 75 181 64 232
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_E - 0.00 0.00 0.00 0 172 0 0 0 0 5
2WW9_D - 0.63 0.40 1.00 4 737 5 0 0 5 6
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WWQ_V 1.00 1.00 1.00 19 1185 4 0 0 4 0
2XKV_B 0.64 0.73 0.57 8 1821 27 0 6 21 3
2XQD_Y 0.85 0.86 0.86 18 1108 4 0 3 1 3
2XXA_G 0.10 0.11 0.12 4 2012 30 2 27 1 31
2Y8W_B - 1.00 1.00 1.00 6 82 3 0 0 3 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 1.00 1.00 1.00 7 537 4 0 0 4 0
2YIE_Z - 0.53 0.63 0.45 5 591 6 4 2 0 3
2ZY6_A - 0.69 0.78 0.64 7 265 5 2 2 1 2
2ZZM_B 0.80 0.80 0.80 12 1343 13 0 3 10 3
2ZZN_D 0.93 0.95 0.91 21 961 3 2 0 1 1
3A2K_C 0.49 0.55 0.46 12 1082 14 3 11 0 10
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.98 0.97 1.00 32 1787 0 0 0 0 1
3AKZ_H 0.44 0.50 0.40 10 1102 18 6 9 3 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.75 0.79 0.71 15 1136 9 0 6 3 4
3DW4_A - -0.03 0.00 0.00 0 136 6 1 3 2 6
3G4S_9 0.45 0.50 0.42 13 2705 24 6 12 6 13
3GCA_A - 0.76 0.71 0.83 5 152 1 1 0 0 2
3GX2_A 0.93 0.89 0.96 25 1423 2 1 0 1 3
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 4 0 0 4 0
3ID5_D - 0.66 0.60 0.75 3 225 6 0 1 5 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.86 0.79 0.96 22 1449 1 0 1 0 6
3IYQ_A 0.28 0.39 0.20 20 22341 95 44 35 16 31
3IZ4_A 0.57 0.58 0.57 55 25440 49 18 23 8 40
3IZF_C 0.89 0.91 0.86 32 2603 11 0 5 6 3
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.28 0.31 0.28 8 2221 26 2 19 5 18
3J0L_g - 0.00 0.00 0.00 0 176 1 0 0 1 2
3J0L_7 - -0.02 0.00 0.00 0 512 7 1 6 0 10
3J0L_h - 0.87 0.88 0.88 28 2108 6 0 4 2 4
3J0L_1 - 0.83 0.77 0.91 10 473 3 0 1 2 3
3J0L_a - 0.53 0.55 0.55 6 400 6 3 2 1 5
3J16_L 0.55 0.57 0.55 12 1137 10 3 7 0 9
3JYV_7 -0.02 0.00 0.00 0 1091 20 4 16 0 20
3JYX_3 0.33 0.47 0.24 7 2349 24 16 6 2 8
3JYX_5 - 0.43 0.50 0.37 240 1968358 697 172 232 293 241
3JYX_4 0.39 0.58 0.27 7 4730 35 13 6 16 5
3KIY_A - 0.70 0.71 0.69 561 1485072 329 73 174 82 225
3KTW_C - 0.56 0.60 0.54 15 1752 20 2 11 7 10
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.98 0.98 0.98 43 3670 6 0 1 5 1
3NKB_B - 0.63 0.68 0.59 13 713 9 0 9 0 6
3NMU_E - 1.00 1.00 1.00 3 213 3 0 0 3 0
3NPB_A 0.86 0.84 0.89 31 2243 9 1 3 5 6
3O58_2 0.92 0.94 0.91 29 2722 10 0 3 7 2
3O58_3 0.32 0.36 0.29 8 4736 21 7 13 1 14
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 1 0 0 1 0
3PDR_A 0.83 0.86 0.80 43 4786 13 5 6 2 7
3R4F_A - 0.81 0.82 0.82 18 884 5 2 2 1 4
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.87 0.83 0.91 20 844 2 2 0 0 4
3SD1_A 0.68 0.69 0.69 20 1504 9 5 4 0 9
3SIU_F - 0.87 0.88 0.88 7 137 1 0 1 0 1
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.81 0.77 0.85 17 1256 5 0 3 2 5
3UZL_B 0.90 0.88 0.93 14 1278 8 0 1 7 2
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_3 0.81 0.81 0.81 30 2726 10 0 7 3 7
4A1C_2 0.21 0.25 0.19 5 4489 33 9 13 11 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.53 0.52 0.56 15 1410 13 3 9 1 14
4ENB_A 0.85 0.73 1.00 11 461 1 0 0 1 4
4ENC_A 0.85 0.73 1.00 11 485 1 0 0 1 4

^top



Performance of PknotsRG - scored lower in this pairwise comparison

1. Total counts & total scores for PknotsRG

Total Base Pair Counts
Total TP 2577
Total TN 5047612
Total FP 2643
Total FP CONTRA 598
Total FP INCONS 1452
Total FP COMP 593
Total FN 1514
Total Scores
MCC 0.592
Average MCC ± 95% Confidence Intervals 0.788 ± 0.051
Sensitivity 0.630
Positive Predictive Value 0.557
Nr of predictions 121

^top



2. Individual counts for PknotsRG [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 0.88 0.89 0.89 16 449 3 0 2 1 2
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.77 0.88 0.70 7 226 3 2 1 0 1
2KGP_A - 0.69 0.67 0.75 6 113 2 0 2 0 3
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.85 0.87 0.83 20 2000 12 4 0 8 3
2KU0_A - 0.96 0.92 1.00 11 281 0 0 0 0 1
2KUR_A 0.89 0.89 0.89 17 448 2 0 2 0 2
2KUU_A 0.88 0.89 0.89 16 429 3 0 2 1 2
2KUV_A 0.89 0.89 0.89 17 420 2 0 2 0 2
2KUW_A 0.94 0.94 0.94 17 452 2 0 1 1 1
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.66 0.61 0.73 11 513 4 2 2 0 7
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.81 0.86 0.76 25 2407 10 6 2 2 4
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.87 1.00 0.78 7 107 2 2 0 0 0
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WDL_A - 0.47 0.49 0.45 389 1444457 514 87 389 38 399
2WRQ_Y 0.94 0.89 1.00 8 1144 14 0 0 14 1
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WW9_F - 0.64 0.75 0.60 6 102 4 1 3 0 2
2WWQ_V 1.00 1.00 1.00 19 1185 5 0 0 5 0
2XKV_B 0.64 0.73 0.57 8 1821 24 0 6 18 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.43 0.57 0.33 4 532 10 2 6 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 2 2 0 0 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.35 0.36 0.36 12 1786 21 1 20 0 21
3AKZ_H 0.46 0.50 0.43 10 1104 15 6 7 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - -0.03 0.00 0.00 0 137 3 0 3 0 6
3G4S_9 0.83 0.85 0.81 22 2709 12 2 3 7 4
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.80 0.79 0.81 22 1422 6 2 3 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.54 0.60 0.50 3 223 6 1 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.21 0.21 0.24 6 1447 19 1 18 0 22
3IYQ_A 0.24 0.33 0.17 17 22340 94 45 38 11 34
3IZ4_A 0.56 0.60 0.52 57 25426 57 25 28 4 38
3IZF_C 0.75 0.77 0.73 27 2603 15 0 10 5 8
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.29 0.31 0.29 8 2222 27 2 18 7 18
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_7 - 0.38 0.50 0.31 5 503 12 3 8 1 5
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_a - 0.18 0.18 0.22 2 402 8 3 4 1 9
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3JYV_7 -0.02 0.00 0.00 0 1089 22 4 18 0 20
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3JYX_5 - 0.37 0.46 0.29 220 1968248 778 241 293 244 261
3JYX_4 0.61 0.83 0.45 10 4734 33 10 2 21 2
3KIY_A - 0.47 0.50 0.45 393 1485010 525 91 386 48 393
3KTW_C - 1.00 1.00 1.00 25 1755 7 0 0 7 0
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.93 0.93 0.93 41 3670 9 0 3 6 3
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.93 0.86 1.00 32 2246 4 0 0 4 5
3O58_2 0.72 0.74 0.70 23 2721 19 0 10 9 8
3O58_3 0.41 0.50 0.34 11 4732 35 6 15 14 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.90 0.90 0.90 45 4790 7 2 3 2 5
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.73 0.72 0.75 21 1505 7 4 3 0 8
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_3 0.85 0.84 0.86 31 2727 8 0 5 3 6
4A1C_2 0.18 0.25 0.14 5 4480 43 11 20 12 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.