CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of UNAFold - scored higher in this pairwise comparison

  4. Performance of PknotsRG - scored lower in this pairwise comparison

  5. Compile and download dataset for UNAFold & PknotsRG [.zip] - may take several seconds...


Overview

Metric UNAFold PknotsRG
MCC 0.610 > 0.592
Average MCC ± 95% Confidence Intervals 0.801 ± 0.050 > 0.788 ± 0.051
Sensitivity 0.641 > 0.630
Positive Predictive Value 0.581 > 0.557
Total TP 2623 > 2577
Total TN 5047728 > 5047612
Total FP 2495 < 2643
Total FP CONTRA 565 < 598
Total FP INCONS 1323 < 1452
Total FP COMP 607 > 593
Total FN 1468 < 1514
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of UNAFold and PknotsRG. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for UNAFold and PknotsRG).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for UNAFold and PknotsRG).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for UNAFold and PknotsRG. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for UNAFold and PknotsRG).

^top





Performance of UNAFold - scored higher in this pairwise comparison

1. Total counts & total scores for UNAFold

Total Base Pair Counts
Total TP 2623
Total TN 5047728
Total FP 2495
Total FP CONTRA 565
Total FP INCONS 1323
Total FP COMP 607
Total FN 1468
Total Scores
MCC 0.610
Average MCC ± 95% Confidence Intervals 0.801 ± 0.050
Sensitivity 0.641
Positive Predictive Value 0.581
Nr of predictions 121

^top



2. Individual counts for UNAFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.95 0.91 1.00 10 176 0 0 0 0 1
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 1.00 1.00 1.00 18 449 1 0 0 1 0
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - -0.03 0.00 0.00 0 229 7 1 6 0 8
2KGP_A - 1.00 1.00 1.00 9 112 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.91 0.87 0.95 20 2003 9 1 0 8 3
2KU0_A - 0.96 0.92 1.00 11 281 0 0 0 0 1
2KUR_A 1.00 1.00 1.00 19 448 0 0 0 0 0
2KUU_A 1.00 1.00 1.00 18 429 1 0 0 1 0
2KUV_A 1.00 1.00 1.00 19 420 0 0 0 0 0
2KUW_A 1.00 1.00 1.00 18 452 1 0 0 1 0
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A -0.03 0.00 0.00 0 513 15 3 12 0 18
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 0.95 0.91 1.00 10 152 0 0 0 0 1
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 1.00 1.00 1.00 29 2411 6 0 0 6 0
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.84 0.71 1.00 5 111 0 0 0 0 2
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WDL_A - 0.49 0.50 0.47 396 1444479 488 87 360 41 392
2WRQ_Y 1.00 1.00 1.00 9 1143 14 0 0 14 0
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_D - 0.13 0.10 0.20 1 736 9 1 3 5 9
2WW9_F - 0.68 0.75 0.67 6 103 4 1 2 1 2
2WWQ_V 1.00 1.00 1.00 19 1185 3 0 0 3 0
2XKV_B 0.58 0.64 0.54 7 1822 23 0 6 17 4
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.75 0.78 0.74 14 887 8 0 5 3 4
2YIE_X - 0.53 0.57 0.50 4 536 6 1 3 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 2 2 0 0 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.86 0.85 0.88 28 1787 4 0 4 0 5
3AKZ_H 0.48 0.50 0.48 10 1106 13 4 7 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - 0.91 0.83 1.00 5 135 2 0 0 2 1
3G4S_9 0.78 0.77 0.80 20 2711 13 2 3 8 6
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.81 0.79 0.85 22 1423 5 2 2 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 0.97 0.94 1.00 15 594 2 0 0 2 1
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.59 0.60 0.60 3 224 5 0 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.80 0.79 0.81 22 1445 6 1 4 1 6
3IYQ_A 0.23 0.33 0.17 17 22339 96 46 38 12 34
3IZ4_A 0.58 0.60 0.57 57 25436 53 12 31 10 38
3IZF_C 0.89 0.91 0.86 32 2603 9 0 5 4 3
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.30 0.31 0.31 8 2224 25 2 16 7 18
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_7 - 0.66 0.70 0.64 7 508 5 2 2 1 3
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_1 - 0.75 0.69 0.82 9 473 4 0 2 2 4
3J0L_a - 0.55 0.64 0.50 7 397 8 5 2 1 4
3J16_L 0.31 0.33 0.30 7 1136 16 3 13 0 14
3JYV_7 0.24 0.25 0.25 5 1091 16 7 8 1 15
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3JYX_5 - 0.36 0.45 0.29 216 1968262 763 245 279 239 265
3JYX_4 0.61 0.83 0.45 10 4734 31 10 2 19 2
3KIY_A - 0.50 0.52 0.48 410 1485034 486 79 357 50 376
3KTW_C - 1.00 1.00 1.00 25 1755 7 0 0 7 0
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 1.00 1.00 1.00 44 3670 4 0 0 4 0
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.85 0.78 0.94 29 2247 6 0 2 4 8
3O58_2 0.87 0.87 0.87 27 2723 12 0 4 8 4
3O58_3 0.42 0.50 0.35 11 4733 34 5 15 14 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.93 0.92 0.94 46 4791 5 1 2 2 4
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.77 0.76 0.79 22 1505 6 4 2 0 7
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_3 0.88 0.86 0.89 32 2727 7 0 4 3 5
4A1C_2 0.19 0.25 0.15 5 4483 42 11 17 14 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 0.85 0.73 1.00 11 461 2 0 0 2 4
4ENC_A 0.37 0.33 0.45 5 485 7 0 6 1 10

^top



Performance of PknotsRG - scored lower in this pairwise comparison

1. Total counts & total scores for PknotsRG

Total Base Pair Counts
Total TP 2577
Total TN 5047612
Total FP 2643
Total FP CONTRA 598
Total FP INCONS 1452
Total FP COMP 593
Total FN 1514
Total Scores
MCC 0.592
Average MCC ± 95% Confidence Intervals 0.788 ± 0.051
Sensitivity 0.630
Positive Predictive Value 0.557
Nr of predictions 121

^top



2. Individual counts for PknotsRG [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 175 0 0 0 0 0
2K63_A - 1.00 1.00 1.00 9 159 0 0 0 0 0
2K66_A - 1.00 1.00 1.00 9 80 0 0 0 0 0
2KD8_A - 1.00 1.00 1.00 9 99 0 0 0 0 0
2KDQ_B 1.00 1.00 1.00 10 164 0 0 0 0 0
2KE6_A 0.88 0.89 0.89 16 449 3 0 2 1 2
2KEZ_A - 1.00 1.00 1.00 8 94 0 0 0 0 0
2KFC_A - 0.77 0.88 0.70 7 226 3 2 1 0 1
2KGP_A - 0.69 0.67 0.75 6 113 2 0 2 0 3
2KMJ_A - 1.00 1.00 1.00 11 157 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 201 0 0 0 0 1
2KRL_A - 0.85 0.87 0.83 20 2000 12 4 0 8 3
2KU0_A - 0.96 0.92 1.00 11 281 0 0 0 0 1
2KUR_A 0.89 0.89 0.89 17 448 2 0 2 0 2
2KUU_A 0.88 0.89 0.89 16 429 3 0 2 1 2
2KUV_A 0.89 0.89 0.89 17 420 2 0 2 0 2
2KUW_A 0.94 0.94 0.94 17 452 2 0 1 1 1
2KX8_A 1.00 1.00 1.00 16 355 0 0 0 0 0
2KXM_A - 0.93 0.88 1.00 7 142 0 0 0 0 1
2KZL_A - 1.00 1.00 1.00 13 507 1 0 0 1 0
2L1F_B 1.00 1.00 1.00 24 767 0 0 0 0 0
2L1F_A 1.00 1.00 1.00 23 740 0 0 0 0 0
2L2K_A - 1.00 1.00 1.00 17 332 0 0 0 0 0
2L3C_B - 1.00 1.00 1.00 14 225 0 0 0 0 0
2L3E_A - 1.00 1.00 1.00 12 228 1 0 0 1 0
2L3J_B 1.00 1.00 1.00 30 955 0 0 0 0 0
2L5Z_A - 1.00 1.00 1.00 9 102 0 0 0 0 0
2L94_A 1.00 1.00 1.00 18 339 1 0 0 1 0
2LA5_A - 0.77 0.71 0.83 5 269 3 0 1 2 2
2LBS_A - 1.00 1.00 1.00 14 193 0 0 0 0 0
2LC8_A 0.66 0.61 0.73 11 513 4 2 2 0 7
2LDL_A - 1.00 1.00 1.00 9 131 1 0 0 1 0
2LDT_A - 1.00 1.00 1.00 11 151 0 0 0 0 0
2LHP_A - 1.00 1.00 1.00 15 246 0 0 0 0 0
2LI4_A - 1.00 1.00 1.00 14 175 0 0 0 0 0
2LJJ_A - 1.00 1.00 1.00 7 123 3 0 0 3 0
2LK3_A - 1.00 1.00 1.00 9 91 0 0 0 0 0
2LKR_A - 0.81 0.86 0.76 25 2407 10 6 2 2 4
2LQZ_A - 1.00 1.00 1.00 8 124 2 0 0 2 0
2LWK_A - 0.95 0.91 1.00 10 197 1 0 0 1 1
2RP0_A - 0.87 1.00 0.78 7 107 2 2 0 0 0
2RPT_A - 1.00 1.00 1.00 7 65 0 0 0 0 0
2RRC_A - 0.83 0.71 1.00 5 70 0 0 0 0 2
2WDL_A - 0.47 0.49 0.45 389 1444457 514 87 389 38 399
2WRQ_Y 0.94 0.89 1.00 8 1144 14 0 0 14 1
2WW9_E - 0.89 0.80 1.00 4 168 0 0 0 0 1
2WW9_D - 0.08 0.10 0.08 1 729 15 2 9 4 9
2WW9_F - 0.64 0.75 0.60 6 102 4 1 3 0 2
2WWQ_V 1.00 1.00 1.00 19 1185 5 0 0 5 0
2XKV_B 0.64 0.73 0.57 8 1821 24 0 6 18 3
2XQD_Y 0.98 0.95 1.00 20 1109 1 0 0 1 1
2XXA_G 0.35 0.34 0.38 12 2013 21 1 19 1 23
2Y8W_B - 1.00 1.00 1.00 6 82 1 0 0 1 0
2Y9C_V - 0.79 0.83 0.75 15 886 8 0 5 3 3
2YIE_X - 0.43 0.57 0.33 4 532 10 2 6 2 3
2YIE_Z - -0.02 0.00 0.00 0 587 15 6 9 0 8
2ZY6_A - 0.66 0.78 0.58 7 264 5 3 2 0 2
2ZZM_B 0.25 0.27 0.25 4 1342 20 3 9 8 11
2ZZN_D 0.93 0.95 0.91 21 961 2 2 0 0 1
3A2K_C 0.47 0.50 0.46 11 1084 13 3 10 0 11
3A3A_A 0.97 0.93 1.00 28 1472 0 0 0 0 2
3ADB_C - 0.35 0.36 0.36 12 1786 21 1 20 0 21
3AKZ_H 0.46 0.50 0.43 10 1104 15 6 7 2 10
3AM1_B - 0.74 0.76 0.73 22 1436 8 3 5 0 7
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3DW4_A - -0.03 0.00 0.00 0 137 3 0 3 0 6
3G4S_9 0.83 0.85 0.81 22 2709 12 2 3 7 4
3GCA_A - 0.84 0.71 1.00 5 153 0 0 0 0 2
3GX2_A 0.80 0.79 0.81 22 1422 6 2 3 1 6
3HAY_E - 1.00 1.00 1.00 14 910 7 0 0 7 0
3HJW_D - 1.00 1.00 1.00 16 593 2 0 0 2 0
3IAB_R - 1.00 1.00 1.00 12 373 0 0 0 0 0
3ID5_D - 0.54 0.60 0.50 3 223 6 1 2 3 2
3IVN_B 0.91 0.83 1.00 19 884 0 0 0 0 4
3IWN_A 0.21 0.21 0.24 6 1447 19 1 18 0 22
3IYQ_A 0.24 0.33 0.17 17 22340 94 45 38 11 34
3IZ4_A 0.56 0.60 0.52 57 25426 57 25 28 4 38
3IZF_C 0.75 0.77 0.73 27 2603 15 0 10 5 8
3J0L_8 - 1.00 1.00 1.00 7 69 0 0 0 0 0
3J0L_2 - 0.29 0.31 0.29 8 2222 27 2 18 7 18
3J0L_g - -0.01 0.00 0.00 0 174 4 0 2 2 2
3J0L_7 - 0.38 0.50 0.31 5 503 12 3 8 1 5
3J0L_h - 0.87 0.81 0.93 26 2112 4 1 1 2 6
3J0L_1 - 0.83 0.77 0.91 10 473 4 0 1 3 3
3J0L_a - 0.18 0.18 0.22 2 402 8 3 4 1 9
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3JYV_7 -0.02 0.00 0.00 0 1089 22 4 18 0 20
3JYX_3 0.77 0.80 0.75 12 2362 27 0 4 23 3
3JYX_5 - 0.37 0.46 0.29 220 1968248 778 241 293 244 261
3JYX_4 0.61 0.83 0.45 10 4734 33 10 2 21 2
3KIY_A - 0.47 0.50 0.45 393 1485010 525 91 386 48 393
3KTW_C - 1.00 1.00 1.00 25 1755 7 0 0 7 0
3LA5_A 0.91 0.84 1.00 21 933 0 0 0 0 4
3NDB_M - 0.93 0.93 0.93 41 3670 9 0 3 6 3
3NKB_B - 0.69 0.74 0.67 14 714 7 0 7 0 5
3NMU_E - 1.00 1.00 1.00 3 213 5 0 0 5 0
3NPB_A 0.93 0.86 1.00 32 2246 4 0 0 4 5
3O58_2 0.72 0.74 0.70 23 2721 19 0 10 9 8
3O58_3 0.41 0.50 0.34 11 4732 35 6 15 14 11
3OVB_D - 1.00 1.00 1.00 11 215 1 0 0 1 0
3OVS_D - 1.00 1.00 1.00 12 204 0 0 0 0 0
3P22_G - 1.00 1.00 1.00 11 301 0 0 0 0 0
3PDR_A 0.90 0.90 0.90 45 4790 7 2 3 2 5
3R4F_A - 1.00 1.00 1.00 22 884 1 0 0 1 0
3R9X_C - 1.00 1.00 1.00 8 221 2 0 0 2 0
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.73 0.72 0.75 21 1505 7 4 3 0 8
3SIU_F - 1.00 1.00 1.00 8 137 0 0 0 0 0
3SN2_B 0.95 0.92 1.00 11 143 0 0 0 0 1
3TRZ_Z - 1.00 1.00 1.00 5 87 1 0 0 1 0
3TS0_U - 1.00 1.00 1.00 6 112 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 103 0 0 0 0 0
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3UZL_B 0.49 0.50 0.50 8 1277 15 0 8 7 8
3VJR_D - 1.00 1.00 1.00 12 239 0 0 0 0 0
4A1C_3 0.85 0.84 0.86 31 2727 8 0 5 3 6
4A1C_2 0.18 0.25 0.14 5 4480 43 11 20 12 15
4A4U_A - 1.00 1.00 1.00 9 96 0 0 0 0 0
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.