CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of RNAshapes - scored higher in this pairwise comparison

  4. Performance of RNAsubopt - scored lower in this pairwise comparison

  5. Compile and download dataset for RNAshapes & RNAsubopt [.zip] - may take several seconds...


Overview

Metric RNAshapes RNAsubopt
MCC 0.650 > 0.635
Average MCC ± 95% Confidence Intervals 0.704 ± 0.052 > 0.690 ± 0.054
Sensitivity 0.580 > 0.574
Positive Predictive Value 0.733 > 0.708
Total TP 1630 > 1612
Total TN 399863 > 399810
Total FP 719 < 776
Total FP CONTRA 81 < 92
Total FP INCONS 513 < 573
Total FP COMP 125 > 111
Total FN 1178 < 1196
P-value 3.56938820447e-08

^top




Performance plots


^top





Performance of RNAshapes - scored higher in this pairwise comparison

1. Total counts & total scores for RNAshapes

Total Base Pair Counts
Total TP 1630
Total TN 399863
Total FP 719
Total FP CONTRA 81
Total FP INCONS 513
Total FP COMP 125
Total FN 1178
Total Scores
MCC 0.650
Average MCC ± 95% Confidence Intervals 0.704 ± 0.052
Sensitivity 0.580
Positive Predictive Value 0.733
Nr of predictions 118

^top



2. Individual counts for RNAshapes [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 0 0 0 0 0
2K63_A - 0.95 0.90 1.00 9 397 0 0 0 0 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - -0.02 0.00 0.00 0 623 7 0 7 0 20
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - 0.83 0.69 1.00 11 692 0 0 0 0 5
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.91 0.83 1.00 15 846 0 0 0 0 3
2KXM_A - 0.83 0.70 1.00 7 344 0 0 0 0 3
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_A 0.87 0.83 0.91 20 2058 2 0 2 0 4
2L1F_B 0.96 0.92 1.00 23 2122 0 0 0 0 2
2L2K_A - 0.92 0.84 1.00 16 845 0 0 0 0 3
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.92 0.85 1.00 29 2456 0 0 0 0 5
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.61 0.55 0.69 11 1524 5 0 5 0 9
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.93 0.87 1.00 34 6071 1 0 0 1 5
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.59 0.59 0.59 10 2833 12 4 3 5 7
2WW9_D - 0.16 0.12 0.25 3 1941 9 4 5 0 23
2WW9_E - 0.53 0.29 1.00 4 557 0 0 0 0 10
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WWQ_V 0.76 0.68 0.86 19 2904 5 0 3 2 9
2XKV_B 0.51 0.50 0.53 10 4541 22 0 9 13 10
2XQD_Y 0.64 0.56 0.75 15 2830 5 0 5 0 12
2XXA_G 0.93 0.86 1.00 36 5115 0 0 0 0 6
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.65 0.63 0.68 15 2256 8 1 6 1 9
2YIE_X - 0.25 0.25 0.27 3 1367 11 1 7 3 9
2YIE_Z - -0.01 0.00 0.00 0 1525 15 3 12 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 0 5 0 5
2ZZM_B 0.10 0.09 0.12 3 3460 23 0 23 0 29
2ZZN_D 0.69 0.63 0.77 17 2463 5 1 4 0 10
3A2K_C 0.42 0.39 0.46 11 2902 13 2 11 0 17
3A3A_A 0.76 0.65 0.89 24 3628 3 0 3 0 13
3ADB_C - 0.67 0.61 0.74 23 4155 8 0 8 0 15
3AKZ_H 0.68 0.61 0.77 17 2679 6 1 4 1 11
3AM1_B - 0.64 0.60 0.70 21 3210 9 1 8 0 14
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3DW4_A - -0.02 0.00 0.00 0 348 3 0 3 0 10
3G4S_9 0.41 0.32 0.53 18 7347 16 1 15 0 39
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.76 0.63 0.93 25 4344 3 0 2 1 15
3HAY_E - 0.91 0.83 1.00 15 2263 4 0 0 4 3
3HJW_D - 0.94 0.89 1.00 16 1637 0 0 0 0 2
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.44 0.33 0.60 3 491 5 0 2 3 6
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.76 0.67 0.88 22 4253 3 0 3 0 11
3IYQ_A 0.26 0.28 0.24 26 60619 83 21 60 2 68
3IZ4_A 0.52 0.45 0.60 60 70776 45 6 34 5 72
3IZF_C 0.59 0.52 0.68 28 6862 13 1 12 0 26
3J0L_h - 0.75 0.60 0.93 26 6077 4 0 2 2 17
3J0L_g - -0.01 0.00 0.00 0 461 4 1 3 0 4
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.23 0.19 0.30 3 1118 7 1 6 0 13
3J0L_2 - 0.38 0.36 0.40 12 6186 20 3 15 2 21
3J0L_1 - 0.76 0.63 0.92 12 1212 2 0 1 1 7
3J0L_7 - -0.01 0.00 0.00 0 1213 12 0 12 0 17
3J16_L 0.50 0.40 0.63 12 2756 7 0 7 0 18
3JYV_7 -0.01 0.00 0.00 0 2830 20 0 20 0 32
3JYX_3 0.63 0.63 0.63 17 6301 21 1 9 11 10
3JYX_4 0.31 0.30 0.31 10 12214 35 5 17 13 23
3KTW_C - 0.39 0.35 0.45 15 4527 19 1 17 1 28
3LA5_A 0.76 0.59 1.00 20 2465 0 0 0 0 14
3NDB_M - 0.82 0.72 0.94 44 9133 4 0 3 1 17
3NKB_B - 0.61 0.54 0.70 14 1996 6 0 6 0 12
3NMU_E - 0.54 0.50 0.60 3 556 5 0 2 3 3
3NPB_A 0.77 0.63 0.94 29 6990 4 1 1 2 17
3O58_2 0.60 0.61 0.59 23 7221 18 4 12 2 15
3O58_3 0.34 0.34 0.34 12 12368 37 2 21 14 23
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.67 0.56 0.80 40 12830 12 0 10 2 32
3R4F_A - 0.96 0.92 1.00 23 2122 0 0 0 0 2
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.64 0.52 0.79 22 3888 6 1 5 0 20
3SIU_F - 0.85 0.73 1.00 8 370 0 0 0 0 3
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3UZL_B 0.39 0.32 0.48 12 3545 13 2 11 0 25
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_2 0.14 0.15 0.14 5 11746 41 5 25 11 28
4A1C_3 0.68 0.57 0.82 31 7102 7 1 6 0 23
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.50 0.40 0.63 17 4344 11 2 8 1 25
4ENB_A 0.70 0.58 0.85 11 1262 2 1 1 0 8
4ENC_A 0.32 0.26 0.42 5 1314 7 1 6 0 14

^top



Performance of RNAsubopt - scored lower in this pairwise comparison

1. Total counts & total scores for RNAsubopt

Total Base Pair Counts
Total TP 1612
Total TN 399810
Total FP 776
Total FP CONTRA 92
Total FP INCONS 573
Total FP COMP 111
Total FN 1196
Total Scores
MCC 0.635
Average MCC ± 95% Confidence Intervals 0.690 ± 0.054
Sensitivity 0.574
Positive Predictive Value 0.708
Nr of predictions 118

^top



2. Individual counts for RNAsubopt [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 0.95 0.91 1.00 10 396 0 0 0 0 1
2K63_A - 0.89 0.80 1.00 8 398 0 0 0 0 2
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 0.88 0.78 1.00 7 224 0 0 0 0 2
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.86 0.84 0.89 16 1110 3 0 2 1 3
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - -0.02 0.00 0.00 0 623 7 0 7 0 20
2KGP_A - 1.00 1.00 1.00 9 291 0 0 0 0 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.52 0.43 0.65 17 5125 12 3 6 3 23
2KU0_A - 0.83 0.69 1.00 11 692 0 0 0 0 5
2KUR_A 0.85 0.81 0.89 17 1109 2 0 2 0 4
2KUU_A 0.82 0.76 0.89 16 1110 3 0 2 1 5
2KUV_A 0.83 0.77 0.89 17 1109 2 0 2 0 5
2KUW_A 0.90 0.86 0.95 18 1109 1 0 1 0 3
2KX8_A 0.91 0.83 1.00 15 846 0 0 0 0 3
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_A 0.93 0.88 1.00 21 2059 0 0 0 0 3
2L1F_B 0.94 0.88 1.00 22 2123 0 0 0 0 3
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.87 0.76 1.00 13 548 0 0 0 0 4
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A -0.01 0.00 0.00 0 1525 15 2 13 0 20
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.81 0.77 0.86 30 6070 9 0 5 4 9
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.83 0.70 1.00 7 224 0 0 0 0 3
2WRQ_Y 0.57 0.59 0.56 10 2832 12 5 3 4 7
2WW9_D - 0.15 0.12 0.21 3 1939 11 4 7 0 23
2WW9_E - -0.01 0.00 0.00 0 556 5 0 5 0 14
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WWQ_V 0.76 0.68 0.86 19 2904 5 0 3 2 9
2XKV_B 0.51 0.50 0.53 10 4541 23 0 9 14 10
2XQD_Y 0.75 0.67 0.86 18 2829 3 0 3 0 9
2XXA_G 0.47 0.43 0.53 18 5117 16 1 15 0 24
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.66 0.63 0.71 15 2257 8 1 5 2 9
2YIE_X - 0.82 0.67 1.00 8 1370 3 0 0 3 4
2YIE_Z - -0.01 0.00 0.00 0 1525 15 3 12 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 1 4 0 5
2ZZM_B 0.23 0.22 0.26 7 3459 20 2 18 0 25
2ZZN_D 0.48 0.44 0.52 12 2462 11 1 10 0 15
3A2K_C 0.42 0.39 0.46 11 2902 13 2 11 0 17
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3ADB_C - 0.92 0.84 1.00 32 4154 0 0 0 0 6
3AKZ_H 0.39 0.36 0.43 10 2678 13 2 11 0 18
3AM1_B - 0.91 0.83 1.00 29 3211 0 0 0 0 6
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3DW4_A - -0.02 0.00 0.00 0 345 6 0 6 0 10
3G4S_9 0.28 0.23 0.35 13 7344 24 1 23 0 44
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.44 0.38 0.54 15 4343 14 1 12 1 25
3HAY_E - 0.91 0.83 1.00 15 2263 3 0 0 3 3
3HJW_D - 0.91 0.83 1.00 15 1638 2 0 0 2 3
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.40 0.33 0.50 3 490 5 0 3 2 6
3IVN_B 0.76 0.58 1.00 18 2328 0 0 0 0 13
3IWN_A 0.63 0.58 0.70 19 4251 9 0 8 1 14
3IYQ_A 0.33 0.36 0.30 34 60614 82 17 61 4 60
3IZ4_A 0.51 0.47 0.56 62 70766 49 8 40 1 70
3IZF_C 0.70 0.61 0.80 33 6862 8 1 7 0 21
3J0L_h - 0.75 0.60 0.93 26 6077 4 0 2 2 17
3J0L_g - -0.01 0.00 0.00 0 460 5 1 4 0 4
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.35 0.31 0.42 5 1116 7 1 6 0 11
3J0L_2 - 0.25 0.24 0.28 8 6187 25 0 21 4 25
3J0L_1 - 0.76 0.63 0.92 12 1212 2 0 1 1 7
3J0L_7 - -0.01 0.00 0.00 0 1213 12 0 12 0 17
3J16_L 0.41 0.37 0.48 11 2752 12 1 11 0 19
3JYV_7 -0.01 0.00 0.00 0 2829 21 0 21 0 32
3JYX_3 0.63 0.63 0.63 17 6301 21 1 9 11 10
3JYX_4 0.19 0.21 0.17 7 12204 38 11 24 3 26
3KTW_C - 0.75 0.65 0.88 28 4528 5 2 2 1 15
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.77 0.69 0.86 42 9131 8 0 7 1 19
3NKB_B - 0.57 0.50 0.65 13 1996 7 0 7 0 13
3NMU_E - 0.54 0.50 0.60 3 556 5 0 2 3 3
3NPB_A 0.75 0.65 0.86 30 6986 7 1 4 2 16
3O58_2 0.72 0.74 0.70 28 7220 13 4 8 1 10
3O58_3 0.34 0.34 0.34 12 12368 34 2 21 11 23
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.75 0.63 0.90 45 12830 7 1 4 2 27
3R4F_A - 0.96 0.92 1.00 23 2122 0 0 0 0 2
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.61 0.50 0.75 21 3888 7 1 6 0 21
3SIU_F - 0.85 0.73 1.00 8 370 0 0 0 0 3
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3UZL_B 0.48 0.38 0.61 14 3547 9 0 9 0 23
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_2 0.14 0.15 0.13 5 11742 43 5 29 9 28
4A1C_3 0.70 0.59 0.82 32 7101 7 1 6 0 22
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.52 0.43 0.64 18 4343 11 2 8 1 24
4ENB_A 0.70 0.58 0.85 11 1262 2 1 1 0 8
4ENC_A 0.32 0.26 0.42 5 1314 7 0 7 0 14

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.