CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Sfold - scored higher in this pairwise comparison

  4. Performance of RNAshapes - scored lower in this pairwise comparison

  5. Compile and download dataset for Sfold & RNAshapes [.zip] - may take several seconds...


Overview

Metric Sfold RNAshapes
MCC 0.669 > 0.650
Average MCC ± 95% Confidence Intervals 0.716 ± 0.050 > 0.704 ± 0.052
Sensitivity 0.581 > 0.580
Positive Predictive Value 0.775 > 0.733
Total TP 1631 > 1630
Total TN 399982 > 399863
Total FP 581 < 719
Total FP CONTRA 65 < 81
Total FP INCONS 409 < 513
Total FP COMP 107 < 125
Total FN 1177 < 1178
P-value 3.56938820447e-08

^top




Performance plots


^top





Performance of Sfold - scored higher in this pairwise comparison

1. Total counts & total scores for Sfold

Total Base Pair Counts
Total TP 1631
Total TN 399982
Total FP 581
Total FP CONTRA 65
Total FP INCONS 409
Total FP COMP 107
Total FN 1177
Total Scores
MCC 0.669
Average MCC ± 95% Confidence Intervals 0.716 ± 0.050
Sensitivity 0.581
Positive Predictive Value 0.775
Nr of predictions 118

^top



2. Individual counts for Sfold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 0 0 0 0 0
2K63_A - 0.95 0.90 1.00 9 397 0 0 0 0 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.92 0.89 0.94 17 1110 2 0 1 1 2
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - -0.02 0.00 0.00 0 623 7 0 7 0 20
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - 0.83 0.69 1.00 11 692 0 0 0 0 5
2KUR_A 0.90 0.86 0.95 18 1109 1 0 1 0 3
2KUU_A 0.87 0.81 0.94 17 1110 2 0 1 1 4
2KUV_A 0.88 0.82 0.95 18 1109 1 0 1 0 4
2KUW_A 0.90 0.86 0.95 18 1109 1 0 1 0 3
2KX8_A 0.94 0.89 1.00 16 845 0 0 0 0 2
2KXM_A - 0.89 0.80 1.00 8 343 0 0 0 0 2
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_A 0.98 0.96 1.00 23 2057 0 0 0 0 1
2L1F_B 0.98 0.96 1.00 24 2121 0 0 0 0 1
2L2K_A - 0.94 0.89 1.00 17 844 0 0 0 0 2
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.94 0.88 1.00 30 2455 0 0 0 0 4
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.45 0.35 0.58 7 1528 5 0 5 0 13
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.89 0.79 1.00 31 6074 1 0 0 1 8
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.88 0.85 0.92 11 484 1 0 1 0 2
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.83 0.70 1.00 7 224 0 0 0 0 3
2WRQ_Y 0.53 0.53 0.53 9 2833 13 5 3 5 8
2WW9_D - 0.29 0.12 0.75 3 1949 1 0 1 0 23
2WW9_E - 0.53 0.29 1.00 4 557 0 0 0 0 10
2WW9_F - 0.69 0.70 0.70 7 290 3 0 3 0 3
2WWQ_V 0.80 0.68 0.95 19 2906 2 0 1 1 9
2XKV_B 0.51 0.50 0.53 10 4541 23 0 9 14 10
2XQD_Y 0.77 0.67 0.90 18 2830 2 0 2 0 9
2XXA_G 0.36 0.31 0.43 13 5121 17 1 16 0 29
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.65 0.63 0.68 15 2256 8 1 6 1 9
2YIE_X - 0.58 0.33 1.00 4 1374 0 0 0 0 8
2YIE_Z - -0.01 0.00 0.00 0 1525 15 3 12 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 1 4 0 5
2ZZM_B 0.12 0.09 0.17 3 3468 15 0 15 0 29
2ZZN_D 0.82 0.78 0.88 21 2461 3 0 3 0 6
3A2K_C 0.42 0.39 0.46 11 2902 13 2 11 0 17
3A3A_A 0.87 0.76 1.00 28 3627 0 0 0 0 9
3ADB_C - 0.62 0.55 0.70 21 4156 9 0 9 0 17
3AKZ_H 0.44 0.39 0.50 11 2679 11 2 9 0 17
3AM1_B - 0.69 0.63 0.76 22 3211 7 1 6 0 13
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3DW4_A - -0.02 0.00 0.00 0 345 6 0 6 0 10
3G4S_9 0.51 0.39 0.69 22 7349 10 1 9 0 35
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.72 0.55 0.96 22 4348 2 0 1 1 18
3HAY_E - 0.91 0.83 1.00 15 2263 6 0 0 6 3
3HJW_D - 0.94 0.89 1.00 16 1637 2 0 0 2 2
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.49 0.33 0.75 3 492 1 0 1 0 6
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.63 0.58 0.70 19 4251 9 0 8 1 14
3IYQ_A 0.33 0.34 0.32 32 60627 71 19 48 4 62
3IZ4_A 0.52 0.42 0.65 56 70790 31 3 27 1 76
3IZF_C 0.71 0.61 0.83 33 6863 7 0 7 0 21
3J0L_h - 0.75 0.60 0.93 26 6077 4 0 2 2 17
3J0L_g - -0.01 0.00 0.00 0 460 5 1 4 0 4
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.51 0.50 0.53 8 1113 7 3 4 0 8
3J0L_2 - 0.36 0.33 0.39 11 6188 19 2 15 2 22
3J0L_1 - 0.76 0.63 0.92 12 1212 3 0 1 2 7
3J0L_7 - -0.01 0.00 0.00 0 1222 3 0 3 0 17
3J16_L 0.29 0.23 0.37 7 2756 12 0 12 0 23
3JYV_7 -0.01 0.00 0.00 0 2834 16 0 16 0 32
3JYX_3 0.62 0.63 0.61 17 6300 19 1 10 8 10
3JYX_4 0.35 0.30 0.42 10 12222 23 3 11 9 23
3KTW_C - 0.74 0.63 0.87 27 4529 5 2 2 1 16
3LA5_A 0.78 0.62 1.00 21 2464 0 0 0 0 13
3NDB_M - 0.81 0.69 0.95 42 9136 3 0 2 1 19
3NKB_B - 0.64 0.54 0.78 14 1998 4 0 4 0 12
3NMU_E - 0.61 0.50 0.75 3 557 4 0 1 3 3
3NPB_A 0.77 0.65 0.91 30 6988 6 1 2 3 16
3O58_2 0.74 0.74 0.74 28 7222 11 3 7 1 10
3O58_3 0.43 0.34 0.55 12 12381 17 0 10 7 23
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.69 0.56 0.85 40 12833 9 0 7 2 32
3R4F_A - 0.96 0.92 1.00 23 2122 0 0 0 0 2
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.65 0.50 0.84 21 3891 4 1 3 0 21
3SIU_F - 0.85 0.73 1.00 8 370 0 0 0 0 3
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 2 0 0 2 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3UZL_B 0.51 0.32 0.80 12 3555 3 2 1 0 25
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_2 0.16 0.15 0.17 5 11751 34 2 23 9 28
4A1C_3 0.69 0.57 0.84 31 7103 6 1 5 0 23
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.50 0.40 0.63 17 4344 11 2 8 1 25
4ENB_A 0.76 0.58 1.00 11 1264 0 0 0 0 8
4ENC_A 0.51 0.26 1.00 5 1321 0 0 0 0 14

^top



Performance of RNAshapes - scored lower in this pairwise comparison

1. Total counts & total scores for RNAshapes

Total Base Pair Counts
Total TP 1630
Total TN 399863
Total FP 719
Total FP CONTRA 81
Total FP INCONS 513
Total FP COMP 125
Total FN 1178
Total Scores
MCC 0.650
Average MCC ± 95% Confidence Intervals 0.704 ± 0.052
Sensitivity 0.580
Positive Predictive Value 0.733
Nr of predictions 118

^top



2. Individual counts for RNAshapes [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2K5Z_A - 1.00 1.00 1.00 11 395 0 0 0 0 0
2K63_A - 0.95 0.90 1.00 9 397 0 0 0 0 1
2K66_A - 0.95 0.90 1.00 9 222 0 0 0 0 1
2KD8_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
2KDQ_B 0.95 0.91 1.00 10 396 0 0 0 0 1
2KE6_A 0.97 0.95 1.00 18 1110 1 0 0 1 1
2KEZ_A - 0.89 0.80 1.00 8 268 0 0 0 0 2
2KFC_A - -0.02 0.00 0.00 0 623 7 0 7 0 20
2KGP_A - 1.00 1.00 1.00 9 291 1 0 0 1 0
2KMJ_A - 1.00 1.00 1.00 11 367 0 0 0 0 0
2KPV_A - 0.96 0.92 1.00 12 549 0 0 0 0 1
2KRL_A - 0.76 0.60 0.96 24 5126 5 0 1 4 16
2KU0_A - 0.83 0.69 1.00 11 692 0 0 0 0 5
2KUR_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KUU_A 0.92 0.86 1.00 18 1110 1 0 0 1 3
2KUV_A 0.93 0.86 1.00 19 1109 0 0 0 0 3
2KUW_A 0.95 0.90 1.00 19 1109 0 0 0 0 2
2KX8_A 0.91 0.83 1.00 15 846 0 0 0 0 3
2KXM_A - 0.83 0.70 1.00 7 344 0 0 0 0 3
2KZL_A - 0.86 0.74 1.00 14 1471 0 0 0 0 5
2L1F_A 0.87 0.83 0.91 20 2058 2 0 2 0 4
2L1F_B 0.96 0.92 1.00 23 2122 0 0 0 0 2
2L2K_A - 0.92 0.84 1.00 16 845 0 0 0 0 3
2L3C_B - 0.91 0.82 1.00 14 547 0 0 0 0 3
2L3E_A - 0.96 0.92 1.00 12 583 1 0 0 1 1
2L3J_B 0.92 0.85 1.00 29 2456 0 0 0 0 5
2L5Z_A - 0.86 0.75 1.00 9 316 0 0 0 0 3
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2LA5_A - 0.39 0.26 0.63 5 622 3 1 2 0 14
2LBS_A - 1.00 1.00 1.00 14 482 0 0 0 0 0
2LC8_A 0.61 0.55 0.69 11 1524 5 0 5 0 9
2LDL_A - 0.90 0.82 1.00 9 342 1 0 0 1 2
2LDT_A - 0.85 0.73 1.00 11 454 0 0 0 0 4
2LHP_A - 0.97 0.94 1.00 15 651 0 0 0 0 1
2LI4_A - 0.93 0.88 1.00 14 482 0 0 0 0 2
2LJJ_A - 1.00 1.00 1.00 8 343 2 0 0 2 0
2LK3_A - 0.95 0.90 1.00 9 267 0 0 0 0 1
2LKR_A - 0.93 0.87 1.00 34 6071 1 0 0 1 5
2LQZ_A - 0.85 0.82 0.90 9 341 1 1 0 0 2
2LWK_A - 0.83 0.77 0.91 10 485 1 0 1 0 3
2RP0_A - 0.64 0.42 1.00 5 346 0 0 0 0 7
2RPT_A - 1.00 1.00 1.00 7 183 0 0 0 0 0
2RRC_A - 0.70 0.50 1.00 5 226 0 0 0 0 5
2WRQ_Y 0.59 0.59 0.59 10 2833 12 4 3 5 7
2WW9_D - 0.16 0.12 0.25 3 1941 9 4 5 0 23
2WW9_E - 0.53 0.29 1.00 4 557 0 0 0 0 10
2WW9_F - 0.59 0.60 0.60 6 290 4 0 4 0 4
2WWQ_V 0.76 0.68 0.86 19 2904 5 0 3 2 9
2XKV_B 0.51 0.50 0.53 10 4541 22 0 9 13 10
2XQD_Y 0.64 0.56 0.75 15 2830 5 0 5 0 12
2XXA_G 0.93 0.86 1.00 36 5115 0 0 0 0 6
2Y8W_B - 1.00 1.00 1.00 6 184 1 0 0 1 0
2Y9C_V - 0.65 0.63 0.68 15 2256 8 1 6 1 9
2YIE_X - 0.25 0.25 0.27 3 1367 11 1 7 3 9
2YIE_Z - -0.01 0.00 0.00 0 1525 15 3 12 0 12
2ZY6_A - 0.58 0.58 0.58 7 691 5 0 5 0 5
2ZZM_B 0.10 0.09 0.12 3 3460 23 0 23 0 29
2ZZN_D 0.69 0.63 0.77 17 2463 5 1 4 0 10
3A2K_C 0.42 0.39 0.46 11 2902 13 2 11 0 17
3A3A_A 0.76 0.65 0.89 24 3628 3 0 3 0 13
3ADB_C - 0.67 0.61 0.74 23 4155 8 0 8 0 15
3AKZ_H 0.68 0.61 0.77 17 2679 6 1 4 1 11
3AM1_B - 0.64 0.60 0.70 21 3210 9 1 8 0 14
3AMU_B 0.70 0.59 0.84 16 2984 5 0 3 2 11
3DW4_A - -0.02 0.00 0.00 0 348 3 0 3 0 10
3G4S_9 0.41 0.32 0.53 18 7347 16 1 15 0 39
3GCA_A - 0.54 0.29 1.00 5 523 0 0 0 0 12
3GX2_A 0.76 0.63 0.93 25 4344 3 0 2 1 15
3HAY_E - 0.91 0.83 1.00 15 2263 4 0 0 4 3
3HJW_D - 0.94 0.89 1.00 16 1637 0 0 0 0 2
3IAB_R - 0.92 0.86 1.00 12 1023 0 0 0 0 2
3ID5_D - 0.44 0.33 0.60 3 491 5 0 2 3 6
3IVN_B 0.78 0.61 1.00 19 2327 0 0 0 0 12
3IWN_A 0.76 0.67 0.88 22 4253 3 0 3 0 11
3IYQ_A 0.26 0.28 0.24 26 60619 83 21 60 2 68
3IZ4_A 0.52 0.45 0.60 60 70776 45 6 34 5 72
3IZF_C 0.59 0.52 0.68 28 6862 13 1 12 0 26
3J0L_h - 0.75 0.60 0.93 26 6077 4 0 2 2 17
3J0L_g - -0.01 0.00 0.00 0 461 4 1 3 0 4
3J0L_8 - 0.93 0.88 1.00 7 183 0 0 0 0 1
3J0L_a - 0.23 0.19 0.30 3 1118 7 1 6 0 13
3J0L_2 - 0.38 0.36 0.40 12 6186 20 3 15 2 21
3J0L_1 - 0.76 0.63 0.92 12 1212 2 0 1 1 7
3J0L_7 - -0.01 0.00 0.00 0 1213 12 0 12 0 17
3J16_L 0.50 0.40 0.63 12 2756 7 0 7 0 18
3JYV_7 -0.01 0.00 0.00 0 2830 20 0 20 0 32
3JYX_3 0.63 0.63 0.63 17 6301 21 1 9 11 10
3JYX_4 0.31 0.30 0.31 10 12214 35 5 17 13 23
3KTW_C - 0.39 0.35 0.45 15 4527 19 1 17 1 28
3LA5_A 0.76 0.59 1.00 20 2465 0 0 0 0 14
3NDB_M - 0.82 0.72 0.94 44 9133 4 0 3 1 17
3NKB_B - 0.61 0.54 0.70 14 1996 6 0 6 0 12
3NMU_E - 0.54 0.50 0.60 3 556 5 0 2 3 3
3NPB_A 0.77 0.63 0.94 29 6990 4 1 1 2 17
3O58_2 0.60 0.61 0.59 23 7221 18 4 12 2 15
3O58_3 0.34 0.34 0.34 12 12368 37 2 21 14 23
3OVB_D - 1.00 1.00 1.00 11 584 1 0 0 1 0
3OVS_D - 0.96 0.92 1.00 12 549 0 0 0 0 1
3P22_G - 0.96 0.92 1.00 11 730 0 0 0 0 1
3PDR_A 0.67 0.56 0.80 40 12830 12 0 10 2 32
3R4F_A - 0.96 0.92 1.00 23 2122 0 0 0 0 2
3R9X_C - 0.84 0.80 0.89 8 586 2 0 1 1 2
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.64 0.52 0.79 22 3888 6 1 5 0 20
3SIU_F - 0.85 0.73 1.00 8 370 0 0 0 0 3
3SN2_B 0.96 0.92 1.00 11 395 0 0 0 0 1
3TRZ_Z - 0.91 0.83 1.00 5 205 1 0 0 1 1
3TS0_U - 1.00 1.00 1.00 6 247 1 0 0 1 0
3TS2_V - 1.00 1.00 1.00 5 271 0 0 0 0 0
3U4M_B - 0.38 0.32 0.46 12 3134 14 0 14 0 25
3UZL_B 0.39 0.32 0.48 12 3545 13 2 11 0 25
3VJR_D - 0.96 0.92 1.00 12 618 0 0 0 0 1
4A1C_2 0.14 0.15 0.14 5 11746 41 5 25 11 28
4A1C_3 0.68 0.57 0.82 31 7102 7 1 6 0 23
4A4U_A - 1.00 1.00 1.00 9 222 0 0 0 0 0
4AOB_A 0.50 0.40 0.63 17 4344 11 2 8 1 25
4ENB_A 0.70 0.58 0.85 11 1262 2 1 1 0 8
4ENC_A 0.32 0.26 0.42 5 1314 7 1 6 0 14

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.