CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Cylofold - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for Cylofold & NanoFolder [.zip] - may take several seconds...


Overview

Metric Cylofold NanoFolder
MCC 0.567 > 0.349
Average MCC ± 95% Confidence Intervals 0.593 ± 0.059 > 0.395 ± 0.056
Sensitivity 0.536 > 0.400
Positive Predictive Value 0.606 > 0.313
Total TP 1298 > 969
Total TN 328911 > 327957
Total FP 902 < 2231
Total FP CONTRA 120 < 413
Total FP INCONS 723 < 1713
Total FP COMP 59 < 105
Total FN 1125 < 1454
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Cylofold and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Cylofold and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Cylofold and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Cylofold and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Cylofold and NanoFolder).

^top





Performance of Cylofold - scored higher in this pairwise comparison

1. Total counts & total scores for Cylofold

Total Base Pair Counts
Total TP 1298
Total TN 328911
Total FP 902
Total FP CONTRA 120
Total FP INCONS 723
Total FP COMP 59
Total FN 1125
Total Scores
MCC 0.567
Average MCC ± 95% Confidence Intervals 0.593 ± 0.059
Sensitivity 0.536
Positive Predictive Value 0.606
Nr of predictions 109

^top



2. Individual counts for Cylofold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01535 0.40 0.38 0.42 14 7107 21 2 17 2 23
CRW_01539 0.47 0.46 0.49 17 7105 21 1 17 3 20
CRW_01563 0.40 0.32 0.50 12 6879 13 0 12 1 25
CRW_01583 0.18 0.16 0.20 6 7110 24 1 23 0 31
CRW_01590 0.08 0.08 0.09 3 7105 32 4 28 0 34
CRW_01593 0.26 0.24 0.29 9 7109 22 5 17 0 28
CRW_01603 0.50 0.46 0.55 17 7109 14 2 12 0 20
CRW_01617 0.25 0.24 0.27 9 7107 24 4 20 0 28
PDB_00011 - 0.91 0.83 1.00 10 266 0 0 0 0 2
PDB_00017 - 0.77 0.60 1.00 9 694 0 0 0 0 6
PDB_00036 - -0.01 0.00 0.00 0 3464 22 1 21 0 27
PDB_00070 0.59 0.58 0.61 14 2827 9 1 8 0 10
PDB_00113 - 0.94 0.89 1.00 8 223 0 0 0 0 1
PDB_00119 - 0.95 0.90 1.00 9 244 0 0 0 0 1
PDB_00136 - 0.85 0.77 0.95 20 2394 1 0 1 0 6
PDB_00137 - 0.87 0.77 1.00 10 396 0 0 0 0 3
PDB_00160 - 0.94 0.89 1.00 8 268 0 0 0 0 1
PDB_00178 - 0.79 0.64 1.00 7 521 0 0 0 0 4
PDB_00221 - 0.88 0.78 1.00 7 246 0 0 0 0 2
PDB_00312 - 1.00 1.00 1.00 9 426 0 0 0 0 0
PDB_00326 - 0.80 0.64 1.00 18 3723 0 0 0 0 10
PDB_00341 - 0.38 0.31 0.50 5 693 5 0 5 0 11
PDB_00349 - 0.24 0.29 0.22 4 928 14 4 10 0 10
PDB_00430 0.73 0.72 0.75 18 3216 6 3 3 0 7
PDB_00527 - -0.01 0.00 0.00 0 296 4 1 3 0 5
PDB_00565 - 0.53 0.46 0.61 23 8090 16 1 14 1 27
PDB_00570 - 0.28 0.23 0.36 5 2264 9 1 8 0 17
PDB_00571 0.87 0.80 0.95 20 3300 1 1 0 0 5
PDB_00573 - 0.63 0.53 0.75 18 6646 10 0 6 4 16
PDB_00584 - 0.84 0.71 1.00 12 1473 0 0 0 0 5
PDB_00698 - 0.94 0.89 1.00 17 844 0 0 0 0 2
PDB_00725 - 0.89 0.80 1.00 12 583 0 0 0 0 3
PDB_00734 - 0.83 0.70 1.00 7 224 0 0 0 0 3
PDB_00823 - -0.04 0.00 0.00 0 183 7 0 7 0 9
PDB_00828 0.88 0.78 1.00 21 2464 0 0 0 0 6
PDB_00851 - 0.45 0.39 0.53 16 4723 14 0 14 0 25
PDB_00890 - 0.71 0.71 0.71 10 689 4 2 2 0 4
PDB_00923 - 1.00 1.00 1.00 6 184 0 0 0 0 0
PDB_01009 0.90 0.90 0.90 19 2464 5 0 2 3 2
PDB_01051 0.79 0.69 0.90 9 893 2 0 1 1 4
PDB_01074 - 0.61 0.60 0.63 15 2751 10 2 7 1 10
PDB_01130 - 0.23 0.20 0.29 4 1756 10 2 8 0 16
PDB_01199 - -0.01 0.00 0.00 0 2130 15 2 13 0 24
RFA_00416 1.00 1.00 1.00 15 1470 3 0 0 3 0
RFA_00434 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00436 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00442 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00446 0.97 0.93 1.00 14 1471 2 0 0 2 1
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00674 -0.01 0.00 0.00 0 1117 11 0 11 0 14
RFA_00675 -0.01 0.00 0.00 0 981 9 1 8 0 14
RFA_00677 0.69 0.64 0.75 9 978 5 0 3 2 5
RFA_00684 0.84 0.71 1.00 10 980 1 0 0 1 4
RFA_00703 0.30 0.36 0.25 5 4258 19 4 11 4 9
RFA_00716 -0.01 0.00 0.00 0 939 7 0 7 0 14
RFA_00717 0.35 0.29 0.44 4 894 5 0 5 0 10
RFA_00730 0.51 0.42 0.63 5 895 4 1 2 1 7
RFA_00763 0.43 0.42 0.45 5 892 7 0 6 1 7
RFA_00765 0.51 0.42 0.63 5 895 4 0 3 1 7
RFA_00770 0.88 0.78 1.00 14 2002 3 0 0 3 4
RFA_00773 0.97 1.00 0.95 18 1934 4 1 0 3 0
SPR_00030 - 0.31 0.24 0.42 5 2473 7 0 7 0 16
SPR_00066 0.93 1.00 0.88 21 2826 3 3 0 0 0
SPR_00228 0.89 1.00 0.80 20 3803 6 5 0 1 0
SPR_00314 0.71 0.76 0.67 16 2902 8 6 2 0 5
SPR_00331 0.83 0.85 0.81 17 2754 4 2 2 0 3
SPR_00333 0.86 0.86 0.86 18 2754 3 0 3 0 3
SPR_00353 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00398 - -0.01 0.00 0.00 0 2197 14 2 12 0 13
SPR_00452 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00456 0.68 0.76 0.62 16 2824 11 4 6 1 5
SPR_00462 0.72 0.74 0.70 14 2395 7 3 3 1 5
SPR_00476 0.76 0.76 0.76 16 2754 5 0 5 0 5
SPR_00496 0.70 0.76 0.64 16 2750 9 4 5 0 5
SPR_00497 0.76 0.76 0.76 16 2754 5 0 5 0 5
SPR_00502 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_00509 0.71 0.75 0.68 15 2904 7 2 5 0 5
SPR_00512 0.71 0.75 0.68 15 2904 7 2 5 0 5
SPR_00542 0.84 0.86 0.82 18 3548 7 2 2 3 3
SPR_00557 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00832 - 0.29 0.27 0.33 4 1641 8 3 5 0 11
SPR_00868 0.95 0.95 0.95 20 2905 2 0 1 1 1
SPR_00932 0.78 0.76 0.80 16 2830 5 1 3 1 5
SPR_00957 0.83 0.81 0.85 17 2983 4 0 3 1 4
SPR_01040 0.17 0.19 0.16 4 2750 21 0 21 0 17
SPR_01143 0.41 0.48 0.37 10 2529 17 8 9 0 11
SRP_00005 - 0.23 0.25 0.21 6 3977 22 4 18 0 18
SRP_00038 0.36 0.36 0.36 12 5017 21 0 21 0 21
SRP_00046 0.38 0.38 0.40 12 4723 19 1 17 1 20
SRP_00138 - 0.92 0.85 1.00 23 3982 3 0 0 3 4
SRP_00170 0.36 0.36 0.38 9 4071 16 2 13 1 16
SRP_00193 - 0.28 0.28 0.29 10 6869 25 1 23 1 26
SRP_00194 0.56 0.48 0.67 10 3225 5 0 5 0 11
SRP_00200 0.61 0.56 0.67 20 6873 10 1 9 0 16
SRP_00216 0.75 0.67 0.86 18 3382 3 1 2 0 9
SRP_00229 0.37 0.37 0.37 10 5226 17 5 12 0 17
SRP_00233 0.36 0.33 0.39 11 5123 17 0 17 0 22
SRP_00243 0.37 0.29 0.47 9 5132 10 1 9 0 22
SRP_00244 0.55 0.49 0.63 19 6873 11 0 11 0 20
SRP_00266 0.51 0.45 0.59 13 4634 9 0 9 0 16
SRP_00269 0.44 0.42 0.47 14 5020 16 3 13 0 19
SRP_00273 0.56 0.48 0.66 19 6757 10 0 10 0 21
SRP_00285 0.93 0.87 1.00 26 3715 0 0 0 0 4
SRP_00303 0.27 0.25 0.30 8 6643 19 1 18 0 24
SRP_00333 0.54 0.50 0.59 17 5122 12 2 10 0 17
SRP_00338 0.61 0.56 0.67 20 5430 10 0 10 0 16
SRP_00342 0.38 0.36 0.41 9 3983 13 2 11 0 16
SRP_00358 0.48 0.46 0.52 16 6297 15 0 15 0 19
SRP_00383 0.11 0.13 0.12 3 3214 23 2 21 0 21

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 969
Total TN 327957
Total FP 2231
Total FP CONTRA 413
Total FP INCONS 1713
Total FP COMP 105
Total FN 1454
Total Scores
MCC 0.349
Average MCC ± 95% Confidence Intervals 0.395 ± 0.056
Sensitivity 0.400
Positive Predictive Value 0.313
Nr of predictions 109

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01535 0.20 0.24 0.17 9 7088 43 7 36 0 28
CRW_01539 0.10 0.11 0.10 4 7099 40 1 36 3 33
CRW_01563 0.11 0.14 0.11 5 6856 43 5 37 1 32
CRW_01583 0.36 0.41 0.33 15 7095 31 8 22 1 22
CRW_01590 -0.01 0.00 0.00 0 7093 50 7 40 3 37
CRW_01593 0.32 0.38 0.29 14 7091 36 5 30 1 23
CRW_01603 0.41 0.46 0.38 17 7095 29 6 22 1 20
CRW_01617 0.20 0.24 0.17 9 7087 45 7 37 1 28
PDB_00011 - 0.96 0.92 1.00 11 265 0 0 0 0 1
PDB_00017 - 0.66 0.67 0.67 10 688 5 1 4 0 5
PDB_00036 - -0.01 0.00 0.00 0 3457 29 1 28 0 27
PDB_00070 0.25 0.29 0.23 7 2819 24 4 20 0 17
PDB_00113 - 0.94 0.89 1.00 8 223 0 0 0 0 1
PDB_00119 - 0.95 0.90 1.00 9 244 0 0 0 0 1
PDB_00136 - 0.70 0.73 0.68 19 2387 9 1 8 0 7
PDB_00137 - 0.87 0.77 1.00 10 396 0 0 0 0 3
PDB_00160 - 0.89 0.89 0.89 8 267 2 0 1 1 1
PDB_00178 - -0.02 0.00 0.00 0 517 11 1 10 0 11
PDB_00221 - 0.94 0.89 1.00 8 245 0 0 0 0 1
PDB_00312 - 1.00 1.00 1.00 9 426 1 0 0 1 0
PDB_00326 - 0.58 0.64 0.53 18 3707 16 4 12 0 10
PDB_00341 - 0.47 0.44 0.54 7 690 6 0 6 0 9
PDB_00349 - 0.10 0.14 0.10 2 926 18 0 18 0 12
PDB_00430 0.38 0.44 0.34 11 3208 21 6 15 0 14
PDB_00527 - -0.02 0.00 0.00 0 292 8 2 6 0 5
PDB_00565 - 0.24 0.24 0.24 12 8079 37 1 36 0 38
PDB_00570 - -0.01 0.00 0.00 0 2257 21 0 21 0 22
PDB_00571 -0.01 0.00 0.00 0 3286 36 6 29 1 25
PDB_00573 - -0.01 0.00 0.00 0 6622 48 5 43 0 34
PDB_00584 - 0.47 0.53 0.43 9 1464 12 1 11 0 8
PDB_00698 - 0.97 0.95 1.00 18 843 0 0 0 0 1
PDB_00725 - 0.82 0.80 0.86 12 581 2 0 2 0 3
PDB_00734 - 0.83 0.70 1.00 7 224 0 0 0 0 3
PDB_00823 - -0.04 0.00 0.00 0 183 7 0 7 0 9
PDB_00828 0.30 0.33 0.28 9 2453 23 0 23 0 18
PDB_00851 - 0.16 0.17 0.17 7 4711 35 3 32 0 34
PDB_00890 - -0.02 0.00 0.00 0 690 13 2 11 0 14
PDB_00923 - 1.00 1.00 1.00 6 184 2 0 0 2 0
PDB_01009 0.27 0.33 0.23 7 2455 25 3 20 2 14
PDB_01051 -0.02 0.00 0.00 0 885 18 1 17 0 13
PDB_01074 - -0.01 0.00 0.00 0 2747 28 3 25 0 25
PDB_01130 - -0.01 0.00 0.00 0 1742 28 5 23 0 20
PDB_01199 - -0.01 0.00 0.00 0 2116 29 2 27 0 24
RFA_00416 0.94 1.00 0.88 15 1468 6 2 0 4 0
RFA_00434 0.84 1.00 0.71 15 1410 8 6 0 2 0
RFA_00436 0.86 1.00 0.75 15 1411 7 5 0 2 0
RFA_00442 0.94 1.00 0.88 15 1414 4 2 0 2 0
RFA_00446 0.85 0.93 0.78 14 1467 7 3 1 3 1
RFA_00664 -0.01 0.00 0.00 0 978 13 0 12 1 14
RFA_00674 0.12 0.14 0.13 2 1112 15 0 14 1 12
RFA_00675 0.28 0.29 0.29 4 976 10 1 9 0 10
RFA_00677 0.62 0.64 0.60 9 975 10 0 6 4 5
RFA_00684 0.74 0.71 0.77 10 977 5 2 1 2 4
RFA_00703 0.26 0.36 0.19 5 4252 32 5 16 11 9
RFA_00716 -0.02 0.00 0.00 0 929 17 0 17 0 14
RFA_00717 0.64 0.64 0.64 9 889 5 2 3 0 5
RFA_00730 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00763 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00765 0.39 0.42 0.38 5 890 9 2 6 1 7
RFA_00770 -0.01 0.00 0.00 0 1996 24 0 20 4 18
RFA_00773 0.93 1.00 0.86 18 1932 9 3 0 6 0
SPR_00030 - 0.47 0.52 0.42 11 2459 16 2 13 1 10
SPR_00066 0.47 0.57 0.39 12 2819 20 4 15 1 9
SPR_00228 0.44 0.60 0.32 12 3791 26 15 10 1 8
SPR_00314 0.47 0.57 0.39 12 2895 21 9 10 2 9
SPR_00331 0.48 0.60 0.39 12 2744 19 6 13 0 8
SPR_00333 0.25 0.33 0.21 7 2741 27 10 17 0 14
SPR_00353 0.64 0.81 0.52 17 2817 16 8 8 0 4
SPR_00398 - -0.01 0.00 0.00 0 2185 28 10 16 2 13
SPR_00452 0.27 0.33 0.23 7 2820 24 7 16 1 14
SPR_00456 0.49 0.57 0.43 12 2822 19 5 11 3 9
SPR_00462 0.65 0.74 0.58 14 2391 11 4 6 1 5
SPR_00476 0.47 0.52 0.42 11 2749 17 4 11 2 10
SPR_00496 0.47 0.57 0.40 12 2745 22 7 11 4 9
SPR_00497 0.27 0.33 0.23 7 2745 23 5 18 0 14
SPR_00502 0.27 0.33 0.23 7 2896 26 2 21 3 14
SPR_00509 0.23 0.30 0.19 6 2895 25 8 17 0 14
SPR_00512 0.61 0.75 0.50 15 2896 15 6 9 0 5
SPR_00542 0.26 0.33 0.22 7 3538 28 4 21 3 14
SPR_00557 0.42 0.52 0.34 11 2818 22 8 13 1 10
SPR_00832 - 0.53 0.67 0.43 10 1630 13 5 8 0 5
SPR_00868 0.49 0.57 0.43 12 2898 19 5 11 3 9
SPR_00932 0.43 0.52 0.35 11 2819 21 7 13 1 10
SPR_00957 0.63 0.76 0.53 16 2973 15 7 7 1 5
SPR_01040 0.15 0.19 0.13 4 2744 27 4 23 0 17
SPR_01143 0.41 0.48 0.36 10 2528 18 8 10 0 11
SRP_00005 - -0.01 0.00 0.00 0 3966 39 6 33 0 24
SRP_00038 0.23 0.27 0.20 9 5005 36 1 35 0 24
SRP_00046 0.33 0.38 0.29 12 4712 30 3 26 1 20
SRP_00138 - 0.44 0.52 0.39 14 3969 24 8 14 2 13
SRP_00170 0.29 0.36 0.25 9 4059 28 4 23 1 16
SRP_00193 - 0.16 0.19 0.15 7 6855 43 4 37 2 29
SRP_00194 0.27 0.33 0.23 7 3209 27 5 19 3 14
SRP_00200 0.41 0.47 0.35 17 6855 31 4 27 0 19
SRP_00216 0.44 0.48 0.41 13 3371 19 4 15 0 14
SRP_00229 0.42 0.52 0.35 14 5213 26 8 18 0 13
SRP_00233 0.25 0.30 0.22 10 5106 35 4 31 0 23
SRP_00243 0.47 0.55 0.40 17 5109 25 7 18 0 14
SRP_00244 0.38 0.44 0.34 17 6853 33 2 31 0 22
SRP_00266 0.55 0.66 0.46 19 4615 22 8 14 0 10
SRP_00269 0.46 0.52 0.43 17 5010 24 6 17 1 16
SRP_00273 0.36 0.40 0.33 16 6737 34 6 27 1 24
SRP_00285 0.38 0.43 0.35 13 3704 24 6 18 0 17
SRP_00303 0.36 0.41 0.33 13 6630 27 3 24 0 19
SRP_00333 0.20 0.24 0.19 8 5108 35 3 32 0 26
SRP_00338 0.52 0.56 0.49 20 5419 22 4 17 1 16
SRP_00342 0.20 0.24 0.17 6 3970 29 7 22 0 19
SRP_00358 0.32 0.37 0.28 13 6282 33 4 29 0 22
SRP_00383 -0.01 0.00 0.00 0 3205 35 9 26 0 24

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.