CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of NanoFolder - scored higher in this pairwise comparison

  4. Performance of Carnac(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for NanoFolder & Carnac(seed) [.zip] - may take several seconds...


Overview

Metric NanoFolder Carnac(seed)
MCC 0.197 > 0.094
Average MCC ± 95% Confidence Intervals 0.198 ± 0.067 > 0.047 ± 0.083
Sensitivity 0.233 > 0.013
Positive Predictive Value 0.170 < 0.677
Total TP 372 > 21
Total TN 850223 < 852381
Total FP 1858 > 13
Total FP CONTRA 320 > 2
Total FP INCONS 1497 > 8
Total FP COMP 41 > 3
Total FN 1227 < 1578
P-value 3.05041766982e-08

^top




Performance plots


  1. Comparison of performance of NanoFolder and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for NanoFolder and Carnac(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for NanoFolder and Carnac(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for NanoFolder and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for NanoFolder and Carnac(seed)).

^top





Performance of NanoFolder - scored higher in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 372
Total TN 850223
Total FP 1858
Total FP CONTRA 320
Total FP INCONS 1497
Total FP COMP 41
Total FN 1227
Total Scores
MCC 0.197
Average MCC ± 95% Confidence Intervals 0.198 ± 0.067
Sensitivity 0.233
Positive Predictive Value 0.170
Nr of predictions 17

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00191 0.36 0.43 0.30 26 29559 67 15 46 6 35
ASE_00196 0.49 0.57 0.42 43 31524 62 16 43 3 32
ASE_00417 0.32 0.36 0.28 36 54486 98 18 75 5 64
CRW_00671 0.18 0.21 0.16 24 62682 130 23 106 1 93
CRW_00672 0.36 0.43 0.31 48 72233 114 35 74 5 63
CRW_00674 0.16 0.19 0.14 23 83274 142 19 120 3 101
CRW_00676 0.11 0.14 0.09 16 93347 169 29 136 4 99
RFA_00620 0.16 0.23 0.11 9 21864 80 29 43 8 30
RFA_00816 0.00 0.00 0.00 0 23133 89 30 57 2 41
SRP_00144 0.05 0.06 0.05 6 46538 122 17 104 1 88
SRP_00197 0.12 0.14 0.11 14 49324 117 16 101 0 87
SRP_00198 0.07 0.08 0.06 8 50903 129 15 114 0 95
SRP_00276 0.19 0.22 0.18 24 50585 112 10 102 0 86
SRP_00278 0.33 0.36 0.30 38 45928 90 14 76 0 67
SRP_00308 0.14 0.17 0.12 15 36189 112 18 93 1 73
SRP_00322 0.22 0.25 0.21 27 48385 105 8 96 1 82
SRP_00332 0.12 0.14 0.11 15 50269 120 8 111 1 91

^top



Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP 21
Total TN 852381
Total FP 13
Total FP CONTRA 2
Total FP INCONS 8
Total FP COMP 3
Total FN 1578
Total Scores
MCC 0.094
Average MCC ± 95% Confidence Intervals 0.047 ± 0.083
Sensitivity 0.013
Positive Predictive Value 0.677
Nr of predictions 17

^top



2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00191 0.00 0.00 0.00 0 29646 0 0 0 0 61
ASE_00196 0.00 0.00 0.00 0 31626 0 0 0 0 75
ASE_00417 0.00 0.00 0.00 0 54615 0 0 0 0 100
CRW_00671 0.00 0.00 0.00 0 62835 0 0 0 0 117
CRW_00672 0.00 0.00 0.00 0 72390 0 0 0 0 111
CRW_00674 0.00 0.00 0.00 0 83436 0 0 0 0 124
CRW_00676 0.00 0.00 0.00 0 93528 0 0 0 0 115
RFA_00620 0.13 0.08 0.23 3 21932 11 2 8 1 36
RFA_00816 0.66 0.44 1.00 18 23202 2 0 0 2 23
SRP_00144 0.00 0.00 0.00 0 46665 0 0 0 0 94
SRP_00197 0.00 0.00 0.00 0 49455 0 0 0 0 101
SRP_00198 0.00 0.00 0.00 0 51040 0 0 0 0 103
SRP_00276 0.00 0.00 0.00 0 50721 0 0 0 0 110
SRP_00278 0.00 0.00 0.00 0 46056 0 0 0 0 105
SRP_00308 0.00 0.00 0.00 0 36315 0 0 0 0 88
SRP_00322 0.00 0.00 0.00 0 48516 0 0 0 0 109
SRP_00332 0.00 0.00 0.00 0 50403 0 0 0 0 106

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.