CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of TurboFold(seed) - scored higher in this pairwise comparison

  4. Performance of Carnac(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for TurboFold(seed) & Carnac(seed) [.zip] - may take several seconds...


Overview

Metric TurboFold(seed) Carnac(seed)
MCC 0.787 > 0.435
Average MCC ± 95% Confidence Intervals 0.805 ± 0.066 > 0.359 ± 0.180
Sensitivity 0.760 > 0.217
Positive Predictive Value 0.815 < 0.875
Total TP 1052 > 300
Total TN 797739 < 798687
Total FP 383 > 72
Total FP CONTRA 40 > 0
Total FP INCONS 199 > 43
Total FP COMP 144 > 29
Total FN 332 < 1084
P-value 2.94377603181e-08

^top




Performance plots


  1. Comparison of performance of TurboFold(seed) and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for TurboFold(seed) and Carnac(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for TurboFold(seed) and Carnac(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for TurboFold(seed) and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for TurboFold(seed) and Carnac(seed)).

^top





Performance of TurboFold(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for TurboFold(seed)

Total Base Pair Counts
Total TP 1052
Total TN 797739
Total FP 383
Total FP CONTRA 40
Total FP INCONS 199
Total FP COMP 144
Total FN 332
Total Scores
MCC 0.787
Average MCC ± 95% Confidence Intervals 0.805 ± 0.066
Sensitivity 0.760
Positive Predictive Value 0.815
Nr of predictions 18

^top



2. Individual counts for TurboFold(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00210 0.71 0.68 0.75 44 27202 16 4 11 1 21
RFA_00626 0.90 0.85 0.95 74 56538 17 0 4 13 13
RFA_00627 0.85 0.82 0.89 71 56873 23 0 9 14 16
RFA_00628 0.92 0.90 0.95 77 57210 17 0 4 13 9
RFA_00630 0.83 0.79 0.87 69 56874 22 0 10 12 18
RFA_00805 0.87 0.82 0.93 123 133254 13 0 9 4 27
RFA_00814 0.92 0.85 1.00 35 25165 13 0 0 13 6
RFA_00816 1.00 1.00 1.00 41 23179 14 0 0 14 0
RFA_00817 0.95 0.90 1.00 37 21908 7 0 0 7 4
RFA_00818 0.85 0.80 0.89 33 20264 11 1 3 7 8
RFA_00819 0.98 0.95 1.00 39 27927 27 0 0 27 2
SRP_00241 0.56 0.59 0.54 48 45967 43 12 29 2 34
SRP_00257 0.78 0.72 0.85 78 50948 16 4 10 2 30
SRP_00308 0.76 0.74 0.77 65 36231 20 2 17 1 23
SRP_00328 0.69 0.67 0.70 57 39540 27 5 19 3 28
SRP_00329 0.65 0.64 0.67 54 39822 30 3 24 3 31
SRP_00331 0.67 0.66 0.68 57 37591 29 3 24 2 30
SRP_00340 0.61 0.61 0.61 50 41246 38 6 26 6 32

^top



Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP 300
Total TN 798687
Total FP 72
Total FP CONTRA 0
Total FP INCONS 43
Total FP COMP 29
Total FN 1084
Total Scores
MCC 0.435
Average MCC ± 95% Confidence Intervals 0.359 ± 0.180
Sensitivity 0.217
Positive Predictive Value 0.875
Nr of predictions 18

^top



2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00210 0.00 0.00 0.00 0 27261 0 0 0 0 65
RFA_00626 0.67 0.52 0.87 45 56564 11 0 7 4 42
RFA_00627 0.69 0.55 0.86 48 56897 10 0 8 2 39
RFA_00628 0.66 0.43 1.00 37 57254 6 0 0 6 49
RFA_00630 0.79 0.67 0.94 58 56891 12 0 4 8 29
RFA_00805 0.09 0.04 0.21 6 133357 23 0 23 0 144
RFA_00814 0.66 0.44 1.00 18 25182 2 0 0 2 23
RFA_00816 0.66 0.44 1.00 18 23202 2 0 0 2 23
RFA_00817 0.80 0.66 0.96 27 21917 1 0 1 0 14
RFA_00818 0.70 0.49 1.00 20 20281 3 0 0 3 21
RFA_00819 0.75 0.56 1.00 23 27943 2 0 0 2 18
SRP_00241 0.00 0.00 0.00 0 46056 0 0 0 0 82
SRP_00257 0.00 0.00 0.00 0 51040 0 0 0 0 108
SRP_00308 0.00 0.00 0.00 0 36315 0 0 0 0 88
SRP_00328 0.00 0.00 0.00 0 39621 0 0 0 0 85
SRP_00329 0.00 0.00 0.00 0 39903 0 0 0 0 85
SRP_00331 0.00 0.00 0.00 0 37675 0 0 0 0 87
SRP_00340 0.00 0.00 0.00 0 41328 0 0 0 0 82

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.