CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of TurboFold(seed) - scored higher in this pairwise comparison

  4. Performance of Mastr(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for TurboFold(seed) & Mastr(seed) [.zip] - may take several seconds...


Overview

Metric TurboFold(seed) Mastr(seed)
MCC 0.787 > 0.393
Average MCC ± 95% Confidence Intervals 0.805 ± 0.066 > 0.174 ± 0.168
Sensitivity 0.760 > 0.173
Positive Predictive Value 0.815 < 0.895
Total TP 1052 > 239
Total TN 797739 < 798763
Total FP 383 > 63
Total FP CONTRA 40 > 2
Total FP INCONS 199 > 26
Total FP COMP 144 > 35
Total FN 332 < 1145
P-value 3.16042554876e-08

^top




Performance plots


  1. Comparison of performance of TurboFold(seed) and Mastr(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for TurboFold(seed) and Mastr(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for TurboFold(seed) and Mastr(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for TurboFold(seed) and Mastr(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for TurboFold(seed) and Mastr(seed)).

^top





Performance of TurboFold(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for TurboFold(seed)

Total Base Pair Counts
Total TP 1052
Total TN 797739
Total FP 383
Total FP CONTRA 40
Total FP INCONS 199
Total FP COMP 144
Total FN 332
Total Scores
MCC 0.787
Average MCC ± 95% Confidence Intervals 0.805 ± 0.066
Sensitivity 0.760
Positive Predictive Value 0.815
Nr of predictions 18

^top



2. Individual counts for TurboFold(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00210 0.71 0.68 0.75 44 27202 16 4 11 1 21
RFA_00626 0.90 0.85 0.95 74 56538 17 0 4 13 13
RFA_00627 0.85 0.82 0.89 71 56873 23 0 9 14 16
RFA_00628 0.92 0.90 0.95 77 57210 17 0 4 13 9
RFA_00630 0.83 0.79 0.87 69 56874 22 0 10 12 18
RFA_00805 0.87 0.82 0.93 123 133254 13 0 9 4 27
RFA_00814 0.92 0.85 1.00 35 25165 13 0 0 13 6
RFA_00816 1.00 1.00 1.00 41 23179 14 0 0 14 0
RFA_00817 0.95 0.90 1.00 37 21908 7 0 0 7 4
RFA_00818 0.85 0.80 0.89 33 20264 11 1 3 7 8
RFA_00819 0.98 0.95 1.00 39 27927 27 0 0 27 2
SRP_00241 0.56 0.59 0.54 48 45967 43 12 29 2 34
SRP_00257 0.78 0.72 0.85 78 50948 16 4 10 2 30
SRP_00308 0.76 0.74 0.77 65 36231 20 2 17 1 23
SRP_00328 0.69 0.67 0.70 57 39540 27 5 19 3 28
SRP_00329 0.65 0.64 0.67 54 39822 30 3 24 3 31
SRP_00331 0.67 0.66 0.68 57 37591 29 3 24 2 30
SRP_00340 0.61 0.61 0.61 50 41246 38 6 26 6 32

^top



Performance of Mastr(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Mastr(seed)

Total Base Pair Counts
Total TP 239
Total TN 798763
Total FP 63
Total FP CONTRA 2
Total FP INCONS 26
Total FP COMP 35
Total FN 1145
Total Scores
MCC 0.393
Average MCC ± 95% Confidence Intervals 0.174 ± 0.168
Sensitivity 0.173
Positive Predictive Value 0.895
Nr of predictions 18

^top



2. Individual counts for Mastr(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00210 0.00 0.00 0.00 0 27261 0 0 0 0 65
RFA_00626 0.61 0.51 0.75 44 56557 22 2 13 7 43
RFA_00627 0.84 0.75 0.94 65 56884 16 0 4 12 22
RFA_00628 0.88 0.80 0.96 69 57219 11 0 3 8 17
RFA_00630 0.80 0.70 0.91 61 56886 14 0 6 8 26
RFA_00805 0.00 0.00 0.00 0 133386 0 0 0 0 150
RFA_00814 0.00 0.00 0.00 0 25200 0 0 0 0 41
RFA_00816 0.00 0.00 0.00 0 23220 0 0 0 0 41
RFA_00817 0.00 0.00 0.00 0 21945 0 0 0 0 41
RFA_00818 0.00 0.00 0.00 0 20301 0 0 0 0 41
RFA_00819 0.00 0.00 0.00 0 27966 0 0 0 0 41
SRP_00241 0.00 0.00 0.00 0 46056 0 0 0 0 82
SRP_00257 0.00 0.00 0.00 0 51040 0 0 0 0 108
SRP_00308 0.00 0.00 0.00 0 36315 0 0 0 0 88
SRP_00328 0.00 0.00 0.00 0 39621 0 0 0 0 85
SRP_00329 0.00 0.00 0.00 0 39903 0 0 0 0 85
SRP_00331 0.00 0.00 0.00 0 37675 0 0 0 0 87
SRP_00340 0.00 0.00 0.00 0 41328 0 0 0 0 82

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.