CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of RNASampler(seed) - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for RNASampler(seed) & NanoFolder [.zip] - may take several seconds...


Overview

Metric RNASampler(seed) NanoFolder
MCC 0.461 > 0.413
Average MCC ± 95% Confidence Intervals 0.351 ± 0.142 < 0.403 ± 0.171
Sensitivity 0.281 < 0.464
Positive Predictive Value 0.763 > 0.375
Total TP 74 < 122
Total TN 35730 > 35502
Total FP 37 < 253
Total FP CONTRA 11 < 45
Total FP INCONS 12 < 158
Total FP COMP 14 < 50
Total FN 189 > 141
P-value 4.30260184036e-08

^top




Performance plots


  1. Comparison of performance of RNASampler(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNASampler(seed) and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNASampler(seed) and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for RNASampler(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNASampler(seed) and NanoFolder).

^top





Performance of RNASampler(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for RNASampler(seed)

Total Base Pair Counts
Total TP 74
Total TN 35730
Total FP 37
Total FP CONTRA 11
Total FP INCONS 12
Total FP COMP 14
Total FN 189
Total Scores
MCC 0.461
Average MCC ± 95% Confidence Intervals 0.351 ± 0.142
Sensitivity 0.281
Positive Predictive Value 0.763
Nr of predictions 17

^top



2. Individual counts for RNASampler(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_01051 0.45 0.31 0.67 4 897 6 0 2 4 9
RFA_00416 0.63 0.40 1.00 6 1479 1 0 0 1 9
RFA_00604 0.73 0.66 0.82 23 13502 14 4 1 9 12
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00684 0.53 0.29 1.00 4 986 0 0 0 0 10
RFA_00703 0.46 0.21 1.00 3 4275 0 0 0 0 11
RFA_00716 -0.01 0.00 0.00 0 943 3 0 3 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00763 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00765 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00770 0.52 0.39 0.70 7 2006 3 3 0 0 11
RFA_00773 0.59 0.50 0.69 9 1940 4 4 0 0 9
RFA_00808 0.58 0.56 0.60 9 2001 6 0 6 0 7

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 122
Total TN 35502
Total FP 253
Total FP CONTRA 45
Total FP INCONS 158
Total FP COMP 50
Total FN 141
Total Scores
MCC 0.413
Average MCC ± 95% Confidence Intervals 0.403 ± 0.171
Sensitivity 0.464
Positive Predictive Value 0.375
Nr of predictions 17

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_01051 -0.02 0.00 0.00 0 885 18 1 17 0 13
RFA_00416 0.94 1.00 0.88 15 1468 6 2 0 4 0
RFA_00604 0.42 0.54 0.33 19 13472 53 14 25 14 16
RFA_00664 -0.01 0.00 0.00 0 978 13 0 12 1 14
RFA_00674 0.12 0.14 0.13 2 1112 15 0 14 1 12
RFA_00675 0.28 0.29 0.29 4 976 10 1 9 0 10
RFA_00677 0.62 0.64 0.60 9 975 10 0 6 4 5
RFA_00684 0.74 0.71 0.77 10 977 5 2 1 2 4
RFA_00703 0.26 0.36 0.19 5 4252 32 5 16 11 9
RFA_00716 -0.02 0.00 0.00 0 929 17 0 17 0 14
RFA_00717 0.64 0.64 0.64 9 889 5 2 3 0 5
RFA_00730 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00763 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00765 0.39 0.42 0.38 5 890 9 2 6 1 7
RFA_00770 -0.01 0.00 0.00 0 1996 24 0 20 4 18
RFA_00773 0.93 1.00 0.86 18 1932 9 3 0 6 0
RFA_00808 0.83 1.00 0.70 16 1993 7 7 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.