CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of RNASampler(20) - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for RNASampler(20) & NanoFolder [.zip] - may take several seconds...


Overview

Metric RNASampler(20) NanoFolder
MCC 0.618 > 0.097
Average MCC ± 95% Confidence Intervals 0.614 ± 0.090 > 0.097 ± 0.053
Sensitivity 0.506 > 0.120
Positive Predictive Value 0.755 > 0.081
Total TP 401 > 95
Total TN 513389 > 512742
Total FP 165 < 1117
Total FP CONTRA 47 < 202
Total FP INCONS 83 < 881
Total FP COMP 35 > 34
Total FN 391 < 697
P-value 0.0

^top




Performance plots


  1. Comparison of performance of RNASampler(20) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNASampler(20) and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNASampler(20) and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for RNASampler(20) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNASampler(20) and NanoFolder).

^top





Performance of RNASampler(20) - scored higher in this pairwise comparison

1. Total counts & total scores for RNASampler(20)

Total Base Pair Counts
Total TP 401
Total TN 513389
Total FP 165
Total FP CONTRA 47
Total FP INCONS 83
Total FP COMP 35
Total FN 391
Total Scores
MCC 0.618
Average MCC ± 95% Confidence Intervals 0.614 ± 0.090
Sensitivity 0.506
Positive Predictive Value 0.755
Nr of predictions 8

^top



2. Individual counts for RNASampler(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
TMR_00123 0.60 0.46 0.79 46 66372 17 3 9 5 55
TMR_00443 0.56 0.41 0.75 43 67104 14 6 8 0 61
TMR_00469 0.72 0.61 0.86 61 64549 11 4 6 1 39
TMR_00519 0.50 0.40 0.62 38 63129 27 14 9 4 57
TMR_00528 0.49 0.40 0.59 39 63124 31 12 15 4 58
TMR_00571 0.77 0.71 0.83 70 60642 22 2 12 8 29
TMR_00584 0.71 0.65 0.79 63 60995 25 3 14 8 34
TMR_00703 0.56 0.41 0.76 41 67474 18 3 10 5 58

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 95
Total TN 512742
Total FP 1117
Total FP CONTRA 202
Total FP INCONS 881
Total FP COMP 34
Total FN 697
Total Scores
MCC 0.097
Average MCC ± 95% Confidence Intervals 0.097 ± 0.053
Sensitivity 0.120
Positive Predictive Value 0.081
Nr of predictions 8

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
TMR_00123 0.19 0.23 0.16 23 66283 126 20 104 2 78
TMR_00443 0.17 0.20 0.15 21 67018 126 24 98 4 83
TMR_00469 0.10 0.12 0.08 12 64469 145 27 112 6 88
TMR_00519 0.04 0.05 0.03 5 63046 141 33 106 2 90
TMR_00528 0.13 0.15 0.11 15 63051 131 20 104 7 82
TMR_00571 0.05 0.06 0.04 6 60577 149 19 124 6 93
TMR_00584 0.01 0.01 0.01 1 60922 159 33 119 7 96
TMR_00703 0.10 0.12 0.08 12 67376 140 26 114 0 87

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.