CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CMfinder(seed) - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for CMfinder(seed) & NanoFolder [.zip] - may take several seconds...


Overview

Metric CMfinder(seed) NanoFolder
MCC 0.430 > 0.188
Average MCC ± 95% Confidence Intervals 0.411 ± 0.108 > 0.179 ± 0.072
Sensitivity 0.239 > 0.221
Positive Predictive Value 0.775 > 0.162
Total TP 331 > 305
Total TN 747369 > 745918
Total FP 98 < 1600
Total FP CONTRA 2 < 269
Total FP INCONS 94 < 1304
Total FP COMP 2 < 27
Total FN 1052 < 1078
P-value 1.50834591464e-08

^top




Performance plots


  1. Comparison of performance of CMfinder(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CMfinder(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).

^top





Performance of CMfinder(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 331
Total TN 747369
Total FP 98
Total FP CONTRA 2
Total FP INCONS 94
Total FP COMP 2
Total FN 1052
Total Scores
MCC 0.430
Average MCC ± 95% Confidence Intervals 0.411 ± 0.108
Sensitivity 0.239
Positive Predictive Value 0.775
Nr of predictions 14

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00674 0.40 0.17 0.95 21 83414 1 0 1 0 103
CRW_00676 0.12 0.04 0.33 5 93513 10 1 9 0 110
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 305
Total TN 745918
Total FP 1600
Total FP CONTRA 269
Total FP INCONS 1304
Total FP COMP 27
Total FN 1078
Total Scores
MCC 0.188
Average MCC ± 95% Confidence Intervals 0.179 ± 0.072
Sensitivity 0.221
Positive Predictive Value 0.162
Nr of predictions 14

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00671 0.18 0.21 0.16 24 62682 130 23 106 1 93
CRW_00672 0.36 0.43 0.31 48 72233 114 35 74 5 63
CRW_00674 0.16 0.19 0.14 23 83274 142 19 120 3 101
CRW_00676 0.11 0.14 0.09 16 93347 169 29 136 4 99
RFA_00620 0.16 0.23 0.11 9 21864 80 29 43 8 30
RFA_00816 0.00 0.00 0.00 0 23133 89 30 57 2 41
SRP_00144 0.05 0.06 0.05 6 46538 122 17 104 1 88
SRP_00197 0.12 0.14 0.11 14 49324 117 16 101 0 87
SRP_00198 0.07 0.08 0.06 8 50903 129 15 114 0 95
SRP_00276 0.19 0.22 0.18 24 50585 112 10 102 0 86
SRP_00278 0.33 0.36 0.30 38 45928 90 14 76 0 67
SRP_00321 0.44 0.49 0.40 53 47453 81 16 64 1 55
SRP_00322 0.22 0.25 0.21 27 48385 105 8 96 1 82
SRP_00332 0.12 0.14 0.11 15 50269 120 8 111 1 91

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.