CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Murlet(20) - scored higher in this pairwise comparison

  4. Performance of MXScarna(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for Murlet(20) & MXScarna(seed) [.zip] - may take several seconds...


Overview

Metric Murlet(20) MXScarna(seed)
MCC 0.649 > 0.632
Average MCC ± 95% Confidence Intervals 0.649 ± 0.171 > 0.638 ± 0.084
Sensitivity 0.576 > 0.567
Positive Predictive Value 0.731 > 0.706
Total TP 2208 > 2172
Total TN 10476787 > 10476730
Total FP 970 < 1251
Total FP CONTRA 123 < 134
Total FP INCONS 689 < 771
Total FP COMP 158 < 346
Total FN 1624 < 1660
P-value 0.0

^top




Performance plots


  1. Comparison of performance of Murlet(20) and MXScarna(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Murlet(20) and MXScarna(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Murlet(20) and MXScarna(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Murlet(20) and MXScarna(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Murlet(20) and MXScarna(seed)).

^top





Performance of Murlet(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Murlet(20)

Total Base Pair Counts
Total TP 2208
Total TN 10476787
Total FP 970
Total FP CONTRA 123
Total FP INCONS 689
Total FP COMP 158
Total FN 1624
Total Scores
MCC 0.649
Average MCC ± 95% Confidence Intervals 0.649 ± 0.171
Sensitivity 0.576
Positive Predictive Value 0.731
Nr of predictions 8

^top



2. Individual counts for Murlet(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00177 0.75 0.70 0.81 330 1178473 93 14 63 16 143
CRW_00250 0.75 0.68 0.84 325 1198537 73 14 50 9 153
CRW_00278 0.75 0.67 0.83 302 1129893 75 12 49 14 148
CRW_00280 0.78 0.72 0.85 328 1117874 75 11 47 17 125
CRW_00411 0.31 0.25 0.40 122 1615195 206 23 163 20 374
CRW_00414 0.33 0.29 0.36 146 1665818 288 38 223 27 349
CRW_00789 0.78 0.71 0.86 318 1137416 79 8 44 27 129
PDB_01105 0.73 0.62 0.86 337 1433581 81 3 50 28 203

^top



Performance of MXScarna(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for MXScarna(seed)

Total Base Pair Counts
Total TP 2172
Total TN 10476730
Total FP 1251
Total FP CONTRA 134
Total FP INCONS 771
Total FP COMP 346
Total FN 1660
Total Scores
MCC 0.632
Average MCC ± 95% Confidence Intervals 0.638 ± 0.084
Sensitivity 0.567
Positive Predictive Value 0.706
Nr of predictions 8

^top



2. Individual counts for MXScarna(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00177 0.68 0.61 0.77 287 1178505 117 12 76 29 186
CRW_00250 0.71 0.64 0.78 307 1198533 120 11 75 34 171
CRW_00278 0.68 0.60 0.78 271 1129907 113 9 69 35 179
CRW_00280 0.70 0.63 0.79 284 1117899 114 10 67 37 169
CRW_00411 0.49 0.47 0.52 231 1615062 273 41 169 63 265
CRW_00414 0.46 0.43 0.49 213 1665792 286 43 177 66 282
CRW_00789 0.71 0.63 0.80 280 1137437 111 7 62 42 167
PDB_01105 0.66 0.55 0.80 299 1433595 117 1 76 40 241

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.