CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MXScarna(seed) - scored higher in this pairwise comparison

  4. Performance of Mastr(20) - scored lower in this pairwise comparison

  5. Compile and download dataset for MXScarna(seed) & Mastr(20) [.zip] - may take several seconds...


Overview

Metric MXScarna(seed) Mastr(20)
MCC 0.472 > 0.000
Average MCC ± 95% Confidence Intervals 0.476 ± 0.055 > 0.000 ± 0.000
Sensitivity 0.437 > 0.000
Positive Predictive Value 0.511 > 0.000
Total TP 1905 > 0
Total TN 14159920 < 14163647
Total FP 2377 > 0
Total FP CONTRA 337 > 0
Total FP INCONS 1485 > 0
Total FP COMP 555 > 0
Total FN 2459 < 4364
P-value 0.0

^top




Performance plots


  1. Comparison of performance of MXScarna(seed) and Mastr(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MXScarna(seed) and Mastr(20)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MXScarna(seed) and Mastr(20)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for MXScarna(seed) and Mastr(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MXScarna(seed) and Mastr(20)).

^top





Performance of MXScarna(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for MXScarna(seed)

Total Base Pair Counts
Total TP 1905
Total TN 14159920
Total FP 2377
Total FP CONTRA 337
Total FP INCONS 1485
Total FP COMP 555
Total FN 2459
Total Scores
MCC 0.472
Average MCC ± 95% Confidence Intervals 0.476 ± 0.055
Sensitivity 0.437
Positive Predictive Value 0.511
Nr of predictions 9

^top



2. Individual counts for MXScarna(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00306 0.42 0.39 0.46 181 1445456 267 46 167 54 281
CRW_00366 0.48 0.45 0.51 215 1579331 275 41 166 68 267
CRW_00371 0.50 0.47 0.53 232 1611469 276 42 167 67 264
CRW_00379 0.36 0.32 0.41 170 2126540 318 50 193 75 364
CRW_00411 0.49 0.47 0.52 231 1615062 273 41 169 63 265
CRW_00759 0.45 0.42 0.48 201 1636722 295 39 183 73 280
CRW_00772 0.63 0.55 0.72 223 945690 133 6 81 46 181
CRW_01153 0.47 0.44 0.50 221 1606085 276 37 185 54 282
CRW_01171 0.49 0.46 0.53 231 1593565 264 35 174 55 275

^top



Performance of Mastr(20) - scored lower in this pairwise comparison

1. Total counts & total scores for Mastr(20)

Total Base Pair Counts
Total TP 0
Total TN 14163647
Total FP 0
Total FP CONTRA 0
Total FP INCONS 0
Total FP COMP 0
Total FN 4364
Total Scores
MCC 0.000
Average MCC ± 95% Confidence Intervals 0.000 ± 0.000
Sensitivity 0.000
Positive Predictive Value 0.000
Nr of predictions 9

^top



2. Individual counts for Mastr(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00306 0.00 0.00 0.00 0 1445850 0 0 0 0 462
CRW_00366 0.00 0.00 0.00 0 1579753 0 0 0 0 482
CRW_00371 0.00 0.00 0.00 0 1611910 0 0 0 0 496
CRW_00379 0.00 0.00 0.00 0 2126953 0 0 0 0 534
CRW_00411 0.00 0.00 0.00 0 1615503 0 0 0 0 496
CRW_00759 0.00 0.00 0.00 0 1637145 0 0 0 0 481
CRW_00772 0.00 0.00 0.00 0 946000 0 0 0 0 404
CRW_01153 0.00 0.00 0.00 0 1606528 0 0 0 0 503
CRW_01171 0.00 0.00 0.00 0 1594005 0 0 0 0 506

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.