CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PPfold(20) - scored higher in this pairwise comparison

  4. Performance of MXScarna(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for PPfold(20) & MXScarna(seed) [.zip] - may take several seconds...


Overview

Metric PPfold(20) MXScarna(seed)
MCC 0.820 > 0.713
Average MCC ± 95% Confidence Intervals 0.822 ± 0.054 > 0.714 ± 0.054
Sensitivity 0.766 > 0.629
Positive Predictive Value 0.879 > 0.808
Total TP 1463 > 1202
Total TN 4652238 < 4652415
Total FP 286 < 421
Total FP CONTRA 39 > 36
Total FP INCONS 162 < 249
Total FP COMP 85 < 136
Total FN 448 < 709
P-value 0.0

^top




Performance plots


  1. Comparison of performance of PPfold(20) and MXScarna(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PPfold(20) and MXScarna(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PPfold(20) and MXScarna(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PPfold(20) and MXScarna(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PPfold(20) and MXScarna(seed)).

^top





Performance of PPfold(20) - scored higher in this pairwise comparison

1. Total counts & total scores for PPfold(20)

Total Base Pair Counts
Total TP 1463
Total TN 4652238
Total FP 286
Total FP CONTRA 39
Total FP INCONS 162
Total FP COMP 85
Total FN 448
Total Scores
MCC 0.820
Average MCC ± 95% Confidence Intervals 0.822 ± 0.054
Sensitivity 0.766
Positive Predictive Value 0.879
Nr of predictions 4

^top



2. Individual counts for PPfold(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00256 0.82 0.78 0.87 363 1192324 70 13 40 17 105
CRW_00590 0.85 0.81 0.90 367 1119348 69 4 37 28 85
CRW_00997 0.83 0.81 0.86 374 1135845 75 15 44 16 90
PDB_01120 0.78 0.68 0.88 359 1204721 72 7 41 24 168

^top



Performance of MXScarna(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for MXScarna(seed)

Total Base Pair Counts
Total TP 1202
Total TN 4652415
Total FP 421
Total FP CONTRA 36
Total FP INCONS 249
Total FP COMP 136
Total FN 709
Total Scores
MCC 0.713
Average MCC ± 95% Confidence Intervals 0.714 ± 0.054
Sensitivity 0.629
Positive Predictive Value 0.808
Nr of predictions 4

^top



2. Individual counts for MXScarna(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00256 0.74 0.66 0.82 311 1192362 98 11 56 31 157
CRW_00590 0.73 0.65 0.82 294 1119399 104 6 57 41 158
CRW_00997 0.72 0.65 0.80 301 1135902 107 11 64 32 163
PDB_01120 0.66 0.56 0.79 296 1204752 112 8 72 32 231

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.