CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PPfold(20) - scored higher in this pairwise comparison

  4. Performance of RSpredict(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for PPfold(20) & RSpredict(seed) [.zip] - may take several seconds...


Overview

Metric PPfold(20) RSpredict(seed)
MCC 0.820 > 0.012
Average MCC ± 95% Confidence Intervals 0.822 ± 0.054 > 0.012 ± 0.017
Sensitivity 0.766 > 0.011
Positive Predictive Value 0.879 > 0.014
Total TP 1463 > 21
Total TN 4652238 < 4652360
Total FP 286 < 1540
Total FP CONTRA 39 < 220
Total FP INCONS 162 < 1301
Total FP COMP 85 > 19
Total FN 448 < 1890
P-value 0.0

^top




Performance plots


  1. Comparison of performance of PPfold(20) and RSpredict(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PPfold(20) and RSpredict(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PPfold(20) and RSpredict(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PPfold(20) and RSpredict(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PPfold(20) and RSpredict(seed)).

^top





Performance of PPfold(20) - scored higher in this pairwise comparison

1. Total counts & total scores for PPfold(20)

Total Base Pair Counts
Total TP 1463
Total TN 4652238
Total FP 286
Total FP CONTRA 39
Total FP INCONS 162
Total FP COMP 85
Total FN 448
Total Scores
MCC 0.820
Average MCC ± 95% Confidence Intervals 0.822 ± 0.054
Sensitivity 0.766
Positive Predictive Value 0.879
Nr of predictions 4

^top



2. Individual counts for PPfold(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00256 0.82 0.78 0.87 363 1192324 70 13 40 17 105
CRW_00590 0.85 0.81 0.90 367 1119348 69 4 37 28 85
CRW_00997 0.83 0.81 0.86 374 1135845 75 15 44 16 90
PDB_01120 0.78 0.68 0.88 359 1204721 72 7 41 24 168

^top



Performance of RSpredict(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for RSpredict(seed)

Total Base Pair Counts
Total TP 21
Total TN 4652360
Total FP 1540
Total FP CONTRA 220
Total FP INCONS 1301
Total FP COMP 19
Total FN 1890
Total Scores
MCC 0.012
Average MCC ± 95% Confidence Intervals 0.012 ± 0.017
Sensitivity 0.011
Positive Predictive Value 0.014
Nr of predictions 4

^top



2. Individual counts for RSpredict(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00256 0.00 0.00 0.00 0 1192358 394 66 316 12 468
CRW_00590 0.02 0.02 0.02 9 1119377 371 50 320 1 443
CRW_00997 0.02 0.02 0.02 9 1135900 370 64 305 1 455
PDB_01120 0.01 0.01 0.01 3 1204725 405 40 360 5 524

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.