CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PPfold(20) - scored higher in this pairwise comparison

  4. Performance of RSpredict(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for PPfold(20) & RSpredict(seed) [.zip] - may take several seconds...


Overview

Metric PPfold(20) RSpredict(seed)
MCC 0.820 > 0.633
Average MCC ± 95% Confidence Intervals 0.822 ± 0.054 > 0.633 ± 0.016
Sensitivity 0.766 > 0.540
Positive Predictive Value 0.879 > 0.743
Total TP 1463 > 1032
Total TN 4652238 < 4652513
Total FP 286 < 465
Total FP CONTRA 39 < 48
Total FP INCONS 162 < 309
Total FP COMP 85 < 108
Total FN 448 < 879
P-value 0.0

^top




Performance plots


  1. Comparison of performance of PPfold(20) and RSpredict(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PPfold(20) and RSpredict(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PPfold(20) and RSpredict(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PPfold(20) and RSpredict(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PPfold(20) and RSpredict(seed)).

^top





Performance of PPfold(20) - scored higher in this pairwise comparison

1. Total counts & total scores for PPfold(20)

Total Base Pair Counts
Total TP 1463
Total TN 4652238
Total FP 286
Total FP CONTRA 39
Total FP INCONS 162
Total FP COMP 85
Total FN 448
Total Scores
MCC 0.820
Average MCC ± 95% Confidence Intervals 0.822 ± 0.054
Sensitivity 0.766
Positive Predictive Value 0.879
Nr of predictions 4

^top



2. Individual counts for PPfold(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00256 0.82 0.78 0.87 363 1192324 70 13 40 17 105
CRW_00590 0.85 0.81 0.90 367 1119348 69 4 37 28 85
CRW_00997 0.83 0.81 0.86 374 1135845 75 15 44 16 90
PDB_01120 0.78 0.68 0.88 359 1204721 72 7 41 24 168

^top



Performance of RSpredict(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for RSpredict(seed)

Total Base Pair Counts
Total TP 1032
Total TN 4652513
Total FP 465
Total FP CONTRA 48
Total FP INCONS 309
Total FP COMP 108
Total FN 879
Total Scores
MCC 0.633
Average MCC ± 95% Confidence Intervals 0.633 ± 0.016
Sensitivity 0.540
Positive Predictive Value 0.743
Nr of predictions 4

^top



2. Individual counts for RSpredict(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00256 0.62 0.54 0.72 251 1192392 119 17 80 22 217
CRW_00590 0.64 0.54 0.76 245 1119432 103 8 71 24 207
CRW_00997 0.63 0.55 0.72 256 1135921 123 14 87 22 208
PDB_01120 0.64 0.53 0.78 280 1204768 120 9 71 40 247

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.