CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Murlet(20) - scored higher in this pairwise comparison

  4. Performance of RSpredict(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for Murlet(20) & RSpredict(seed) [.zip] - may take several seconds...


Overview

Metric Murlet(20) RSpredict(seed)
MCC 0.636 > 0.466
Average MCC ± 95% Confidence Intervals 0.635 ± 0.200 > 0.457 ± 0.290
Sensitivity 0.564 > 0.389
Positive Predictive Value 0.717 > 0.558
Total TP 1906 > 1316
Total TN 9346894 < 9347192
Total FP 895 < 1203
Total FP CONTRA 111 < 166
Total FP INCONS 640 < 877
Total FP COMP 144 < 160
Total FN 1476 < 2066
P-value 0.0

^top




Performance plots


  1. Comparison of performance of Murlet(20) and RSpredict(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Murlet(20) and RSpredict(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Murlet(20) and RSpredict(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Murlet(20) and RSpredict(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Murlet(20) and RSpredict(seed)).

^top





Performance of Murlet(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Murlet(20)

Total Base Pair Counts
Total TP 1906
Total TN 9346894
Total FP 895
Total FP CONTRA 111
Total FP INCONS 640
Total FP COMP 144
Total FN 1476
Total Scores
MCC 0.636
Average MCC ± 95% Confidence Intervals 0.635 ± 0.200
Sensitivity 0.564
Positive Predictive Value 0.717
Nr of predictions 7

^top



2. Individual counts for Murlet(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00177 0.75 0.70 0.81 330 1178473 93 14 63 16 143
CRW_00250 0.75 0.68 0.84 325 1198537 73 14 50 9 153
CRW_00280 0.78 0.72 0.85 328 1117874 75 11 47 17 125
CRW_00411 0.31 0.25 0.40 122 1615195 206 23 163 20 374
CRW_00414 0.33 0.29 0.36 146 1665818 288 38 223 27 349
CRW_00789 0.78 0.71 0.86 318 1137416 79 8 44 27 129
PDB_01105 0.73 0.62 0.86 337 1433581 81 3 50 28 203

^top



Performance of RSpredict(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for RSpredict(seed)

Total Base Pair Counts
Total TP 1316
Total TN 9347192
Total FP 1203
Total FP CONTRA 166
Total FP INCONS 877
Total FP COMP 160
Total FN 2066
Total Scores
MCC 0.466
Average MCC ± 95% Confidence Intervals 0.457 ± 0.290
Sensitivity 0.389
Positive Predictive Value 0.558
Nr of predictions 7

^top



2. Individual counts for RSpredict(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00177 0.67 0.58 0.77 274 1178524 103 15 67 21 199
CRW_00250 0.67 0.60 0.74 285 1198543 124 18 80 26 193
CRW_00280 0.61 0.53 0.70 240 1117918 122 15 87 20 213
CRW_00411 0.00 0.00 0.00 0 1615206 309 45 252 12 496
CRW_00414 0.00 0.00 0.00 0 1665933 312 53 239 20 495
CRW_00789 0.64 0.56 0.74 249 1137451 114 13 73 28 198
PDB_01105 0.61 0.50 0.76 268 1433617 119 7 79 33 272

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.