CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PETfold_pre2.0(20) - scored higher in this pairwise comparison

  4. Performance of PPfold(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for PETfold_pre2.0(20) & PPfold(seed) [.zip] - may take several seconds...


Overview

Metric PETfold_pre2.0(20) PPfold(seed)
MCC 0.669 > 0.453
Average MCC ± 95% Confidence Intervals 0.641 ± 0.138 > 0.289 ± 0.172
Sensitivity 0.473 > 0.224
Positive Predictive Value 0.948 > 0.923
Total TP 328 > 155
Total TN 132920 < 133098
Total FP 39 > 29
Total FP CONTRA 0 = 0
Total FP INCONS 18 > 13
Total FP COMP 21 > 16
Total FN 365 < 538
P-value 4.69811216434e-08

^top




Performance plots


  1. Comparison of performance of PETfold_pre2.0(20) and PPfold(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(20) and PPfold(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(20) and PPfold(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(20) and PPfold(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(20) and PPfold(seed)).

^top





Performance of PETfold_pre2.0(20) - scored higher in this pairwise comparison

1. Total counts & total scores for PETfold_pre2.0(20)

Total Base Pair Counts
Total TP 328
Total TN 132920
Total FP 39
Total FP CONTRA 0
Total FP INCONS 18
Total FP COMP 21
Total FN 365
Total Scores
MCC 0.669
Average MCC ± 95% Confidence Intervals 0.641 ± 0.138
Sensitivity 0.473
Positive Predictive Value 0.948
Nr of predictions 24

^top



2. Individual counts for PETfold_pre2.0(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
RFA_00390 0.97 0.93 1.00 14 1417 2 0 0 2 1
RFA_00416 0.97 0.93 1.00 14 1471 2 0 0 2 1
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00658 0.00 0.00 0.00 0 1126 3 0 2 1 14
RFA_00664 -0.01 0.00 0.00 0 986 4 0 4 0 14
RFA_00749 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00764 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
SPR_00277 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_00721 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_01157 1.00 1.00 1.00 21 2754 1 0 0 1 0
SRP_00020 0.58 0.33 1.00 11 5554 0 0 0 0 22
SRP_00058 0.54 0.29 1.00 10 5768 0 0 0 0 24
SRP_00084 0.45 0.21 1.00 7 5453 0 0 0 0 27
SRP_00105 0.45 0.21 1.00 7 5558 0 0 0 0 27
SRP_00134 0.41 0.28 0.62 8 5982 6 0 5 1 21
SRP_00141 0.38 0.15 1.00 6 6322 1 0 0 1 35
SRP_00182 0.92 0.90 0.94 91 45959 15 0 6 9 10
SRP_00231 0.42 0.21 0.88 7 5452 1 0 1 0 27
SRP_00273 0.52 0.28 1.00 11 6775 0 0 0 0 29
SRP_00274 0.43 0.18 1.00 7 6896 0 0 0 0 31
SRP_00338 0.44 0.19 1.00 7 5453 0 0 0 0 29
SRP_00341 0.47 0.23 1.00 7 4746 0 0 0 0 24
SRP_00357 0.44 0.19 1.00 7 5349 0 0 0 0 29

^top



Performance of PPfold(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for PPfold(seed)

Total Base Pair Counts
Total TP 155
Total TN 133098
Total FP 29
Total FP CONTRA 0
Total FP INCONS 13
Total FP COMP 16
Total FN 538
Total Scores
MCC 0.453
Average MCC ± 95% Confidence Intervals 0.289 ± 0.172
Sensitivity 0.224
Positive Predictive Value 0.923
Nr of predictions 24

^top



2. Individual counts for PPfold(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
RFA_00390 1.00 1.00 1.00 15 1416 1 0 0 1 0
RFA_00416 1.00 1.00 1.00 15 1470 1 0 0 1 0
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00658 0.38 0.14 1.00 2 1126 1 0 0 1 12
RFA_00664 0.38 0.14 1.00 2 988 1 0 0 1 12
RFA_00749 0.78 0.75 0.82 9 892 2 0 2 0 3
RFA_00764 0.78 0.75 0.82 9 892 2 0 2 0 3
RFA_00765 0.78 0.75 0.82 9 892 2 0 2 0 3
SPR_00277 0.00 0.00 0.00 0 2926 0 0 0 0 21
SPR_00721 0.00 0.00 0.00 0 2926 0 0 0 0 21
SPR_01157 0.00 0.00 0.00 0 2775 0 0 0 0 21
SRP_00020 0.00 0.00 0.00 0 5565 0 0 0 0 33
SRP_00058 0.00 0.00 0.00 0 5778 0 0 0 0 34
SRP_00084 0.00 0.00 0.00 0 5460 0 0 0 0 34
SRP_00105 0.00 0.00 0.00 0 5565 0 0 0 0 34
SRP_00134 0.00 0.00 0.00 0 5995 0 0 0 0 29
SRP_00141 0.00 0.00 0.00 0 6328 0 0 0 0 41
SRP_00182 0.85 0.78 0.92 79 45970 17 0 7 10 22
SRP_00231 0.00 0.00 0.00 0 5460 0 0 0 0 34
SRP_00273 0.00 0.00 0.00 0 6786 0 0 0 0 40
SRP_00274 0.00 0.00 0.00 0 6903 0 0 0 0 38
SRP_00338 0.00 0.00 0.00 0 5460 0 0 0 0 36
SRP_00341 0.00 0.00 0.00 0 4753 0 0 0 0 31
SRP_00357 0.00 0.00 0.00 0 5356 0 0 0 0 36

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.