CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CMfinder(seed) - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for CMfinder(seed) & NanoFolder [.zip] - may take several seconds...


Overview

Metric CMfinder(seed) NanoFolder
MCC 0.575 > 0.392
Average MCC ± 95% Confidence Intervals 0.552 ± 0.150 > 0.377 ± 0.178
Sensitivity 0.432 < 0.436
Positive Predictive Value 0.771 > 0.362
Total TP 101 < 102
Total TN 29933 > 29782
Total FP 43 < 215
Total FP CONTRA 0 < 38
Total FP INCONS 30 < 142
Total FP COMP 13 < 35
Total FN 133 > 132
P-value 1.82627677697e-08

^top




Performance plots


  1. Comparison of performance of CMfinder(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CMfinder(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).

^top





Performance of CMfinder(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 101
Total TN 29933
Total FP 43
Total FP CONTRA 0
Total FP INCONS 30
Total FP COMP 13
Total FN 133
Total Scores
MCC 0.575
Average MCC ± 95% Confidence Intervals 0.552 ± 0.150
Sensitivity 0.432
Positive Predictive Value 0.771
Nr of predictions 15

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 102
Total TN 29782
Total FP 215
Total FP CONTRA 38
Total FP INCONS 142
Total FP COMP 35
Total FN 132
Total Scores
MCC 0.392
Average MCC ± 95% Confidence Intervals 0.377 ± 0.178
Sensitivity 0.436
Positive Predictive Value 0.362
Nr of predictions 15

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_01051 -0.02 0.00 0.00 0 885 18 1 17 0 13
RFA_00604 0.42 0.54 0.33 19 13472 53 14 25 14 16
RFA_00664 -0.01 0.00 0.00 0 978 13 0 12 1 14
RFA_00674 0.12 0.14 0.13 2 1112 15 0 14 1 12
RFA_00675 0.28 0.29 0.29 4 976 10 1 9 0 10
RFA_00677 0.62 0.64 0.60 9 975 10 0 6 4 5
RFA_00684 0.74 0.71 0.77 10 977 5 2 1 2 4
RFA_00716 -0.02 0.00 0.00 0 929 17 0 17 0 14
RFA_00717 0.64 0.64 0.64 9 889 5 2 3 0 5
RFA_00730 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00763 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00765 0.39 0.42 0.38 5 890 9 2 6 1 7
RFA_00770 -0.01 0.00 0.00 0 1996 24 0 20 4 18
RFA_00773 0.93 1.00 0.86 18 1932 9 3 0 6 0
RFA_00808 0.83 1.00 0.70 16 1993 7 7 0 0 0

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.