CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of ContextFold - scored higher in this pairwise comparison

  4. Performance of RSpredict(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for ContextFold & RSpredict(seed) [.zip] - may take several seconds...


Overview

Metric ContextFold RSpredict(seed)
MCC 0.800 > 0.480
Average MCC ± 95% Confidence Intervals 0.709 ± 0.115 > 0.123 ± 0.092
Sensitivity 0.779 > 0.353
Positive Predictive Value 0.823 > 0.655
Total TP 774 > 351
Total TN 470748 < 471153
Total FP 291 > 253
Total FP CONTRA 67 > 34
Total FP INCONS 100 < 151
Total FP COMP 124 > 68
Total FN 219 < 642
P-value 5.19332990918e-08

^top




Performance plots


  1. Comparison of performance of ContextFold and RSpredict(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for ContextFold and RSpredict(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for ContextFold and RSpredict(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for ContextFold and RSpredict(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for ContextFold and RSpredict(seed)).

^top





Performance of ContextFold - scored higher in this pairwise comparison

1. Total counts & total scores for ContextFold

Total Base Pair Counts
Total TP 774
Total TN 470748
Total FP 291
Total FP CONTRA 67
Total FP INCONS 100
Total FP COMP 124
Total FN 219
Total Scores
MCC 0.800
Average MCC ± 95% Confidence Intervals 0.709 ± 0.115
Sensitivity 0.779
Positive Predictive Value 0.823
Nr of predictions 25

^top



2. Individual counts for ContextFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A 0.58 0.56 0.63 10 512 6 0 6 0 8
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3J20_0 0.95 0.95 0.95 20 1198 2 1 0 1 1
3J20_1 1.00 1.00 1.00 20 1092 1 0 0 1 0
3J20_2 0.89 0.88 0.89 362 421963 88 15 28 45 50
3J2L_3 0.95 0.91 1.00 31 2989 4 0 0 4 3
3J3D_C 0.79 0.79 0.79 15 949 4 3 1 0 4
3J3E_7 0.94 0.88 1.00 30 2711 2 0 0 2 4
3J3E_8 -0.01 0.00 0.00 0 2722 29 12 8 9 15
3J3F_8 0.36 0.42 0.31 8 4735 32 9 9 14 11
3J3F_7 0.96 0.92 1.00 33 2901 2 0 0 2 3
3UZL_B 0.93 0.88 1.00 14 1279 7 0 0 7 2
3W1K_J 0.90 0.87 0.93 27 1649 2 1 1 0 4
3W3S_B 0.87 0.85 0.90 28 1958 4 0 3 1 5
3ZEX_D 0.92 0.86 1.00 30 2766 4 0 0 4 5
3ZEX_C 0.44 0.41 0.48 12 5349 23 4 9 10 17
3ZND_W 0.26 0.38 0.19 3 1175 22 6 7 9 5
4A1C_2 0.26 0.25 0.28 5 4498 26 3 10 13 15
4A1C_3 0.96 0.92 1.00 34 2729 1 0 0 1 3
4AOB_A 0.62 0.59 0.68 17 1412 9 2 6 1 12
4ENB_A 0.77 0.60 1.00 9 463 0 0 0 0 6
4ENC_A 0.73 0.60 0.90 9 486 1 1 0 0 6
4FRG_B 0.77 0.71 0.85 17 1182 3 2 1 0 7
4FRN_A 0.42 0.39 0.46 11 1824 13 6 7 0 17
4JF2_A 0.57 0.50 0.67 12 1064 6 2 4 0 12

^top



Performance of RSpredict(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for RSpredict(seed)

Total Base Pair Counts
Total TP 351
Total TN 471153
Total FP 253
Total FP CONTRA 34
Total FP INCONS 151
Total FP COMP 68
Total FN 642
Total Scores
MCC 0.480
Average MCC ± 95% Confidence Intervals 0.123 ± 0.092
Sensitivity 0.353
Positive Predictive Value 0.655
Nr of predictions 25

^top



2. Individual counts for RSpredict(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A -0.03 0.00 0.00 0 515 13 0 13 0 18
3J16_L 0.00 0.00 0.00 0 1158 1 0 1 0 21
3J20_0 -0.01 0.00 0.00 0 1215 4 0 4 0 21
3J20_1 -0.01 0.00 0.00 0 1108 5 0 4 1 20
3J20_2 0.77 0.76 0.78 312 421966 152 18 72 62 100
3J2L_3 0.00 0.00 0.00 0 3015 5 0 5 0 34
3J3D_C -0.01 0.00 0.00 0 964 4 1 3 0 19
3J3E_7 -0.01 0.00 0.00 0 2733 8 1 7 0 34
3J3E_8 0.00 0.00 0.00 0 2742 0 0 0 0 15
3J3F_8 0.19 0.11 0.33 2 4755 5 3 1 1 17
3J3F_7 0.00 0.00 0.00 0 2930 4 1 3 0 36
3UZL_B -0.01 0.00 0.00 0 1287 6 3 3 0 16
3W1K_J 0.57 0.35 0.92 11 1666 1 0 1 0 20
3W3S_B 0.45 0.36 0.57 12 1968 10 1 8 1 21
3ZEX_D 0.00 0.00 0.00 0 2794 2 0 2 0 35
3ZEX_C 0.13 0.07 0.25 2 5366 6 2 4 0 27
3ZND_W 0.00 0.00 0.00 0 1189 4 0 2 2 8
4A1C_2 0.00 0.00 0.00 0 4512 5 2 2 1 20
4A1C_3 0.00 0.00 0.00 0 2759 4 2 2 0 37
4AOB_A 0.42 0.21 0.86 6 1430 1 0 1 0 23
4ENB_A 0.34 0.20 0.60 3 467 2 0 2 0 12
4ENC_A 0.34 0.20 0.60 3 491 2 0 2 0 12
4FRG_B -0.01 0.00 0.00 0 1200 2 0 2 0 24
4FRN_A 0.00 0.00 0.00 0 1845 3 0 3 0 28
4JF2_A -0.01 0.00 0.00 0 1078 4 0 4 0 24

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.