CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of ProbKnot - scored higher in this pairwise comparison

  4. Performance of Carnac(20) - scored lower in this pairwise comparison

  5. Compile and download dataset for ProbKnot & Carnac(20) [.zip] - may take several seconds...


Overview

Metric ProbKnot Carnac(20)
MCC 0.578 > 0.557
Average MCC ± 95% Confidence Intervals 0.590 ± 0.087 > 0.497 ± 0.108
Sensitivity 0.516 > 0.348
Positive Predictive Value 0.653 < 0.896
Total TP 513 > 346
Total TN 203485 < 203885
Total FP 334 > 52
Total FP CONTRA 36 > 2
Total FP INCONS 237 > 38
Total FP COMP 61 > 12
Total FN 482 < 649
P-value 6.48688953717e-08

^top




Performance plots


  1. Comparison of performance of ProbKnot and Carnac(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for ProbKnot and Carnac(20)).

  2. Comparison of performance of ProbKnot and Carnac(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for ProbKnot and Carnac(20)).

  3. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for ProbKnot and Carnac(20)).

  4. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for ProbKnot and Carnac(20)).

  5. Comparison of average Matthews Correlation Coefficients (MCCs) for ProbKnot and Carnac(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for ProbKnot and Carnac(20)).

  6. Comparison of average Matthews Correlation Coefficients (MCCs) for ProbKnot and Carnac(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for ProbKnot and Carnac(20)).

^top





Performance of ProbKnot - scored higher in this pairwise comparison

1. Total counts & total scores for ProbKnot

Total Base Pair Counts
Total TP 513
Total TN 203485
Total FP 334
Total FP CONTRA 36
Total FP INCONS 237
Total FP COMP 61
Total FN 482
Total Scores
MCC 0.578
Average MCC ± 95% Confidence Intervals 0.590 ± 0.087
Sensitivity 0.516
Positive Predictive Value 0.653
Nr of predictions 24

^top



2. Individual counts for ProbKnot [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2L94_A 0.97 0.95 1.00 19 971 0 0 0 0 1
2XKV_B 0.51 0.50 0.53 10 4541 24 0 9 15 10
2XQD_Y 0.90 0.81 1.00 22 2828 1 0 0 1 5
2XXA_G 0.26 0.24 0.29 10 5116 25 1 24 0 32
3AMU_B 0.65 0.59 0.73 16 2981 8 0 6 2 11
3IZ4_A 0.52 0.46 0.60 61 70774 46 6 35 5 71
3IZF_C 0.72 0.61 0.85 33 6864 6 0 6 0 21
3J20_0 0.45 0.40 0.52 12 2827 12 0 11 1 18
3J20_1 0.71 0.70 0.73 16 2904 7 0 6 1 7
3J2L_3 0.60 0.49 0.74 26 7840 11 0 9 2 27
3NPB_A 0.72 0.61 0.85 28 6988 8 1 4 3 18
3O58_3 0.31 0.34 0.29 12 12362 41 4 25 12 23
3O58_2 0.76 0.76 0.76 29 7222 10 3 6 1 9
3PDR_A 0.74 0.64 0.85 46 12826 10 1 7 2 26
3RKF_A 0.73 0.59 0.91 20 2189 2 1 1 0 14
3SD1_A 0.55 0.48 0.65 20 3885 11 2 9 0 22
3ZEX_C 0.38 0.31 0.47 16 14162 21 2 16 3 36
3ZEX_D 0.77 0.67 0.89 33 6984 4 0 4 0 16
4A1C_3 0.73 0.61 0.87 33 7102 6 1 4 1 21
4A1C_2 0.13 0.15 0.12 5 11738 49 7 31 11 28
4AOB_A 0.50 0.40 0.63 17 4344 11 2 8 1 25
4ENB_A 0.73 0.58 0.92 11 1263 1 1 0 0 8
4ENC_A 0.45 0.42 0.50 8 1310 8 1 7 0 11
4FRG_B 0.37 0.31 0.45 10 3464 12 3 9 0 22

^top



Performance of Carnac(20) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(20)

Total Base Pair Counts
Total TP 346
Total TN 203885
Total FP 52
Total FP CONTRA 2
Total FP INCONS 38
Total FP COMP 12
Total FN 649
Total Scores
MCC 0.557
Average MCC ± 95% Confidence Intervals 0.497 ± 0.108
Sensitivity 0.348
Positive Predictive Value 0.896
Nr of predictions 24

^top



2. Individual counts for Carnac(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2L94_A 0.00 0.00 0.00 0 990 0 0 0 0 20
2XKV_B 0.32 0.10 1.00 2 4558 1 0 0 1 18
2XQD_Y 0.79 0.63 1.00 17 2833 0 0 0 0 10
2XXA_G 0.27 0.07 1.00 3 5148 0 0 0 0 39
3AMU_B 0.69 0.59 0.80 16 2983 6 0 4 2 11
3IZ4_A 0.47 0.23 0.94 31 70843 2 0 2 0 101
3IZF_C 0.73 0.56 0.97 30 6872 1 0 1 0 24
3J20_0 0.62 0.53 0.73 16 2828 7 0 6 1 14
3J20_1 0.86 0.74 1.00 17 2909 0 0 0 0 6
3J2L_3 0.46 0.34 0.64 18 7847 11 0 10 1 35
3NPB_A 0.44 0.20 1.00 9 7012 2 0 0 2 37
3O58_3 0.51 0.26 1.00 9 12394 0 0 0 0 26
3O58_2 0.83 0.71 0.96 27 7232 2 0 1 1 11
3PDR_A 0.62 0.40 0.97 29 12850 3 0 1 2 43
3RKF_A 0.76 0.59 1.00 20 2191 0 0 0 0 14
3SD1_A 0.60 0.38 0.94 16 3899 1 0 1 0 26
3ZEX_C 0.31 0.13 0.70 7 14186 3 1 2 0 45
3ZEX_D 0.73 0.53 1.00 26 6995 0 0 0 0 23
4A1C_3 0.67 0.52 0.88 28 7108 4 0 4 0 26
4A1C_2 0.33 0.15 0.71 5 11774 3 0 2 1 28
4AOB_A 0.49 0.33 0.74 14 4352 6 1 4 1 28
4ENB_A 0.00 0.00 0.00 0 1275 0 0 0 0 19
4ENC_A 0.00 0.00 0.00 0 1326 0 0 0 0 19
4FRG_B 0.43 0.19 1.00 6 3480 0 0 0 0 26

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.