CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PknotsRG - scored higher in this pairwise comparison

  4. Performance of RNAwolf - scored lower in this pairwise comparison

  5. Compile and download dataset for PknotsRG & RNAwolf [.zip] - may take several seconds...


Overview

Metric PknotsRG RNAwolf
MCC 0.492 > 0.332
Average MCC ± 95% Confidence Intervals 0.596 ± 0.119 > 0.427 ± 0.129
Sensitivity 0.531 > 0.344
Positive Predictive Value 0.458 > 0.323
Total TP 429 > 278
Total TN 456199 < 456276
Total FP 604 < 652
Total FP CONTRA 151 < 172
Total FP INCONS 357 < 410
Total FP COMP 96 > 70
Total FN 379 < 530
P-value 5.1503931209e-08

^top




Performance plots


  1. Comparison of performance of PknotsRG and RNAwolf. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PknotsRG and RNAwolf).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PknotsRG and RNAwolf).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PknotsRG and RNAwolf. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PknotsRG and RNAwolf).

^top





Performance of PknotsRG - scored higher in this pairwise comparison

1. Total counts & total scores for PknotsRG

Total Base Pair Counts
Total TP 429
Total TN 456199
Total FP 604
Total FP CONTRA 151
Total FP INCONS 357
Total FP COMP 96
Total FN 379
Total Scores
MCC 0.492
Average MCC ± 95% Confidence Intervals 0.596 ± 0.119
Sensitivity 0.531
Positive Predictive Value 0.458
Nr of predictions 23

^top



2. Individual counts for PknotsRG [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A 0.66 0.61 0.73 11 513 4 2 2 0 7
2M58_A - 0.96 1.00 0.92 12 531 1 1 0 0 0
3J0L_a - 0.18 0.18 0.22 2 402 8 3 4 1 9
3J20_0 0.51 0.57 0.48 12 1194 14 3 10 1 9
3J3D_C 0.63 0.68 0.59 13 946 9 3 6 0 6
3J3E_8 0.26 0.33 0.21 5 2718 30 6 13 11 10
3J3F_8 0.32 0.42 0.24 8 4728 40 12 13 15 11
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.73 0.72 0.75 21 1505 7 4 3 0 8
3U4M_B - 0.49 0.55 0.46 12 1250 14 3 11 0 10
3W1K_J 0.97 0.97 0.97 30 1647 1 1 0 0 1
3W3S_B 0.64 0.64 0.66 21 1957 12 4 7 1 12
3ZEX_B - 0.36 0.40 0.33 140 420927 323 78 207 38 206
3ZEX_C 0.30 0.34 0.26 10 5336 44 5 23 16 19
4A1C_2 0.18 0.25 0.14 5 4480 43 11 20 12 15
4AOB_A 0.60 0.59 0.63 17 1410 11 4 6 1 12
4ATO_G - 0.88 1.00 0.78 7 211 2 2 0 0 0
4ENB_A 1.00 1.00 1.00 15 457 0 0 0 0 0
4ENC_A 0.97 1.00 0.94 15 480 1 1 0 0 0
4FRG_B 0.36 0.38 0.38 9 1178 15 2 13 0 15
4FRN_A 0.59 0.57 0.62 16 1822 10 2 8 0 12
4JF2_A 0.88 0.92 0.85 22 1056 4 4 0 0 2
4JRC_A - 0.34 0.35 0.35 6 605 11 0 11 0 11

^top



Performance of RNAwolf - scored lower in this pairwise comparison

1. Total counts & total scores for RNAwolf

Total Base Pair Counts
Total TP 278
Total TN 456276
Total FP 652
Total FP CONTRA 172
Total FP INCONS 410
Total FP COMP 70
Total FN 530
Total Scores
MCC 0.332
Average MCC ± 95% Confidence Intervals 0.427 ± 0.129
Sensitivity 0.344
Positive Predictive Value 0.323
Nr of predictions 23

^top



2. Individual counts for RNAwolf [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A 0.52 0.50 0.56 9 512 7 0 7 0 9
2M58_A - 0.60 0.58 0.64 7 533 4 1 3 0 5
3J0L_a - 0.14 0.18 0.14 2 397 13 4 8 1 9
3J20_0 0.54 0.57 0.52 12 1196 12 2 9 1 9
3J3D_C 0.92 0.95 0.90 18 948 3 2 0 1 1
3J3E_8 -0.01 0.00 0.00 0 2719 37 6 17 14 15
3J3F_8 0.28 0.37 0.23 7 4730 39 9 15 15 12
3RKF_A 0.89 0.83 0.95 20 845 1 0 1 0 4
3SD1_A 0.58 0.59 0.59 17 1504 12 2 10 0 12
3U4M_B - 0.50 0.50 0.52 11 1255 12 1 9 2 11
3W1K_J 0.81 0.77 0.86 24 1650 5 0 4 1 7
3W3S_B 0.70 0.70 0.72 23 1957 10 0 9 1 10
3ZEX_B - 0.18 0.19 0.17 65 420964 343 98 225 20 281
3ZEX_C 0.08 0.10 0.07 3 5328 43 18 25 0 26
4A1C_2 0.12 0.15 0.10 3 4487 38 12 14 12 17
4AOB_A 0.26 0.24 0.30 7 1414 17 2 14 1 22
4ATO_G - -0.03 0.00 0.00 0 213 7 4 3 0 7
4ENB_A 0.45 0.40 0.55 6 461 5 1 4 0 9
4ENC_A 0.34 0.33 0.38 5 483 9 0 8 1 10
4FRG_B 0.54 0.50 0.60 12 1182 8 3 5 0 12
4FRN_A -0.01 0.00 0.00 0 1828 20 1 19 0 28
4JF2_A 0.72 0.67 0.80 16 1062 4 4 0 0 8
4JRC_A - 0.71 0.65 0.79 11 608 3 2 1 0 6

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.