CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MXScarna(seed) - scored higher in this pairwise comparison

  4. Performance of IPknot - scored lower in this pairwise comparison

  5. Compile and download dataset for MXScarna(seed) & IPknot [.zip] - may take several seconds...


Overview

Metric MXScarna(seed) IPknot
MCC 0.781 > 0.745
Average MCC ± 95% Confidence Intervals 0.706 ± 0.119 < 0.730 ± 0.095
Sensitivity 0.745 > 0.717
Positive Predictive Value 0.819 > 0.775
Total TP 655 > 630
Total TN 456348 > 456335
Total FP 287 > 281
Total FP CONTRA 35 < 37
Total FP INCONS 110 < 146
Total FP COMP 142 > 98
Total FN 224 < 249
P-value 2.00017354801e-07

^top




Performance plots


  1. Comparison of performance of MXScarna(seed) and IPknot. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MXScarna(seed) and IPknot).

  2. Comparison of performance of MXScarna(seed) and IPknot. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MXScarna(seed) and IPknot).

  3. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MXScarna(seed) and IPknot).

  4. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MXScarna(seed) and IPknot).

  5. Comparison of average Matthews Correlation Coefficients (MCCs) for MXScarna(seed) and IPknot. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MXScarna(seed) and IPknot).

  6. Comparison of average Matthews Correlation Coefficients (MCCs) for MXScarna(seed) and IPknot. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MXScarna(seed) and IPknot).

^top





Performance of MXScarna(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for MXScarna(seed)

Total Base Pair Counts
Total TP 655
Total TN 456348
Total FP 287
Total FP CONTRA 35
Total FP INCONS 110
Total FP COMP 142
Total FN 224
Total Scores
MCC 0.781
Average MCC ± 95% Confidence Intervals 0.706 ± 0.119
Sensitivity 0.745
Positive Predictive Value 0.819
Nr of predictions 20

^top



2. Individual counts for MXScarna(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A -0.03 0.00 0.00 0 518 12 0 10 2 18
3AMU_B 0.97 0.95 1.00 18 1139 2 0 0 2 1
3J16_L 0.98 0.95 1.00 20 1139 0 0 0 0 1
3J20_2 0.83 0.82 0.84 337 421969 147 14 48 85 75
3J20_0 0.90 0.86 0.95 18 1200 1 1 0 0 3
3J20_1 0.97 0.95 1.00 19 1093 1 0 0 1 1
3J2L_3 0.79 0.76 0.81 26 2988 11 1 5 5 8
3RKF_A 0.86 0.75 1.00 18 848 0 0 0 0 6
3SD1_A 0.74 0.76 0.73 22 1503 9 4 4 1 7
3UZL_B 0.83 0.75 0.92 12 1280 8 0 1 7 4
3W3S_B 0.74 0.70 0.79 23 1960 7 0 6 1 10
3ZEX_C 0.39 0.31 0.50 9 5356 21 3 6 12 20
3ZEX_D 0.87 0.86 0.88 30 2762 11 0 4 7 5
4A1C_3 0.92 0.92 0.92 34 2726 6 0 3 3 3
4A1C_2 0.45 0.40 0.50 8 4500 20 4 4 12 12
4AOB_A 0.71 0.69 0.74 20 1410 10 2 5 3 9
4ENB_A 0.54 0.40 0.75 6 464 2 0 2 0 9
4ENC_A 0.48 0.40 0.60 6 486 4 1 3 0 9
4FRG_B 0.49 0.42 0.59 10 1185 8 2 5 1 14
4FRN_A 0.70 0.68 0.73 19 1822 7 3 4 0 9

^top



Performance of IPknot - scored lower in this pairwise comparison

1. Total counts & total scores for IPknot

Total Base Pair Counts
Total TP 630
Total TN 456335
Total FP 281
Total FP CONTRA 37
Total FP INCONS 146
Total FP COMP 98
Total FN 249
Total Scores
MCC 0.745
Average MCC ± 95% Confidence Intervals 0.730 ± 0.095
Sensitivity 0.717
Positive Predictive Value 0.775
Nr of predictions 20

^top



2. Individual counts for IPknot [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
2LC8_A 0.41 0.33 0.55 6 517 5 1 4 0 12
3AMU_B 0.81 0.79 0.83 15 1139 6 0 3 3 4
3J16_L 0.90 0.81 1.00 17 1142 0 0 0 0 4
3J20_2 0.75 0.74 0.76 305 421966 159 17 80 62 107
3J20_0 0.54 0.57 0.52 12 1196 12 3 8 1 9
3J20_1 1.00 1.00 1.00 20 1092 1 0 0 1 0
3J2L_3 0.82 0.82 0.82 28 2986 10 0 6 4 6
3RKF_A 0.91 0.83 1.00 20 846 0 0 0 0 4
3SD1_A 0.74 0.69 0.80 20 1508 5 0 5 0 9
3UZL_B 0.93 0.88 1.00 14 1279 7 0 0 7 2
3W3S_B 0.94 0.91 0.97 30 1958 2 0 1 1 3
3ZEX_C 0.51 0.34 0.77 10 5361 7 1 2 4 19
3ZEX_D 0.81 0.80 0.82 28 2762 10 0 6 4 7
4A1C_3 0.83 0.81 0.86 30 2728 7 0 5 2 7
4A1C_2 0.23 0.25 0.22 5 4493 26 8 10 8 15
4AOB_A 0.50 0.48 0.54 14 1411 13 3 9 1 15
4ENB_A 0.85 0.73 1.00 11 461 0 0 0 0 4
4ENC_A 0.59 0.53 0.67 8 484 4 0 4 0 7
4FRG_B 0.75 0.71 0.81 17 1181 4 3 1 0 7
4FRN_A 0.79 0.71 0.87 20 1825 3 1 2 0 8

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.