CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of MXScarna(seed) - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for MXScarna(seed) & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric MXScarna(seed) Murlet(seed)
MCC 0.731 > 0.540
Average MCC ± 95% Confidence Intervals 0.725 ± 0.037 > 0.396 ± 0.065
Sensitivity 0.700 > 0.360
Positive Predictive Value 0.763 < 0.811
Total TP 3382 > 1738
Total TN 2787206 < 2789495
Total FP 1740 > 489
Total FP CONTRA 155 > 21
Total FP INCONS 896 > 385
Total FP COMP 689 > 83
Total FN 1446 < 3090
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of MXScarna(seed) and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for MXScarna(seed) and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for MXScarna(seed) and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for MXScarna(seed) and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for MXScarna(seed) and Murlet(seed)).

^top





Performance of MXScarna(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for MXScarna(seed)

Total Base Pair Counts
Total TP 3382
Total TN 2787206
Total FP 1740
Total FP CONTRA 155
Total FP INCONS 896
Total FP COMP 689
Total FN 1446
Total Scores
MCC 0.731
Average MCC ± 95% Confidence Intervals 0.725 ± 0.037
Sensitivity 0.700
Positive Predictive Value 0.763
Nr of predictions 123

^top



2. Individual counts for MXScarna(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.73 0.68 0.78 294 977526 128 10 71 47 140
PDB_00005 0.59 0.36 1.00 5 941 0 0 0 0 9
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.81 0.73 0.91 30 5017 3 0 3 0 11
PDB_00553 0.90 0.82 1.00 9 456 0 0 0 0 2
PDB_00716 -0.01 0.00 0.00 0 2681 21 1 19 1 23
PDB_00810 0.77 0.65 0.92 11 1069 2 0 1 1 6
PDB_01050 0.78 0.62 1.00 8 622 2 0 0 2 5
PDB_01051 0.64 0.54 0.78 7 894 5 0 2 3 6
PDB_01092 0.70 0.63 0.79 33 10111 12 2 7 3 19
PDB_01152 0.85 0.79 0.92 11 549 1 0 1 0 3
RFA_00390 0.97 0.93 1.00 14 1417 1 0 0 1 1
RFA_00416 0.97 0.93 1.00 14 1471 1 0 0 1 1
RFA_00433 1.00 1.00 1.00 15 1416 1 0 0 1 0
RFA_00587 1.00 1.00 1.00 15 4836 19 0 0 19 0
RFA_00603 0.74 0.74 0.74 26 13495 17 3 6 8 9
RFA_00604 0.72 0.63 0.81 22 13503 13 3 2 8 13
RFA_00605 0.26 0.23 0.30 8 15198 22 5 14 3 27
RFA_00606 0.53 0.51 0.56 20 21285 25 4 12 9 19
RFA_00607 0.77 0.72 0.81 26 17734 16 1 5 10 10
RFA_00609 0.67 0.65 0.69 24 17920 19 2 9 8 13
RFA_00610 0.76 0.76 0.76 28 17168 19 1 8 10 9
RFA_00611 0.75 0.74 0.76 26 13007 13 4 4 5 9
RFA_00613 0.76 0.71 0.81 25 12849 11 4 2 5 10
RFA_00615 0.75 0.74 0.76 26 13332 15 4 4 7 9
RFA_00620 0.20 0.18 0.23 7 21915 33 7 16 10 32
RFA_00626 0.89 0.87 0.92 76 56533 29 2 5 22 11
RFA_00627 0.83 0.79 0.87 69 56874 36 1 9 26 18
RFA_00628 0.85 0.81 0.89 70 57212 38 2 7 29 16
RFA_00630 0.82 0.78 0.86 68 56874 33 2 9 22 19
RFA_00632 0.25 0.25 0.27 7 4069 19 1 18 0 21
RFA_00636 0.35 0.36 0.36 10 3977 18 2 16 0 18
RFA_00639 0.31 0.31 0.31 27 54527 69 10 51 8 60
RFA_00642 0.56 0.50 0.64 9 2912 7 0 5 2 9
RFA_00643 0.69 0.61 0.79 11 2197 5 0 3 2 7
RFA_00644 0.69 0.61 0.79 11 2687 5 0 3 2 7
RFA_00645 0.30 0.28 0.33 5 2400 11 0 10 1 13
RFA_00649 0.73 0.67 0.80 12 2130 5 0 3 2 6
RFA_00651 0.73 0.67 0.80 12 2065 5 0 3 2 6
RFA_00653 0.67 0.61 0.73 11 2130 6 0 4 2 7
RFA_00654 0.73 0.67 0.80 12 2400 5 0 3 2 6
RFA_00658 0.61 0.57 0.67 8 1116 5 0 4 1 6
RFA_00659 0.46 0.43 0.50 6 1116 7 0 6 1 8
RFA_00664 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00667 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00668 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00672 0.72 0.69 0.75 9 891 4 0 3 1 4
RFA_00673 0.61 0.57 0.67 8 1116 5 0 4 1 6
RFA_00674 0.69 0.64 0.75 9 1116 4 0 3 1 5
RFA_00675 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00677 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00678 0.66 0.64 0.69 9 933 5 0 4 1 5
RFA_00680 0.61 0.57 0.67 8 1116 5 0 4 1 6
RFA_00684 0.66 0.64 0.69 9 977 5 0 4 1 5
RFA_00685 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00695 0.62 0.57 0.67 8 7009 22 1 3 18 6
RFA_00703 0.62 0.57 0.67 8 4266 8 1 3 4 6
RFA_00704 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00705 0.66 0.64 0.69 9 1022 5 0 4 1 5
RFA_00706 0.69 0.64 0.75 9 1023 4 0 3 1 5
RFA_00707 0.69 0.64 0.75 9 1023 4 0 3 1 5
RFA_00708 0.64 0.57 0.73 8 1024 4 0 3 1 6
RFA_00709 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00710 0.69 0.64 0.75 9 978 4 0 3 1 5
RFA_00711 0.69 0.64 0.75 9 1023 4 0 3 1 5
RFA_00715 0.56 0.50 0.64 7 935 5 0 4 1 7
RFA_00716 0.48 0.43 0.55 6 935 6 0 5 1 8
RFA_00717 0.69 0.64 0.75 9 891 3 0 3 0 5
RFA_00730 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00731 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00733 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00734 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00736 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00737 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00745 1.00 1.00 1.00 12 934 1 0 0 1 0
RFA_00749 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00758 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00762 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00763 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00764 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00767 0.59 0.56 0.63 10 1875 6 2 4 0 8
RFA_00768 0.59 0.56 0.63 10 1875 6 2 4 0 8
RFA_00769 0.55 0.56 0.56 10 1935 8 3 5 0 8
RFA_00770 0.57 0.56 0.59 10 1999 7 2 5 0 8
RFA_00773 0.55 0.56 0.56 10 1935 8 3 5 0 8
RFA_00779 0.57 0.56 0.59 10 1936 7 2 5 0 8
RFA_00781 0.92 0.91 0.94 29 5019 3 2 0 1 3
RFA_00786 0.92 0.91 0.94 29 5019 3 2 0 1 3
RFA_00791 0.87 0.88 0.88 28 5119 5 2 2 1 4
RFA_00792 0.92 0.91 0.94 29 5019 3 2 0 1 3
RFA_00801 0.91 0.91 0.91 29 5018 4 2 1 1 3
RFA_00808 -0.01 0.00 0.00 0 2000 19 2 14 3 16
RFA_00809 0.30 0.31 0.29 5 2128 12 1 11 0 11
RFA_00814 0.95 0.93 0.97 38 25161 22 0 1 21 3
RFA_00815 0.96 0.93 1.00 38 24493 15 0 0 15 3
RFA_00816 0.96 0.93 1.00 38 23182 15 0 0 15 3
RFA_00817 0.94 0.88 1.00 36 21909 12 0 0 12 5
RFA_00818 0.64 0.68 0.61 28 20255 30 11 7 12 13
RFA_00819 0.94 0.90 0.97 37 27928 26 0 1 25 4
RFA_00820 0.88 0.88 0.87 141 151363 51 0 21 30 19
RFA_00830 0.61 0.59 0.63 91 152484 92 2 51 39 63
SRP_00079 0.68 0.66 0.71 58 41823 39 0 24 15 30
SRP_00124 0.60 0.55 0.64 47 37055 30 6 20 4 38
SRP_00137 0.60 0.64 0.57 16 4158 13 3 9 1 9
SRP_00141 0.81 0.73 0.91 30 6295 5 1 2 2 11
SRP_00146 0.82 0.75 0.90 27 5223 5 1 2 2 9
SRP_00182 0.84 0.83 0.85 84 45957 25 0 15 10 17
SRP_00197 0.92 0.90 0.94 91 49358 24 0 6 18 10
SRP_00198 0.90 0.87 0.92 90 50942 24 0 8 16 13
SRP_00199 0.89 0.88 0.91 92 50939 21 0 9 12 13
SRP_00202 0.89 0.85 0.93 100 54178 16 0 7 9 17
SRP_00241 0.50 0.51 0.49 42 45970 54 8 36 10 40
SRP_00255 0.74 0.76 0.73 71 47489 40 1 25 14 23
SRP_00260 0.50 0.47 0.53 47 47806 44 6 36 2 53
SRP_00278 0.73 0.69 0.77 72 45963 26 0 21 5 33
SRP_00285 0.86 0.80 0.92 24 3715 3 0 2 1 6
SRP_00322 0.79 0.75 0.83 82 48417 26 0 17 9 27
SRP_00328 0.64 0.60 0.69 51 39547 31 4 19 8 34
SRP_00329 0.63 0.60 0.65 51 39825 35 4 23 8 34
SRP_00338 0.75 0.72 0.79 26 5427 8 0 7 1 10
SRP_00339 0.78 0.78 0.78 73 44458 27 4 16 7 21
SRP_00340 0.58 0.56 0.60 46 41251 43 4 27 12 36

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1738
Total TN 2789495
Total FP 489
Total FP CONTRA 21
Total FP INCONS 385
Total FP COMP 83
Total FN 3090
Total Scores
MCC 0.540
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065
Sensitivity 0.360
Positive Predictive Value 0.811
Nr of predictions 123

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.