CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Sfold - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for Sfold & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric Sfold Murlet(seed)
MCC 0.613 > 0.540
Average MCC ± 95% Confidence Intervals 0.572 ± 0.055 > 0.396 ± 0.065
Sensitivity 0.572 > 0.360
Positive Predictive Value 0.658 < 0.811
Total TP 2761 > 1738
Total TN 2787445 < 2789495
Total FP 1821 > 489
Total FP CONTRA 233 > 21
Total FP INCONS 1200 > 385
Total FP COMP 388 > 83
Total FN 2067 < 3090
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Sfold and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Sfold and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Sfold and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Sfold and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Sfold and Murlet(seed)).

^top





Performance of Sfold - scored higher in this pairwise comparison

1. Total counts & total scores for Sfold

Total Base Pair Counts
Total TP 2761
Total TN 2787445
Total FP 1821
Total FP CONTRA 233
Total FP INCONS 1200
Total FP COMP 388
Total FN 2067
Total Scores
MCC 0.613
Average MCC ± 95% Confidence Intervals 0.572 ± 0.055
Sensitivity 0.572
Positive Predictive Value 0.658
Nr of predictions 123

^top



2. Individual counts for Sfold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.37 0.34 0.41 147 977541 224 31 182 11 287
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.94 0.88 1.00 36 5014 0 0 0 0 5
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2678 24 1 22 1 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_01050 0.96 0.92 1.00 12 618 2 0 0 2 1
PDB_01051 0.88 0.85 0.92 11 891 5 0 1 4 2
PDB_01092 0.76 0.63 0.92 33 10117 5 0 3 2 19
PDB_01152 0.96 0.93 1.00 13 548 0 0 0 0 1
RFA_00390 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00416 0.93 0.93 0.93 14 1470 4 0 1 3 1
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00587 1.00 1.00 1.00 15 4836 8 0 0 8 0
RFA_00603 0.66 0.69 0.63 24 13492 17 4 10 3 11
RFA_00604 0.71 0.74 0.68 26 13492 24 2 10 12 9
RFA_00605 0.39 0.34 0.44 12 15198 24 3 12 9 23
RFA_00606 0.65 0.59 0.72 23 21289 19 7 2 10 16
RFA_00607 0.91 0.83 1.00 30 17736 7 0 0 7 6
RFA_00609 0.78 0.78 0.78 29 17918 18 2 6 10 8
RFA_00610 0.89 0.84 0.94 31 17172 15 0 2 13 6
RFA_00611 0.81 0.71 0.93 25 13014 5 0 2 3 10
RFA_00613 0.49 0.49 0.50 17 12846 19 7 10 2 18
RFA_00615 0.63 0.57 0.69 20 13337 9 4 5 0 15
RFA_00620 0.55 0.49 0.63 19 21915 22 3 8 11 20
RFA_00626 0.89 0.84 0.95 73 56539 20 1 3 16 14
RFA_00627 0.72 0.68 0.77 59 56876 30 4 14 12 28
RFA_00628 0.90 0.86 0.95 74 57213 28 0 4 24 12
RFA_00630 0.61 0.59 0.65 51 56874 38 7 21 10 36
RFA_00632 0.24 0.25 0.25 7 4067 21 2 19 0 21
RFA_00636 0.40 0.39 0.41 11 3978 16 2 14 0 17
RFA_00639 0.27 0.24 0.30 21 54546 53 9 39 5 66
RFA_00642 0.00 0.00 0.00 0 2926 0 0 0 0 18
RFA_00643 0.24 0.22 0.27 4 2196 12 2 9 1 14
RFA_00644 -0.01 0.00 0.00 0 2689 12 0 12 0 18
RFA_00645 -0.01 0.00 0.00 0 2404 11 3 8 0 18
RFA_00649 0.37 0.33 0.43 6 2131 10 0 8 2 12
RFA_00651 0.14 0.11 0.18 2 2069 9 1 8 0 16
RFA_00653 0.34 0.33 0.35 6 2128 11 3 8 0 12
RFA_00654 -0.01 0.00 0.00 0 2401 14 2 12 0 18
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1122 8 0 6 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.22 0.21 0.25 3 978 10 1 8 1 11
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 -0.01 0.00 0.00 0 896 7 0 7 0 13
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.66 0.64 0.69 9 1115 5 0 4 1 5
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.59 0.57 0.62 8 977 6 0 5 1 6
RFA_00678 0.53 0.29 1.00 4 942 0 0 0 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.59 0.57 0.62 8 977 6 0 5 1 6
RFA_00685 0.47 0.36 0.63 5 982 4 0 3 1 9
RFA_00695 0.56 0.50 0.64 7 7010 23 1 3 19 7
RFA_00703 0.67 0.64 0.69 9 4265 16 1 3 12 5
RFA_00704 -0.01 0.00 0.00 0 981 9 0 9 0 14
RFA_00705 0.77 0.71 0.83 10 1023 2 0 2 0 4
RFA_00706 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00707 -0.01 0.00 0.00 0 1028 7 1 6 0 14
RFA_00708 0.40 0.29 0.57 4 1028 3 0 3 0 10
RFA_00709 -0.01 0.00 0.00 0 980 10 0 10 0 14
RFA_00710 -0.01 0.00 0.00 0 981 9 0 9 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 -0.01 0.00 0.00 0 937 9 0 9 0 14
RFA_00716 -0.01 0.00 0.00 0 935 11 0 11 0 14
RFA_00717 0.72 0.64 0.82 9 892 2 0 2 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00733 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00734 0.61 0.67 0.57 8 889 7 1 5 1 4
RFA_00736 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00737 0.96 0.92 1.00 11 892 1 0 0 1 1
RFA_00745 0.91 0.83 1.00 10 936 1 0 0 1 2
RFA_00749 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00758 0.76 0.58 1.00 7 896 0 0 0 0 5
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00767 0.74 0.56 1.00 10 1881 0 0 0 0 8
RFA_00768 0.48 0.44 0.53 8 1876 7 1 6 0 10
RFA_00769 0.52 0.56 0.50 10 1933 10 4 6 0 8
RFA_00770 0.68 0.56 0.83 10 2004 5 0 2 3 8
RFA_00773 0.57 0.56 0.59 10 1936 7 1 6 0 8
RFA_00779 0.68 0.56 0.83 10 1941 2 0 2 0 8
RFA_00781 0.59 0.59 0.59 19 5018 13 2 11 0 13
RFA_00786 0.27 0.22 0.33 7 5029 14 1 13 0 25
RFA_00791 0.58 0.50 0.67 16 5127 8 1 7 0 16
RFA_00792 0.95 0.94 0.97 30 5019 2 0 1 1 2
RFA_00801 0.79 0.75 0.83 24 5021 6 0 5 1 8
RFA_00808 -0.01 0.00 0.00 0 2000 19 2 14 3 16
RFA_00809 0.37 0.38 0.38 6 2129 10 1 9 0 10
RFA_00814 0.86 0.76 0.97 31 25168 10 0 1 9 10
RFA_00815 0.61 0.61 0.61 25 24490 21 6 10 5 16
RFA_00816 0.76 0.73 0.79 30 23182 12 8 0 4 11
RFA_00817 0.11 0.12 0.11 5 21899 41 14 27 0 36
RFA_00818 0.31 0.22 0.43 9 20280 13 8 4 1 32
RFA_00819 0.88 0.80 0.97 33 27932 25 0 1 24 8
RFA_00820 0.88 0.88 0.89 141 151366 30 1 17 12 19
RFA_00830 0.49 0.45 0.53 69 152498 105 6 55 44 85
SRP_00079 0.68 0.60 0.78 53 41837 20 3 12 5 35
SRP_00124 0.56 0.53 0.59 45 37052 35 4 27 4 40
SRP_00137 0.74 0.84 0.66 21 4154 11 4 7 0 4
SRP_00141 0.91 0.90 0.93 37 6288 4 1 2 1 4
SRP_00146 0.93 0.94 0.92 34 5216 3 1 2 0 2
SRP_00182 0.73 0.70 0.76 71 45963 24 3 19 2 30
SRP_00197 0.43 0.37 0.51 37 49382 36 1 35 0 64
SRP_00198 0.42 0.35 0.51 36 50970 35 2 32 1 67
SRP_00199 0.78 0.72 0.84 76 50950 15 0 14 1 29
SRP_00202 0.91 0.86 0.95 101 54179 7 0 5 2 16
SRP_00241 0.55 0.57 0.53 47 45967 44 11 31 2 35
SRP_00255 0.62 0.62 0.62 58 47492 40 3 33 4 36
SRP_00260 0.71 0.70 0.72 70 47798 29 4 23 2 30
SRP_00278 0.81 0.73 0.89 77 45969 12 0 10 2 28
SRP_00285 0.89 0.83 0.96 25 3715 1 0 1 0 5
SRP_00322 0.71 0.66 0.76 72 48421 24 4 19 1 37
SRP_00328 0.63 0.64 0.64 54 39536 34 7 24 3 31
SRP_00329 0.72 0.71 0.73 60 39821 25 3 19 3 25
SRP_00338 0.94 0.94 0.94 34 5424 2 0 2 0 2
SRP_00339 0.67 0.64 0.70 60 44465 29 5 21 3 34
SRP_00340 0.55 0.59 0.52 48 41236 46 8 36 2 34

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1738
Total TN 2789495
Total FP 489
Total FP CONTRA 21
Total FP INCONS 385
Total FP COMP 83
Total FN 3090
Total Scores
MCC 0.540
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065
Sensitivity 0.360
Positive Predictive Value 0.811
Nr of predictions 123

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.