CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Contrafold - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for Contrafold & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric Contrafold Murlet(seed)
MCC 0.637 > 0.540
Average MCC ± 95% Confidence Intervals 0.598 ± 0.054 > 0.396 ± 0.065
Sensitivity 0.619 > 0.360
Positive Predictive Value 0.657 < 0.811
Total TP 2988 > 1738
Total TN 2787092 < 2789495
Total FP 2113 > 489
Total FP CONTRA 245 > 21
Total FP INCONS 1314 > 385
Total FP COMP 554 > 83
Total FN 1840 < 3090
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Contrafold and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Contrafold and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Contrafold and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Contrafold and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Contrafold and Murlet(seed)).

^top





Performance of Contrafold - scored higher in this pairwise comparison

1. Total counts & total scores for Contrafold

Total Base Pair Counts
Total TP 2988
Total TN 2787092
Total FP 2113
Total FP CONTRA 245
Total FP INCONS 1314
Total FP COMP 554
Total FN 1840
Total Scores
MCC 0.637
Average MCC ± 95% Confidence Intervals 0.598 ± 0.054
Sensitivity 0.619
Positive Predictive Value 0.657
Nr of predictions 123

^top



2. Individual counts for Contrafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.46 0.44 0.48 189 977509 217 30 173 14 245
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.94 0.88 1.00 36 5014 0 0 0 0 5
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 0.33 0.30 0.37 7 2682 13 0 12 1 16
PDB_00810 0.77 0.71 0.86 12 1067 3 0 2 1 5
PDB_01050 0.96 0.92 1.00 12 618 2 0 0 2 1
PDB_01051 0.88 0.85 0.92 11 891 5 0 1 4 2
PDB_01092 0.70 0.65 0.76 34 10108 14 2 9 3 18
PDB_01152 0.96 0.93 1.00 13 548 0 0 0 0 1
RFA_00390 1.00 1.00 1.00 15 1416 3 0 0 3 0
RFA_00416 1.00 1.00 1.00 15 1470 3 0 0 3 0
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00587 1.00 1.00 1.00 15 4836 10 0 0 10 0
RFA_00603 0.73 0.71 0.76 25 13497 8 4 4 0 10
RFA_00604 0.71 0.74 0.68 26 13492 26 2 10 14 9
RFA_00605 0.35 0.34 0.36 12 15192 35 2 19 14 23
RFA_00606 0.53 0.51 0.54 20 21284 25 10 7 8 19
RFA_00607 0.87 0.81 0.94 29 17735 11 0 2 9 7
RFA_00609 0.76 0.78 0.74 29 17916 27 2 8 17 8
RFA_00610 0.90 0.86 0.94 32 17171 20 0 2 18 5
RFA_00611 0.82 0.74 0.90 26 13012 6 0 3 3 9
RFA_00613 0.60 0.51 0.69 18 12854 9 3 5 1 17
RFA_00615 0.63 0.57 0.69 20 13337 9 4 5 0 15
RFA_00620 0.71 0.64 0.78 25 21913 29 0 7 22 14
RFA_00626 0.91 0.87 0.94 76 56535 26 1 4 21 11
RFA_00627 0.86 0.83 0.89 72 56872 28 0 9 19 15
RFA_00628 0.91 0.88 0.94 76 57210 25 0 5 20 10
RFA_00630 0.77 0.77 0.77 67 56866 34 6 14 14 20
RFA_00632 0.38 0.39 0.37 11 4065 19 0 19 0 17
RFA_00636 0.60 0.64 0.56 18 3973 14 4 10 0 10
RFA_00639 0.60 0.59 0.61 51 54532 47 5 27 15 36
RFA_00642 0.00 0.00 0.00 0 2915 11 2 9 0 18
RFA_00643 -0.01 0.00 0.00 0 2202 9 1 8 0 18
RFA_00644 0.00 0.00 0.00 0 2692 9 1 8 0 18
RFA_00645 -0.01 0.00 0.00 0 2402 13 3 10 0 18
RFA_00649 0.26 0.22 0.31 4 2132 9 6 3 0 14
RFA_00651 0.40 0.33 0.50 6 2068 6 1 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2133 12 2 10 0 18
RFA_00654 -0.01 0.00 0.00 0 2400 15 2 13 0 18
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1120 10 1 7 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.29 0.29 0.31 4 977 10 1 8 1 10
RFA_00668 0.53 0.43 0.67 6 981 3 0 3 0 8
RFA_00672 -0.01 0.00 0.00 0 897 6 0 6 0 13
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.96 0.93 1.00 13 1115 1 0 0 1 1
RFA_00675 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00677 0.59 0.57 0.62 8 977 7 0 5 2 6
RFA_00678 0.53 0.29 1.00 4 942 0 0 0 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.66 0.64 0.69 9 977 5 0 4 1 5
RFA_00685 0.44 0.36 0.56 5 981 4 0 4 0 9
RFA_00695 0.44 0.36 0.56 5 7012 29 1 3 25 9
RFA_00703 0.67 0.64 0.69 9 4265 19 1 3 15 5
RFA_00704 0.32 0.21 0.50 3 984 5 0 3 2 11
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 0.37 0.29 0.50 4 1027 4 0 4 0 10
RFA_00707 -0.01 0.00 0.00 0 1027 8 1 7 0 14
RFA_00708 0.40 0.29 0.57 4 1028 3 0 3 0 10
RFA_00709 -0.01 0.00 0.00 0 981 10 0 9 1 14
RFA_00710 -0.01 0.00 0.00 0 979 11 0 11 0 14
RFA_00711 -0.01 0.00 0.00 0 1026 9 0 9 0 14
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 937 9 0 9 0 14
RFA_00717 0.66 0.64 0.69 9 890 4 0 4 0 5
RFA_00730 0.75 0.75 0.75 9 891 5 0 3 2 3
RFA_00731 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00733 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00734 0.83 0.83 0.83 10 891 5 0 2 3 2
RFA_00736 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00737 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00745 0.66 0.67 0.67 8 934 6 1 3 2 4
RFA_00749 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00758 0.60 0.58 0.64 7 892 5 1 3 1 5
RFA_00762 0.64 0.67 0.62 8 890 6 1 4 1 4
RFA_00763 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00764 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00765 0.83 0.83 0.83 10 891 4 0 2 2 2
RFA_00767 0.74 0.56 1.00 10 1881 2 0 0 2 8
RFA_00768 0.45 0.44 0.47 8 1874 9 1 8 0 10
RFA_00769 0.54 0.56 0.53 10 1934 9 4 5 0 8
RFA_00770 0.68 0.56 0.83 10 2004 5 0 2 3 8
RFA_00773 0.71 0.56 0.91 10 1942 4 1 0 3 8
RFA_00779 0.68 0.56 0.83 10 1941 2 0 2 0 8
RFA_00781 0.98 0.97 1.00 31 5019 2 0 0 2 1
RFA_00786 0.48 0.44 0.54 14 5024 14 1 11 2 18
RFA_00791 0.89 0.88 0.90 28 5120 6 2 1 3 4
RFA_00792 0.95 0.94 0.97 30 5019 3 0 1 2 2
RFA_00801 0.81 0.78 0.83 25 5020 8 0 5 3 7
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.40 0.38 0.43 6 2131 8 0 8 0 10
RFA_00814 0.79 0.78 0.80 32 25160 19 1 7 11 9
RFA_00815 0.59 0.61 0.57 25 24487 35 2 17 16 16
RFA_00816 0.72 0.73 0.71 30 23178 20 8 4 8 11
RFA_00817 0.11 0.12 0.10 5 21893 47 12 35 0 36
RFA_00818 0.57 0.61 0.53 25 20254 32 11 11 10 16
RFA_00819 0.46 0.46 0.45 19 27924 47 3 20 24 22
RFA_00820 0.95 0.94 0.95 151 151366 28 0 8 20 9
RFA_00830 0.57 0.52 0.62 80 152499 100 2 47 51 74
SRP_00079 0.62 0.63 0.63 55 41817 43 3 30 10 33
SRP_00124 0.63 0.62 0.63 53 37044 38 3 28 7 32
SRP_00137 0.74 0.84 0.66 21 4154 13 4 7 2 4
SRP_00141 0.73 0.71 0.76 29 6290 10 1 8 1 12
SRP_00146 0.82 0.83 0.81 30 5216 8 1 6 1 6
SRP_00182 0.58 0.58 0.58 59 45955 45 10 32 3 42
SRP_00197 0.24 0.22 0.26 22 49370 63 10 53 0 79
SRP_00198 0.48 0.49 0.48 50 50936 55 2 52 1 53
SRP_00199 0.61 0.60 0.62 63 50939 42 3 35 4 42
SRP_00202 0.89 0.89 0.89 104 54168 16 1 12 3 13
SRP_00241 0.51 0.55 0.47 45 45961 53 14 36 3 37
SRP_00255 0.81 0.82 0.79 77 47489 27 2 18 7 17
SRP_00260 0.71 0.74 0.69 74 47787 37 6 28 3 26
SRP_00278 0.73 0.74 0.72 78 45947 33 4 27 2 27
SRP_00285 0.97 0.97 0.97 29 3711 1 0 1 0 1
SRP_00322 0.89 0.90 0.88 98 48404 17 1 13 3 11
SRP_00328 0.60 0.62 0.58 53 39530 41 10 28 3 32
SRP_00329 0.59 0.61 0.58 52 39813 43 6 32 5 33
SRP_00338 0.97 0.97 0.97 35 5424 2 0 1 1 1
SRP_00339 0.66 0.68 0.64 64 44451 42 4 32 6 30
SRP_00340 0.51 0.56 0.47 46 41230 56 11 41 4 36

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1738
Total TN 2789495
Total FP 489
Total FP CONTRA 21
Total FP INCONS 385
Total FP COMP 83
Total FN 3090
Total Scores
MCC 0.540
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065
Sensitivity 0.360
Positive Predictive Value 0.811
Nr of predictions 123

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.