CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of ContextFold - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for ContextFold & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric ContextFold Murlet(seed)
MCC 0.761 > 0.540
Average MCC ± 95% Confidence Intervals 0.712 ± 0.051 > 0.396 ± 0.065
Sensitivity 0.727 > 0.360
Positive Predictive Value 0.798 < 0.811
Total TP 3510 > 1738
Total TN 2787240 < 2789495
Total FP 1164 > 489
Total FP CONTRA 177 > 21
Total FP INCONS 712 > 385
Total FP COMP 275 > 83
Total FN 1318 < 3090
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of ContextFold and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for ContextFold and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for ContextFold and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for ContextFold and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for ContextFold and Murlet(seed)).

^top





Performance of ContextFold - scored higher in this pairwise comparison

1. Total counts & total scores for ContextFold

Total Base Pair Counts
Total TP 3510
Total TN 2787240
Total FP 1164
Total FP CONTRA 177
Total FP INCONS 712
Total FP COMP 275
Total FN 1318
Total Scores
MCC 0.761
Average MCC ± 95% Confidence Intervals 0.712 ± 0.051
Sensitivity 0.727
Positive Predictive Value 0.798
Nr of predictions 123

^top



2. Individual counts for ContextFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.82 0.76 0.89 328 977531 59 6 36 17 106
PDB_00005 0.42 0.36 0.50 5 936 5 0 5 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.90 0.85 0.95 35 5013 2 0 2 0 6
PDB_00553 0.95 0.91 1.00 10 455 0 0 0 0 1
PDB_00716 -0.01 0.00 0.00 0 2687 14 0 14 0 23
PDB_00810 0.91 0.82 1.00 14 1067 0 0 0 0 3
PDB_01050 0.96 0.92 1.00 12 618 1 0 0 1 1
PDB_01051 0.96 0.92 1.00 12 891 4 0 0 4 1
PDB_01092 0.70 0.63 0.77 33 10110 12 1 9 2 19
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00390 1.00 1.00 1.00 15 1416 0 0 0 0 0
RFA_00416 1.00 1.00 1.00 15 1470 1 0 0 1 0
RFA_00433 1.00 1.00 1.00 15 1416 0 0 0 0 0
RFA_00587 1.00 1.00 1.00 15 4836 11 0 0 11 0
RFA_00603 0.63 0.60 0.66 21 13498 12 5 6 1 14
RFA_00604 0.73 0.69 0.77 24 13499 17 2 5 10 11
RFA_00605 0.20 0.17 0.24 6 15200 26 5 14 7 29
RFA_00606 0.45 0.44 0.47 17 21285 23 10 9 4 22
RFA_00607 0.85 0.75 0.96 27 17738 10 0 1 9 9
RFA_00609 0.68 0.65 0.71 24 17921 17 2 8 7 13
RFA_00610 0.90 0.86 0.94 32 17171 7 0 2 5 5
RFA_00611 0.93 0.86 1.00 30 13011 0 0 0 0 5
RFA_00613 0.68 0.66 0.70 23 12847 11 4 6 1 12
RFA_00615 0.67 0.60 0.75 21 13338 7 6 1 0 14
RFA_00620 0.58 0.54 0.62 21 21911 27 3 10 14 18
RFA_00626 0.80 0.75 0.87 65 56541 19 1 9 9 22
RFA_00627 0.76 0.71 0.81 62 56876 16 3 12 1 25
RFA_00628 0.60 0.56 0.65 48 57217 28 3 23 2 38
RFA_00630 0.62 0.60 0.65 52 56873 35 3 25 7 35
RFA_00632 0.25 0.25 0.26 7 4068 20 1 19 0 21
RFA_00636 0.52 0.43 0.63 12 3986 7 0 7 0 16
RFA_00639 0.64 0.63 0.65 55 54531 39 7 22 10 32
RFA_00642 0.44 0.39 0.50 7 2912 7 2 5 0 11
RFA_00643 0.47 0.39 0.58 7 2199 5 0 5 0 11
RFA_00644 0.77 0.83 0.71 15 2680 6 3 3 0 3
RFA_00645 -0.01 0.00 0.00 0 2402 13 2 11 0 18
RFA_00649 0.88 0.83 0.94 15 2129 1 0 1 0 3
RFA_00651 0.59 0.50 0.69 9 2067 4 0 4 0 9
RFA_00653 0.25 0.22 0.29 4 2131 10 2 8 0 14
RFA_00654 0.71 0.67 0.75 12 2399 5 1 3 1 6
RFA_00658 0.89 0.79 1.00 11 1117 0 0 0 0 3
RFA_00659 0.53 0.43 0.67 6 1119 3 0 3 0 8
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.33 0.29 0.40 4 980 7 0 6 1 10
RFA_00668 0.67 0.57 0.80 8 980 2 0 2 0 6
RFA_00672 -0.01 0.00 0.00 0 895 8 0 8 0 13
RFA_00673 0.80 0.64 1.00 9 1119 0 0 0 0 5
RFA_00674 0.92 0.86 1.00 12 1116 1 0 0 1 2
RFA_00675 0.80 0.64 1.00 9 981 0 0 0 0 5
RFA_00677 0.67 0.57 0.80 8 980 2 0 2 0 6
RFA_00678 0.84 0.71 1.00 10 936 0 0 0 0 4
RFA_00680 0.92 0.86 1.00 12 1116 1 0 0 1 2
RFA_00684 0.77 0.71 0.83 10 978 2 0 2 0 4
RFA_00685 0.71 0.57 0.89 8 981 1 0 1 0 6
RFA_00695 0.00 0.00 0.00 0 7006 23 2 13 8 14
RFA_00703 0.46 0.43 0.50 6 4266 14 0 6 8 8
RFA_00704 0.50 0.43 0.60 6 980 5 0 4 1 8
RFA_00705 0.84 0.71 1.00 10 1025 0 0 0 0 4
RFA_00706 0.84 0.71 1.00 10 1025 0 0 0 0 4
RFA_00707 0.89 0.79 1.00 11 1024 0 0 0 0 3
RFA_00708 -0.01 0.00 0.00 0 1029 6 0 6 0 14
RFA_00709 0.16 0.14 0.20 2 980 9 0 8 1 12
RFA_00710 0.40 0.36 0.45 5 979 6 0 6 0 9
RFA_00711 -0.01 0.00 0.00 0 1024 12 0 11 1 14
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 938 8 0 8 0 14
RFA_00717 0.88 0.79 1.00 11 892 0 0 0 0 3
RFA_00730 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00731 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00733 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00734 0.78 0.75 0.82 9 892 3 0 2 1 3
RFA_00736 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00737 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00745 1.00 1.00 1.00 12 934 1 0 0 1 0
RFA_00749 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00758 0.96 0.92 1.00 11 892 0 0 0 0 1
RFA_00762 0.96 0.92 1.00 11 892 0 0 0 0 1
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.59 0.56 0.63 10 1875 6 3 3 0 8
RFA_00768 0.61 0.56 0.67 10 1876 5 2 3 0 8
RFA_00769 0.55 0.56 0.56 10 1935 8 4 4 0 8
RFA_00770 0.61 0.56 0.67 10 2001 5 2 3 0 8
RFA_00773 0.59 0.56 0.63 10 1937 6 1 5 0 8
RFA_00779 0.61 0.56 0.67 10 1938 5 2 3 0 8
RFA_00781 1.00 1.00 1.00 32 5018 0 0 0 0 0
RFA_00786 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00791 0.94 0.88 1.00 28 5123 1 0 0 1 4
RFA_00792 0.76 0.75 0.77 24 5019 7 2 5 0 8
RFA_00801 0.97 0.94 1.00 30 5020 1 0 0 1 2
RFA_00808 0.58 0.44 0.78 7 2007 2 0 2 0 9
RFA_00809 0.39 0.31 0.50 5 2135 6 0 5 1 11
RFA_00814 0.94 0.93 0.95 38 25160 18 0 2 16 3
RFA_00815 0.74 0.83 0.65 34 24479 21 13 5 3 7
RFA_00816 0.67 0.71 0.63 29 23174 22 13 4 5 12
RFA_00817 0.58 0.59 0.59 24 21904 17 5 12 0 17
RFA_00818 0.61 0.66 0.56 27 20253 29 10 11 8 14
RFA_00819 0.96 0.95 0.98 39 27926 15 0 1 14 2
RFA_00820 0.94 0.92 0.97 147 151373 9 1 4 4 13
RFA_00830 0.54 0.47 0.62 73 152510 80 5 40 35 81
SRP_00079 0.87 0.85 0.89 75 41821 17 0 9 8 13
SRP_00124 0.90 0.87 0.94 74 37049 9 0 5 4 11
SRP_00137 0.77 0.80 0.74 20 4159 7 3 4 0 5
SRP_00141 0.99 0.98 1.00 40 6288 0 0 0 0 1
SRP_00146 1.00 1.00 1.00 36 5217 0 0 0 0 0
SRP_00182 0.86 0.83 0.88 84 45961 11 2 9 0 17
SRP_00197 0.91 0.88 0.95 89 49361 5 0 5 0 12
SRP_00198 0.91 0.88 0.95 91 50944 5 0 5 0 12
SRP_00199 0.86 0.84 0.89 88 50941 12 0 11 1 17
SRP_00202 0.97 0.97 0.98 113 54170 3 0 2 1 4
SRP_00241 0.55 0.59 0.53 48 45965 44 14 29 1 34
SRP_00255 0.85 0.86 0.84 81 47489 19 0 16 3 13
SRP_00260 0.68 0.68 0.68 68 47795 34 5 27 2 32
SRP_00278 0.83 0.82 0.85 86 45955 15 0 15 0 19
SRP_00285 1.00 1.00 1.00 30 3711 0 0 0 0 0
SRP_00322 0.94 0.94 0.94 102 48408 6 0 6 0 7
SRP_00328 0.82 0.81 0.82 69 39537 17 4 11 2 16
SRP_00329 0.70 0.69 0.70 59 39819 28 3 22 3 26
SRP_00338 0.96 0.94 0.97 34 5425 1 0 1 0 2
SRP_00339 0.83 0.82 0.84 77 44459 18 1 14 3 17
SRP_00340 0.70 0.70 0.70 57 41247 26 8 16 2 25

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1738
Total TN 2789495
Total FP 489
Total FP CONTRA 21
Total FP INCONS 385
Total FP COMP 83
Total FN 3090
Total Scores
MCC 0.540
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065
Sensitivity 0.360
Positive Predictive Value 0.811
Nr of predictions 123

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.