CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Murlet(seed) - scored higher in this pairwise comparison

  4. Performance of McQFold - scored lower in this pairwise comparison

  5. Compile and download dataset for Murlet(seed) & McQFold [.zip] - may take several seconds...


Overview

Metric Murlet(seed) McQFold
MCC 0.530 > 0.500
Average MCC ± 95% Confidence Intervals 0.394 ± 0.065 < 0.504 ± 0.056
Sensitivity 0.336 < 0.480
Positive Predictive Value 0.839 > 0.523
Total TP 1475 < 2111
Total TN 1811980 > 1809698
Total FP 344 < 2218
Total FP CONTRA 15 < 337
Total FP INCONS 268 < 1592
Total FP COMP 61 < 289
Total FN 2919 > 2283
P-value 3.85233844192e-08

^top




Performance plots


  1. Comparison of performance of Murlet(seed) and McQFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Murlet(seed) and McQFold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Murlet(seed) and McQFold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Murlet(seed) and McQFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Murlet(seed) and McQFold).

^top





Performance of Murlet(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1475
Total TN 1811980
Total FP 344
Total FP CONTRA 15
Total FP INCONS 268
Total FP COMP 61
Total FN 2919
Total Scores
MCC 0.530
Average MCC ± 95% Confidence Intervals 0.394 ± 0.065
Sensitivity 0.336
Positive Predictive Value 0.839
Nr of predictions 122

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top



Performance of McQFold - scored lower in this pairwise comparison

1. Total counts & total scores for McQFold

Total Base Pair Counts
Total TP 2111
Total TN 1809698
Total FP 2218
Total FP CONTRA 337
Total FP INCONS 1592
Total FP COMP 289
Total FN 2283
Total Scores
MCC 0.500
Average MCC ± 95% Confidence Intervals 0.504 ± 0.056
Sensitivity 0.480
Positive Predictive Value 0.523
Nr of predictions 122

^top



2. Individual counts for McQFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.72 0.68 0.76 28 5013 9 0 9 0 13
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 0.68 0.70 0.67 16 2677 9 0 8 1 7
PDB_00810 0.51 0.41 0.64 7 1070 4 2 2 0 10
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.79 0.69 0.90 9 893 2 0 1 1 4
PDB_01092 0.81 0.75 0.89 39 10109 6 1 4 1 13
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00390 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00416 1.00 1.00 1.00 15 1470 4 0 0 4 0
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00587 1.00 1.00 1.00 15 4836 8 0 0 8 0
RFA_00603 0.34 0.29 0.42 10 13506 15 4 10 1 25
RFA_00604 0.71 0.66 0.77 23 13500 14 2 5 7 12
RFA_00605 0.36 0.34 0.38 12 15193 31 3 17 11 23
RFA_00606 0.32 0.33 0.32 13 21280 34 12 16 6 26
RFA_00607 0.65 0.61 0.69 22 17734 17 3 7 7 14
RFA_00609 0.77 0.76 0.78 28 17919 18 2 6 10 9
RFA_00610 0.56 0.57 0.55 21 17167 25 2 15 8 16
RFA_00611 0.58 0.54 0.61 19 13010 12 3 9 0 16
RFA_00613 0.15 0.14 0.16 5 12848 27 6 21 0 30
RFA_00615 0.32 0.29 0.37 10 13339 18 4 13 1 25
RFA_00620 0.36 0.38 0.34 15 21901 37 4 25 8 24
RFA_00626 0.21 0.22 0.21 19 56526 76 15 56 5 68
RFA_00627 0.42 0.40 0.44 35 56873 47 13 32 2 52
RFA_00628 0.42 0.41 0.43 35 57209 50 7 40 3 51
RFA_00630 0.63 0.63 0.63 55 56865 41 6 27 8 32
RFA_00632 0.88 0.89 0.86 25 4066 4 3 1 0 3
RFA_00636 0.73 0.71 0.74 20 3978 7 5 2 0 8
RFA_00639 0.37 0.39 0.36 34 54520 66 13 48 5 53
RFA_00642 -0.01 0.00 0.00 0 2913 13 1 12 0 18
RFA_00643 -0.01 0.00 0.00 0 2196 15 1 14 0 18
RFA_00644 -0.01 0.00 0.00 0 2684 17 3 14 0 18
RFA_00645 -0.01 0.00 0.00 0 2403 12 3 9 0 18
RFA_00649 0.78 0.67 0.92 12 2132 1 0 1 0 6
RFA_00651 0.40 0.33 0.50 6 2068 6 0 6 0 12
RFA_00653 0.33 0.33 0.33 6 2127 12 3 9 0 12
RFA_00654 -0.01 0.00 0.00 0 2403 12 2 10 0 18
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1125 5 0 3 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 -0.01 0.00 0.00 0 981 10 1 8 1 14
RFA_00668 0.50 0.43 0.60 6 980 4 0 4 0 8
RFA_00672 -0.01 0.00 0.00 0 895 10 0 8 2 13
RFA_00673 0.47 0.36 0.63 5 1120 4 0 3 1 9
RFA_00674 0.96 0.93 1.00 13 1115 1 0 0 1 1
RFA_00675 0.28 0.29 0.29 4 976 10 1 9 0 10
RFA_00677 0.74 0.71 0.77 10 977 6 0 3 3 4
RFA_00678 0.53 0.29 1.00 4 942 0 0 0 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.84 0.71 1.00 10 980 1 0 0 1 4
RFA_00685 0.47 0.36 0.63 5 982 4 0 3 1 9
RFA_00695 0.00 0.00 0.00 0 7007 29 4 10 15 14
RFA_00703 0.44 0.43 0.46 6 4265 19 1 6 12 8
RFA_00704 -0.01 0.00 0.00 0 983 8 0 7 1 14
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00707 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 -0.01 0.00 0.00 0 984 7 0 6 1 14
RFA_00710 -0.01 0.00 0.00 0 980 10 0 10 0 14
RFA_00711 -0.01 0.00 0.00 0 1023 12 0 12 0 14
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00717 0.69 0.64 0.75 9 891 3 0 3 0 5
RFA_00730 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00731 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00733 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00734 0.67 0.58 0.78 7 894 3 0 2 1 5
RFA_00736 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00737 0.43 0.42 0.45 5 892 7 0 6 1 7
RFA_00745 0.43 0.42 0.45 5 935 7 1 5 1 7
RFA_00749 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00758 0.40 0.33 0.50 4 895 4 2 2 0 8
RFA_00762 0.31 0.33 0.31 4 890 9 2 7 0 8
RFA_00763 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00764 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00765 0.51 0.42 0.63 5 895 4 0 3 1 7
RFA_00767 1.00 1.00 1.00 18 1873 0 0 0 0 0
RFA_00768 1.00 1.00 1.00 18 1873 0 0 0 0 0
RFA_00769 0.85 0.94 0.77 17 1931 5 4 1 0 1
RFA_00770 0.88 0.78 1.00 14 2002 4 0 0 4 4
RFA_00773 0.97 1.00 0.95 18 1934 5 1 0 4 0
RFA_00779 0.97 0.94 1.00 17 1936 0 0 0 0 1
RFA_00781 0.97 0.94 1.00 30 5020 1 0 0 1 2
RFA_00786 0.21 0.19 0.25 6 5026 19 2 16 1 26
RFA_00791 0.60 0.56 0.64 18 5123 11 0 10 1 14
RFA_00792 0.97 0.94 1.00 30 5020 1 0 0 1 2
RFA_00801 0.69 0.63 0.77 20 5024 8 0 6 2 12
RFA_00808 1.00 1.00 1.00 16 2000 0 0 0 0 0
RFA_00809 0.79 0.81 0.76 13 2128 4 0 4 0 3
RFA_00814 0.79 0.80 0.77 33 25157 21 0 10 11 8
RFA_00815 0.56 0.59 0.53 24 24486 27 1 20 6 17
RFA_00816 0.82 0.85 0.80 35 23176 12 4 5 3 6
RFA_00817 0.35 0.39 0.31 16 21894 39 9 26 4 25
RFA_00818 0.28 0.27 0.29 11 20263 30 12 15 3 30
RFA_00819 0.30 0.29 0.31 12 27927 50 4 23 23 29
RFA_00820 0.64 0.61 0.67 98 151379 53 13 35 5 62
RFA_00830 0.39 0.36 0.43 55 152499 100 12 62 26 99
SRP_00079 0.42 0.41 0.44 36 41823 55 1 45 9 52
SRP_00124 0.43 0.40 0.46 34 37054 43 6 34 3 51
SRP_00137 0.37 0.40 0.36 10 4158 18 7 11 0 15
SRP_00141 0.84 0.80 0.87 33 6290 5 1 4 0 8
SRP_00146 0.37 0.33 0.41 12 5224 17 2 15 0 24
SRP_00182 0.70 0.69 0.71 70 45957 32 7 22 3 31
SRP_00197 0.21 0.20 0.22 20 49363 73 10 62 1 81
SRP_00198 0.08 0.08 0.09 8 50954 78 7 71 0 95
SRP_00199 0.13 0.11 0.14 12 50956 72 6 66 0 93
SRP_00202 0.88 0.89 0.88 104 54167 15 5 9 1 13
SRP_00241 0.30 0.30 0.29 25 45971 62 12 48 2 57
SRP_00255 0.26 0.28 0.25 26 47484 76 14 62 0 68
SRP_00260 0.56 0.52 0.61 52 47810 36 7 26 3 48
SRP_00278 0.75 0.74 0.76 78 45953 28 5 20 3 27
SRP_00285 0.38 0.37 0.41 11 3714 16 1 15 0 19
SRP_00322 0.68 0.69 0.68 75 48405 36 4 32 0 34
SRP_00328 0.49 0.49 0.49 42 39536 43 10 33 0 43
SRP_00329 0.38 0.38 0.40 32 39822 50 8 41 1 53
SRP_00338 0.75 0.75 0.75 27 5424 9 3 6 0 9
SRP_00339 0.43 0.44 0.43 41 44455 55 8 47 0 53
SRP_00340 0.48 0.50 0.46 41 41238 51 9 40 2 41

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.