CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Afold - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for Afold & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric Afold Murlet(seed)
MCC 0.595 > 0.553
Average MCC ± 95% Confidence Intervals 0.595 ± 0.049 > 0.394 ± 0.072
Sensitivity 0.589 > 0.371
Positive Predictive Value 0.601 < 0.823
Total TP 2433 > 1534
Total TN 2505389 < 2507572
Total FP 2090 > 397
Total FP CONTRA 269 > 19
Total FP INCONS 1344 > 310
Total FP COMP 477 > 68
Total FN 1697 < 2596
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Afold and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Afold and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Afold and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Afold and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Afold and Murlet(seed)).

^top





Performance of Afold - scored higher in this pairwise comparison

1. Total counts & total scores for Afold

Total Base Pair Counts
Total TP 2433
Total TN 2505389
Total FP 2090
Total FP CONTRA 269
Total FP INCONS 1344
Total FP COMP 477
Total FN 1697
Total Scores
MCC 0.595
Average MCC ± 95% Confidence Intervals 0.595 ± 0.049
Sensitivity 0.589
Positive Predictive Value 0.601
Nr of predictions 108

^top



2. Individual counts for Afold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.37 0.37 0.37 160 977470 284 42 229 13 274
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.87 0.80 0.94 33 5015 2 0 2 0 8
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2677 25 1 23 1 23
PDB_00810 0.77 0.71 0.86 12 1067 2 0 2 0 5
PDB_01050 0.96 0.92 1.00 12 618 2 0 0 2 1
PDB_01051 0.75 0.69 0.82 9 892 6 0 2 4 4
PDB_01092 0.69 0.60 0.79 31 10114 11 1 7 3 21
PDB_01152 0.96 0.93 1.00 13 548 0 0 0 0 1
RFA_00390 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00416 0.93 0.93 0.93 14 1470 4 0 1 3 1
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00587 0.87 0.87 0.87 13 4836 14 0 2 12 2
RFA_00603 0.64 0.69 0.60 24 13490 20 4 12 4 11
RFA_00604 0.72 0.74 0.70 26 13493 24 2 9 13 9
RFA_00606 0.67 0.67 0.68 26 21283 26 7 5 14 13
RFA_00607 0.57 0.61 0.54 22 17725 25 6 13 6 14
RFA_00609 0.75 0.76 0.74 28 17917 28 2 8 18 9
RFA_00610 0.77 0.76 0.78 28 17169 23 2 6 15 9
RFA_00611 0.79 0.77 0.82 27 13008 13 0 6 7 8
RFA_00613 0.61 0.60 0.62 21 12846 17 4 9 4 14
RFA_00615 0.51 0.51 0.51 18 13331 18 8 9 1 17
RFA_00620 0.64 0.62 0.67 24 21909 26 3 9 14 15
RFA_00626 0.89 0.86 0.93 75 56535 31 2 4 25 12
RFA_00627 0.48 0.47 0.48 41 56868 50 10 34 6 46
RFA_00628 0.83 0.80 0.85 69 57210 38 1 11 26 17
RFA_00630 0.60 0.60 0.60 52 56866 46 8 27 11 35
RFA_00632 0.40 0.39 0.41 11 4068 18 0 16 2 17
RFA_00636 0.42 0.43 0.43 12 3977 17 2 14 1 16
RFA_00639 0.49 0.49 0.49 43 54528 61 8 36 17 44
RFA_00644 -0.01 0.00 0.00 0 2681 21 4 16 1 18
RFA_00645 -0.01 0.00 0.00 0 2401 15 3 11 1 18
RFA_00649 0.65 0.61 0.69 11 2129 8 1 4 3 7
RFA_00653 0.34 0.33 0.35 6 2128 12 3 8 1 12
RFA_00654 0.27 0.28 0.28 5 2397 14 2 11 1 13
RFA_00658 0.69 0.64 0.75 9 1116 5 0 3 2 5
RFA_00659 0.30 0.29 0.33 4 1116 10 1 7 2 10
RFA_00664 0.07 0.07 0.09 1 979 11 0 10 1 13
RFA_00667 0.30 0.29 0.33 4 978 8 1 7 0 10
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 0.17 0.15 0.20 2 893 9 0 8 1 11
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.69 0.71 0.67 10 1113 6 0 5 1 4
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.42 0.43 0.43 6 976 9 0 8 1 8
RFA_00678 0.66 0.64 0.69 9 933 4 0 4 0 5
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.66 0.64 0.69 9 977 5 0 4 1 5
RFA_00685 0.42 0.36 0.50 5 980 6 0 5 1 9
RFA_00695 0.67 0.64 0.69 9 7008 29 1 3 25 5
RFA_00703 0.67 0.64 0.69 9 4265 18 1 3 14 5
RFA_00704 0.48 0.43 0.55 6 979 7 0 5 2 8
RFA_00705 0.69 0.71 0.67 10 1020 5 0 5 0 4
RFA_00706 0.44 0.36 0.56 5 1026 4 0 4 0 9
RFA_00707 0.07 0.07 0.10 1 1025 9 3 6 0 13
RFA_00708 0.47 0.36 0.63 5 1027 3 0 3 0 9
RFA_00709 0.16 0.14 0.20 2 980 9 0 8 1 12
RFA_00710 0.23 0.21 0.27 3 979 8 0 8 0 11
RFA_00711 0.40 0.36 0.45 5 1024 6 0 6 0 9
RFA_00716 0.06 0.07 0.08 1 933 12 0 12 0 13
RFA_00717 0.64 0.64 0.64 9 889 5 0 5 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00733 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00734 0.61 0.67 0.57 8 889 8 1 5 2 4
RFA_00736 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00737 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00745 0.66 0.67 0.67 8 934 5 1 3 1 4
RFA_00749 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00758 0.96 0.92 1.00 11 892 1 0 0 1 1
RFA_00762 0.64 0.67 0.62 8 890 6 1 4 1 4
RFA_00763 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00764 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00767 0.63 0.56 0.71 10 1877 5 0 4 1 8
RFA_00768 0.48 0.44 0.53 8 1876 8 1 6 1 10
RFA_00769 0.55 0.56 0.56 10 1935 10 3 5 2 8
RFA_00770 0.68 0.56 0.83 10 2004 6 0 2 4 8
RFA_00773 0.71 0.56 0.91 10 1942 5 1 0 4 8
RFA_00781 0.55 0.56 0.55 18 5017 17 2 13 2 14
RFA_00786 0.71 0.72 0.70 23 5017 12 2 8 2 9
RFA_00791 0.50 0.50 0.52 16 5120 16 3 12 1 16
RFA_00792 0.90 0.88 0.93 28 5020 3 0 2 1 4
RFA_00801 0.59 0.59 0.59 19 5018 14 0 13 1 13
RFA_00808 -0.01 0.00 0.00 0 2001 16 2 13 1 16
RFA_00809 0.36 0.38 0.35 6 2128 12 1 10 1 10
RFA_00814 0.35 0.37 0.33 15 25155 42 7 23 12 26
RFA_00815 0.56 0.61 0.51 25 24482 35 8 16 11 16
RFA_00816 0.59 0.66 0.53 27 23169 33 13 11 9 14
RFA_00818 0.19 0.22 0.17 9 20247 48 19 26 3 32
RFA_00819 0.96 0.95 0.98 39 27926 32 0 1 31 2
RFA_00820 0.90 0.91 0.88 145 151361 34 1 18 15 15
RFA_00830 0.50 0.47 0.53 73 152490 110 5 60 45 81
SRP_00079 0.52 0.55 0.50 48 41809 53 8 40 5 40
SRP_00124 0.53 0.52 0.55 44 37048 38 5 31 2 41
SRP_00141 0.86 0.85 0.88 35 6288 5 1 4 0 6
SRP_00146 0.93 0.94 0.92 34 5216 3 1 2 0 2
SRP_00182 0.76 0.73 0.79 74 45962 21 3 17 1 27
SRP_00197 0.24 0.24 0.25 24 49359 72 7 65 0 77
SRP_00199 0.67 0.66 0.68 69 50939 34 2 30 2 36
SRP_00202 0.95 0.93 0.96 109 54172 6 0 4 2 8
SRP_00241 0.54 0.59 0.51 48 45961 50 11 36 3 34
SRP_00278 0.72 0.72 0.71 76 45949 33 4 27 2 29
SRP_00322 0.60 0.60 0.61 65 48409 43 2 40 1 44
SRP_00338 0.92 0.92 0.92 33 5424 4 0 3 1 3
SRP_00339 0.56 0.56 0.56 53 44456 45 5 37 3 41
SRP_00340 0.55 0.59 0.52 48 41235 47 9 36 2 34

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1534
Total TN 2507572
Total FP 397
Total FP CONTRA 19
Total FP INCONS 310
Total FP COMP 68
Total FN 2596
Total Scores
MCC 0.553
Average MCC ± 95% Confidence Intervals 0.394 ± 0.072
Sensitivity 0.371
Positive Predictive Value 0.823
Nr of predictions 108

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.