CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Murlet(seed) - scored higher in this pairwise comparison

  4. Performance of CRWrnafold - scored lower in this pairwise comparison

  5. Compile and download dataset for Murlet(seed) & CRWrnafold [.zip] - may take several seconds...


Overview

Metric Murlet(seed) CRWrnafold
MCC 0.540 > 0.513
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065 < 0.492 ± 0.053
Sensitivity 0.360 < 0.507
Positive Predictive Value 0.811 > 0.520
Total TP 1738 < 2449
Total TN 2789495 > 2786932
Total FP 489 < 2710
Total FP CONTRA 21 < 419
Total FP INCONS 385 < 1839
Total FP COMP 83 < 452
Total FN 3090 > 2379
P-value 6.06606238843e-08

^top




Performance plots


  1. Comparison of performance of Murlet(seed) and CRWrnafold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Murlet(seed) and CRWrnafold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Murlet(seed) and CRWrnafold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Murlet(seed) and CRWrnafold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Murlet(seed) and CRWrnafold).

^top





Performance of Murlet(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1738
Total TN 2789495
Total FP 489
Total FP CONTRA 21
Total FP INCONS 385
Total FP COMP 83
Total FN 3090
Total Scores
MCC 0.540
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065
Sensitivity 0.360
Positive Predictive Value 0.811
Nr of predictions 123

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top



Performance of CRWrnafold - scored lower in this pairwise comparison

1. Total counts & total scores for CRWrnafold

Total Base Pair Counts
Total TP 2449
Total TN 2786932
Total FP 2710
Total FP CONTRA 419
Total FP INCONS 1839
Total FP COMP 452
Total FN 2379
Total Scores
MCC 0.513
Average MCC ± 95% Confidence Intervals 0.492 ± 0.053
Sensitivity 0.507
Positive Predictive Value 0.520
Nr of predictions 123

^top



2. Individual counts for CRWrnafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.37 0.36 0.38 158 977489 271 43 211 17 276
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.92 0.85 1.00 35 5015 0 0 0 0 6
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2680 22 0 21 1 23
PDB_00810 0.67 0.59 0.77 10 1068 3 0 3 0 7
PDB_01050 0.92 0.85 1.00 11 619 1 0 0 1 2
PDB_01051 0.79 0.69 0.90 9 893 5 0 1 4 4
PDB_01092 0.68 0.62 0.76 32 10111 12 1 9 2 20
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00390 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00416 1.00 1.00 1.00 15 1470 3 0 0 3 0
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00587 1.00 1.00 1.00 15 4836 12 0 0 12 0
RFA_00603 0.27 0.29 0.26 10 13491 31 8 21 2 25
RFA_00604 0.61 0.63 0.59 22 13493 26 2 13 11 13
RFA_00605 0.34 0.34 0.34 12 15190 36 4 19 13 23
RFA_00606 0.41 0.46 0.37 18 21272 38 17 14 7 21
RFA_00607 0.29 0.31 0.28 11 17726 39 5 24 10 25
RFA_00609 0.52 0.57 0.48 21 17911 36 8 15 13 16
RFA_00610 0.52 0.54 0.50 20 17165 33 4 16 13 17
RFA_00611 0.45 0.49 0.43 17 13001 27 8 15 4 18
RFA_00613 0.43 0.43 0.44 15 12846 26 8 11 7 20
RFA_00615 0.27 0.29 0.26 10 13328 29 7 21 1 25
RFA_00620 0.55 0.59 0.51 23 21900 40 4 18 18 16
RFA_00626 0.64 0.63 0.65 55 56532 42 8 21 13 32
RFA_00627 0.65 0.63 0.67 55 56871 42 6 21 15 32
RFA_00628 0.89 0.85 0.94 73 57213 30 0 5 25 13
RFA_00630 0.57 0.57 0.57 50 56865 51 12 26 13 37
RFA_00632 0.37 0.36 0.38 10 4069 19 0 16 3 18
RFA_00636 0.51 0.50 0.52 14 3978 13 3 10 0 14
RFA_00639 0.11 0.11 0.11 10 54524 86 13 68 5 77
RFA_00642 0.17 0.17 0.18 3 2909 17 0 14 3 15
RFA_00643 0.24 0.22 0.27 4 2196 12 2 9 1 14
RFA_00644 0.21 0.22 0.21 4 2682 15 4 11 0 14
RFA_00645 -0.01 0.00 0.00 0 2401 14 2 12 0 18
RFA_00649 0.57 0.56 0.59 10 2128 7 1 6 0 8
RFA_00651 0.35 0.33 0.38 6 2064 10 1 9 0 12
RFA_00653 0.35 0.33 0.38 6 2129 10 2 8 0 12
RFA_00654 0.26 0.28 0.26 5 2396 14 2 12 0 13
RFA_00658 0.59 0.50 0.70 7 1118 4 0 3 1 7
RFA_00659 0.18 0.14 0.25 2 1120 8 0 6 2 12
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.30 0.29 0.33 4 978 9 1 7 1 10
RFA_00668 0.46 0.43 0.50 6 978 6 0 6 0 8
RFA_00672 -0.01 0.00 0.00 0 897 6 0 6 0 13
RFA_00673 0.47 0.36 0.63 5 1120 4 0 3 1 9
RFA_00674 -0.01 0.00 0.00 0 1118 10 0 10 0 14
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.69 0.64 0.75 9 978 5 0 3 2 5
RFA_00678 -0.01 0.00 0.00 0 941 5 0 5 0 14
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.72 0.64 0.82 9 979 3 0 2 1 5
RFA_00685 0.42 0.36 0.50 5 980 6 0 5 1 9
RFA_00695 0.67 0.64 0.69 9 7008 28 1 3 24 5
RFA_00703 0.67 0.64 0.69 9 4265 18 1 3 14 5
RFA_00704 -0.01 0.00 0.00 0 981 10 0 9 1 14
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 0.33 0.29 0.40 4 1025 6 0 6 0 10
RFA_00707 -0.01 0.00 0.00 0 1024 11 3 8 0 14
RFA_00708 0.35 0.29 0.44 4 1026 6 0 5 1 10
RFA_00709 -0.01 0.00 0.00 0 982 9 0 8 1 14
RFA_00710 -0.01 0.00 0.00 0 980 10 0 10 0 14
RFA_00711 -0.01 0.00 0.00 0 1024 11 0 11 0 14
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 939 7 0 7 0 14
RFA_00717 0.31 0.29 0.36 4 892 8 0 7 1 10
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.41 0.42 0.42 5 891 8 3 4 1 7
RFA_00733 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00734 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00736 0.41 0.42 0.42 5 891 8 3 4 1 7
RFA_00737 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00745 0.69 0.67 0.73 8 935 4 1 2 1 4
RFA_00749 0.41 0.42 0.42 5 891 8 3 4 1 7
RFA_00758 0.63 0.58 0.70 7 893 3 1 2 0 5
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 0.69 0.67 0.73 8 892 3 1 2 0 4
RFA_00764 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00765 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00767 0.59 0.56 0.63 10 1875 6 3 3 0 8
RFA_00768 0.47 0.44 0.50 8 1875 8 1 7 0 10
RFA_00769 0.54 0.56 0.53 10 1934 9 4 5 0 8
RFA_00770 -0.01 0.00 0.00 0 2002 17 1 13 3 18
RFA_00773 0.59 0.56 0.63 10 1937 6 1 5 0 8
RFA_00779 0.59 0.56 0.63 10 1937 6 3 3 0 8
RFA_00781 0.74 0.75 0.73 24 5017 9 2 7 0 8
RFA_00786 0.17 0.16 0.19 5 5024 21 1 20 0 27
RFA_00791 0.46 0.44 0.48 14 5122 15 0 15 0 18
RFA_00792 0.94 0.94 0.94 30 5018 4 0 2 2 2
RFA_00801 0.38 0.34 0.42 11 5024 16 1 14 1 21
RFA_00808 -0.01 0.00 0.00 0 2000 18 2 14 2 16
RFA_00809 0.40 0.38 0.43 6 2131 8 1 7 0 10
RFA_00814 0.24 0.27 0.21 11 25148 48 10 31 7 30
RFA_00815 0.36 0.41 0.31 17 24477 49 6 31 12 24
RFA_00816 0.21 0.24 0.19 10 23166 50 20 24 6 31
RFA_00817 0.10 0.12 0.08 5 21883 57 17 40 0 36
RFA_00818 0.00 0.00 0.00 0 20249 56 19 33 4 41
RFA_00819 0.44 0.44 0.44 18 27925 57 0 23 34 23
RFA_00820 0.65 0.64 0.66 103 151368 68 8 46 14 57
RFA_00830 0.54 0.50 0.59 77 152498 101 6 47 48 77
SRP_00079 0.47 0.47 0.48 41 41819 53 3 42 8 47
SRP_00124 0.30 0.29 0.30 25 37046 57 8 49 0 60
SRP_00137 0.75 0.84 0.68 21 4155 10 4 6 0 4
SRP_00141 0.92 0.93 0.90 38 6286 4 1 3 0 3
SRP_00146 0.92 0.92 0.92 33 5217 3 1 2 0 3
SRP_00182 0.72 0.72 0.72 73 45954 32 9 20 3 28
SRP_00197 0.15 0.15 0.16 15 49362 78 9 69 0 86
SRP_00198 0.39 0.39 0.39 40 50938 63 4 58 1 63
SRP_00199 0.70 0.70 0.70 73 50936 32 3 28 1 32
SRP_00202 0.85 0.84 0.87 98 54172 16 2 13 1 19
SRP_00241 0.54 0.57 0.51 47 45963 48 17 29 2 35
SRP_00255 0.63 0.65 0.61 61 47486 41 7 32 2 33
SRP_00260 0.73 0.73 0.74 73 47796 28 7 19 2 27
SRP_00278 0.77 0.76 0.77 80 45952 26 5 19 2 25
SRP_00285 0.98 0.97 1.00 29 3712 0 0 0 0 1
SRP_00322 0.82 0.82 0.82 89 48407 20 3 17 0 20
SRP_00328 0.55 0.55 0.55 47 39536 41 8 30 3 38
SRP_00329 0.25 0.25 0.25 21 39818 65 10 54 1 64
SRP_00338 0.96 0.94 0.97 34 5425 1 0 1 0 2
SRP_00339 0.58 0.61 0.56 57 44449 46 5 40 1 37
SRP_00340 0.48 0.50 0.47 41 41240 50 8 39 3 41

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.