CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Murlet(seed) - scored higher in this pairwise comparison

  4. Performance of RNASLOpt - scored lower in this pairwise comparison

  5. Compile and download dataset for Murlet(seed) & RNASLOpt [.zip] - may take several seconds...


Overview

Metric Murlet(seed) RNASLOpt
MCC 0.519 > 0.506
Average MCC ± 95% Confidence Intervals 0.389 ± 0.066 < 0.490 ± 0.054
Sensitivity 0.322 < 0.472
Positive Predictive Value 0.839 > 0.545
Total TP 1315 < 1926
Total TN 1508018 > 1506051
Total FP 308 < 1845
Total FP CONTRA 9 < 308
Total FP INCONS 243 < 1300
Total FP COMP 56 < 237
Total FN 2765 > 2154
P-value 2.71856013146e-06

^top




Performance plots


  1. Comparison of performance of Murlet(seed) and RNASLOpt. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Murlet(seed) and RNASLOpt).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Murlet(seed) and RNASLOpt).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Murlet(seed) and RNASLOpt. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Murlet(seed) and RNASLOpt).

^top





Performance of Murlet(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1315
Total TN 1508018
Total FP 308
Total FP CONTRA 9
Total FP INCONS 243
Total FP COMP 56
Total FN 2765
Total Scores
MCC 0.519
Average MCC ± 95% Confidence Intervals 0.389 ± 0.066
Sensitivity 0.322
Positive Predictive Value 0.839
Nr of predictions 120

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top



Performance of RNASLOpt - scored lower in this pairwise comparison

1. Total counts & total scores for RNASLOpt

Total Base Pair Counts
Total TP 1926
Total TN 1506051
Total FP 1845
Total FP CONTRA 308
Total FP INCONS 1300
Total FP COMP 237
Total FN 2154
Total Scores
MCC 0.506
Average MCC ± 95% Confidence Intervals 0.490 ± 0.054
Sensitivity 0.472
Positive Predictive Value 0.545
Nr of predictions 120

^top



2. Individual counts for RNASLOpt [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.90 0.80 1.00 33 5017 0 0 0 0 8
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2680 22 0 21 1 23
PDB_00810 0.51 0.41 0.64 7 1070 4 2 2 0 10
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.79 0.69 0.90 9 893 2 0 1 1 4
PDB_01092 0.74 0.63 0.87 33 10115 7 1 4 2 19
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00390 0.77 0.60 1.00 9 1422 1 0 0 1 6
RFA_00416 0.77 0.60 1.00 9 1476 1 0 0 1 6
RFA_00433 0.77 0.60 1.00 9 1422 1 0 0 1 6
RFA_00587 1.00 1.00 1.00 15 4836 13 0 0 13 0
RFA_00603 0.54 0.51 0.56 18 13498 15 4 10 1 17
RFA_00604 0.73 0.66 0.82 23 13502 12 4 1 7 12
RFA_00605 0.36 0.34 0.39 12 15194 27 3 16 8 23
RFA_00606 0.18 0.18 0.18 7 21281 39 4 29 6 32
RFA_00607 0.90 0.83 0.97 30 17735 12 0 1 11 6
RFA_00609 0.92 0.89 0.94 33 17920 10 0 2 8 4
RFA_00610 0.90 0.86 0.94 32 17171 10 0 2 8 5
RFA_00611 0.69 0.66 0.72 23 13009 9 2 7 0 12
RFA_00613 0.16 0.14 0.18 5 12852 27 9 14 4 30
RFA_00615 0.47 0.43 0.52 15 13337 14 8 6 0 20
RFA_00620 0.66 0.64 0.68 25 21908 22 2 10 10 14
RFA_00626 0.68 0.64 0.72 56 56538 30 6 16 8 31
RFA_00627 0.61 0.60 0.62 52 56869 41 11 21 9 35
RFA_00628 0.89 0.84 0.95 72 57215 15 0 4 11 14
RFA_00630 0.69 0.67 0.72 58 56872 32 5 18 9 29
RFA_00632 0.59 0.57 0.62 16 4069 10 2 8 0 12
RFA_00636 0.65 0.64 0.67 18 3978 9 2 7 0 10
RFA_00639 0.29 0.29 0.29 25 54528 67 12 50 5 62
RFA_00642 -0.01 0.00 0.00 0 2914 12 1 11 0 18
RFA_00643 -0.01 0.00 0.00 0 2199 12 1 11 0 18
RFA_00644 -0.01 0.00 0.00 0 2680 21 6 15 0 18
RFA_00645 -0.01 0.00 0.00 0 2403 12 3 9 0 18
RFA_00649 0.33 0.33 0.33 6 2127 12 6 6 0 12
RFA_00651 0.27 0.22 0.33 4 2068 8 1 7 0 14
RFA_00653 0.33 0.33 0.33 6 2127 12 3 9 0 12
RFA_00654 0.59 0.50 0.69 9 2402 5 1 3 1 9
RFA_00658 0.40 0.29 0.57 4 1121 5 0 3 2 10
RFA_00659 -0.01 0.00 0.00 0 1125 5 0 3 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.29 0.29 0.31 4 977 10 1 8 1 10
RFA_00668 0.50 0.43 0.60 6 980 4 0 4 0 8
RFA_00672 -0.01 0.00 0.00 0 897 6 0 6 0 13
RFA_00673 0.47 0.36 0.63 5 1120 4 0 3 1 9
RFA_00674 0.74 0.71 0.77 10 1115 4 0 3 1 4
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.74 0.71 0.77 10 977 5 0 3 2 4
RFA_00678 0.53 0.29 1.00 4 942 0 0 0 0 10
RFA_00680 0.84 0.71 1.00 10 1118 2 0 0 2 4
RFA_00684 0.84 0.71 1.00 10 980 1 0 0 1 4
RFA_00685 0.47 0.36 0.63 5 982 4 0 3 1 9
RFA_00695 0.30 0.29 0.31 4 7008 25 2 7 16 10
RFA_00703 0.41 0.43 0.40 6 4263 20 3 6 11 8
RFA_00704 -0.01 0.00 0.00 0 982 8 0 8 0 14
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 0.37 0.29 0.50 4 1027 4 0 4 0 10
RFA_00707 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00708 0.53 0.29 1.00 4 1031 0 0 0 0 10
RFA_00709 -0.01 0.00 0.00 0 984 7 0 6 1 14
RFA_00710 -0.01 0.00 0.00 0 984 6 0 6 0 14
RFA_00711 -0.01 0.00 0.00 0 1026 9 0 9 0 14
RFA_00715 -0.01 0.00 0.00 0 939 7 0 7 0 14
RFA_00716 -0.01 0.00 0.00 0 939 7 0 7 0 14
RFA_00717 -0.01 0.00 0.00 0 894 9 0 9 0 14
RFA_00730 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00731 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00733 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00734 0.67 0.58 0.78 7 894 3 0 2 1 5
RFA_00736 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00737 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00745 -0.01 0.00 0.00 0 941 5 1 4 0 12
RFA_00749 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00758 -0.01 0.00 0.00 0 898 5 0 5 0 12
RFA_00762 -0.01 0.00 0.00 0 894 9 2 7 0 12
RFA_00763 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00764 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00765 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00767 0.63 0.56 0.71 10 1877 4 0 4 0 8
RFA_00768 0.61 0.56 0.67 10 1876 5 0 5 0 8
RFA_00769 0.59 0.56 0.63 10 1937 6 1 5 0 8
RFA_00770 0.47 0.39 0.58 7 2004 5 1 4 0 11
RFA_00773 0.59 0.56 0.63 10 1937 6 1 5 0 8
RFA_00779 0.74 0.56 1.00 10 1943 0 0 0 0 8
RFA_00781 0.51 0.50 0.53 16 5020 14 2 12 0 16
RFA_00786 -0.01 0.00 0.00 0 5022 28 4 24 0 32
RFA_00791 0.44 0.41 0.48 13 5124 14 4 10 0 19
RFA_00792 0.89 0.84 0.93 27 5021 2 0 2 0 5
RFA_00801 0.82 0.75 0.89 24 5023 4 0 3 1 8
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.40 0.38 0.43 6 2131 8 1 7 0 10
RFA_00814 0.51 0.51 0.51 21 25159 29 9 11 9 20
RFA_00815 0.41 0.41 0.41 17 24490 31 1 23 7 24
RFA_00816 0.58 0.63 0.53 26 23171 26 13 10 3 15
RFA_00817 0.24 0.27 0.22 11 21896 42 12 26 4 30
RFA_00818 0.21 0.22 0.21 9 20258 39 14 20 5 32
RFA_00819 0.60 0.68 0.53 28 27913 34 16 9 9 13
SRP_00079 0.62 0.57 0.68 50 41832 24 3 20 1 38
SRP_00124 0.39 0.36 0.43 31 37056 41 8 33 0 54
SRP_00137 0.67 0.72 0.62 18 4157 11 4 7 0 7
SRP_00141 0.84 0.80 0.87 33 6290 5 1 4 0 8
SRP_00146 0.82 0.67 1.00 24 5229 0 0 0 0 12
SRP_00182 0.54 0.51 0.57 52 45965 39 11 28 0 49
SRP_00197 0.23 0.21 0.25 21 49370 64 9 55 0 80
SRP_00198 0.41 0.38 0.45 39 50954 47 5 42 0 64
SRP_00199 0.24 0.23 0.26 24 50949 67 3 64 0 81
SRP_00202 0.52 0.49 0.56 57 54184 44 6 38 0 60
SRP_00241 0.26 0.27 0.26 22 45970 66 14 50 2 60
SRP_00255 0.29 0.30 0.29 28 47489 69 11 58 0 66
SRP_00260 0.69 0.66 0.72 66 47803 28 8 18 2 34
SRP_00278 0.68 0.65 0.72 68 45961 29 4 23 2 37
SRP_00285 0.87 0.77 1.00 23 3718 0 0 0 0 7
SRP_00322 0.57 0.54 0.61 59 48419 38 1 37 0 50
SRP_00328 0.60 0.60 0.61 51 39537 36 7 26 3 34
SRP_00329 0.34 0.32 0.36 27 39829 50 4 43 3 58
SRP_00338 0.93 0.86 1.00 31 5429 0 0 0 0 5
SRP_00339 0.44 0.43 0.45 40 44462 51 6 43 2 54
SRP_00340 0.30 0.32 0.29 26 41238 64 12 52 0 56

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.