CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CentroidAlifold(seed) - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for CentroidAlifold(seed) & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric CentroidAlifold(seed) Murlet(seed)
MCC 0.817 > 0.540
Average MCC ± 95% Confidence Intervals 0.776 ± 0.040 > 0.396 ± 0.065
Sensitivity 0.728 > 0.360
Positive Predictive Value 0.916 > 0.811
Total TP 3516 > 1738
Total TN 2787802 < 2789495
Total FP 610 > 489
Total FP CONTRA 44 > 21
Total FP INCONS 277 < 385
Total FP COMP 289 > 83
Total FN 1312 < 3090
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CentroidAlifold(seed) and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CentroidAlifold(seed) and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CentroidAlifold(seed) and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CentroidAlifold(seed) and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CentroidAlifold(seed) and Murlet(seed)).

^top





Performance of CentroidAlifold(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for CentroidAlifold(seed)

Total Base Pair Counts
Total TP 3516
Total TN 2787802
Total FP 610
Total FP CONTRA 44
Total FP INCONS 277
Total FP COMP 289
Total FN 1312
Total Scores
MCC 0.817
Average MCC ± 95% Confidence Intervals 0.776 ± 0.040
Sensitivity 0.728
Positive Predictive Value 0.916
Nr of predictions 123

^top



2. Individual counts for CentroidAlifold(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.86 0.81 0.92 351 977520 61 3 27 31 83
PDB_00005 0.80 0.64 1.00 9 937 0 0 0 0 5
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.95 0.91 1.00 10 455 0 0 0 0 1
PDB_00716 0.75 0.61 0.93 14 2686 1 0 1 0 9
PDB_00810 0.87 0.76 1.00 13 1068 1 0 0 1 4
PDB_01050 0.68 0.46 1.00 6 624 2 0 0 2 7
PDB_01051 0.96 0.92 1.00 12 891 4 0 0 4 1
PDB_01092 0.72 0.58 0.91 30 10120 4 0 3 1 22
PDB_01152 0.80 0.64 1.00 9 552 0 0 0 0 5
RFA_00390 0.97 0.93 1.00 14 1417 2 0 0 2 1
RFA_00416 0.97 0.93 1.00 14 1471 2 0 0 2 1
RFA_00433 0.97 0.93 1.00 14 1417 2 0 0 2 1
RFA_00587 1.00 1.00 1.00 15 4836 3 0 0 3 0
RFA_00603 0.96 0.91 1.00 32 13498 0 0 0 0 3
RFA_00604 0.97 0.94 1.00 33 13497 0 0 0 0 2
RFA_00605 0.97 0.94 1.00 33 15192 0 0 0 0 2
RFA_00606 0.91 0.82 1.00 32 21289 0 0 0 0 7
RFA_00607 0.94 0.89 1.00 32 17734 0 0 0 0 4
RFA_00609 0.93 0.86 1.00 32 17923 0 0 0 0 5
RFA_00610 0.93 0.86 1.00 32 17173 0 0 0 0 5
RFA_00611 0.96 0.91 1.00 32 13009 0 0 0 0 3
RFA_00613 0.97 0.94 1.00 33 12847 0 0 0 0 2
RFA_00615 0.85 0.80 0.90 28 13335 3 0 3 0 7
RFA_00620 0.91 0.82 1.00 32 21913 0 0 0 0 7
RFA_00626 0.98 0.95 1.00 83 56533 8 0 0 8 4
RFA_00627 0.98 0.97 1.00 84 56869 10 0 0 10 3
RFA_00628 0.98 0.95 1.00 82 57209 12 0 0 12 4
RFA_00630 0.93 0.90 0.96 78 56872 16 0 3 13 9
RFA_00632 0.74 0.64 0.86 18 4074 3 2 1 0 10
RFA_00636 0.78 0.68 0.90 19 3984 2 2 0 0 9
RFA_00639 0.67 0.48 0.93 42 54570 3 0 3 0 45
RFA_00642 0.41 0.17 1.00 3 2923 0 0 0 0 15
RFA_00643 0.00 0.00 0.00 0 2211 0 0 0 0 18
RFA_00644 0.00 0.00 0.00 0 2701 0 0 0 0 18
RFA_00645 0.23 0.06 1.00 1 2414 0 0 0 0 17
RFA_00649 0.00 0.00 0.00 0 2145 0 0 0 0 18
RFA_00651 0.00 0.00 0.00 0 2080 0 0 0 0 18
RFA_00653 0.33 0.11 1.00 2 2143 0 0 0 0 16
RFA_00654 0.00 0.00 0.00 0 2415 0 0 0 0 18
RFA_00658 0.71 0.57 0.89 8 1119 2 0 1 1 6
RFA_00659 0.65 0.43 1.00 6 1122 0 0 0 0 8
RFA_00664 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00667 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00668 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00672 0.78 0.62 1.00 8 895 0 0 0 0 5
RFA_00673 0.65 0.43 1.00 6 1122 0 0 0 0 8
RFA_00674 0.60 0.43 0.86 6 1121 1 0 1 0 8
RFA_00675 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00677 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00678 0.65 0.43 1.00 6 940 0 0 0 0 8
RFA_00680 0.65 0.43 1.00 6 1122 0 0 0 0 8
RFA_00684 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00685 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00695 0.80 0.64 1.00 9 7012 0 0 0 0 5
RFA_00703 0.80 0.64 1.00 9 4269 0 0 0 0 5
RFA_00704 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00705 0.76 0.64 0.90 9 1025 2 0 1 1 5
RFA_00706 0.65 0.43 1.00 6 1029 0 0 0 0 8
RFA_00707 0.65 0.43 1.00 6 1029 0 0 0 0 8
RFA_00708 0.71 0.57 0.89 8 1026 2 0 1 1 6
RFA_00709 0.65 0.43 1.00 6 984 0 0 0 0 8
RFA_00710 0.72 0.64 0.82 9 979 3 0 2 1 5
RFA_00711 0.65 0.43 1.00 6 1029 0 0 0 0 8
RFA_00715 0.65 0.43 1.00 6 940 0 0 0 0 8
RFA_00716 0.65 0.43 1.00 6 940 0 0 0 0 8
RFA_00717 0.75 0.57 1.00 8 895 0 0 0 0 6
RFA_00730 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00731 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00733 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00734 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00736 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00737 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00745 1.00 1.00 1.00 12 934 2 0 0 2 0
RFA_00749 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00758 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00762 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00763 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00764 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00765 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00767 0.74 0.56 1.00 10 1881 0 0 0 0 8
RFA_00768 0.74 0.56 1.00 10 1881 0 0 0 0 8
RFA_00769 0.74 0.56 1.00 10 1943 0 0 0 0 8
RFA_00770 0.74 0.56 1.00 10 2006 0 0 0 0 8
RFA_00773 0.74 0.56 1.00 10 1943 0 0 0 0 8
RFA_00779 0.74 0.56 1.00 10 1943 0 0 0 0 8
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00791 0.97 0.94 1.00 30 5121 0 0 0 0 2
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00808 0.65 0.56 0.75 9 2004 3 1 2 0 7
RFA_00809 0.43 0.38 0.50 6 2133 6 0 6 0 10
RFA_00814 0.91 0.83 1.00 34 25166 1 0 0 1 7
RFA_00815 0.94 0.88 1.00 36 24495 1 0 0 1 5
RFA_00816 0.94 0.88 1.00 36 23184 1 0 0 1 5
RFA_00817 0.92 0.85 1.00 35 21910 1 0 0 1 6
RFA_00818 0.94 0.88 1.00 36 20265 1 0 0 1 5
RFA_00819 0.94 0.88 1.00 36 27930 1 0 0 1 5
RFA_00820 0.90 0.83 0.99 133 151390 14 0 2 12 27
RFA_00830 0.76 0.68 0.84 105 152503 32 1 19 12 49
SRP_00079 0.74 0.68 0.81 60 41831 29 0 14 15 28
SRP_00124 0.64 0.58 0.72 49 37060 31 3 16 12 36
SRP_00137 0.66 0.68 0.65 17 4160 10 3 6 1 8
SRP_00141 0.78 0.66 0.93 27 6299 3 1 1 1 14
SRP_00146 0.83 0.75 0.93 27 5224 3 1 1 1 9
SRP_00182 0.87 0.79 0.95 80 45972 11 0 4 7 21
SRP_00197 0.90 0.83 0.97 84 49368 12 0 3 9 17
SRP_00198 0.90 0.83 0.97 86 50951 11 0 3 8 17
SRP_00199 0.87 0.81 0.93 85 50949 12 0 6 6 20
SRP_00202 0.86 0.76 0.97 89 54193 7 0 3 4 28
SRP_00241 0.53 0.51 0.55 42 45979 47 8 27 12 40
SRP_00255 0.78 0.73 0.82 69 47502 25 1 14 10 25
SRP_00260 0.58 0.52 0.64 52 47814 34 5 24 5 48
SRP_00278 0.83 0.74 0.92 78 45971 10 0 7 3 27
SRP_00285 0.87 0.77 1.00 23 3718 2 0 0 2 7
SRP_00322 0.83 0.74 0.93 81 48429 10 0 6 4 28
SRP_00328 0.59 0.54 0.65 46 39550 30 5 20 5 39
SRP_00329 0.71 0.64 0.79 54 39835 24 2 12 10 31
SRP_00338 0.90 0.81 1.00 29 5431 1 0 0 1 7
SRP_00339 0.81 0.74 0.88 70 44471 17 1 9 7 24
SRP_00340 0.55 0.52 0.59 43 41255 43 5 25 13 39

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1738
Total TN 2789495
Total FP 489
Total FP CONTRA 21
Total FP INCONS 385
Total FP COMP 83
Total FN 3090
Total Scores
MCC 0.540
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065
Sensitivity 0.360
Positive Predictive Value 0.811
Nr of predictions 123

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.