CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PknotsRG - scored higher in this pairwise comparison

  4. Performance of Murlet(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for PknotsRG & Murlet(seed) [.zip] - may take several seconds...


Overview

Metric PknotsRG Murlet(seed)
MCC 0.598 > 0.540
Average MCC ± 95% Confidence Intervals 0.603 ± 0.057 > 0.396 ± 0.065
Sensitivity 0.597 > 0.360
Positive Predictive Value 0.601 < 0.811
Total TP 2881 > 1738
Total TN 2786848 < 2789495
Total FP 2394 > 489
Total FP CONTRA 317 > 21
Total FP INCONS 1593 > 385
Total FP COMP 484 > 83
Total FN 1947 < 3090
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of PknotsRG and Murlet(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PknotsRG and Murlet(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PknotsRG and Murlet(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PknotsRG and Murlet(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PknotsRG and Murlet(seed)).

^top





Performance of PknotsRG - scored higher in this pairwise comparison

1. Total counts & total scores for PknotsRG

Total Base Pair Counts
Total TP 2881
Total TN 2786848
Total FP 2394
Total FP CONTRA 317
Total FP INCONS 1593
Total FP COMP 484
Total FN 1947
Total Scores
MCC 0.598
Average MCC ± 95% Confidence Intervals 0.603 ± 0.057
Sensitivity 0.597
Positive Predictive Value 0.601
Nr of predictions 123

^top



2. Individual counts for PknotsRG [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.34 0.33 0.34 145 977474 293 46 236 11 289
PDB_00005 1.00 1.00 1.00 14 932 0 0 0 0 0
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.87 0.83 0.92 34 5013 3 2 1 0 7
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2675 27 1 25 1 23
PDB_00810 0.91 0.82 1.00 14 1067 0 0 0 0 3
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.88 0.85 0.92 11 891 5 0 1 4 2
PDB_01092 0.71 0.67 0.74 35 10106 13 2 10 1 17
PDB_01152 0.96 0.93 1.00 13 548 0 0 0 0 1
RFA_00390 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00416 0.93 0.93 0.93 14 1470 4 0 1 3 1
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00587 1.00 1.00 1.00 15 4836 12 0 0 12 0
RFA_00603 0.69 0.74 0.63 26 13489 18 4 11 3 9
RFA_00604 0.70 0.71 0.69 25 13494 22 2 9 11 10
RFA_00605 0.33 0.34 0.32 12 15188 34 4 21 9 23
RFA_00606 0.66 0.67 0.65 26 21281 28 6 8 14 13
RFA_00607 0.83 0.83 0.83 30 17730 18 4 2 12 6
RFA_00609 0.75 0.76 0.74 28 17917 28 2 8 18 9
RFA_00610 0.77 0.76 0.78 28 17169 23 2 6 15 9
RFA_00611 0.87 0.86 0.88 30 13007 9 0 4 5 5
RFA_00613 0.66 0.66 0.66 23 12845 14 4 8 2 12
RFA_00615 0.63 0.66 0.61 23 13328 19 5 10 4 12
RFA_00620 0.63 0.62 0.65 24 21908 26 4 9 13 15
RFA_00626 0.89 0.86 0.93 75 56535 28 2 4 22 12
RFA_00627 0.73 0.70 0.76 61 56873 35 4 15 16 26
RFA_00628 0.78 0.74 0.81 64 57212 40 0 15 25 22
RFA_00630 0.60 0.60 0.60 52 56866 44 8 27 9 35
RFA_00632 0.88 0.89 0.86 25 4066 6 2 2 2 3
RFA_00636 0.95 0.96 0.93 27 3976 4 2 0 2 1
RFA_00639 0.23 0.24 0.21 21 54517 81 16 61 4 66
RFA_00642 -0.01 0.00 0.00 0 2910 16 3 13 0 18
RFA_00643 -0.01 0.00 0.00 0 2196 15 1 14 0 18
RFA_00644 -0.01 0.00 0.00 0 2677 24 4 20 0 18
RFA_00645 0.19 0.17 0.23 3 2402 11 1 9 1 15
RFA_00649 0.22 0.22 0.22 4 2127 16 0 14 2 14
RFA_00651 0.35 0.33 0.38 6 2064 10 1 9 0 12
RFA_00653 0.33 0.33 0.33 6 2127 12 3 9 0 12
RFA_00654 -0.01 0.00 0.00 0 2398 17 2 15 0 18
RFA_00658 0.33 0.29 0.40 4 1118 8 0 6 2 10
RFA_00659 -0.01 0.00 0.00 0 1120 10 0 8 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.29 0.29 0.31 4 977 10 1 8 1 10
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 -0.01 0.00 0.00 0 895 9 0 8 1 13
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.74 0.71 0.77 10 1115 4 0 3 1 4
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.42 0.43 0.43 6 976 9 0 8 1 8
RFA_00678 0.33 0.29 0.40 4 936 6 0 6 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.59 0.57 0.62 8 977 6 0 5 1 6
RFA_00685 0.42 0.36 0.50 5 980 6 0 5 1 9
RFA_00695 0.56 0.50 0.64 7 7010 29 1 3 25 7
RFA_00703 0.67 0.64 0.69 9 4265 18 1 3 14 5
RFA_00704 -0.01 0.00 0.00 0 983 8 0 7 1 14
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 0.33 0.29 0.40 4 1025 7 0 6 1 10
RFA_00707 -0.01 0.00 0.00 0 1025 10 1 9 0 14
RFA_00708 0.35 0.29 0.44 4 1026 6 0 5 1 10
RFA_00709 -0.01 0.00 0.00 0 979 12 0 11 1 14
RFA_00710 -0.01 0.00 0.00 0 981 9 0 9 0 14
RFA_00711 0.33 0.29 0.40 4 1025 6 0 6 0 10
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 935 11 0 11 0 14
RFA_00717 0.66 0.64 0.69 9 890 4 0 4 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00733 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00734 0.18 0.17 0.22 2 894 8 0 7 1 10
RFA_00736 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00737 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00745 1.00 1.00 1.00 12 934 1 0 0 1 0
RFA_00749 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00758 0.96 0.92 1.00 11 892 0 0 0 0 1
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00767 1.00 1.00 1.00 18 1873 3 0 0 3 0
RFA_00768 1.00 1.00 1.00 18 1873 2 0 0 2 0
RFA_00769 0.97 1.00 0.95 18 1934 3 1 0 2 0
RFA_00770 0.88 0.78 1.00 14 2002 3 0 0 3 4
RFA_00773 0.97 1.00 0.95 18 1934 4 1 0 3 0
RFA_00779 0.97 0.94 1.00 17 1936 2 0 0 2 1
RFA_00781 0.58 0.59 0.58 19 5017 14 2 12 0 13
RFA_00786 0.71 0.72 0.70 23 5017 10 2 8 0 9
RFA_00791 0.50 0.50 0.50 16 5119 16 5 11 0 16
RFA_00792 0.88 0.91 0.85 29 5016 7 2 3 2 3
RFA_00801 0.81 0.81 0.81 26 5018 7 1 5 1 6
RFA_00808 1.00 1.00 1.00 16 2000 0 0 0 0 0
RFA_00809 0.79 0.81 0.76 13 2128 4 0 4 0 3
RFA_00814 0.91 0.95 0.87 39 25155 23 4 2 17 2
RFA_00815 0.66 0.71 0.62 29 24484 33 5 13 15 12
RFA_00816 0.68 0.73 0.63 30 23172 26 12 6 8 11
RFA_00817 0.10 0.12 0.08 5 21885 56 16 39 1 36
RFA_00818 0.00 0.00 0.00 0 20245 56 20 36 0 41
RFA_00819 0.79 0.80 0.79 33 27924 34 0 9 25 8
RFA_00820 0.86 0.87 0.85 139 151361 39 1 24 14 21
RFA_00830 0.50 0.47 0.54 73 152492 108 5 58 45 81
SRP_00079 0.57 0.59 0.56 52 41812 46 3 38 5 36
SRP_00124 0.57 0.56 0.57 48 37044 38 6 30 2 37
SRP_00137 0.74 0.84 0.66 21 4154 13 4 7 2 4
SRP_00141 0.89 0.88 0.90 36 6288 4 1 3 0 5
SRP_00146 0.93 0.94 0.92 34 5216 3 1 2 0 2
SRP_00182 0.74 0.72 0.77 73 45961 23 5 17 1 28
SRP_00197 0.38 0.38 0.39 38 49357 61 4 56 1 63
SRP_00198 0.38 0.38 0.38 39 50937 65 4 60 1 64
SRP_00199 0.31 0.30 0.34 31 50948 64 4 57 3 74
SRP_00202 0.84 0.84 0.84 98 54169 22 2 16 4 19
SRP_00241 0.54 0.59 0.50 48 45960 50 12 36 2 34
SRP_00255 0.60 0.62 0.59 58 47487 47 3 38 6 36
SRP_00260 0.75 0.78 0.72 78 47787 32 8 22 2 22
SRP_00278 0.72 0.72 0.72 76 45951 31 4 25 2 29
SRP_00285 0.98 0.97 1.00 29 3712 0 0 0 0 1
SRP_00322 0.61 0.61 0.60 67 48405 45 6 38 1 42
SRP_00328 0.69 0.71 0.68 60 39533 31 7 21 3 25
SRP_00329 0.75 0.74 0.75 63 39819 24 3 18 3 22
SRP_00338 0.94 0.94 0.94 34 5424 2 0 2 0 2
SRP_00339 0.56 0.59 0.53 55 44448 50 6 42 2 39
SRP_00340 0.54 0.57 0.52 47 41237 46 8 36 2 35

^top



Performance of Murlet(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP 1738
Total TN 2789495
Total FP 489
Total FP CONTRA 21
Total FP INCONS 385
Total FP COMP 83
Total FN 3090
Total Scores
MCC 0.540
Average MCC ± 95% Confidence Intervals 0.396 ± 0.065
Sensitivity 0.360
Positive Predictive Value 0.811
Nr of predictions 123

^top



2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_01343 0.64 0.61 0.68 263 977515 145 6 117 22 171
PDB_00005 0.53 0.29 1.00 4 942 2 0 0 2 10
PDB_00012 0.65 0.43 1.00 3 403 0 0 0 0 4
PDB_00213 0.73 0.59 0.92 24 5024 2 0 2 0 17
PDB_00553 0.45 0.27 0.75 3 461 1 0 1 0 8
PDB_00716 0.23 0.13 0.43 3 2694 4 0 4 0 20
PDB_00810 0.63 0.53 0.75 9 1069 3 0 3 0 8
PDB_01050 0.55 0.31 1.00 4 626 1 0 0 1 9
PDB_01051 0.00 0.00 0.00 0 903 0 0 0 0 13
PDB_01092 0.69 0.48 1.00 25 10128 0 0 0 0 27
PDB_01152 0.76 0.64 0.90 9 551 1 0 1 0 5
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.00 0.00 0.00 0 1485 0 0 0 0 15
RFA_00433 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00587 0.00 0.00 0.00 0 4851 0 0 0 0 15
RFA_00603 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00604 0.00 0.00 0.00 0 13530 0 0 0 0 35
RFA_00605 0.00 0.00 0.00 0 15225 0 0 0 0 35
RFA_00606 0.00 0.00 0.00 0 21321 0 0 0 0 39
RFA_00607 0.00 0.00 0.00 0 17766 0 0 0 0 36
RFA_00609 0.00 0.00 0.00 0 17955 0 0 0 0 37
RFA_00610 0.00 0.00 0.00 0 17205 0 0 0 0 37
RFA_00611 0.00 0.00 0.00 0 13041 0 0 0 0 35
RFA_00613 0.00 0.00 0.00 0 12880 0 0 0 0 35
RFA_00615 0.00 0.00 0.00 0 13366 0 0 0 0 35
RFA_00620 0.00 0.00 0.00 0 21945 0 0 0 0 39
RFA_00626 0.81 0.66 1.00 57 56559 4 0 0 4 30
RFA_00627 0.84 0.70 1.00 61 56892 5 0 0 5 26
RFA_00628 0.83 0.69 1.00 59 57232 8 0 0 8 27
RFA_00630 0.82 0.67 1.00 58 56895 4 0 0 4 29
RFA_00632 0.21 0.11 0.43 3 4088 4 0 4 0 25
RFA_00636 0.21 0.11 0.43 3 3998 4 0 4 0 25
RFA_00639 0.21 0.11 0.40 10 54590 15 4 11 0 77
RFA_00642 0.57 0.44 0.73 8 2915 3 0 3 0 10
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.42 0.33 0.55 6 2690 5 0 5 0 12
RFA_00645 0.00 0.00 0.00 0 2407 8 0 8 0 18
RFA_00649 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00651 0.42 0.33 0.55 6 2069 5 0 5 0 12
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.57 0.44 0.73 8 2404 3 0 3 0 10
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.00 0.00 0.00 0 4278 0 0 0 0 14
RFA_00704 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00731 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00733 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00734 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00736 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00737 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00745 1.00 1.00 1.00 12 934 0 0 0 0 0
RFA_00749 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00758 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00762 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00765 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00767 0.47 0.33 0.67 6 1882 3 0 3 0 12
RFA_00768 0.41 0.28 0.63 5 1883 3 0 3 0 13
RFA_00769 0.57 0.44 0.73 8 1942 3 0 3 0 10
RFA_00770 0.57 0.44 0.73 8 2005 3 0 3 0 10
RFA_00773 0.41 0.28 0.63 5 1945 3 0 3 0 13
RFA_00779 0.47 0.33 0.67 6 1944 3 0 3 0 12
RFA_00781 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00786 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00791 0.83 0.75 0.92 24 5125 2 0 2 0 8
RFA_00792 0.83 0.75 0.92 24 5024 2 0 2 0 8
RFA_00801 0.80 0.72 0.88 23 5024 3 0 3 0 9
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.50 0.38 0.67 6 2136 3 0 3 0 10
RFA_00814 0.75 0.63 0.90 26 25171 4 0 3 1 15
RFA_00815 0.83 0.68 1.00 28 24503 1 0 0 1 13
RFA_00816 0.75 0.63 0.90 26 23191 4 0 3 1 15
RFA_00817 0.83 0.68 1.00 28 21917 1 0 0 1 13
RFA_00818 0.83 0.68 1.00 28 20273 1 0 0 1 13
RFA_00819 0.83 0.68 1.00 28 27938 0 0 0 0 13
RFA_00820 0.71 0.56 0.89 90 151424 13 3 8 2 70
RFA_00830 0.59 0.45 0.78 70 152538 23 3 17 3 84
SRP_00079 0.54 0.33 0.88 29 41872 6 0 4 2 59
SRP_00124 0.15 0.09 0.24 8 37094 27 3 23 1 77
SRP_00137 0.49 0.24 1.00 6 4180 0 0 0 0 19
SRP_00141 0.35 0.12 1.00 5 6323 1 0 0 1 36
SRP_00146 0.41 0.17 1.00 6 5247 0 0 0 0 30
SRP_00182 0.59 0.35 1.00 35 46021 1 0 0 1 66
SRP_00197 0.47 0.28 0.80 28 49420 8 0 7 1 73
SRP_00198 0.47 0.27 0.82 28 51006 7 0 6 1 75
SRP_00199 0.47 0.24 0.93 25 51013 3 0 2 1 80
SRP_00202 0.48 0.25 0.94 29 54254 2 0 2 0 88
SRP_00241 0.51 0.33 0.79 27 46022 9 0 7 2 55
SRP_00255 0.65 0.43 1.00 40 47546 2 0 0 2 54
SRP_00260 0.21 0.14 0.33 14 47852 29 1 28 0 86
SRP_00278 0.53 0.30 0.94 31 46023 3 0 2 1 74
SRP_00285 0.45 0.20 1.00 6 3735 0 0 0 0 24
SRP_00322 0.44 0.25 0.79 27 48482 8 0 7 1 82
SRP_00328 0.49 0.34 0.71 29 39580 16 1 11 4 56
SRP_00329 0.50 0.34 0.73 29 39863 16 0 11 5 56
SRP_00338 0.41 0.17 1.00 6 5454 0 0 0 0 30
SRP_00339 0.49 0.32 0.77 30 44512 9 0 9 0 64
SRP_00340 0.44 0.27 0.73 22 41298 12 0 8 4 60

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.