CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of UNAFold - scored higher in this pairwise comparison

  4. Performance of CMfinder(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for UNAFold & CMfinder(seed) [.zip] - may take several seconds...


Overview

Metric UNAFold CMfinder(seed)
MCC 0.610 > 0.486
Average MCC ± 95% Confidence Intervals 0.570 ± 0.051 > 0.505 ± 0.044
Sensitivity 0.602 > 0.312
Positive Predictive Value 0.620 < 0.757
Total TP 3670 > 1905
Total TN 2877835 < 2881233
Total FP 2760 > 688
Total FP CONTRA 433 > 40
Total FP INCONS 1812 > 572
Total FP COMP 515 > 76
Total FN 2427 < 4192
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of UNAFold and CMfinder(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for UNAFold and CMfinder(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for UNAFold and CMfinder(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for UNAFold and CMfinder(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for UNAFold and CMfinder(seed)).

^top





Performance of UNAFold - scored higher in this pairwise comparison

1. Total counts & total scores for UNAFold

Total Base Pair Counts
Total TP 3670
Total TN 2877835
Total FP 2760
Total FP CONTRA 433
Total FP INCONS 1812
Total FP COMP 515
Total FN 2427
Total Scores
MCC 0.610
Average MCC ± 95% Confidence Intervals 0.570 ± 0.051
Sensitivity 0.602
Positive Predictive Value 0.620
Nr of predictions 129

^top



2. Individual counts for UNAFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.24 0.24 0.24 28 103624 107 18 70 19 87
CRW_00016 0.62 0.57 0.67 68 77320 45 5 28 12 52
CRW_00610 0.45 0.44 0.46 36 36236 43 6 37 0 45
CRW_00613 0.63 0.58 0.68 45 34914 29 1 20 8 33
CRW_00614 0.09 0.11 0.08 6 121694 140 20 51 69 51
CRW_00618 0.18 0.20 0.17 12 56210 83 10 48 25 49
CRW_00633 0.69 0.67 0.71 71 63446 33 8 21 4 35
CRW_00634 0.33 0.34 0.33 32 64522 70 14 52 4 62
CRW_00670 0.79 0.74 0.83 89 70018 21 6 12 3 31
CRW_00671 0.65 0.62 0.68 72 62729 39 7 27 5 45
CRW_00672 0.48 0.49 0.47 54 72275 65 19 42 4 57
CRW_00674 0.58 0.56 0.62 69 83324 45 7 36 2 55
CRW_00676 0.54 0.55 0.53 63 93410 69 15 40 14 52
CRW_00692 0.20 0.21 0.19 19 68537 84 24 55 5 71
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.92 0.85 1.00 35 5015 0 0 0 0 6
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2677 24 5 19 0 23
PDB_00810 0.91 0.82 1.00 14 1067 0 0 0 0 3
PDB_00827 0.58 0.52 0.66 42 26964 23 1 21 1 39
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.88 0.85 0.92 11 891 5 0 1 4 2
PDB_01092 0.74 0.65 0.83 34 10112 8 1 6 1 18
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.69 0.74 0.63 26 13489 18 4 11 3 9
RFA_00604 0.70 0.71 0.69 25 13494 22 2 9 11 10
RFA_00605 0.37 0.34 0.40 12 15195 27 3 15 9 23
RFA_00606 0.67 0.67 0.68 26 21283 25 7 5 13 13
RFA_00607 0.83 0.83 0.83 30 17730 18 4 2 12 6
RFA_00609 0.75 0.76 0.74 28 17917 27 2 8 17 9
RFA_00610 0.77 0.76 0.78 28 17169 23 2 6 15 9
RFA_00611 0.84 0.83 0.85 29 13007 10 0 5 5 6
RFA_00613 0.66 0.66 0.66 23 12845 15 4 8 3 12
RFA_00615 0.65 0.69 0.62 24 13327 15 4 11 0 11
RFA_00620 0.67 0.64 0.69 25 21909 24 3 8 13 14
RFA_00626 0.87 0.84 0.91 73 56536 32 2 5 25 14
RFA_00627 0.70 0.67 0.74 58 56875 36 4 16 16 29
RFA_00628 0.86 0.83 0.89 71 57211 31 1 8 22 15
RFA_00630 0.59 0.59 0.59 51 56866 48 8 28 12 36
RFA_00632 0.34 0.36 0.33 10 4065 20 2 18 0 18
RFA_00636 0.43 0.43 0.44 12 3978 15 2 13 0 16
RFA_00639 0.17 0.18 0.16 16 54518 83 18 63 2 71
RFA_00642 0.25 0.22 0.29 4 2912 10 2 8 0 14
RFA_00643 0.24 0.22 0.27 4 2196 12 2 9 1 14
RFA_00644 -0.01 0.00 0.00 0 2681 20 4 16 0 18
RFA_00645 -0.01 0.00 0.00 0 2402 13 2 11 0 18
RFA_00649 0.68 0.67 0.71 12 2128 7 1 4 2 6
RFA_00651 0.35 0.33 0.38 6 2064 10 1 9 0 12
RFA_00653 0.33 0.33 0.33 6 2127 12 3 9 0 12
RFA_00654 0.27 0.28 0.28 5 2397 13 2 11 0 13
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1120 10 1 7 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.30 0.29 0.33 4 978 8 1 7 0 10
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 -0.01 0.00 0.00 0 895 9 0 8 1 13
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.71 0.71 0.71 10 1114 5 0 4 1 4
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.42 0.43 0.43 6 976 9 0 8 1 8
RFA_00678 0.43 0.29 0.67 4 940 2 0 2 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.66 0.64 0.69 9 977 5 0 4 1 5
RFA_00685 0.42 0.36 0.50 5 980 6 0 5 1 9
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.69 0.71 0.67 10 1020 5 0 5 0 4
RFA_00706 0.37 0.29 0.50 4 1027 4 0 4 0 10
RFA_00707 -0.01 0.00 0.00 0 1026 9 3 6 0 14
RFA_00708 0.40 0.29 0.57 4 1028 3 0 3 0 10
RFA_00709 -0.01 0.00 0.00 0 982 9 0 8 1 14
RFA_00710 -0.01 0.00 0.00 0 981 9 0 9 0 14
RFA_00711 0.33 0.29 0.40 4 1025 6 0 6 0 10
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 935 11 0 11 0 14
RFA_00717 0.64 0.64 0.64 9 889 5 0 5 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00733 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00734 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00736 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00737 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00745 1.00 1.00 1.00 12 934 1 0 0 1 0
RFA_00749 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00758 0.96 0.92 1.00 11 892 0 0 0 0 1
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00769 0.54 0.56 0.53 10 1934 9 4 5 0 8
RFA_00770 0.61 0.56 0.67 10 2001 8 0 5 3 8
RFA_00773 0.71 0.56 0.91 10 1942 4 1 0 3 8
RFA_00779 0.55 0.56 0.56 10 1935 8 3 5 0 8
RFA_00781 0.58 0.59 0.58 19 5017 14 2 12 0 13
RFA_00786 0.71 0.72 0.70 23 5017 10 2 8 0 9
RFA_00791 0.95 0.91 1.00 29 5122 1 0 0 1 3
RFA_00792 0.94 0.94 0.94 30 5018 4 0 2 2 2
RFA_00801 0.85 0.81 0.90 26 5021 4 0 3 1 6
RFA_00808 -0.01 0.00 0.00 0 2001 15 2 13 0 16
RFA_00809 0.37 0.38 0.38 6 2129 10 1 9 0 10
RFA_00814 0.77 0.78 0.76 32 25158 23 4 6 13 9
RFA_00815 0.56 0.61 0.51 25 24482 36 8 16 12 16
RFA_00816 0.60 0.66 0.55 27 23171 30 13 9 8 14
RFA_00817 0.10 0.12 0.08 5 21882 58 18 40 0 36
RFA_00818 0.19 0.22 0.17 9 20247 46 19 26 1 32
RFA_00819 0.96 0.95 0.98 39 27926 28 0 1 27 2
SRP_00016 0.91 0.87 0.94 95 47794 8 0 6 2 14
SRP_00130 0.27 0.27 0.27 26 49675 69 10 59 0 71
SRP_00142 0.59 0.60 0.58 56 45960 42 5 35 2 37
SRP_00143 0.85 0.84 0.86 85 49671 19 1 13 5 16
SRP_00144 0.74 0.73 0.75 69 46573 30 1 22 7 25
SRP_00197 0.44 0.43 0.45 43 49360 53 4 48 1 58
SRP_00198 0.34 0.34 0.35 35 50940 66 4 61 1 68
SRP_00199 0.65 0.65 0.66 68 50937 36 3 32 1 37
SRP_00201 0.94 0.92 0.95 102 47788 7 0 5 2 9
SRP_00202 0.97 0.95 0.98 111 54172 4 0 2 2 6
SRP_00203 0.73 0.75 0.72 79 48718 37 3 28 6 26
SRP_00204 0.60 0.61 0.60 62 49038 42 7 34 1 40
SRP_00206 0.82 0.81 0.83 87 47790 19 5 13 1 21
SRP_00208 0.85 0.81 0.88 82 46572 14 1 10 3 19
SRP_00255 0.60 0.62 0.59 58 47487 46 3 38 5 36
SRP_00276 0.87 0.85 0.88 94 50614 14 5 8 1 16
SRP_00278 0.77 0.76 0.78 80 45954 23 5 17 1 25
SRP_00321 0.64 0.64 0.65 69 47480 38 7 30 1 39
SRP_00322 0.61 0.61 0.60 67 48405 45 6 38 1 42
SRP_00323 0.93 0.92 0.94 98 47174 8 0 6 2 9
SRP_00330 0.63 0.64 0.63 61 49989 40 4 32 4 34
SRP_00332 0.84 0.85 0.83 90 50295 19 6 12 1 16
SRP_00339 0.64 0.64 0.64 60 44457 37 6 28 3 34

^top



Performance of CMfinder(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 1905
Total TN 2881233
Total FP 688
Total FP CONTRA 40
Total FP INCONS 572
Total FP COMP 76
Total FN 4192
Total Scores
MCC 0.486
Average MCC ± 95% Confidence Intervals 0.505 ± 0.044
Sensitivity 0.312
Positive Predictive Value 0.757
Nr of predictions 129

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.37 0.14 1.00 16 103724 0 0 0 0 99
CRW_00016 0.36 0.14 0.89 17 77402 2 0 2 0 103
CRW_00610 0.49 0.31 0.78 25 36283 7 1 6 0 56
CRW_00613 0.54 0.33 0.87 26 34950 4 0 4 0 52
CRW_00614 0.46 0.25 0.88 14 121755 9 0 2 7 43
CRW_00618 0.36 0.13 1.00 8 56272 0 0 0 0 53
CRW_00633 0.32 0.12 0.81 13 63530 3 1 2 0 93
CRW_00634 0.40 0.17 0.94 16 64603 1 0 1 0 78
CRW_00670 0.32 0.18 0.55 22 70085 18 0 18 0 98
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00674 0.40 0.17 0.95 21 83414 1 0 1 0 103
CRW_00676 0.12 0.04 0.33 5 93513 10 1 9 0 110
CRW_00692 0.41 0.17 1.00 15 68620 0 0 0 0 75
PDB_00005 0.47 0.36 0.63 5 938 3 0 3 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.00 0.00 0.00 0 465 0 0 0 0 11
PDB_00716 -0.01 0.00 0.00 0 2688 13 0 13 0 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.25 0.10 0.62 8 27015 5 0 5 0 73
PDB_01050 0.68 0.46 1.00 6 624 1 0 0 1 7
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
PDB_01092 0.38 0.21 0.69 11 10137 6 0 5 1 41
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.70 0.60 0.81 21 13504 7 0 5 2 14
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00605 0.14 0.06 0.33 2 15219 5 0 4 1 33
RFA_00606 0.34 0.21 0.57 8 21307 8 2 4 2 31
RFA_00607 0.08 0.06 0.13 2 17750 15 3 11 1 34
RFA_00609 0.32 0.22 0.47 8 17938 12 2 7 3 29
RFA_00610 0.39 0.30 0.52 11 17184 13 2 8 3 26
RFA_00611 0.65 0.51 0.82 18 13019 5 0 4 1 17
RFA_00613 0.70 0.57 0.87 20 12857 4 0 3 1 15
RFA_00615 0.57 0.49 0.68 17 13341 10 0 8 2 18
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00626 0.44 0.32 0.60 28 56569 22 2 17 3 59
RFA_00627 0.53 0.39 0.72 34 56906 15 3 10 2 53
RFA_00628 0.58 0.43 0.77 37 57243 14 3 8 3 49
RFA_00630 0.45 0.32 0.64 28 56909 17 3 13 1 59
RFA_00632 0.27 0.25 0.29 7 4071 17 0 17 0 21
RFA_00636 0.71 0.64 0.78 18 3982 5 2 3 0 10
RFA_00639 0.00 0.00 0.00 0 54575 44 6 34 4 87
RFA_00642 0.44 0.28 0.71 5 2919 2 0 2 0 13
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.00 0.00 0.00 0 2693 8 0 8 0 18
RFA_00645 0.29 0.17 0.50 3 2409 3 0 3 0 15
RFA_00649 0.35 0.22 0.57 4 2138 3 0 3 0 14
RFA_00651 0.35 0.22 0.57 4 2073 3 0 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.38 0.22 0.67 4 2409 2 0 2 0 14
RFA_00658 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00659 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00667 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00668 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00672 0.41 0.31 0.57 4 896 3 0 3 0 9
RFA_00673 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00685 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00706 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00707 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00710 0.32 0.21 0.50 3 984 3 0 3 0 11
RFA_00711 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00715 0.30 0.14 0.67 2 943 2 0 1 1 12
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00731 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00733 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00737 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00745 0.91 0.83 1.00 10 936 0 0 0 0 2
RFA_00749 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00769 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00779 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00791 0.92 0.88 0.97 28 5122 1 0 1 0 4
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.41 0.38 0.46 6 2132 7 1 6 0 10
RFA_00814 0.68 0.51 0.91 21 25177 2 0 2 0 20
RFA_00815 0.68 0.51 0.91 21 24508 2 0 2 0 20
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
RFA_00817 0.68 0.56 0.82 23 21917 5 0 5 0 18
RFA_00818 0.60 0.49 0.74 20 20274 8 1 6 1 21
RFA_00819 0.62 0.51 0.75 21 27938 7 0 7 0 20
SRP_00016 0.54 0.35 0.83 38 47849 8 0 8 0 71
SRP_00130 0.50 0.32 0.78 31 49730 10 1 8 1 66
SRP_00142 0.46 0.37 0.58 34 45997 26 1 24 1 59
SRP_00143 0.50 0.30 0.86 30 49735 5 0 5 0 71
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00199 0.54 0.35 0.82 37 50995 8 0 8 0 68
SRP_00201 0.54 0.35 0.83 39 47848 8 0 8 0 72
SRP_00202 0.43 0.21 0.86 25 54256 4 0 4 0 92
SRP_00203 0.51 0.34 0.77 36 48781 12 0 11 1 69
SRP_00204 0.52 0.34 0.80 35 49097 9 1 8 0 67
SRP_00206 0.55 0.36 0.83 39 47848 8 0 8 0 69
SRP_00208 0.52 0.33 0.83 33 46625 7 1 6 0 68
SRP_00255 0.49 0.27 0.89 25 47558 3 0 3 0 69
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00323 0.54 0.34 0.88 36 47237 5 0 5 0 71
SRP_00330 0.44 0.23 0.85 22 50060 5 0 4 1 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79
SRP_00339 0.56 0.39 0.79 37 44504 10 2 8 0 57

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.