CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of Sfold - scored higher in this pairwise comparison

  4. Performance of CMfinder(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for Sfold & CMfinder(seed) [.zip] - may take several seconds...


Overview

Metric Sfold CMfinder(seed)
MCC 0.645 > 0.486
Average MCC ± 95% Confidence Intervals 0.562 ± 0.053 > 0.505 ± 0.044
Sensitivity 0.594 > 0.312
Positive Predictive Value 0.703 < 0.757
Total TP 3622 > 1905
Total TN 2878595 < 2881233
Total FP 1986 > 688
Total FP CONTRA 276 > 40
Total FP INCONS 1257 > 572
Total FP COMP 453 > 76
Total FN 2475 < 4192
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of Sfold and CMfinder(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Sfold and CMfinder(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Sfold and CMfinder(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for Sfold and CMfinder(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Sfold and CMfinder(seed)).

^top





Performance of Sfold - scored higher in this pairwise comparison

1. Total counts & total scores for Sfold

Total Base Pair Counts
Total TP 3622
Total TN 2878595
Total FP 1986
Total FP CONTRA 276
Total FP INCONS 1257
Total FP COMP 453
Total FN 2475
Total Scores
MCC 0.645
Average MCC ± 95% Confidence Intervals 0.562 ± 0.053
Sensitivity 0.594
Positive Predictive Value 0.703
Nr of predictions 129

^top



2. Individual counts for Sfold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.36 0.27 0.48 31 103675 57 8 26 23 84
CRW_00016 0.80 0.65 0.98 78 77341 14 0 2 12 42
CRW_00610 0.51 0.49 0.52 40 36238 37 9 28 0 41
CRW_00613 0.74 0.67 0.83 52 34917 19 0 11 8 26
CRW_00614 0.19 0.19 0.19 11 121713 102 12 35 55 46
CRW_00618 0.13 0.13 0.14 8 56223 68 13 36 19 53
CRW_00633 0.65 0.49 0.85 52 63485 11 2 7 2 54
CRW_00634 0.54 0.51 0.58 48 64537 40 6 29 5 46
CRW_00670 0.80 0.74 0.87 89 70023 15 6 7 2 31
CRW_00671 0.75 0.70 0.81 82 62734 23 5 14 4 35
CRW_00672 0.78 0.71 0.85 79 72297 20 6 8 6 32
CRW_00674 0.60 0.54 0.66 67 83335 36 6 28 2 57
CRW_00676 0.61 0.57 0.66 65 93430 48 9 24 15 50
CRW_00692 0.54 0.42 0.69 38 68580 27 6 11 10 52
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.94 0.88 1.00 36 5014 0 0 0 0 5
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2678 24 1 22 1 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.64 0.47 0.86 38 26984 8 0 6 2 43
PDB_01050 0.96 0.92 1.00 12 618 2 0 0 2 1
PDB_01051 0.88 0.85 0.92 11 891 5 0 1 4 2
PDB_01092 0.76 0.63 0.92 33 10117 5 0 3 2 19
PDB_01152 0.96 0.93 1.00 13 548 0 0 0 0 1
RFA_00603 0.66 0.69 0.63 24 13492 17 4 10 3 11
RFA_00604 0.71 0.74 0.68 26 13492 24 2 10 12 9
RFA_00605 0.39 0.34 0.44 12 15198 24 3 12 9 23
RFA_00606 0.65 0.59 0.72 23 21289 19 7 2 10 16
RFA_00607 0.91 0.83 1.00 30 17736 7 0 0 7 6
RFA_00609 0.78 0.78 0.78 29 17918 18 2 6 10 8
RFA_00610 0.89 0.84 0.94 31 17172 15 0 2 13 6
RFA_00611 0.81 0.71 0.93 25 13014 5 0 2 3 10
RFA_00613 0.49 0.49 0.50 17 12846 19 7 10 2 18
RFA_00615 0.63 0.57 0.69 20 13337 9 4 5 0 15
RFA_00620 0.55 0.49 0.63 19 21915 22 3 8 11 20
RFA_00626 0.89 0.84 0.95 73 56539 20 1 3 16 14
RFA_00627 0.72 0.68 0.77 59 56876 30 4 14 12 28
RFA_00628 0.90 0.86 0.95 74 57213 28 0 4 24 12
RFA_00630 0.61 0.59 0.65 51 56874 38 7 21 10 36
RFA_00632 0.24 0.25 0.25 7 4067 21 2 19 0 21
RFA_00636 0.40 0.39 0.41 11 3978 16 2 14 0 17
RFA_00639 0.27 0.24 0.30 21 54546 53 9 39 5 66
RFA_00642 0.00 0.00 0.00 0 2926 0 0 0 0 18
RFA_00643 0.24 0.22 0.27 4 2196 12 2 9 1 14
RFA_00644 -0.01 0.00 0.00 0 2689 12 0 12 0 18
RFA_00645 -0.01 0.00 0.00 0 2404 11 3 8 0 18
RFA_00649 0.37 0.33 0.43 6 2131 10 0 8 2 12
RFA_00651 0.14 0.11 0.18 2 2069 9 1 8 0 16
RFA_00653 0.34 0.33 0.35 6 2128 11 3 8 0 12
RFA_00654 -0.01 0.00 0.00 0 2401 14 2 12 0 18
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1122 8 0 6 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.22 0.21 0.25 3 978 10 1 8 1 11
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 -0.01 0.00 0.00 0 896 7 0 7 0 13
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.66 0.64 0.69 9 1115 5 0 4 1 5
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.59 0.57 0.62 8 977 6 0 5 1 6
RFA_00678 0.53 0.29 1.00 4 942 0 0 0 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.59 0.57 0.62 8 977 6 0 5 1 6
RFA_00685 0.47 0.36 0.63 5 982 4 0 3 1 9
RFA_00704 -0.01 0.00 0.00 0 981 9 0 9 0 14
RFA_00705 0.77 0.71 0.83 10 1023 2 0 2 0 4
RFA_00706 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00707 -0.01 0.00 0.00 0 1028 7 1 6 0 14
RFA_00708 0.40 0.29 0.57 4 1028 3 0 3 0 10
RFA_00709 -0.01 0.00 0.00 0 980 10 0 10 0 14
RFA_00710 -0.01 0.00 0.00 0 981 9 0 9 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 -0.01 0.00 0.00 0 937 9 0 9 0 14
RFA_00716 -0.01 0.00 0.00 0 935 11 0 11 0 14
RFA_00717 0.72 0.64 0.82 9 892 2 0 2 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00733 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00734 0.61 0.67 0.57 8 889 7 1 5 1 4
RFA_00736 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00737 0.96 0.92 1.00 11 892 1 0 0 1 1
RFA_00745 0.91 0.83 1.00 10 936 1 0 0 1 2
RFA_00749 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00758 0.76 0.58 1.00 7 896 0 0 0 0 5
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.66 0.67 0.67 8 891 5 1 3 1 4
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00769 0.52 0.56 0.50 10 1933 10 4 6 0 8
RFA_00770 0.68 0.56 0.83 10 2004 5 0 2 3 8
RFA_00773 0.57 0.56 0.59 10 1936 7 1 6 0 8
RFA_00779 0.68 0.56 0.83 10 1941 2 0 2 0 8
RFA_00781 0.59 0.59 0.59 19 5018 13 2 11 0 13
RFA_00786 0.27 0.22 0.33 7 5029 14 1 13 0 25
RFA_00791 0.58 0.50 0.67 16 5127 8 1 7 0 16
RFA_00792 0.95 0.94 0.97 30 5019 2 0 1 1 2
RFA_00801 0.79 0.75 0.83 24 5021 6 0 5 1 8
RFA_00808 -0.01 0.00 0.00 0 2000 19 2 14 3 16
RFA_00809 0.37 0.38 0.38 6 2129 10 1 9 0 10
RFA_00814 0.86 0.76 0.97 31 25168 10 0 1 9 10
RFA_00815 0.61 0.61 0.61 25 24490 21 6 10 5 16
RFA_00816 0.76 0.73 0.79 30 23182 12 8 0 4 11
RFA_00817 0.11 0.12 0.11 5 21899 41 14 27 0 36
RFA_00818 0.31 0.22 0.43 9 20280 13 8 4 1 32
RFA_00819 0.88 0.80 0.97 33 27932 25 0 1 24 8
SRP_00016 0.92 0.89 0.95 97 47793 7 0 5 2 12
SRP_00130 0.36 0.32 0.40 31 49693 46 7 39 0 66
SRP_00142 0.63 0.59 0.68 55 45975 28 2 24 2 38
SRP_00143 0.80 0.78 0.81 79 49673 25 1 17 7 22
SRP_00144 0.74 0.73 0.75 69 46573 27 1 22 4 25
SRP_00197 0.43 0.37 0.51 37 49382 36 1 35 0 64
SRP_00198 0.42 0.35 0.51 36 50970 35 2 32 1 67
SRP_00199 0.78 0.72 0.84 76 50950 15 0 14 1 29
SRP_00201 0.87 0.84 0.89 93 47791 13 5 6 2 18
SRP_00202 0.91 0.86 0.95 101 54179 7 0 5 2 16
SRP_00203 0.80 0.81 0.79 85 48720 29 1 22 6 20
SRP_00204 0.67 0.66 0.68 67 49043 32 6 25 1 35
SRP_00206 0.84 0.83 0.85 90 47789 17 5 11 1 18
SRP_00208 0.71 0.64 0.79 65 46583 18 2 15 1 36
SRP_00255 0.62 0.62 0.62 58 47492 40 3 33 4 36
SRP_00276 0.87 0.85 0.89 94 50615 13 5 7 1 16
SRP_00278 0.81 0.73 0.89 77 45969 12 0 10 2 28
SRP_00321 0.77 0.75 0.79 81 47483 24 3 19 2 27
SRP_00322 0.71 0.66 0.76 72 48421 24 4 19 1 37
SRP_00323 0.92 0.92 0.93 98 47173 11 0 7 4 9
SRP_00330 0.50 0.44 0.57 42 50012 33 1 31 1 53
SRP_00332 0.84 0.81 0.87 86 50304 16 6 7 3 20
SRP_00339 0.67 0.64 0.70 60 44465 29 5 21 3 34

^top



Performance of CMfinder(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 1905
Total TN 2881233
Total FP 688
Total FP CONTRA 40
Total FP INCONS 572
Total FP COMP 76
Total FN 4192
Total Scores
MCC 0.486
Average MCC ± 95% Confidence Intervals 0.505 ± 0.044
Sensitivity 0.312
Positive Predictive Value 0.757
Nr of predictions 129

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.37 0.14 1.00 16 103724 0 0 0 0 99
CRW_00016 0.36 0.14 0.89 17 77402 2 0 2 0 103
CRW_00610 0.49 0.31 0.78 25 36283 7 1 6 0 56
CRW_00613 0.54 0.33 0.87 26 34950 4 0 4 0 52
CRW_00614 0.46 0.25 0.88 14 121755 9 0 2 7 43
CRW_00618 0.36 0.13 1.00 8 56272 0 0 0 0 53
CRW_00633 0.32 0.12 0.81 13 63530 3 1 2 0 93
CRW_00634 0.40 0.17 0.94 16 64603 1 0 1 0 78
CRW_00670 0.32 0.18 0.55 22 70085 18 0 18 0 98
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00674 0.40 0.17 0.95 21 83414 1 0 1 0 103
CRW_00676 0.12 0.04 0.33 5 93513 10 1 9 0 110
CRW_00692 0.41 0.17 1.00 15 68620 0 0 0 0 75
PDB_00005 0.47 0.36 0.63 5 938 3 0 3 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.00 0.00 0.00 0 465 0 0 0 0 11
PDB_00716 -0.01 0.00 0.00 0 2688 13 0 13 0 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.25 0.10 0.62 8 27015 5 0 5 0 73
PDB_01050 0.68 0.46 1.00 6 624 1 0 0 1 7
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
PDB_01092 0.38 0.21 0.69 11 10137 6 0 5 1 41
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.70 0.60 0.81 21 13504 7 0 5 2 14
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00605 0.14 0.06 0.33 2 15219 5 0 4 1 33
RFA_00606 0.34 0.21 0.57 8 21307 8 2 4 2 31
RFA_00607 0.08 0.06 0.13 2 17750 15 3 11 1 34
RFA_00609 0.32 0.22 0.47 8 17938 12 2 7 3 29
RFA_00610 0.39 0.30 0.52 11 17184 13 2 8 3 26
RFA_00611 0.65 0.51 0.82 18 13019 5 0 4 1 17
RFA_00613 0.70 0.57 0.87 20 12857 4 0 3 1 15
RFA_00615 0.57 0.49 0.68 17 13341 10 0 8 2 18
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00626 0.44 0.32 0.60 28 56569 22 2 17 3 59
RFA_00627 0.53 0.39 0.72 34 56906 15 3 10 2 53
RFA_00628 0.58 0.43 0.77 37 57243 14 3 8 3 49
RFA_00630 0.45 0.32 0.64 28 56909 17 3 13 1 59
RFA_00632 0.27 0.25 0.29 7 4071 17 0 17 0 21
RFA_00636 0.71 0.64 0.78 18 3982 5 2 3 0 10
RFA_00639 0.00 0.00 0.00 0 54575 44 6 34 4 87
RFA_00642 0.44 0.28 0.71 5 2919 2 0 2 0 13
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.00 0.00 0.00 0 2693 8 0 8 0 18
RFA_00645 0.29 0.17 0.50 3 2409 3 0 3 0 15
RFA_00649 0.35 0.22 0.57 4 2138 3 0 3 0 14
RFA_00651 0.35 0.22 0.57 4 2073 3 0 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.38 0.22 0.67 4 2409 2 0 2 0 14
RFA_00658 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00659 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00667 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00668 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00672 0.41 0.31 0.57 4 896 3 0 3 0 9
RFA_00673 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00685 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00706 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00707 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00710 0.32 0.21 0.50 3 984 3 0 3 0 11
RFA_00711 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00715 0.30 0.14 0.67 2 943 2 0 1 1 12
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00731 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00733 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00737 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00745 0.91 0.83 1.00 10 936 0 0 0 0 2
RFA_00749 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00769 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00779 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00791 0.92 0.88 0.97 28 5122 1 0 1 0 4
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.41 0.38 0.46 6 2132 7 1 6 0 10
RFA_00814 0.68 0.51 0.91 21 25177 2 0 2 0 20
RFA_00815 0.68 0.51 0.91 21 24508 2 0 2 0 20
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
RFA_00817 0.68 0.56 0.82 23 21917 5 0 5 0 18
RFA_00818 0.60 0.49 0.74 20 20274 8 1 6 1 21
RFA_00819 0.62 0.51 0.75 21 27938 7 0 7 0 20
SRP_00016 0.54 0.35 0.83 38 47849 8 0 8 0 71
SRP_00130 0.50 0.32 0.78 31 49730 10 1 8 1 66
SRP_00142 0.46 0.37 0.58 34 45997 26 1 24 1 59
SRP_00143 0.50 0.30 0.86 30 49735 5 0 5 0 71
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00199 0.54 0.35 0.82 37 50995 8 0 8 0 68
SRP_00201 0.54 0.35 0.83 39 47848 8 0 8 0 72
SRP_00202 0.43 0.21 0.86 25 54256 4 0 4 0 92
SRP_00203 0.51 0.34 0.77 36 48781 12 0 11 1 69
SRP_00204 0.52 0.34 0.80 35 49097 9 1 8 0 67
SRP_00206 0.55 0.36 0.83 39 47848 8 0 8 0 69
SRP_00208 0.52 0.33 0.83 33 46625 7 1 6 0 68
SRP_00255 0.49 0.27 0.89 25 47558 3 0 3 0 69
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00323 0.54 0.34 0.88 36 47237 5 0 5 0 71
SRP_00330 0.44 0.23 0.85 22 50060 5 0 4 1 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79
SRP_00339 0.56 0.39 0.79 37 44504 10 2 8 0 57

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.