CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of McQFold - scored higher in this pairwise comparison

  4. Performance of CMfinder(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for McQFold & CMfinder(seed) [.zip] - may take several seconds...


Overview

Metric McQFold CMfinder(seed)
MCC 0.502 > 0.486
Average MCC ± 95% Confidence Intervals 0.488 ± 0.051 < 0.505 ± 0.044
Sensitivity 0.485 > 0.312
Positive Predictive Value 0.520 < 0.757
Total TP 2960 > 1905
Total TN 2878063 < 2881233
Total FP 3091 > 688
Total FP CONTRA 463 > 40
Total FP INCONS 2264 > 572
Total FP COMP 364 > 76
Total FN 3137 < 4192
P-value 1.70738736246e-07

^top




Performance plots


  1. Comparison of performance of McQFold and CMfinder(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for McQFold and CMfinder(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for McQFold and CMfinder(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for McQFold and CMfinder(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for McQFold and CMfinder(seed)).

^top





Performance of McQFold - scored higher in this pairwise comparison

1. Total counts & total scores for McQFold

Total Base Pair Counts
Total TP 2960
Total TN 2878063
Total FP 3091
Total FP CONTRA 463
Total FP INCONS 2264
Total FP COMP 364
Total FN 3137
Total Scores
MCC 0.502
Average MCC ± 95% Confidence Intervals 0.488 ± 0.051
Sensitivity 0.485
Positive Predictive Value 0.520
Nr of predictions 129

^top



2. Individual counts for McQFold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.10 0.10 0.11 11 103641 107 9 79 19 104
CRW_00016 0.66 0.61 0.72 73 77320 38 5 23 10 47
CRW_00610 0.50 0.48 0.51 39 36239 38 6 31 1 42
CRW_00613 0.81 0.73 0.89 57 34916 15 0 7 8 21
CRW_00614 0.07 0.09 0.06 5 121689 132 26 51 55 52
CRW_00618 0.37 0.39 0.34 24 56210 60 15 31 14 37
CRW_00633 0.74 0.72 0.77 76 63447 26 8 15 3 30
CRW_00634 0.44 0.41 0.47 39 64537 48 18 26 4 55
CRW_00670 0.54 0.53 0.55 63 70011 53 6 45 2 57
CRW_00671 0.66 0.59 0.74 69 62742 28 5 19 4 48
CRW_00672 0.52 0.50 0.56 55 72291 48 9 35 4 56
CRW_00674 0.43 0.42 0.44 52 83319 69 14 51 4 72
CRW_00676 0.30 0.30 0.31 35 93414 85 9 70 6 80
CRW_00692 0.26 0.27 0.25 24 68538 77 23 50 4 66
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.72 0.68 0.76 28 5013 9 0 9 0 13
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 0.68 0.70 0.67 16 2677 9 0 8 1 7
PDB_00810 0.51 0.41 0.64 7 1070 4 2 2 0 10
PDB_00827 0.68 0.59 0.79 48 26967 15 0 13 2 33
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.79 0.69 0.90 9 893 2 0 1 1 4
PDB_01092 0.81 0.75 0.89 39 10109 6 1 4 1 13
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00603 0.34 0.29 0.42 10 13506 15 4 10 1 25
RFA_00604 0.71 0.66 0.77 23 13500 14 2 5 7 12
RFA_00605 0.36 0.34 0.38 12 15193 31 3 17 11 23
RFA_00606 0.32 0.33 0.32 13 21280 34 12 16 6 26
RFA_00607 0.65 0.61 0.69 22 17734 17 3 7 7 14
RFA_00609 0.77 0.76 0.78 28 17919 18 2 6 10 9
RFA_00610 0.56 0.57 0.55 21 17167 25 2 15 8 16
RFA_00611 0.58 0.54 0.61 19 13010 12 3 9 0 16
RFA_00613 0.15 0.14 0.16 5 12848 27 6 21 0 30
RFA_00615 0.32 0.29 0.37 10 13339 18 4 13 1 25
RFA_00620 0.36 0.38 0.34 15 21901 37 4 25 8 24
RFA_00626 0.21 0.22 0.21 19 56526 76 15 56 5 68
RFA_00627 0.42 0.40 0.44 35 56873 47 13 32 2 52
RFA_00628 0.42 0.41 0.43 35 57209 50 7 40 3 51
RFA_00630 0.63 0.63 0.63 55 56865 41 6 27 8 32
RFA_00632 0.88 0.89 0.86 25 4066 4 3 1 0 3
RFA_00636 0.73 0.71 0.74 20 3978 7 5 2 0 8
RFA_00639 0.37 0.39 0.36 34 54520 66 13 48 5 53
RFA_00642 -0.01 0.00 0.00 0 2913 13 1 12 0 18
RFA_00643 -0.01 0.00 0.00 0 2196 15 1 14 0 18
RFA_00644 -0.01 0.00 0.00 0 2684 17 3 14 0 18
RFA_00645 -0.01 0.00 0.00 0 2403 12 3 9 0 18
RFA_00649 0.78 0.67 0.92 12 2132 1 0 1 0 6
RFA_00651 0.40 0.33 0.50 6 2068 6 0 6 0 12
RFA_00653 0.33 0.33 0.33 6 2127 12 3 9 0 12
RFA_00654 -0.01 0.00 0.00 0 2403 12 2 10 0 18
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1125 5 0 3 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 -0.01 0.00 0.00 0 981 10 1 8 1 14
RFA_00668 0.50 0.43 0.60 6 980 4 0 4 0 8
RFA_00672 -0.01 0.00 0.00 0 895 10 0 8 2 13
RFA_00673 0.47 0.36 0.63 5 1120 4 0 3 1 9
RFA_00674 0.96 0.93 1.00 13 1115 1 0 0 1 1
RFA_00675 0.28 0.29 0.29 4 976 10 1 9 0 10
RFA_00677 0.74 0.71 0.77 10 977 6 0 3 3 4
RFA_00678 0.53 0.29 1.00 4 942 0 0 0 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.84 0.71 1.00 10 980 1 0 0 1 4
RFA_00685 0.47 0.36 0.63 5 982 4 0 3 1 9
RFA_00704 -0.01 0.00 0.00 0 983 8 0 7 1 14
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00707 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 -0.01 0.00 0.00 0 984 7 0 6 1 14
RFA_00710 -0.01 0.00 0.00 0 980 10 0 10 0 14
RFA_00711 -0.01 0.00 0.00 0 1023 12 0 12 0 14
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00717 0.69 0.64 0.75 9 891 3 0 3 0 5
RFA_00730 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00731 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00733 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00734 0.67 0.58 0.78 7 894 3 0 2 1 5
RFA_00736 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00737 0.43 0.42 0.45 5 892 7 0 6 1 7
RFA_00745 0.43 0.42 0.45 5 935 7 1 5 1 7
RFA_00749 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00758 0.40 0.33 0.50 4 895 4 2 2 0 8
RFA_00762 0.31 0.33 0.31 4 890 9 2 7 0 8
RFA_00763 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00764 0.43 0.42 0.45 5 892 7 1 5 1 7
RFA_00765 0.51 0.42 0.63 5 895 4 0 3 1 7
RFA_00769 0.85 0.94 0.77 17 1931 5 4 1 0 1
RFA_00770 0.88 0.78 1.00 14 2002 4 0 0 4 4
RFA_00773 0.97 1.00 0.95 18 1934 5 1 0 4 0
RFA_00779 0.97 0.94 1.00 17 1936 0 0 0 0 1
RFA_00781 0.97 0.94 1.00 30 5020 1 0 0 1 2
RFA_00786 0.21 0.19 0.25 6 5026 19 2 16 1 26
RFA_00791 0.60 0.56 0.64 18 5123 11 0 10 1 14
RFA_00792 0.97 0.94 1.00 30 5020 1 0 0 1 2
RFA_00801 0.69 0.63 0.77 20 5024 8 0 6 2 12
RFA_00808 1.00 1.00 1.00 16 2000 0 0 0 0 0
RFA_00809 0.79 0.81 0.76 13 2128 4 0 4 0 3
RFA_00814 0.79 0.80 0.77 33 25157 21 0 10 11 8
RFA_00815 0.56 0.59 0.53 24 24486 27 1 20 6 17
RFA_00816 0.82 0.85 0.80 35 23176 12 4 5 3 6
RFA_00817 0.35 0.39 0.31 16 21894 39 9 26 4 25
RFA_00818 0.28 0.27 0.29 11 20263 30 12 15 3 30
RFA_00819 0.30 0.29 0.31 12 27927 50 4 23 23 29
SRP_00016 0.49 0.49 0.50 53 47789 54 4 49 1 56
SRP_00130 0.54 0.52 0.56 50 49681 44 3 36 5 47
SRP_00142 0.14 0.14 0.14 13 45961 82 8 74 0 80
SRP_00143 0.54 0.53 0.56 54 49673 48 2 41 5 47
SRP_00144 0.64 0.65 0.64 61 46569 37 4 31 2 33
SRP_00197 0.21 0.20 0.22 20 49363 73 10 62 1 81
SRP_00198 0.08 0.08 0.09 8 50954 78 7 71 0 95
SRP_00199 0.13 0.11 0.14 12 50956 72 6 66 0 93
SRP_00201 0.73 0.72 0.74 80 47787 30 2 26 2 31
SRP_00202 0.88 0.89 0.88 104 54167 15 5 9 1 13
SRP_00203 0.73 0.74 0.71 78 48718 39 2 30 7 27
SRP_00204 0.80 0.77 0.83 79 49046 17 5 11 1 23
SRP_00206 0.76 0.72 0.80 78 47797 21 7 13 1 30
SRP_00208 0.61 0.58 0.64 59 46573 35 6 27 2 42
SRP_00255 0.26 0.28 0.25 26 47484 76 14 62 0 68
SRP_00276 0.54 0.54 0.55 59 50614 49 8 40 1 51
SRP_00278 0.75 0.74 0.76 78 45953 28 5 20 3 27
SRP_00321 0.62 0.62 0.63 67 47479 42 8 32 2 41
SRP_00322 0.68 0.69 0.68 75 48405 36 4 32 0 34
SRP_00323 0.60 0.61 0.60 65 47169 46 4 40 2 42
SRP_00330 0.10 0.11 0.10 10 49988 88 11 77 0 85
SRP_00332 0.49 0.50 0.49 53 50295 56 3 52 1 53
SRP_00339 0.43 0.44 0.43 41 44455 55 8 47 0 53

^top



Performance of CMfinder(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 1905
Total TN 2881233
Total FP 688
Total FP CONTRA 40
Total FP INCONS 572
Total FP COMP 76
Total FN 4192
Total Scores
MCC 0.486
Average MCC ± 95% Confidence Intervals 0.505 ± 0.044
Sensitivity 0.312
Positive Predictive Value 0.757
Nr of predictions 129

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.37 0.14 1.00 16 103724 0 0 0 0 99
CRW_00016 0.36 0.14 0.89 17 77402 2 0 2 0 103
CRW_00610 0.49 0.31 0.78 25 36283 7 1 6 0 56
CRW_00613 0.54 0.33 0.87 26 34950 4 0 4 0 52
CRW_00614 0.46 0.25 0.88 14 121755 9 0 2 7 43
CRW_00618 0.36 0.13 1.00 8 56272 0 0 0 0 53
CRW_00633 0.32 0.12 0.81 13 63530 3 1 2 0 93
CRW_00634 0.40 0.17 0.94 16 64603 1 0 1 0 78
CRW_00670 0.32 0.18 0.55 22 70085 18 0 18 0 98
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00674 0.40 0.17 0.95 21 83414 1 0 1 0 103
CRW_00676 0.12 0.04 0.33 5 93513 10 1 9 0 110
CRW_00692 0.41 0.17 1.00 15 68620 0 0 0 0 75
PDB_00005 0.47 0.36 0.63 5 938 3 0 3 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.00 0.00 0.00 0 465 0 0 0 0 11
PDB_00716 -0.01 0.00 0.00 0 2688 13 0 13 0 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.25 0.10 0.62 8 27015 5 0 5 0 73
PDB_01050 0.68 0.46 1.00 6 624 1 0 0 1 7
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
PDB_01092 0.38 0.21 0.69 11 10137 6 0 5 1 41
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.70 0.60 0.81 21 13504 7 0 5 2 14
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00605 0.14 0.06 0.33 2 15219 5 0 4 1 33
RFA_00606 0.34 0.21 0.57 8 21307 8 2 4 2 31
RFA_00607 0.08 0.06 0.13 2 17750 15 3 11 1 34
RFA_00609 0.32 0.22 0.47 8 17938 12 2 7 3 29
RFA_00610 0.39 0.30 0.52 11 17184 13 2 8 3 26
RFA_00611 0.65 0.51 0.82 18 13019 5 0 4 1 17
RFA_00613 0.70 0.57 0.87 20 12857 4 0 3 1 15
RFA_00615 0.57 0.49 0.68 17 13341 10 0 8 2 18
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00626 0.44 0.32 0.60 28 56569 22 2 17 3 59
RFA_00627 0.53 0.39 0.72 34 56906 15 3 10 2 53
RFA_00628 0.58 0.43 0.77 37 57243 14 3 8 3 49
RFA_00630 0.45 0.32 0.64 28 56909 17 3 13 1 59
RFA_00632 0.27 0.25 0.29 7 4071 17 0 17 0 21
RFA_00636 0.71 0.64 0.78 18 3982 5 2 3 0 10
RFA_00639 0.00 0.00 0.00 0 54575 44 6 34 4 87
RFA_00642 0.44 0.28 0.71 5 2919 2 0 2 0 13
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.00 0.00 0.00 0 2693 8 0 8 0 18
RFA_00645 0.29 0.17 0.50 3 2409 3 0 3 0 15
RFA_00649 0.35 0.22 0.57 4 2138 3 0 3 0 14
RFA_00651 0.35 0.22 0.57 4 2073 3 0 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.38 0.22 0.67 4 2409 2 0 2 0 14
RFA_00658 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00659 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00667 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00668 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00672 0.41 0.31 0.57 4 896 3 0 3 0 9
RFA_00673 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00685 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00706 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00707 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00710 0.32 0.21 0.50 3 984 3 0 3 0 11
RFA_00711 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00715 0.30 0.14 0.67 2 943 2 0 1 1 12
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00731 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00733 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00737 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00745 0.91 0.83 1.00 10 936 0 0 0 0 2
RFA_00749 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00769 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00779 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00791 0.92 0.88 0.97 28 5122 1 0 1 0 4
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.41 0.38 0.46 6 2132 7 1 6 0 10
RFA_00814 0.68 0.51 0.91 21 25177 2 0 2 0 20
RFA_00815 0.68 0.51 0.91 21 24508 2 0 2 0 20
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
RFA_00817 0.68 0.56 0.82 23 21917 5 0 5 0 18
RFA_00818 0.60 0.49 0.74 20 20274 8 1 6 1 21
RFA_00819 0.62 0.51 0.75 21 27938 7 0 7 0 20
SRP_00016 0.54 0.35 0.83 38 47849 8 0 8 0 71
SRP_00130 0.50 0.32 0.78 31 49730 10 1 8 1 66
SRP_00142 0.46 0.37 0.58 34 45997 26 1 24 1 59
SRP_00143 0.50 0.30 0.86 30 49735 5 0 5 0 71
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00199 0.54 0.35 0.82 37 50995 8 0 8 0 68
SRP_00201 0.54 0.35 0.83 39 47848 8 0 8 0 72
SRP_00202 0.43 0.21 0.86 25 54256 4 0 4 0 92
SRP_00203 0.51 0.34 0.77 36 48781 12 0 11 1 69
SRP_00204 0.52 0.34 0.80 35 49097 9 1 8 0 67
SRP_00206 0.55 0.36 0.83 39 47848 8 0 8 0 69
SRP_00208 0.52 0.33 0.83 33 46625 7 1 6 0 68
SRP_00255 0.49 0.27 0.89 25 47558 3 0 3 0 69
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00323 0.54 0.34 0.88 36 47237 5 0 5 0 71
SRP_00330 0.44 0.23 0.85 22 50060 5 0 4 1 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79
SRP_00339 0.56 0.39 0.79 37 44504 10 2 8 0 57

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.