CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PknotsRG - scored higher in this pairwise comparison

  4. Performance of CMfinder(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for PknotsRG & CMfinder(seed) [.zip] - may take several seconds...


Overview

Metric PknotsRG CMfinder(seed)
MCC 0.604 > 0.486
Average MCC ± 95% Confidence Intervals 0.573 ± 0.054 > 0.505 ± 0.044
Sensitivity 0.604 > 0.312
Positive Predictive Value 0.606 < 0.757
Total TP 3683 > 1905
Total TN 2877674 < 2881233
Total FP 2915 > 688
Total FP CONTRA 437 > 40
Total FP INCONS 1956 > 572
Total FP COMP 522 > 76
Total FN 2414 < 4192
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of PknotsRG and CMfinder(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PknotsRG and CMfinder(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PknotsRG and CMfinder(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PknotsRG and CMfinder(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PknotsRG and CMfinder(seed)).

^top





Performance of PknotsRG - scored higher in this pairwise comparison

1. Total counts & total scores for PknotsRG

Total Base Pair Counts
Total TP 3683
Total TN 2877674
Total FP 2915
Total FP CONTRA 437
Total FP INCONS 1956
Total FP COMP 522
Total FN 2414
Total Scores
MCC 0.604
Average MCC ± 95% Confidence Intervals 0.573 ± 0.054
Sensitivity 0.604
Positive Predictive Value 0.606
Nr of predictions 129

^top



2. Individual counts for PknotsRG [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.16 0.17 0.15 19 103617 114 25 79 10 96
CRW_00016 0.64 0.62 0.66 74 77309 50 5 33 12 46
CRW_00610 0.49 0.49 0.49 40 36233 43 7 35 1 41
CRW_00613 0.64 0.59 0.69 46 34913 29 1 20 8 32
CRW_00614 0.09 0.11 0.07 6 121690 145 21 54 70 51
CRW_00618 0.11 0.11 0.10 7 56211 83 13 49 21 54
CRW_00633 0.50 0.48 0.52 51 63447 50 10 38 2 55
CRW_00634 0.56 0.60 0.52 56 64513 53 17 34 2 38
CRW_00670 0.75 0.74 0.77 89 70009 30 5 22 3 31
CRW_00671 0.77 0.76 0.78 89 62721 30 6 19 5 28
CRW_00672 0.53 0.55 0.52 61 72273 60 21 35 4 50
CRW_00674 0.52 0.52 0.52 65 83310 63 12 49 2 59
CRW_00676 0.56 0.60 0.52 69 93396 67 13 50 4 46
CRW_00692 0.43 0.46 0.41 41 68536 66 18 40 8 49
PDB_00005 1.00 1.00 1.00 14 932 0 0 0 0 0
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.87 0.83 0.92 34 5013 3 2 1 0 7
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2675 27 1 25 1 23
PDB_00810 0.91 0.82 1.00 14 1067 0 0 0 0 3
PDB_00827 0.50 0.46 0.55 37 26961 31 1 29 1 44
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.88 0.85 0.92 11 891 5 0 1 4 2
PDB_01092 0.71 0.67 0.74 35 10106 13 2 10 1 17
PDB_01152 0.96 0.93 1.00 13 548 0 0 0 0 1
RFA_00603 0.69 0.74 0.63 26 13489 18 4 11 3 9
RFA_00604 0.70 0.71 0.69 25 13494 22 2 9 11 10
RFA_00605 0.33 0.34 0.32 12 15188 34 4 21 9 23
RFA_00606 0.66 0.67 0.65 26 21281 28 6 8 14 13
RFA_00607 0.83 0.83 0.83 30 17730 18 4 2 12 6
RFA_00609 0.75 0.76 0.74 28 17917 28 2 8 18 9
RFA_00610 0.77 0.76 0.78 28 17169 23 2 6 15 9
RFA_00611 0.87 0.86 0.88 30 13007 9 0 4 5 5
RFA_00613 0.66 0.66 0.66 23 12845 14 4 8 2 12
RFA_00615 0.63 0.66 0.61 23 13328 19 5 10 4 12
RFA_00620 0.63 0.62 0.65 24 21908 26 4 9 13 15
RFA_00626 0.89 0.86 0.93 75 56535 28 2 4 22 12
RFA_00627 0.73 0.70 0.76 61 56873 35 4 15 16 26
RFA_00628 0.78 0.74 0.81 64 57212 40 0 15 25 22
RFA_00630 0.60 0.60 0.60 52 56866 44 8 27 9 35
RFA_00632 0.88 0.89 0.86 25 4066 6 2 2 2 3
RFA_00636 0.95 0.96 0.93 27 3976 4 2 0 2 1
RFA_00639 0.23 0.24 0.21 21 54517 81 16 61 4 66
RFA_00642 -0.01 0.00 0.00 0 2910 16 3 13 0 18
RFA_00643 -0.01 0.00 0.00 0 2196 15 1 14 0 18
RFA_00644 -0.01 0.00 0.00 0 2677 24 4 20 0 18
RFA_00645 0.19 0.17 0.23 3 2402 11 1 9 1 15
RFA_00649 0.22 0.22 0.22 4 2127 16 0 14 2 14
RFA_00651 0.35 0.33 0.38 6 2064 10 1 9 0 12
RFA_00653 0.33 0.33 0.33 6 2127 12 3 9 0 12
RFA_00654 -0.01 0.00 0.00 0 2398 17 2 15 0 18
RFA_00658 0.33 0.29 0.40 4 1118 8 0 6 2 10
RFA_00659 -0.01 0.00 0.00 0 1120 10 0 8 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.29 0.29 0.31 4 977 10 1 8 1 10
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 -0.01 0.00 0.00 0 895 9 0 8 1 13
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.74 0.71 0.77 10 1115 4 0 3 1 4
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.42 0.43 0.43 6 976 9 0 8 1 8
RFA_00678 0.33 0.29 0.40 4 936 6 0 6 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.59 0.57 0.62 8 977 6 0 5 1 6
RFA_00685 0.42 0.36 0.50 5 980 6 0 5 1 9
RFA_00704 -0.01 0.00 0.00 0 983 8 0 7 1 14
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 0.33 0.29 0.40 4 1025 7 0 6 1 10
RFA_00707 -0.01 0.00 0.00 0 1025 10 1 9 0 14
RFA_00708 0.35 0.29 0.44 4 1026 6 0 5 1 10
RFA_00709 -0.01 0.00 0.00 0 979 12 0 11 1 14
RFA_00710 -0.01 0.00 0.00 0 981 9 0 9 0 14
RFA_00711 0.33 0.29 0.40 4 1025 6 0 6 0 10
RFA_00715 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00716 -0.01 0.00 0.00 0 935 11 0 11 0 14
RFA_00717 0.66 0.64 0.69 9 890 4 0 4 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00733 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00734 0.18 0.17 0.22 2 894 8 0 7 1 10
RFA_00736 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00737 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00745 1.00 1.00 1.00 12 934 1 0 0 1 0
RFA_00749 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00758 0.96 0.92 1.00 11 892 0 0 0 0 1
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00769 0.97 1.00 0.95 18 1934 3 1 0 2 0
RFA_00770 0.88 0.78 1.00 14 2002 3 0 0 3 4
RFA_00773 0.97 1.00 0.95 18 1934 4 1 0 3 0
RFA_00779 0.97 0.94 1.00 17 1936 2 0 0 2 1
RFA_00781 0.58 0.59 0.58 19 5017 14 2 12 0 13
RFA_00786 0.71 0.72 0.70 23 5017 10 2 8 0 9
RFA_00791 0.50 0.50 0.50 16 5119 16 5 11 0 16
RFA_00792 0.88 0.91 0.85 29 5016 7 2 3 2 3
RFA_00801 0.81 0.81 0.81 26 5018 7 1 5 1 6
RFA_00808 1.00 1.00 1.00 16 2000 0 0 0 0 0
RFA_00809 0.79 0.81 0.76 13 2128 4 0 4 0 3
RFA_00814 0.91 0.95 0.87 39 25155 23 4 2 17 2
RFA_00815 0.66 0.71 0.62 29 24484 33 5 13 15 12
RFA_00816 0.68 0.73 0.63 30 23172 26 12 6 8 11
RFA_00817 0.10 0.12 0.08 5 21885 56 16 39 1 36
RFA_00818 0.00 0.00 0.00 0 20245 56 20 36 0 41
RFA_00819 0.79 0.80 0.79 33 27924 34 0 9 25 8
SRP_00016 0.85 0.83 0.88 91 47791 15 2 11 2 18
SRP_00130 0.23 0.23 0.24 22 49677 71 10 61 0 75
SRP_00142 0.67 0.67 0.67 62 45963 35 2 29 4 31
SRP_00143 0.85 0.84 0.86 85 49671 19 1 13 5 16
SRP_00144 0.74 0.73 0.74 69 46572 31 1 23 7 25
SRP_00197 0.38 0.38 0.39 38 49357 61 4 56 1 63
SRP_00198 0.38 0.38 0.38 39 50937 65 4 60 1 64
SRP_00199 0.31 0.30 0.34 31 50948 64 4 57 3 74
SRP_00201 0.77 0.77 0.78 85 47786 26 5 19 2 26
SRP_00202 0.84 0.84 0.84 98 54169 22 2 16 4 19
SRP_00203 0.74 0.75 0.72 79 48719 36 3 27 6 26
SRP_00204 0.78 0.77 0.78 79 49040 24 4 18 2 23
SRP_00206 0.84 0.83 0.84 90 47788 18 5 12 1 18
SRP_00208 0.67 0.67 0.67 68 46564 35 7 26 2 33
SRP_00255 0.60 0.62 0.59 58 47487 47 3 38 6 36
SRP_00276 0.86 0.85 0.86 94 50612 16 5 10 1 16
SRP_00278 0.72 0.72 0.72 76 45951 31 4 25 2 29
SRP_00321 0.78 0.77 0.79 83 47481 24 3 19 2 25
SRP_00322 0.61 0.61 0.60 67 48405 45 6 38 1 42
SRP_00323 0.92 0.92 0.93 98 47173 10 0 7 3 9
SRP_00330 0.49 0.49 0.48 47 49988 53 3 48 2 48
SRP_00332 0.85 0.86 0.84 91 50295 18 6 11 1 15
SRP_00339 0.56 0.59 0.53 55 44448 50 6 42 2 39

^top



Performance of CMfinder(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 1905
Total TN 2881233
Total FP 688
Total FP CONTRA 40
Total FP INCONS 572
Total FP COMP 76
Total FN 4192
Total Scores
MCC 0.486
Average MCC ± 95% Confidence Intervals 0.505 ± 0.044
Sensitivity 0.312
Positive Predictive Value 0.757
Nr of predictions 129

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.37 0.14 1.00 16 103724 0 0 0 0 99
CRW_00016 0.36 0.14 0.89 17 77402 2 0 2 0 103
CRW_00610 0.49 0.31 0.78 25 36283 7 1 6 0 56
CRW_00613 0.54 0.33 0.87 26 34950 4 0 4 0 52
CRW_00614 0.46 0.25 0.88 14 121755 9 0 2 7 43
CRW_00618 0.36 0.13 1.00 8 56272 0 0 0 0 53
CRW_00633 0.32 0.12 0.81 13 63530 3 1 2 0 93
CRW_00634 0.40 0.17 0.94 16 64603 1 0 1 0 78
CRW_00670 0.32 0.18 0.55 22 70085 18 0 18 0 98
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00674 0.40 0.17 0.95 21 83414 1 0 1 0 103
CRW_00676 0.12 0.04 0.33 5 93513 10 1 9 0 110
CRW_00692 0.41 0.17 1.00 15 68620 0 0 0 0 75
PDB_00005 0.47 0.36 0.63 5 938 3 0 3 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.00 0.00 0.00 0 465 0 0 0 0 11
PDB_00716 -0.01 0.00 0.00 0 2688 13 0 13 0 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.25 0.10 0.62 8 27015 5 0 5 0 73
PDB_01050 0.68 0.46 1.00 6 624 1 0 0 1 7
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
PDB_01092 0.38 0.21 0.69 11 10137 6 0 5 1 41
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.70 0.60 0.81 21 13504 7 0 5 2 14
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00605 0.14 0.06 0.33 2 15219 5 0 4 1 33
RFA_00606 0.34 0.21 0.57 8 21307 8 2 4 2 31
RFA_00607 0.08 0.06 0.13 2 17750 15 3 11 1 34
RFA_00609 0.32 0.22 0.47 8 17938 12 2 7 3 29
RFA_00610 0.39 0.30 0.52 11 17184 13 2 8 3 26
RFA_00611 0.65 0.51 0.82 18 13019 5 0 4 1 17
RFA_00613 0.70 0.57 0.87 20 12857 4 0 3 1 15
RFA_00615 0.57 0.49 0.68 17 13341 10 0 8 2 18
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00626 0.44 0.32 0.60 28 56569 22 2 17 3 59
RFA_00627 0.53 0.39 0.72 34 56906 15 3 10 2 53
RFA_00628 0.58 0.43 0.77 37 57243 14 3 8 3 49
RFA_00630 0.45 0.32 0.64 28 56909 17 3 13 1 59
RFA_00632 0.27 0.25 0.29 7 4071 17 0 17 0 21
RFA_00636 0.71 0.64 0.78 18 3982 5 2 3 0 10
RFA_00639 0.00 0.00 0.00 0 54575 44 6 34 4 87
RFA_00642 0.44 0.28 0.71 5 2919 2 0 2 0 13
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.00 0.00 0.00 0 2693 8 0 8 0 18
RFA_00645 0.29 0.17 0.50 3 2409 3 0 3 0 15
RFA_00649 0.35 0.22 0.57 4 2138 3 0 3 0 14
RFA_00651 0.35 0.22 0.57 4 2073 3 0 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.38 0.22 0.67 4 2409 2 0 2 0 14
RFA_00658 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00659 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00667 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00668 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00672 0.41 0.31 0.57 4 896 3 0 3 0 9
RFA_00673 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00685 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00706 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00707 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00710 0.32 0.21 0.50 3 984 3 0 3 0 11
RFA_00711 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00715 0.30 0.14 0.67 2 943 2 0 1 1 12
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00731 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00733 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00737 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00745 0.91 0.83 1.00 10 936 0 0 0 0 2
RFA_00749 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00769 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00779 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00791 0.92 0.88 0.97 28 5122 1 0 1 0 4
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.41 0.38 0.46 6 2132 7 1 6 0 10
RFA_00814 0.68 0.51 0.91 21 25177 2 0 2 0 20
RFA_00815 0.68 0.51 0.91 21 24508 2 0 2 0 20
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
RFA_00817 0.68 0.56 0.82 23 21917 5 0 5 0 18
RFA_00818 0.60 0.49 0.74 20 20274 8 1 6 1 21
RFA_00819 0.62 0.51 0.75 21 27938 7 0 7 0 20
SRP_00016 0.54 0.35 0.83 38 47849 8 0 8 0 71
SRP_00130 0.50 0.32 0.78 31 49730 10 1 8 1 66
SRP_00142 0.46 0.37 0.58 34 45997 26 1 24 1 59
SRP_00143 0.50 0.30 0.86 30 49735 5 0 5 0 71
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00199 0.54 0.35 0.82 37 50995 8 0 8 0 68
SRP_00201 0.54 0.35 0.83 39 47848 8 0 8 0 72
SRP_00202 0.43 0.21 0.86 25 54256 4 0 4 0 92
SRP_00203 0.51 0.34 0.77 36 48781 12 0 11 1 69
SRP_00204 0.52 0.34 0.80 35 49097 9 1 8 0 67
SRP_00206 0.55 0.36 0.83 39 47848 8 0 8 0 69
SRP_00208 0.52 0.33 0.83 33 46625 7 1 6 0 68
SRP_00255 0.49 0.27 0.89 25 47558 3 0 3 0 69
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00323 0.54 0.34 0.88 36 47237 5 0 5 0 71
SRP_00330 0.44 0.23 0.85 22 50060 5 0 4 1 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79
SRP_00339 0.56 0.39 0.79 37 44504 10 2 8 0 57

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.