CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of ProbKnot - scored higher in this pairwise comparison

  4. Performance of CMfinder(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for ProbKnot & CMfinder(seed) [.zip] - may take several seconds...


Overview

Metric ProbKnot CMfinder(seed)
MCC 0.592 > 0.486
Average MCC ± 95% Confidence Intervals 0.550 ± 0.049 > 0.505 ± 0.044
Sensitivity 0.584 > 0.312
Positive Predictive Value 0.601 < 0.757
Total TP 3558 > 1905
Total TN 2877833 < 2881233
Total FP 2948 > 688
Total FP CONTRA 414 > 40
Total FP INCONS 1945 > 572
Total FP COMP 589 > 76
Total FN 2539 < 4192
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of ProbKnot and CMfinder(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for ProbKnot and CMfinder(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for ProbKnot and CMfinder(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for ProbKnot and CMfinder(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for ProbKnot and CMfinder(seed)).

^top





Performance of ProbKnot - scored higher in this pairwise comparison

1. Total counts & total scores for ProbKnot

Total Base Pair Counts
Total TP 3558
Total TN 2877833
Total FP 2948
Total FP CONTRA 414
Total FP INCONS 1945
Total FP COMP 589
Total FN 2539
Total Scores
MCC 0.592
Average MCC ± 95% Confidence Intervals 0.550 ± 0.049
Sensitivity 0.584
Positive Predictive Value 0.601
Nr of predictions 129

^top



2. Individual counts for ProbKnot [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.39 0.37 0.42 43 103637 84 9 51 24 72
CRW_00016 0.48 0.45 0.50 54 77314 68 6 47 15 66
CRW_00610 0.44 0.44 0.43 36 36232 49 8 39 2 45
CRW_00613 0.84 0.82 0.86 64 34906 18 0 10 8 14
CRW_00614 0.14 0.16 0.12 9 121698 120 17 47 56 48
CRW_00618 0.18 0.20 0.16 12 56207 83 15 46 22 49
CRW_00633 0.62 0.59 0.65 63 63449 38 9 25 4 43
CRW_00634 0.50 0.49 0.52 46 64531 48 10 33 5 48
CRW_00670 0.57 0.58 0.56 70 70001 57 11 43 3 50
CRW_00671 0.79 0.77 0.80 90 62723 28 6 16 6 27
CRW_00672 0.76 0.78 0.74 87 72272 41 10 21 10 24
CRW_00674 0.68 0.67 0.70 83 83317 41 8 28 5 41
CRW_00676 0.59 0.58 0.60 67 93416 63 8 37 18 48
CRW_00692 0.33 0.34 0.32 31 68537 84 18 49 17 59
PDB_00005 0.78 0.79 0.79 11 932 3 0 3 0 3
PDB_00012 1.00 1.00 1.00 7 399 4 0 0 4 0
PDB_00213 0.94 0.88 1.00 36 5014 0 0 0 0 5
PDB_00553 1.00 1.00 1.00 11 454 1 0 0 1 0
PDB_00716 -0.01 0.00 0.00 0 2680 22 0 21 1 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.76 0.67 0.86 54 26965 11 1 8 2 27
PDB_01050 0.88 0.85 0.92 11 618 3 0 1 2 2
PDB_01051 0.88 0.85 0.92 11 891 5 0 1 4 2
PDB_01092 0.73 0.63 0.85 33 10114 8 1 5 2 19
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00603 0.67 0.74 0.60 26 13487 22 4 13 5 9
RFA_00604 0.71 0.74 0.68 26 13492 23 2 10 11 9
RFA_00605 0.30 0.34 0.27 12 15180 46 3 30 13 23
RFA_00606 0.60 0.62 0.59 24 21280 33 7 10 16 15
RFA_00607 0.83 0.83 0.83 30 17730 17 4 2 11 6
RFA_00609 0.53 0.59 0.47 22 17908 35 6 19 10 15
RFA_00610 0.88 0.86 0.89 32 17169 12 0 4 8 5
RFA_00611 0.80 0.80 0.80 28 13006 18 3 4 11 7
RFA_00613 0.71 0.69 0.73 24 12847 17 4 5 8 11
RFA_00615 0.62 0.69 0.56 24 13323 21 8 11 2 11
RFA_00620 0.43 0.49 0.38 19 21895 44 8 23 13 20
RFA_00626 0.86 0.80 0.91 70 56539 32 1 6 25 17
RFA_00627 0.69 0.69 0.68 60 56865 47 6 22 19 27
RFA_00628 0.90 0.88 0.93 76 57209 31 1 5 25 10
RFA_00630 0.69 0.66 0.72 57 56874 41 4 18 19 30
RFA_00632 0.40 0.39 0.41 11 4068 16 0 16 0 17
RFA_00636 0.44 0.46 0.43 13 3975 17 0 17 0 15
RFA_00639 0.57 0.57 0.57 50 54528 49 8 29 12 37
RFA_00642 0.39 0.33 0.46 6 2913 7 2 5 0 12
RFA_00643 0.30 0.28 0.33 5 2196 11 2 8 1 13
RFA_00644 -0.01 0.00 0.00 0 2685 16 5 11 0 18
RFA_00645 -0.01 0.00 0.00 0 2404 11 2 9 0 18
RFA_00649 0.75 0.78 0.74 14 2126 8 1 4 3 4
RFA_00651 0.25 0.22 0.29 4 2066 12 0 10 2 14
RFA_00653 0.34 0.33 0.35 6 2128 11 2 9 0 12
RFA_00654 0.23 0.22 0.25 4 2399 12 2 10 0 14
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1122 8 0 6 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.30 0.29 0.33 4 978 8 1 7 0 10
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 -0.01 0.00 0.00 0 895 9 0 8 1 13
RFA_00673 0.44 0.36 0.56 5 1119 5 0 4 1 9
RFA_00674 0.42 0.36 0.50 5 1118 5 0 5 0 9
RFA_00675 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00677 0.46 0.43 0.50 6 978 7 0 6 1 8
RFA_00678 0.37 0.29 0.50 4 938 4 0 4 0 10
RFA_00680 0.74 0.71 0.77 10 1115 7 0 3 4 4
RFA_00684 0.66 0.64 0.69 9 977 6 0 4 2 5
RFA_00685 0.42 0.36 0.50 5 980 6 0 5 1 9
RFA_00704 -0.01 0.00 0.00 0 988 3 0 2 1 14
RFA_00705 0.69 0.71 0.67 10 1020 5 0 5 0 4
RFA_00706 0.25 0.21 0.30 3 1025 7 0 7 0 11
RFA_00707 -0.01 0.00 0.00 0 1027 8 0 8 0 14
RFA_00708 0.33 0.29 0.40 4 1025 6 0 6 0 10
RFA_00709 -0.01 0.00 0.00 0 982 8 0 8 0 14
RFA_00710 0.07 0.07 0.10 1 980 9 0 9 0 13
RFA_00711 -0.01 0.00 0.00 0 1023 12 0 12 0 14
RFA_00715 -0.01 0.00 0.00 0 933 13 0 13 0 14
RFA_00716 -0.01 0.00 0.00 0 936 10 0 10 0 14
RFA_00717 0.64 0.64 0.64 9 889 5 0 5 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00733 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00734 0.73 0.67 0.80 8 893 3 0 2 1 4
RFA_00736 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00737 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00745 0.81 0.67 1.00 8 938 1 0 0 1 4
RFA_00749 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00758 0.76 0.58 1.00 7 896 0 0 0 0 5
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00764 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00765 0.81 0.67 1.00 8 895 1 0 0 1 4
RFA_00769 0.54 0.56 0.53 10 1934 9 4 5 0 8
RFA_00770 0.59 0.56 0.63 10 2000 9 0 6 3 8
RFA_00773 0.55 0.56 0.56 10 1935 8 3 5 0 8
RFA_00779 0.63 0.56 0.71 10 1939 4 3 1 0 8
RFA_00781 0.73 0.75 0.71 24 5016 11 2 8 1 8
RFA_00786 0.64 0.66 0.64 21 5017 12 2 10 0 11
RFA_00791 0.86 0.84 0.87 27 5120 6 2 2 2 5
RFA_00792 0.95 0.94 0.97 30 5019 3 0 1 2 2
RFA_00801 0.76 0.78 0.74 25 5016 11 2 7 2 7
RFA_00808 0.65 0.56 0.75 9 2004 3 0 3 0 7
RFA_00809 0.35 0.38 0.33 6 2127 12 1 11 0 10
RFA_00814 0.32 0.37 0.28 15 25146 45 8 31 6 26
RFA_00815 0.57 0.61 0.54 25 24485 35 5 16 14 16
RFA_00816 0.80 0.88 0.73 36 23171 20 9 4 7 5
RFA_00817 0.10 0.12 0.08 5 21883 57 18 39 0 36
RFA_00818 0.48 0.54 0.43 22 20250 38 13 16 9 19
RFA_00819 0.96 0.95 0.98 39 27926 34 0 1 33 2
SRP_00016 0.79 0.78 0.80 85 47789 22 5 16 1 24
SRP_00130 0.50 0.51 0.49 49 49670 54 9 42 3 48
SRP_00142 0.33 0.32 0.34 30 45967 61 8 51 2 63
SRP_00143 0.37 0.38 0.36 38 49664 70 7 61 2 63
SRP_00144 0.38 0.39 0.38 37 46567 65 9 52 4 57
SRP_00197 0.22 0.21 0.24 21 49368 66 5 61 0 80
SRP_00198 0.25 0.24 0.26 25 50945 71 5 65 1 78
SRP_00199 0.63 0.63 0.63 66 50936 41 3 35 3 39
SRP_00201 0.71 0.70 0.72 78 47786 33 3 28 2 33
SRP_00202 0.88 0.87 0.89 102 54170 16 2 11 3 15
SRP_00203 0.72 0.74 0.70 78 48717 41 2 31 8 27
SRP_00204 0.67 0.67 0.67 68 49040 33 7 26 0 34
SRP_00206 0.86 0.88 0.83 95 47781 21 5 14 2 13
SRP_00208 0.65 0.64 0.65 65 46565 37 9 26 2 36
SRP_00255 0.62 0.62 0.62 58 47492 41 3 33 5 36
SRP_00276 0.85 0.84 0.87 92 50615 17 5 9 3 18
SRP_00278 0.67 0.67 0.67 70 45951 38 3 32 3 35
SRP_00321 0.58 0.58 0.58 63 47477 48 11 35 2 45
SRP_00322 0.66 0.64 0.68 70 48413 34 2 31 1 39
SRP_00323 0.82 0.80 0.83 86 47175 20 0 17 3 21
SRP_00330 0.42 0.39 0.46 37 50006 44 3 40 1 58
SRP_00332 0.69 0.70 0.69 74 50296 35 7 26 2 32
SRP_00339 0.70 0.67 0.73 63 44465 27 4 19 4 31

^top



Performance of CMfinder(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 1905
Total TN 2881233
Total FP 688
Total FP CONTRA 40
Total FP INCONS 572
Total FP COMP 76
Total FN 4192
Total Scores
MCC 0.486
Average MCC ± 95% Confidence Intervals 0.505 ± 0.044
Sensitivity 0.312
Positive Predictive Value 0.757
Nr of predictions 129

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.37 0.14 1.00 16 103724 0 0 0 0 99
CRW_00016 0.36 0.14 0.89 17 77402 2 0 2 0 103
CRW_00610 0.49 0.31 0.78 25 36283 7 1 6 0 56
CRW_00613 0.54 0.33 0.87 26 34950 4 0 4 0 52
CRW_00614 0.46 0.25 0.88 14 121755 9 0 2 7 43
CRW_00618 0.36 0.13 1.00 8 56272 0 0 0 0 53
CRW_00633 0.32 0.12 0.81 13 63530 3 1 2 0 93
CRW_00634 0.40 0.17 0.94 16 64603 1 0 1 0 78
CRW_00670 0.32 0.18 0.55 22 70085 18 0 18 0 98
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00674 0.40 0.17 0.95 21 83414 1 0 1 0 103
CRW_00676 0.12 0.04 0.33 5 93513 10 1 9 0 110
CRW_00692 0.41 0.17 1.00 15 68620 0 0 0 0 75
PDB_00005 0.47 0.36 0.63 5 938 3 0 3 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.00 0.00 0.00 0 465 0 0 0 0 11
PDB_00716 -0.01 0.00 0.00 0 2688 13 0 13 0 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.25 0.10 0.62 8 27015 5 0 5 0 73
PDB_01050 0.68 0.46 1.00 6 624 1 0 0 1 7
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
PDB_01092 0.38 0.21 0.69 11 10137 6 0 5 1 41
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.70 0.60 0.81 21 13504 7 0 5 2 14
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00605 0.14 0.06 0.33 2 15219 5 0 4 1 33
RFA_00606 0.34 0.21 0.57 8 21307 8 2 4 2 31
RFA_00607 0.08 0.06 0.13 2 17750 15 3 11 1 34
RFA_00609 0.32 0.22 0.47 8 17938 12 2 7 3 29
RFA_00610 0.39 0.30 0.52 11 17184 13 2 8 3 26
RFA_00611 0.65 0.51 0.82 18 13019 5 0 4 1 17
RFA_00613 0.70 0.57 0.87 20 12857 4 0 3 1 15
RFA_00615 0.57 0.49 0.68 17 13341 10 0 8 2 18
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00626 0.44 0.32 0.60 28 56569 22 2 17 3 59
RFA_00627 0.53 0.39 0.72 34 56906 15 3 10 2 53
RFA_00628 0.58 0.43 0.77 37 57243 14 3 8 3 49
RFA_00630 0.45 0.32 0.64 28 56909 17 3 13 1 59
RFA_00632 0.27 0.25 0.29 7 4071 17 0 17 0 21
RFA_00636 0.71 0.64 0.78 18 3982 5 2 3 0 10
RFA_00639 0.00 0.00 0.00 0 54575 44 6 34 4 87
RFA_00642 0.44 0.28 0.71 5 2919 2 0 2 0 13
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.00 0.00 0.00 0 2693 8 0 8 0 18
RFA_00645 0.29 0.17 0.50 3 2409 3 0 3 0 15
RFA_00649 0.35 0.22 0.57 4 2138 3 0 3 0 14
RFA_00651 0.35 0.22 0.57 4 2073 3 0 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.38 0.22 0.67 4 2409 2 0 2 0 14
RFA_00658 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00659 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00667 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00668 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00672 0.41 0.31 0.57 4 896 3 0 3 0 9
RFA_00673 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00685 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00706 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00707 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00710 0.32 0.21 0.50 3 984 3 0 3 0 11
RFA_00711 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00715 0.30 0.14 0.67 2 943 2 0 1 1 12
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00731 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00733 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00737 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00745 0.91 0.83 1.00 10 936 0 0 0 0 2
RFA_00749 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00769 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00779 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00791 0.92 0.88 0.97 28 5122 1 0 1 0 4
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.41 0.38 0.46 6 2132 7 1 6 0 10
RFA_00814 0.68 0.51 0.91 21 25177 2 0 2 0 20
RFA_00815 0.68 0.51 0.91 21 24508 2 0 2 0 20
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
RFA_00817 0.68 0.56 0.82 23 21917 5 0 5 0 18
RFA_00818 0.60 0.49 0.74 20 20274 8 1 6 1 21
RFA_00819 0.62 0.51 0.75 21 27938 7 0 7 0 20
SRP_00016 0.54 0.35 0.83 38 47849 8 0 8 0 71
SRP_00130 0.50 0.32 0.78 31 49730 10 1 8 1 66
SRP_00142 0.46 0.37 0.58 34 45997 26 1 24 1 59
SRP_00143 0.50 0.30 0.86 30 49735 5 0 5 0 71
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00199 0.54 0.35 0.82 37 50995 8 0 8 0 68
SRP_00201 0.54 0.35 0.83 39 47848 8 0 8 0 72
SRP_00202 0.43 0.21 0.86 25 54256 4 0 4 0 92
SRP_00203 0.51 0.34 0.77 36 48781 12 0 11 1 69
SRP_00204 0.52 0.34 0.80 35 49097 9 1 8 0 67
SRP_00206 0.55 0.36 0.83 39 47848 8 0 8 0 69
SRP_00208 0.52 0.33 0.83 33 46625 7 1 6 0 68
SRP_00255 0.49 0.27 0.89 25 47558 3 0 3 0 69
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00323 0.54 0.34 0.88 36 47237 5 0 5 0 71
SRP_00330 0.44 0.23 0.85 22 50060 5 0 4 1 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79
SRP_00339 0.56 0.39 0.79 37 44504 10 2 8 0 57

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.