CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CMfinder(seed) - scored higher in this pairwise comparison

  4. Performance of RSpredict(seed) - scored lower in this pairwise comparison

  5. Compile and download dataset for CMfinder(seed) & RSpredict(seed) [.zip] - may take several seconds...


Overview

Metric CMfinder(seed) RSpredict(seed)
MCC 0.486 > 0.484
Average MCC ± 95% Confidence Intervals 0.505 ± 0.044 > 0.495 ± 0.054
Sensitivity 0.312 < 0.331
Positive Predictive Value 0.757 > 0.709
Total TP 1905 < 2020
Total TN 2881233 > 2880901
Total FP 688 < 1032
Total FP CONTRA 40 < 81
Total FP INCONS 572 < 748
Total FP COMP 76 < 203
Total FN 4192 > 4077
P-value 0.375011270663

^top




Performance plots


  1. Comparison of performance of CMfinder(seed) and RSpredict(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CMfinder(seed) and RSpredict(seed)).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CMfinder(seed) and RSpredict(seed)).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CMfinder(seed) and RSpredict(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CMfinder(seed) and RSpredict(seed)).

^top





Performance of CMfinder(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 1905
Total TN 2881233
Total FP 688
Total FP CONTRA 40
Total FP INCONS 572
Total FP COMP 76
Total FN 4192
Total Scores
MCC 0.486
Average MCC ± 95% Confidence Intervals 0.505 ± 0.044
Sensitivity 0.312
Positive Predictive Value 0.757
Nr of predictions 129

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.37 0.14 1.00 16 103724 0 0 0 0 99
CRW_00016 0.36 0.14 0.89 17 77402 2 0 2 0 103
CRW_00610 0.49 0.31 0.78 25 36283 7 1 6 0 56
CRW_00613 0.54 0.33 0.87 26 34950 4 0 4 0 52
CRW_00614 0.46 0.25 0.88 14 121755 9 0 2 7 43
CRW_00618 0.36 0.13 1.00 8 56272 0 0 0 0 53
CRW_00633 0.32 0.12 0.81 13 63530 3 1 2 0 93
CRW_00634 0.40 0.17 0.94 16 64603 1 0 1 0 78
CRW_00670 0.32 0.18 0.55 22 70085 18 0 18 0 98
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00674 0.40 0.17 0.95 21 83414 1 0 1 0 103
CRW_00676 0.12 0.04 0.33 5 93513 10 1 9 0 110
CRW_00692 0.41 0.17 1.00 15 68620 0 0 0 0 75
PDB_00005 0.47 0.36 0.63 5 938 3 0 3 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.00 0.00 0.00 0 465 0 0 0 0 11
PDB_00716 -0.01 0.00 0.00 0 2688 13 0 13 0 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.25 0.10 0.62 8 27015 5 0 5 0 73
PDB_01050 0.68 0.46 1.00 6 624 1 0 0 1 7
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
PDB_01092 0.38 0.21 0.69 11 10137 6 0 5 1 41
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.70 0.60 0.81 21 13504 7 0 5 2 14
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00605 0.14 0.06 0.33 2 15219 5 0 4 1 33
RFA_00606 0.34 0.21 0.57 8 21307 8 2 4 2 31
RFA_00607 0.08 0.06 0.13 2 17750 15 3 11 1 34
RFA_00609 0.32 0.22 0.47 8 17938 12 2 7 3 29
RFA_00610 0.39 0.30 0.52 11 17184 13 2 8 3 26
RFA_00611 0.65 0.51 0.82 18 13019 5 0 4 1 17
RFA_00613 0.70 0.57 0.87 20 12857 4 0 3 1 15
RFA_00615 0.57 0.49 0.68 17 13341 10 0 8 2 18
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00626 0.44 0.32 0.60 28 56569 22 2 17 3 59
RFA_00627 0.53 0.39 0.72 34 56906 15 3 10 2 53
RFA_00628 0.58 0.43 0.77 37 57243 14 3 8 3 49
RFA_00630 0.45 0.32 0.64 28 56909 17 3 13 1 59
RFA_00632 0.27 0.25 0.29 7 4071 17 0 17 0 21
RFA_00636 0.71 0.64 0.78 18 3982 5 2 3 0 10
RFA_00639 0.00 0.00 0.00 0 54575 44 6 34 4 87
RFA_00642 0.44 0.28 0.71 5 2919 2 0 2 0 13
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.00 0.00 0.00 0 2693 8 0 8 0 18
RFA_00645 0.29 0.17 0.50 3 2409 3 0 3 0 15
RFA_00649 0.35 0.22 0.57 4 2138 3 0 3 0 14
RFA_00651 0.35 0.22 0.57 4 2073 3 0 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.38 0.22 0.67 4 2409 2 0 2 0 14
RFA_00658 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00659 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00667 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00668 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00672 0.41 0.31 0.57 4 896 3 0 3 0 9
RFA_00673 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00685 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00706 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00707 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00710 0.32 0.21 0.50 3 984 3 0 3 0 11
RFA_00711 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00715 0.30 0.14 0.67 2 943 2 0 1 1 12
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00731 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00733 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00737 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00745 0.91 0.83 1.00 10 936 0 0 0 0 2
RFA_00749 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00769 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00779 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00791 0.92 0.88 0.97 28 5122 1 0 1 0 4
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.41 0.38 0.46 6 2132 7 1 6 0 10
RFA_00814 0.68 0.51 0.91 21 25177 2 0 2 0 20
RFA_00815 0.68 0.51 0.91 21 24508 2 0 2 0 20
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
RFA_00817 0.68 0.56 0.82 23 21917 5 0 5 0 18
RFA_00818 0.60 0.49 0.74 20 20274 8 1 6 1 21
RFA_00819 0.62 0.51 0.75 21 27938 7 0 7 0 20
SRP_00016 0.54 0.35 0.83 38 47849 8 0 8 0 71
SRP_00130 0.50 0.32 0.78 31 49730 10 1 8 1 66
SRP_00142 0.46 0.37 0.58 34 45997 26 1 24 1 59
SRP_00143 0.50 0.30 0.86 30 49735 5 0 5 0 71
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00199 0.54 0.35 0.82 37 50995 8 0 8 0 68
SRP_00201 0.54 0.35 0.83 39 47848 8 0 8 0 72
SRP_00202 0.43 0.21 0.86 25 54256 4 0 4 0 92
SRP_00203 0.51 0.34 0.77 36 48781 12 0 11 1 69
SRP_00204 0.52 0.34 0.80 35 49097 9 1 8 0 67
SRP_00206 0.55 0.36 0.83 39 47848 8 0 8 0 69
SRP_00208 0.52 0.33 0.83 33 46625 7 1 6 0 68
SRP_00255 0.49 0.27 0.89 25 47558 3 0 3 0 69
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00323 0.54 0.34 0.88 36 47237 5 0 5 0 71
SRP_00330 0.44 0.23 0.85 22 50060 5 0 4 1 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79
SRP_00339 0.56 0.39 0.79 37 44504 10 2 8 0 57

^top



Performance of RSpredict(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for RSpredict(seed)

Total Base Pair Counts
Total TP 2020
Total TN 2880901
Total FP 1032
Total FP CONTRA 81
Total FP INCONS 748
Total FP COMP 203
Total FN 4077
Total Scores
MCC 0.484
Average MCC ± 95% Confidence Intervals 0.495 ± 0.054
Sensitivity 0.331
Positive Predictive Value 0.709
Nr of predictions 129

^top



2. Individual counts for RSpredict(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00013 0.00 0.00 0.00 0 103737 3 0 3 0 115
CRW_00016 0.00 0.00 0.00 0 77418 3 0 3 0 120
CRW_00610 0.00 0.00 0.00 0 36310 5 0 5 0 81
CRW_00613 0.00 0.00 0.00 0 34979 1 0 1 0 78
CRW_00614 0.00 0.00 0.00 0 121764 17 0 7 10 57
CRW_00618 0.00 0.00 0.00 0 56279 1 0 1 0 61
CRW_00633 0.00 0.00 0.00 0 63541 5 0 5 0 106
CRW_00634 0.00 0.00 0.00 0 64620 1 0 0 1 94
CRW_00670 0.00 0.00 0.00 0 70124 4 0 1 3 120
CRW_00671 0.00 0.00 0.00 0 62835 0 0 0 0 117
CRW_00672 0.00 0.00 0.00 0 72386 7 0 4 3 111
CRW_00674 0.00 0.00 0.00 0 83436 1 0 0 1 124
CRW_00676 0.00 0.00 0.00 0 93527 2 0 1 1 115
CRW_00692 0.00 0.00 0.00 0 68635 3 0 0 3 90
PDB_00005 0.71 0.57 0.89 8 937 1 0 1 0 6
PDB_00012 1.00 1.00 1.00 7 399 1 0 0 1 0
PDB_00213 0.56 0.49 0.65 20 5019 11 0 11 0 21
PDB_00553 0.90 0.82 1.00 9 456 0 0 0 0 2
PDB_00716 0.59 0.48 0.73 11 2686 4 0 4 0 12
PDB_00810 0.80 0.65 1.00 11 1070 0 0 0 0 6
PDB_00827 0.00 0.00 0.00 0 27026 2 0 2 0 81
PDB_01050 0.62 0.46 0.86 6 623 2 0 1 1 7
PDB_01051 -0.01 0.00 0.00 0 899 4 2 2 0 13
PDB_01092 0.48 0.37 0.63 19 10123 13 1 10 2 33
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00603 0.00 0.00 0.00 0 13520 11 2 8 1 35
RFA_00604 0.00 0.00 0.00 0 13529 1 0 1 0 35
RFA_00605 0.00 0.00 0.00 0 15215 11 2 8 1 35
RFA_00606 0.00 0.00 0.00 0 21309 12 3 9 0 39
RFA_00607 0.00 0.00 0.00 0 17752 15 0 14 1 36
RFA_00609 0.00 0.00 0.00 0 17949 9 1 5 3 37
RFA_00610 0.00 0.00 0.00 0 17195 10 3 7 0 37
RFA_00611 0.00 0.00 0.00 0 13032 10 0 9 1 35
RFA_00613 0.00 0.00 0.00 0 12879 1 0 1 0 35
RFA_00615 0.00 0.00 0.00 0 13360 6 0 6 0 35
RFA_00620 0.00 0.00 0.00 0 21941 4 0 4 0 39
RFA_00626 0.90 0.80 1.00 70 56546 12 0 0 12 17
RFA_00627 0.90 0.82 0.99 71 56881 11 0 1 10 16
RFA_00628 0.89 0.81 0.97 70 57219 12 0 2 10 16
RFA_00630 0.85 0.75 0.97 65 56886 11 0 2 9 22
RFA_00632 0.47 0.39 0.58 11 4076 8 3 5 0 17
RFA_00636 0.62 0.54 0.71 15 3984 6 3 3 0 13
RFA_00639 0.59 0.48 0.72 42 54557 30 3 13 14 45
RFA_00642 0.31 0.22 0.44 4 2917 5 0 5 0 14
RFA_00643 0.21 0.17 0.27 3 2200 8 0 8 0 15
RFA_00644 0.36 0.33 0.40 6 2686 9 1 8 0 12
RFA_00645 0.31 0.28 0.36 5 2401 9 1 8 0 13
RFA_00649 0.37 0.33 0.43 6 2131 8 0 8 0 12
RFA_00651 0.32 0.28 0.38 5 2067 8 0 8 0 13
RFA_00653 0.36 0.33 0.40 6 2130 9 1 8 0 12
RFA_00654 0.35 0.28 0.45 5 2404 6 0 6 0 13
RFA_00658 0.64 0.57 0.73 8 1117 4 0 3 1 6
RFA_00659 0.35 0.29 0.44 4 1119 6 0 5 1 10
RFA_00664 0.59 0.50 0.70 7 980 4 0 3 1 7
RFA_00667 0.64 0.57 0.73 8 979 4 0 3 1 6
RFA_00668 0.71 0.57 0.89 8 981 1 0 1 0 6
RFA_00672 0.61 0.54 0.70 7 893 3 0 3 0 6
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 0.69 0.64 0.75 9 1116 4 0 3 1 5
RFA_00675 0.62 0.50 0.78 7 981 3 0 2 1 7
RFA_00677 0.72 0.64 0.82 9 979 2 0 2 0 5
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.69 0.64 0.75 9 1116 4 0 3 1 5
RFA_00684 0.80 0.64 1.00 9 981 1 0 0 1 5
RFA_00685 0.64 0.57 0.73 8 979 4 0 3 1 6
RFA_00704 0.35 0.29 0.44 4 981 6 0 5 1 10
RFA_00705 0.80 0.64 1.00 9 1026 1 0 0 1 5
RFA_00706 0.56 0.43 0.75 6 1027 3 0 2 1 8
RFA_00707 0.67 0.57 0.80 8 1025 3 0 2 1 6
RFA_00708 0.75 0.57 1.00 8 1027 1 0 0 1 6
RFA_00709 0.53 0.43 0.67 6 981 3 0 3 0 8
RFA_00710 0.43 0.29 0.67 4 984 2 0 2 0 10
RFA_00711 0.35 0.21 0.60 3 1030 3 0 2 1 11
RFA_00715 0.66 0.50 0.88 7 938 2 0 1 1 7
RFA_00716 0.56 0.43 0.75 6 938 3 0 2 1 8
RFA_00717 0.50 0.36 0.71 5 896 2 0 2 0 9
RFA_00730 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00731 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00733 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00737 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00745 0.91 0.83 1.00 10 936 1 0 0 1 2
RFA_00749 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00764 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00765 0.91 0.83 1.00 10 893 1 0 0 1 2
RFA_00769 0.59 0.56 0.63 10 1937 6 4 2 0 8
RFA_00770 0.54 0.56 0.53 10 1997 9 6 3 0 8
RFA_00773 0.50 0.50 0.50 9 1935 9 6 3 0 9
RFA_00779 0.50 0.50 0.50 9 1935 9 6 3 0 9
RFA_00781 0.61 0.59 0.63 19 5020 12 2 9 1 13
RFA_00786 0.61 0.59 0.63 19 5020 12 2 9 1 13
RFA_00791 0.58 0.53 0.63 17 5124 11 2 8 1 15
RFA_00792 0.64 0.63 0.67 20 5020 11 2 8 1 12
RFA_00801 0.59 0.56 0.62 18 5021 12 2 9 1 14
RFA_00808 0.60 0.50 0.73 8 2005 4 0 3 1 8
RFA_00809 0.37 0.31 0.45 5 2134 7 0 6 1 11
RFA_00814 0.90 0.85 0.95 35 25163 11 0 2 9 6
RFA_00815 0.90 0.85 0.95 35 24494 10 0 2 8 6
RFA_00816 0.90 0.85 0.95 35 23183 8 0 2 6 6
RFA_00817 0.88 0.83 0.94 34 21909 7 0 2 5 7
RFA_00818 0.80 0.71 0.91 29 20269 10 0 3 7 12
RFA_00819 0.90 0.85 0.95 35 27929 13 0 2 11 6
SRP_00016 0.58 0.41 0.82 45 47840 11 0 10 1 64
SRP_00130 0.43 0.31 0.60 30 49720 22 2 18 2 67
SRP_00142 0.55 0.42 0.74 39 46003 17 0 14 3 54
SRP_00143 0.56 0.45 0.70 45 49706 20 0 19 1 56
SRP_00144 0.55 0.43 0.70 40 46608 20 1 16 3 54
SRP_00197 0.51 0.37 0.73 37 49404 15 3 11 1 64
SRP_00198 0.52 0.39 0.70 40 50983 20 2 15 3 63
SRP_00199 0.48 0.37 0.63 39 50978 24 0 23 1 66
SRP_00201 0.51 0.39 0.68 43 47832 21 0 20 1 68
SRP_00202 0.46 0.36 0.58 42 54213 31 2 28 1 75
SRP_00203 0.41 0.31 0.54 33 48767 30 1 27 2 72
SRP_00204 0.56 0.42 0.74 43 49083 17 1 14 2 59
SRP_00206 0.55 0.41 0.73 44 47835 17 0 16 1 64
SRP_00208 0.44 0.32 0.60 32 46612 21 2 19 0 69
SRP_00255 0.52 0.38 0.71 36 47535 17 2 13 2 58
SRP_00276 0.62 0.48 0.79 53 50654 15 1 13 1 57
SRP_00278 0.48 0.35 0.65 37 45999 21 1 19 1 68
SRP_00321 0.37 0.25 0.56 27 47538 22 1 20 1 81
SRP_00322 0.47 0.34 0.66 37 48460 20 2 17 1 72
SRP_00323 0.53 0.36 0.76 39 47227 13 0 12 1 68
SRP_00330 0.38 0.25 0.57 24 50044 20 1 17 2 71
SRP_00332 0.70 0.58 0.86 61 50332 11 0 10 1 45
SRP_00339 0.36 0.26 0.50 24 44503 25 1 23 1 70

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.