CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CMfinder(seed) - scored higher in this pairwise comparison

  4. Performance of RNASLOpt - scored lower in this pairwise comparison

  5. Compile and download dataset for CMfinder(seed) & RNASLOpt [.zip] - may take several seconds...


Overview

Metric CMfinder(seed) RNASLOpt
MCC 0.495 > 0.489
Average MCC ± 95% Confidence Intervals 0.510 ± 0.045 > 0.465 ± 0.051
Sensitivity 0.325 < 0.458
Positive Predictive Value 0.755 > 0.524
Total TP 1849 < 2607
Total TN 2478827 > 2476301
Total FP 668 < 2629
Total FP CONTRA 39 < 396
Total FP INCONS 560 < 1971
Total FP COMP 69 < 262
Total FN 3837 > 3079
P-value 0.00297289556633

^top




Performance plots


  1. Comparison of performance of CMfinder(seed) and RNASLOpt. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CMfinder(seed) and RNASLOpt).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CMfinder(seed) and RNASLOpt).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CMfinder(seed) and RNASLOpt. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CMfinder(seed) and RNASLOpt).

^top





Performance of CMfinder(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for CMfinder(seed)

Total Base Pair Counts
Total TP 1849
Total TN 2478827
Total FP 668
Total FP CONTRA 39
Total FP INCONS 560
Total FP COMP 69
Total FN 3837
Total Scores
MCC 0.495
Average MCC ± 95% Confidence Intervals 0.510 ± 0.045
Sensitivity 0.325
Positive Predictive Value 0.755
Nr of predictions 125

^top



2. Individual counts for CMfinder(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00016 0.36 0.14 0.89 17 77402 2 0 2 0 103
CRW_00610 0.49 0.31 0.78 25 36283 7 1 6 0 56
CRW_00613 0.54 0.33 0.87 26 34950 4 0 4 0 52
CRW_00618 0.36 0.13 1.00 8 56272 0 0 0 0 53
CRW_00633 0.32 0.12 0.81 13 63530 3 1 2 0 93
CRW_00634 0.40 0.17 0.94 16 64603 1 0 1 0 78
CRW_00670 0.32 0.18 0.55 22 70085 18 0 18 0 98
CRW_00671 0.20 0.09 0.41 11 62808 16 0 16 0 106
CRW_00672 0.38 0.15 0.94 17 72372 1 0 1 0 94
CRW_00692 0.41 0.17 1.00 15 68620 0 0 0 0 75
PDB_00005 0.47 0.36 0.63 5 938 3 0 3 0 9
PDB_00012 1.00 1.00 1.00 7 399 2 0 0 2 0
PDB_00213 0.85 0.73 1.00 30 5020 0 0 0 0 11
PDB_00553 0.00 0.00 0.00 0 465 0 0 0 0 11
PDB_00716 -0.01 0.00 0.00 0 2688 13 0 13 0 23
PDB_00810 0.76 0.59 1.00 10 1071 0 0 0 0 7
PDB_00827 0.25 0.10 0.62 8 27015 5 0 5 0 73
PDB_01050 0.68 0.46 1.00 6 624 1 0 0 1 7
PDB_01051 0.83 0.69 1.00 9 894 4 0 0 4 4
PDB_01092 0.38 0.21 0.69 11 10137 6 0 5 1 41
PDB_01152 0.92 0.86 1.00 12 549 0 0 0 0 2
RFA_00603 0.70 0.60 0.81 21 13504 7 0 5 2 14
RFA_00604 0.63 0.51 0.78 18 13507 8 0 5 3 17
RFA_00605 0.14 0.06 0.33 2 15219 5 0 4 1 33
RFA_00606 0.34 0.21 0.57 8 21307 8 2 4 2 31
RFA_00607 0.08 0.06 0.13 2 17750 15 3 11 1 34
RFA_00609 0.32 0.22 0.47 8 17938 12 2 7 3 29
RFA_00610 0.39 0.30 0.52 11 17184 13 2 8 3 26
RFA_00611 0.65 0.51 0.82 18 13019 5 0 4 1 17
RFA_00613 0.70 0.57 0.87 20 12857 4 0 3 1 15
RFA_00615 0.57 0.49 0.68 17 13341 10 0 8 2 18
RFA_00620 0.00 0.00 0.00 0 21939 6 0 6 0 39
RFA_00626 0.44 0.32 0.60 28 56569 22 2 17 3 59
RFA_00627 0.53 0.39 0.72 34 56906 15 3 10 2 53
RFA_00628 0.58 0.43 0.77 37 57243 14 3 8 3 49
RFA_00630 0.45 0.32 0.64 28 56909 17 3 13 1 59
RFA_00632 0.27 0.25 0.29 7 4071 17 0 17 0 21
RFA_00636 0.71 0.64 0.78 18 3982 5 2 3 0 10
RFA_00639 0.00 0.00 0.00 0 54575 44 6 34 4 87
RFA_00642 0.44 0.28 0.71 5 2919 2 0 2 0 13
RFA_00643 -0.01 0.00 0.00 0 2203 8 0 8 0 18
RFA_00644 0.00 0.00 0.00 0 2693 8 0 8 0 18
RFA_00645 0.29 0.17 0.50 3 2409 3 0 3 0 15
RFA_00649 0.35 0.22 0.57 4 2138 3 0 3 0 14
RFA_00651 0.35 0.22 0.57 4 2073 3 0 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2137 8 0 8 0 18
RFA_00654 0.38 0.22 0.67 4 2409 2 0 2 0 14
RFA_00658 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00659 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00664 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00667 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00668 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00672 0.41 0.31 0.57 4 896 3 0 3 0 9
RFA_00673 0.23 0.14 0.40 2 1123 4 0 3 1 12
RFA_00674 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00675 0.35 0.21 0.60 3 985 3 0 2 1 11
RFA_00677 0.40 0.29 0.57 4 983 4 0 3 1 10
RFA_00678 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00680 0.40 0.29 0.57 4 1121 4 0 3 1 10
RFA_00684 0.43 0.29 0.67 4 984 3 0 2 1 10
RFA_00685 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00706 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00707 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00708 0.40 0.29 0.57 4 1028 4 0 3 1 10
RFA_00709 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00710 0.32 0.21 0.50 3 984 3 0 3 0 11
RFA_00711 0.43 0.29 0.67 4 1029 3 0 2 1 10
RFA_00715 0.30 0.14 0.67 2 943 2 0 1 1 12
RFA_00716 0.43 0.29 0.67 4 940 3 0 2 1 10
RFA_00717 0.30 0.21 0.43 3 896 4 0 4 0 11
RFA_00730 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00731 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00733 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00734 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00736 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00737 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00745 0.91 0.83 1.00 10 936 0 0 0 0 2
RFA_00749 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00758 0.86 0.75 1.00 9 894 0 0 0 0 3
RFA_00762 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00763 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00764 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00765 0.91 0.83 1.00 10 893 0 0 0 0 2
RFA_00769 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00770 0.71 0.50 1.00 9 2007 0 0 0 0 9
RFA_00773 0.00 0.00 0.00 0 1949 4 0 4 0 18
RFA_00779 0.71 0.50 1.00 9 1944 0 0 0 0 9
RFA_00781 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00786 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00791 0.92 0.88 0.97 28 5122 1 0 1 0 4
RFA_00792 0.97 0.94 1.00 30 5020 0 0 0 0 2
RFA_00801 0.95 0.91 1.00 29 5021 0 0 0 0 3
RFA_00808 0.68 0.56 0.82 9 2005 2 0 2 0 7
RFA_00809 0.41 0.38 0.46 6 2132 7 1 6 0 10
RFA_00814 0.68 0.51 0.91 21 25177 2 0 2 0 20
RFA_00815 0.68 0.51 0.91 21 24508 2 0 2 0 20
RFA_00816 0.68 0.51 0.91 21 23197 2 0 2 0 20
RFA_00817 0.68 0.56 0.82 23 21917 5 0 5 0 18
RFA_00818 0.60 0.49 0.74 20 20274 8 1 6 1 21
RFA_00819 0.62 0.51 0.75 21 27938 7 0 7 0 20
SRP_00016 0.54 0.35 0.83 38 47849 8 0 8 0 71
SRP_00130 0.50 0.32 0.78 31 49730 10 1 8 1 66
SRP_00142 0.46 0.37 0.58 34 45997 26 1 24 1 59
SRP_00143 0.50 0.30 0.86 30 49735 5 0 5 0 71
SRP_00144 0.54 0.37 0.80 35 46621 11 0 9 2 59
SRP_00197 0.46 0.26 0.84 26 49424 5 0 5 0 75
SRP_00198 0.56 0.37 0.84 38 50995 7 0 7 0 65
SRP_00199 0.54 0.35 0.82 37 50995 8 0 8 0 68
SRP_00201 0.54 0.35 0.83 39 47848 8 0 8 0 72
SRP_00202 0.43 0.21 0.86 25 54256 4 0 4 0 92
SRP_00203 0.51 0.34 0.77 36 48781 12 0 11 1 69
SRP_00204 0.52 0.34 0.80 35 49097 9 1 8 0 67
SRP_00206 0.55 0.36 0.83 39 47848 8 0 8 0 69
SRP_00208 0.52 0.33 0.83 33 46625 7 1 6 0 68
SRP_00255 0.49 0.27 0.89 25 47558 3 0 3 0 69
SRP_00276 0.51 0.33 0.80 36 50676 9 0 9 0 74
SRP_00278 0.52 0.33 0.81 35 46013 8 0 8 0 70
SRP_00321 0.42 0.21 0.82 23 47558 5 1 4 0 85
SRP_00322 0.52 0.33 0.82 36 48472 8 0 8 0 73
SRP_00323 0.54 0.34 0.88 36 47237 5 0 5 0 71
SRP_00330 0.44 0.23 0.85 22 50060 5 0 4 1 73
SRP_00332 0.44 0.25 0.75 27 50367 9 0 9 0 79
SRP_00339 0.56 0.39 0.79 37 44504 10 2 8 0 57

^top



Performance of RNASLOpt - scored lower in this pairwise comparison

1. Total counts & total scores for RNASLOpt

Total Base Pair Counts
Total TP 2607
Total TN 2476301
Total FP 2629
Total FP CONTRA 396
Total FP INCONS 1971
Total FP COMP 262
Total FN 3079
Total Scores
MCC 0.489
Average MCC ± 95% Confidence Intervals 0.465 ± 0.051
Sensitivity 0.458
Positive Predictive Value 0.524
Nr of predictions 125

^top



2. Individual counts for RNASLOpt [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00016 0.48 0.45 0.52 54 77317 58 5 45 8 66
CRW_00610 0.06 0.06 0.07 5 36246 64 9 55 0 76
CRW_00613 0.59 0.50 0.70 39 34924 24 1 16 7 39
CRW_00618 0.21 0.21 0.21 13 56218 62 6 43 13 48
CRW_00633 0.50 0.49 0.51 52 63444 52 5 45 2 54
CRW_00634 0.50 0.50 0.51 47 64527 48 11 35 2 47
CRW_00670 0.45 0.43 0.47 52 70015 58 8 50 0 68
CRW_00671 0.60 0.53 0.68 62 62744 32 2 27 3 55
CRW_00672 0.43 0.41 0.45 45 72291 57 6 48 3 66
CRW_00692 0.33 0.34 0.33 31 68540 77 20 44 13 59
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.90 0.80 1.00 33 5017 0 0 0 0 8
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2680 22 0 21 1 23
PDB_00810 0.51 0.41 0.64 7 1070 4 2 2 0 10
PDB_00827 0.74 0.60 0.91 49 26974 7 0 5 2 32
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.79 0.69 0.90 9 893 2 0 1 1 4
PDB_01092 0.74 0.63 0.87 33 10115 7 1 4 2 19
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00603 0.54 0.51 0.56 18 13498 15 4 10 1 17
RFA_00604 0.73 0.66 0.82 23 13502 12 4 1 7 12
RFA_00605 0.36 0.34 0.39 12 15194 27 3 16 8 23
RFA_00606 0.18 0.18 0.18 7 21281 39 4 29 6 32
RFA_00607 0.90 0.83 0.97 30 17735 12 0 1 11 6
RFA_00609 0.92 0.89 0.94 33 17920 10 0 2 8 4
RFA_00610 0.90 0.86 0.94 32 17171 10 0 2 8 5
RFA_00611 0.69 0.66 0.72 23 13009 9 2 7 0 12
RFA_00613 0.16 0.14 0.18 5 12852 27 9 14 4 30
RFA_00615 0.47 0.43 0.52 15 13337 14 8 6 0 20
RFA_00620 0.66 0.64 0.68 25 21908 22 2 10 10 14
RFA_00626 0.68 0.64 0.72 56 56538 30 6 16 8 31
RFA_00627 0.61 0.60 0.62 52 56869 41 11 21 9 35
RFA_00628 0.89 0.84 0.95 72 57215 15 0 4 11 14
RFA_00630 0.69 0.67 0.72 58 56872 32 5 18 9 29
RFA_00632 0.59 0.57 0.62 16 4069 10 2 8 0 12
RFA_00636 0.65 0.64 0.67 18 3978 9 2 7 0 10
RFA_00639 0.29 0.29 0.29 25 54528 67 12 50 5 62
RFA_00642 -0.01 0.00 0.00 0 2914 12 1 11 0 18
RFA_00643 -0.01 0.00 0.00 0 2199 12 1 11 0 18
RFA_00644 -0.01 0.00 0.00 0 2680 21 6 15 0 18
RFA_00645 -0.01 0.00 0.00 0 2403 12 3 9 0 18
RFA_00649 0.33 0.33 0.33 6 2127 12 6 6 0 12
RFA_00651 0.27 0.22 0.33 4 2068 8 1 7 0 14
RFA_00653 0.33 0.33 0.33 6 2127 12 3 9 0 12
RFA_00654 0.59 0.50 0.69 9 2402 5 1 3 1 9
RFA_00658 0.40 0.29 0.57 4 1121 5 0 3 2 10
RFA_00659 -0.01 0.00 0.00 0 1125 5 0 3 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.29 0.29 0.31 4 977 10 1 8 1 10
RFA_00668 0.50 0.43 0.60 6 980 4 0 4 0 8
RFA_00672 -0.01 0.00 0.00 0 897 6 0 6 0 13
RFA_00673 0.47 0.36 0.63 5 1120 4 0 3 1 9
RFA_00674 0.74 0.71 0.77 10 1115 4 0 3 1 4
RFA_00675 0.29 0.29 0.31 4 977 9 1 8 0 10
RFA_00677 0.74 0.71 0.77 10 977 5 0 3 2 4
RFA_00678 0.53 0.29 1.00 4 942 0 0 0 0 10
RFA_00680 0.84 0.71 1.00 10 1118 2 0 0 2 4
RFA_00684 0.84 0.71 1.00 10 980 1 0 0 1 4
RFA_00685 0.47 0.36 0.63 5 982 4 0 3 1 9
RFA_00704 -0.01 0.00 0.00 0 982 8 0 8 0 14
RFA_00705 0.74 0.71 0.77 10 1022 4 0 3 1 4
RFA_00706 0.37 0.29 0.50 4 1027 4 0 4 0 10
RFA_00707 -0.01 0.00 0.00 0 1030 5 0 5 0 14
RFA_00708 0.53 0.29 1.00 4 1031 0 0 0 0 10
RFA_00709 -0.01 0.00 0.00 0 984 7 0 6 1 14
RFA_00710 -0.01 0.00 0.00 0 984 6 0 6 0 14
RFA_00711 -0.01 0.00 0.00 0 1026 9 0 9 0 14
RFA_00715 -0.01 0.00 0.00 0 939 7 0 7 0 14
RFA_00716 -0.01 0.00 0.00 0 939 7 0 7 0 14
RFA_00717 -0.01 0.00 0.00 0 894 9 0 9 0 14
RFA_00730 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00731 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00733 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00734 0.67 0.58 0.78 7 894 3 0 2 1 5
RFA_00736 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00737 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00745 -0.01 0.00 0.00 0 941 5 1 4 0 12
RFA_00749 0.48 0.42 0.56 5 894 5 1 3 1 7
RFA_00758 -0.01 0.00 0.00 0 898 5 0 5 0 12
RFA_00762 -0.01 0.00 0.00 0 894 9 2 7 0 12
RFA_00763 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00764 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00765 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00769 0.59 0.56 0.63 10 1937 6 1 5 0 8
RFA_00770 0.47 0.39 0.58 7 2004 5 1 4 0 11
RFA_00773 0.59 0.56 0.63 10 1937 6 1 5 0 8
RFA_00779 0.74 0.56 1.00 10 1943 0 0 0 0 8
RFA_00781 0.51 0.50 0.53 16 5020 14 2 12 0 16
RFA_00786 -0.01 0.00 0.00 0 5022 28 4 24 0 32
RFA_00791 0.44 0.41 0.48 13 5124 14 4 10 0 19
RFA_00792 0.89 0.84 0.93 27 5021 2 0 2 0 5
RFA_00801 0.82 0.75 0.89 24 5023 4 0 3 1 8
RFA_00808 0.75 0.56 1.00 9 2007 0 0 0 0 7
RFA_00809 0.40 0.38 0.43 6 2131 8 1 7 0 10
RFA_00814 0.51 0.51 0.51 21 25159 29 9 11 9 20
RFA_00815 0.41 0.41 0.41 17 24490 31 1 23 7 24
RFA_00816 0.58 0.63 0.53 26 23171 26 13 10 3 15
RFA_00817 0.24 0.27 0.22 11 21896 42 12 26 4 30
RFA_00818 0.21 0.22 0.21 9 20258 39 14 20 5 32
RFA_00819 0.60 0.68 0.53 28 27913 34 16 9 9 13
SRP_00016 0.74 0.71 0.77 77 47795 24 5 18 1 32
SRP_00130 0.51 0.48 0.55 47 49684 44 5 34 5 50
SRP_00142 0.18 0.17 0.18 16 45968 72 8 64 0 77
SRP_00143 0.65 0.62 0.68 63 49677 35 0 30 5 38
SRP_00144 0.04 0.04 0.05 4 46578 83 17 66 0 90
SRP_00197 0.23 0.21 0.25 21 49370 64 9 55 0 80
SRP_00198 0.41 0.38 0.45 39 50954 47 5 42 0 64
SRP_00199 0.24 0.23 0.26 24 50949 67 3 64 0 81
SRP_00201 0.73 0.72 0.74 80 47787 30 3 25 2 31
SRP_00202 0.52 0.49 0.56 57 54184 44 6 38 0 60
SRP_00203 0.65 0.64 0.66 67 48727 39 4 30 5 38
SRP_00204 0.56 0.53 0.59 54 49049 39 8 30 1 48
SRP_00206 0.84 0.80 0.90 86 47799 11 5 5 1 22
SRP_00208 0.47 0.45 0.49 45 46574 46 9 37 0 56
SRP_00255 0.29 0.30 0.29 28 47489 69 11 58 0 66
SRP_00276 0.66 0.63 0.70 69 50622 31 7 23 1 41
SRP_00278 0.68 0.65 0.72 68 45961 29 4 23 2 37
SRP_00321 0.48 0.45 0.50 49 47488 51 5 44 2 59
SRP_00322 0.57 0.54 0.61 59 48419 38 1 37 0 50
SRP_00323 0.75 0.69 0.82 74 47188 19 0 16 3 33
SRP_00330 0.19 0.18 0.20 17 50002 67 11 56 0 78
SRP_00332 0.10 0.09 0.10 10 50305 88 5 83 0 96
SRP_00339 0.44 0.43 0.45 40 44462 51 6 43 2 54

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.