CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of RNASampler(seed) - scored higher in this pairwise comparison

  4. Performance of Fold - scored lower in this pairwise comparison

  5. Compile and download dataset for RNASampler(seed) & Fold [.zip] - may take several seconds...


Overview

Metric RNASampler(seed) Fold
MCC 0.629 > 0.601
Average MCC ± 95% Confidence Intervals 0.470 ± 0.066 < 0.569 ± 0.067
Sensitivity 0.468 < 0.589
Positive Predictive Value 0.847 > 0.616
Total TP 1072 < 1348
Total TN 696315 > 695391
Total FP 320 < 1220
Total FP CONTRA 57 < 155
Total FP INCONS 137 < 687
Total FP COMP 126 < 378
Total FN 1217 > 941
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of RNASampler(seed) and Fold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNASampler(seed) and Fold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNASampler(seed) and Fold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for RNASampler(seed) and Fold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNASampler(seed) and Fold).

^top





Performance of RNASampler(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for RNASampler(seed)

Total Base Pair Counts
Total TP 1072
Total TN 696315
Total FP 320
Total FP CONTRA 57
Total FP INCONS 137
Total FP COMP 126
Total FN 1217
Total Scores
MCC 0.629
Average MCC ± 95% Confidence Intervals 0.470 ± 0.066
Sensitivity 0.468
Positive Predictive Value 0.847
Nr of predictions 98

^top



2. Individual counts for RNASampler(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.76 0.59 1.00 24 5026 0 0 0 0 17
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 0.50 0.39 0.64 9 2687 6 0 5 1 14
PDB_00810 0.42 0.18 1.00 3 1078 0 0 0 0 14
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.45 0.31 0.67 4 897 6 0 2 4 9
PDB_01092 0.72 0.62 0.84 32 10115 8 0 6 2 20
PDB_01152 0.84 0.71 1.00 10 551 0 0 0 0 4
RFA_00390 0.00 0.00 0.00 0 1431 0 0 0 0 15
RFA_00416 0.63 0.40 1.00 6 1479 1 0 0 1 9
RFA_00433 0.63 0.40 1.00 6 1425 1 0 0 1 9
RFA_00587 1.00 1.00 1.00 15 4836 5 0 0 5 0
RFA_00603 0.63 0.51 0.78 18 13507 8 3 2 3 17
RFA_00604 0.73 0.66 0.82 23 13502 14 4 1 9 12
RFA_00605 0.41 0.31 0.55 11 15205 11 1 8 2 24
RFA_00606 0.44 0.36 0.54 14 21295 18 7 5 6 25
RFA_00607 0.71 0.56 0.91 20 17744 9 0 2 7 16
RFA_00609 0.82 0.68 1.00 25 17930 6 0 0 6 12
RFA_00610 0.81 0.70 0.93 26 17177 9 0 2 7 11
RFA_00611 0.68 0.57 0.80 20 13016 8 3 2 3 15
RFA_00613 0.58 0.46 0.73 16 12858 6 4 2 0 19
RFA_00615 0.63 0.51 0.78 18 13343 5 3 2 0 17
RFA_00620 0.52 0.44 0.63 17 21918 15 2 8 5 22
RFA_00626 0.80 0.64 1.00 56 56560 5 0 0 5 31
RFA_00627 0.77 0.63 0.93 55 56894 8 0 4 4 32
RFA_00628 0.78 0.64 0.95 55 57233 12 0 3 9 31
RFA_00630 0.77 0.66 0.90 57 56890 15 0 6 9 30
RFA_00632 0.78 0.68 0.90 19 4074 2 1 1 0 9
RFA_00636 0.85 0.75 0.95 21 3983 1 1 0 0 7
RFA_00642 0.58 0.39 0.88 7 2918 1 0 1 0 11
RFA_00643 0.21 0.17 0.27 3 2200 8 0 8 0 15
RFA_00644 0.44 0.33 0.60 6 2691 4 1 3 0 12
RFA_00645 -0.01 0.00 0.00 0 2403 12 3 9 0 18
RFA_00649 0.44 0.33 0.60 6 2135 4 1 3 0 12
RFA_00651 0.33 0.22 0.50 4 2072 4 1 3 0 14
RFA_00653 -0.01 0.00 0.00 0 2133 12 3 9 0 18
RFA_00654 0.39 0.28 0.56 5 2406 4 1 3 0 13
RFA_00658 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00659 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00664 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00667 0.53 0.29 1.00 4 986 0 0 0 0 10
RFA_00668 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00672 0.00 0.00 0.00 0 903 0 0 0 0 13
RFA_00673 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00674 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00675 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00677 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00678 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00680 0.00 0.00 0.00 0 1128 0 0 0 0 14
RFA_00684 0.53 0.29 1.00 4 986 0 0 0 0 10
RFA_00685 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00695 0.00 0.00 0.00 0 7021 0 0 0 0 14
RFA_00703 0.46 0.21 1.00 3 4275 0 0 0 0 11
RFA_00704 -0.01 0.00 0.00 0 987 3 0 3 0 14
RFA_00705 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00706 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00707 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00708 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00709 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00710 0.00 0.00 0.00 0 990 0 0 0 0 14
RFA_00711 0.00 0.00 0.00 0 1035 0 0 0 0 14
RFA_00715 0.00 0.00 0.00 0 946 0 0 0 0 14
RFA_00716 -0.01 0.00 0.00 0 943 3 0 3 0 14
RFA_00717 0.00 0.00 0.00 0 903 0 0 0 0 14
RFA_00730 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00731 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00733 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00734 0.64 0.42 1.00 5 898 0 0 0 0 7
RFA_00736 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00737 0.64 0.42 1.00 5 898 1 0 0 1 7
RFA_00745 0.50 0.25 1.00 3 943 0 0 0 0 9
RFA_00749 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00758 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00762 0.64 0.42 1.00 5 898 0 0 0 0 7
RFA_00763 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00764 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00765 0.50 0.25 1.00 3 900 0 0 0 0 9
RFA_00767 0.65 0.56 0.77 10 1878 3 3 0 0 8
RFA_00768 0.65 0.56 0.77 10 1878 3 3 0 0 8
RFA_00769 0.59 0.56 0.63 10 1937 6 1 5 0 8
RFA_00770 0.52 0.39 0.70 7 2006 3 3 0 0 11
RFA_00773 0.59 0.50 0.69 9 1940 4 4 0 0 9
RFA_00779 0.65 0.56 0.77 10 1940 3 3 0 0 8
RFA_00781 0.85 0.72 1.00 23 5027 0 0 0 0 9
RFA_00786 0.79 0.63 1.00 20 5030 0 0 0 0 12
RFA_00791 0.71 0.56 0.90 18 5131 2 0 2 0 14
RFA_00792 0.81 0.72 0.92 23 5025 2 0 2 0 9
RFA_00801 0.74 0.63 0.87 20 5027 4 0 3 1 12
RFA_00808 0.58 0.56 0.60 9 2001 6 0 6 0 7
RFA_00809 0.36 0.38 0.35 6 2128 11 1 10 0 10
RFA_00814 0.87 0.78 0.97 32 25167 6 0 1 5 9
RFA_00815 0.88 0.78 1.00 32 24499 8 0 0 8 9
RFA_00816 0.86 0.76 0.97 31 23188 6 0 1 5 10
RFA_00817 0.86 0.76 0.97 31 21913 1 0 1 0 10
RFA_00818 0.86 0.73 1.00 30 20271 6 0 0 6 11
RFA_00819 0.88 0.78 1.00 32 27934 5 0 0 5 9

^top



Performance of Fold - scored lower in this pairwise comparison

1. Total counts & total scores for Fold

Total Base Pair Counts
Total TP 1348
Total TN 695391
Total FP 1220
Total FP CONTRA 155
Total FP INCONS 687
Total FP COMP 378
Total FN 941
Total Scores
MCC 0.601
Average MCC ± 95% Confidence Intervals 0.569 ± 0.067
Sensitivity 0.589
Positive Predictive Value 0.616
Nr of predictions 98

^top



2. Individual counts for Fold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
PDB_00005 0.88 0.79 1.00 11 935 0 0 0 0 3
PDB_00012 1.00 1.00 1.00 7 399 3 0 0 3 0
PDB_00213 0.87 0.80 0.94 33 5015 2 0 2 0 8
PDB_00553 1.00 1.00 1.00 11 454 0 0 0 0 0
PDB_00716 -0.01 0.00 0.00 0 2678 24 1 22 1 23
PDB_00810 0.91 0.82 1.00 14 1067 0 0 0 0 3
PDB_01050 0.92 0.85 1.00 11 619 2 0 0 2 2
PDB_01051 0.96 0.92 1.00 12 891 4 0 0 4 1
PDB_01092 0.73 0.63 0.85 33 10114 8 0 6 2 19
PDB_01152 0.88 0.79 1.00 11 550 0 0 0 0 3
RFA_00390 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00416 0.93 0.93 0.93 14 1470 4 0 1 3 1
RFA_00433 1.00 1.00 1.00 15 1416 2 0 0 2 0
RFA_00587 1.00 1.00 1.00 15 4836 11 0 0 11 0
RFA_00603 0.69 0.74 0.63 26 13489 18 4 11 3 9
RFA_00604 0.70 0.71 0.69 25 13494 22 2 9 11 10
RFA_00605 0.36 0.34 0.38 12 15193 29 3 17 9 23
RFA_00606 0.68 0.67 0.70 26 21284 27 7 4 16 13
RFA_00607 0.78 0.83 0.73 30 17725 21 5 6 10 6
RFA_00609 0.52 0.57 0.48 21 17911 32 4 19 9 16
RFA_00610 0.87 0.84 0.91 31 17171 17 0 3 14 6
RFA_00611 0.84 0.83 0.85 29 13007 13 0 5 8 6
RFA_00613 0.73 0.69 0.77 24 12849 13 4 3 6 11
RFA_00615 0.66 0.69 0.63 24 13328 18 5 9 4 11
RFA_00620 0.47 0.49 0.45 19 21903 36 7 16 13 20
RFA_00626 0.86 0.82 0.90 71 56537 35 2 6 27 16
RFA_00627 0.68 0.67 0.69 58 56869 42 7 19 16 29
RFA_00628 0.87 0.84 0.90 72 57211 30 0 8 22 14
RFA_00630 0.62 0.63 0.61 55 56863 52 9 26 17 32
RFA_00632 0.39 0.39 0.39 11 4067 17 2 15 0 17
RFA_00636 0.42 0.43 0.43 12 3977 16 2 14 0 16
RFA_00642 0.36 0.33 0.40 6 2911 9 2 7 0 12
RFA_00643 0.22 0.22 0.24 4 2194 14 2 11 1 14
RFA_00644 -0.01 0.00 0.00 0 2681 20 5 15 0 18
RFA_00645 -0.01 0.00 0.00 0 2402 13 2 11 0 18
RFA_00649 0.72 0.72 0.72 13 2127 7 1 4 2 5
RFA_00651 0.28 0.28 0.29 5 2063 13 0 12 1 13
RFA_00653 0.34 0.33 0.35 6 2128 11 2 9 0 12
RFA_00654 -0.01 0.00 0.00 0 2400 15 2 13 0 18
RFA_00658 0.59 0.50 0.70 7 1118 5 0 3 2 7
RFA_00659 -0.01 0.00 0.00 0 1120 10 1 7 2 14
RFA_00664 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00667 0.30 0.29 0.33 4 978 8 1 7 0 10
RFA_00668 0.41 0.43 0.40 6 975 9 0 9 0 8
RFA_00672 -0.01 0.00 0.00 0 897 6 0 6 0 13
RFA_00673 0.42 0.36 0.50 5 1118 6 0 5 1 9
RFA_00674 -0.01 0.00 0.00 0 1116 13 0 12 1 14
RFA_00675 -0.01 0.00 0.00 0 980 11 0 10 1 14
RFA_00677 0.42 0.43 0.43 6 976 9 0 8 1 8
RFA_00678 0.33 0.29 0.40 4 936 6 0 6 0 10
RFA_00680 0.74 0.71 0.77 10 1115 6 0 3 3 4
RFA_00684 0.66 0.64 0.69 9 977 5 0 4 1 5
RFA_00685 0.42 0.36 0.50 5 980 6 0 5 1 9
RFA_00695 0.56 0.50 0.64 7 7010 29 1 3 25 7
RFA_00703 0.67 0.64 0.69 9 4265 18 1 3 14 5
RFA_00704 0.32 0.21 0.50 3 984 4 0 3 1 11
RFA_00705 0.69 0.71 0.67 10 1020 5 0 5 0 4
RFA_00706 0.37 0.29 0.50 4 1027 4 0 4 0 10
RFA_00707 -0.01 0.00 0.00 0 1028 7 1 6 0 14
RFA_00708 0.40 0.29 0.57 4 1028 3 0 3 0 10
RFA_00709 -0.01 0.00 0.00 0 981 10 0 9 1 14
RFA_00710 0.18 0.14 0.25 2 982 7 0 6 1 12
RFA_00711 -0.01 0.00 0.00 0 1023 12 0 12 0 14
RFA_00715 -0.01 0.00 0.00 0 933 13 0 13 0 14
RFA_00716 -0.01 0.00 0.00 0 934 12 0 12 0 14
RFA_00717 0.64 0.64 0.64 9 889 5 0 5 0 5
RFA_00730 0.69 0.67 0.73 8 892 4 1 2 1 4
RFA_00731 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00733 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00734 0.78 0.75 0.82 9 892 3 0 2 1 3
RFA_00736 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00737 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00745 1.00 1.00 1.00 12 934 1 0 0 1 0
RFA_00749 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00758 0.96 0.92 1.00 11 892 0 0 0 0 1
RFA_00762 0.64 0.67 0.62 8 890 5 1 4 0 4
RFA_00763 1.00 1.00 1.00 12 891 0 0 0 0 0
RFA_00764 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00765 1.00 1.00 1.00 12 891 1 0 0 1 0
RFA_00767 0.44 0.44 0.44 8 1873 10 1 9 0 10
RFA_00768 0.45 0.44 0.47 8 1874 10 1 8 1 10
RFA_00769 0.55 0.56 0.56 10 1935 8 3 5 0 8
RFA_00770 0.61 0.56 0.67 10 2001 8 0 5 3 8
RFA_00773 0.71 0.56 0.91 10 1942 3 1 0 2 8
RFA_00779 0.55 0.56 0.56 10 1935 8 3 5 0 8
RFA_00781 0.66 0.69 0.65 22 5016 12 2 10 0 10
RFA_00786 0.55 0.56 0.55 18 5017 15 2 13 0 14
RFA_00791 0.89 0.84 0.93 27 5122 3 0 2 1 5
RFA_00792 0.94 0.94 0.94 30 5018 4 0 2 2 2
RFA_00801 0.81 0.78 0.83 25 5020 6 0 5 1 7
RFA_00808 -0.01 0.00 0.00 0 2000 18 2 14 2 16
RFA_00809 0.37 0.38 0.38 6 2129 10 1 9 0 10
RFA_00814 0.22 0.24 0.19 10 25148 51 11 31 9 31
RFA_00815 0.57 0.61 0.54 25 24485 37 4 17 16 16
RFA_00816 0.61 0.66 0.56 27 23172 33 13 8 12 14
RFA_00817 0.35 0.41 0.30 17 21889 43 11 28 4 24
RFA_00818 0.46 0.54 0.40 22 20246 38 13 20 5 19
RFA_00819 0.83 0.80 0.85 33 27927 41 0 6 35 8

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.