CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CentroidAlifold(20) - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for CentroidAlifold(20) & NanoFolder [.zip] - may take several seconds...


Overview

Metric CentroidAlifold(20) NanoFolder
MCC 0.731 > 0.189
Average MCC ± 95% Confidence Intervals 0.749 ± 0.033 > 0.259 ± 0.038
Sensitivity 0.584 > 0.218
Positive Predictive Value 0.917 > 0.167
Total TP 4315 > 1613
Total TN 3528766 > 3523826
Total FP 579 < 8227
Total FP CONTRA 63 < 1213
Total FP INCONS 327 < 6819
Total FP COMP 189 < 195
Total FN 3077 < 5779
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CentroidAlifold(20) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CentroidAlifold(20) and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CentroidAlifold(20) and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CentroidAlifold(20) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CentroidAlifold(20) and NanoFolder).

^top





Performance of CentroidAlifold(20) - scored higher in this pairwise comparison

1. Total counts & total scores for CentroidAlifold(20)

Total Base Pair Counts
Total TP 4315
Total TN 3528766
Total FP 579
Total FP CONTRA 63
Total FP INCONS 327
Total FP COMP 189
Total FN 3077
Total Scores
MCC 0.731
Average MCC ± 95% Confidence Intervals 0.749 ± 0.033
Sensitivity 0.584
Positive Predictive Value 0.917
Nr of predictions 116

^top



2. Individual counts for CentroidAlifold(20) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00083 0.73 0.55 0.98 60 62420 1 0 1 0 50
ASE_00185 0.63 0.41 0.96 53 73481 2 1 1 0 75
ASE_00190 0.75 0.56 1.00 50 45401 2 0 0 2 40
ASE_00221 0.84 0.73 0.97 85 64532 3 0 3 0 32
ASE_00228 0.74 0.62 0.88 53 46911 8 3 4 1 33
ASE_00229 0.77 0.60 0.98 52 42433 3 0 1 2 35
ASE_00231 0.64 0.48 0.87 46 47842 7 0 7 0 50
ASE_00234 0.58 0.42 0.82 54 75400 12 5 7 0 75
ASE_00238 0.55 0.40 0.75 46 64200 15 3 12 0 68
ASE_00248 0.84 0.72 0.99 82 62398 2 0 1 1 32
ASE_00254 0.83 0.70 0.98 57 36798 1 0 1 0 25
ASE_00255 0.57 0.41 0.79 53 74624 14 0 14 0 77
ASE_00257 0.66 0.51 0.86 49 50983 8 2 6 0 48
ASE_00267 0.82 0.67 1.00 59 45091 1 0 0 1 29
ASE_00270 0.56 0.35 0.90 45 72340 6 0 5 1 83
ASE_00274 0.65 0.47 0.91 48 55558 5 3 2 0 55
ASE_00277 0.62 0.47 0.83 43 48153 9 2 7 0 48
ASE_00279 0.78 0.63 0.96 64 53234 3 0 3 0 37
ASE_00285 0.73 0.56 0.94 68 68934 4 2 2 0 54
ASE_00287 0.67 0.53 0.83 55 54880 11 0 11 0 48
ASE_00294 0.71 0.52 0.96 89 114388 5 0 4 1 82
ASE_00298 0.72 0.54 0.97 61 67465 2 1 1 0 52
ASE_00318 0.79 0.63 0.99 74 80125 8 0 1 7 44
ASE_00321 0.83 0.69 0.98 61 54553 5 0 1 4 27
ASE_00335 0.74 0.59 0.92 69 75391 13 0 6 7 47
ASE_00340 0.75 0.61 0.93 52 46000 8 0 4 4 33
ASE_00367 0.84 0.71 1.00 61 43010 1 0 0 1 25
ASE_00370 0.76 0.60 0.96 50 41853 2 0 2 0 34
ASE_00372 0.75 0.57 0.98 57 51945 1 0 1 0 43
ASE_00376 0.72 0.55 0.94 58 56891 4 0 4 0 47
ASE_00377 0.66 0.50 0.88 53 57570 7 1 6 0 54
ASE_00386 0.74 0.56 0.98 54 50348 1 0 1 0 42
ASE_00393 0.72 0.58 0.90 54 47835 6 0 6 0 39
ASE_00412 0.54 0.40 0.72 42 58253 16 2 14 0 63
ASE_00413 0.77 0.64 0.93 52 44495 5 3 1 1 29
ASE_00416 0.73 0.58 0.93 74 77341 6 1 5 0 53
ASE_00422 0.69 0.56 0.84 53 46908 11 0 10 1 41
ASE_00427 0.50 0.39 0.64 18 40727 13 2 8 3 28
ASE_00428 0.76 0.64 0.90 80 76547 9 1 8 0 45
ASE_00430 0.74 0.57 0.96 55 46914 2 0 2 0 42
ASE_00437 0.56 0.46 0.68 62 79310 29 0 29 0 73
ASE_00451 0.79 0.66 0.95 82 70790 4 0 4 0 43
CRW_01535 0.92 0.92 0.92 34 7103 5 0 3 2 3
CRW_01539 0.85 0.84 0.86 31 7104 7 0 5 2 6
CRW_01563 0.78 0.78 0.78 29 6866 8 0 8 0 8
CRW_01583 0.93 0.92 0.94 34 7104 4 0 2 2 3
CRW_01590 0.89 0.84 0.94 31 7107 4 0 2 2 6
CRW_01593 0.92 0.89 0.94 33 7105 4 0 2 2 4
CRW_01595 0.79 0.78 0.81 29 8479 9 0 7 2 8
CRW_01603 0.86 0.78 0.94 29 7109 3 0 2 1 8
CRW_01617 0.93 0.92 0.94 34 7104 4 0 2 2 3
CRW_01625 0.89 0.86 0.91 32 7105 5 0 3 2 5
PDB_00070 0.93 0.88 1.00 21 2829 0 0 0 0 3
PDB_00571 0.87 0.80 0.95 20 3300 1 1 0 0 5
PDB_00828 0.88 0.78 1.00 21 2464 0 0 0 0 6
PDB_01236 0.90 0.81 1.00 44 11584 4 0 0 4 10
RFA_00416 0.97 0.93 1.00 14 1471 2 0 0 2 1
RFA_00434 0.97 0.93 1.00 14 1417 2 0 0 2 1
RFA_00436 0.97 0.93 1.00 14 1417 2 0 0 2 1
RFA_00442 0.97 0.93 1.00 14 1417 2 0 0 2 1
RFA_00664 0.50 0.36 0.71 5 983 2 0 2 0 9
RFA_00674 0.56 0.43 0.75 6 1120 3 0 2 1 8
RFA_00675 0.35 0.29 0.44 4 981 5 0 5 0 10
RFA_00677 0.47 0.36 0.63 5 982 3 0 3 0 9
RFA_00717 0.54 0.36 0.83 5 897 1 0 1 0 9
RFA_00730 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00763 1.00 1.00 1.00 12 891 2 0 0 2 0
RFA_00765 1.00 1.00 1.00 12 891 2 0 0 2 0
SPR_00066 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00314 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_00333 1.00 1.00 1.00 21 2754 0 0 0 0 0
SPR_00353 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00452 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00456 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00476 1.00 1.00 1.00 21 2754 1 0 0 1 0
SPR_00496 1.00 1.00 1.00 21 2754 0 0 0 0 0
SPR_00497 1.00 1.00 1.00 21 2754 1 0 0 1 0
SPR_00502 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_00509 1.00 1.00 1.00 20 2906 1 0 0 1 0
SPR_00512 1.00 1.00 1.00 20 2906 1 0 0 1 0
SPR_00557 1.00 1.00 1.00 21 2829 0 0 0 0 0
SPR_00868 1.00 1.00 1.00 21 2905 0 0 0 0 0
SPR_00957 1.00 1.00 1.00 21 2982 0 0 0 0 0
SPR_01040 1.00 1.00 1.00 21 2754 0 0 0 0 0
SRP_00038 0.46 0.21 1.00 7 5043 4 0 0 4 26
SRP_00046 0.47 0.22 1.00 7 4746 4 0 0 4 25
SRP_00103 0.86 0.75 0.97 77 45372 6 0 2 4 25
SRP_00170 0.83 0.76 0.90 19 4074 4 1 1 2 6
SRP_00173 0.67 0.48 0.93 43 38180 9 0 3 6 47
SRP_00194 0.53 0.29 1.00 6 3234 4 0 0 4 15
SRP_00200 0.41 0.17 1.00 6 6897 1 0 0 1 30
SRP_00233 0.46 0.21 1.00 7 5144 4 0 0 4 26
SRP_00234 0.55 0.34 0.91 29 36283 9 0 3 6 57
SRP_00243 0.47 0.23 1.00 7 5144 4 0 0 4 24
SRP_00244 0.39 0.15 1.00 6 6897 5 0 0 5 33
SRP_00266 0.49 0.24 1.00 7 4649 4 0 0 4 22
SRP_00269 0.46 0.21 1.00 7 5043 4 0 0 4 26
SRP_00273 0.79 0.70 0.90 28 6755 7 1 2 4 12
SRP_00303 0.56 0.31 1.00 10 6660 5 0 0 5 22
SRP_00308 0.64 0.45 0.91 40 36271 9 0 4 5 48
SRP_00317 0.87 0.79 0.97 77 45071 9 0 2 7 21
SRP_00333 0.45 0.21 1.00 7 5144 4 0 0 4 27
SRP_00337 0.75 0.62 0.90 56 35449 12 0 6 6 35
SRP_00338 0.44 0.19 1.00 7 5453 4 0 0 4 29
SRP_00342 0.49 0.24 1.00 6 3999 1 0 0 1 19
SRP_00358 0.45 0.20 1.00 7 6321 4 0 0 4 28
SRP_00383 0.54 0.29 1.00 7 3233 0 0 0 0 17
TMR_00123 0.79 0.64 0.97 65 66363 3 1 1 1 36
TMR_00399 0.60 0.49 0.74 46 64918 16 6 10 0 48
TMR_00443 0.73 0.58 0.92 60 67096 6 0 5 1 44
TMR_00469 0.79 0.69 0.91 69 64544 10 6 1 3 31
TMR_00519 0.63 0.53 0.75 50 63123 19 5 12 2 45
TMR_00528 0.67 0.54 0.84 52 63128 15 4 6 5 45
TMR_00571 0.80 0.71 0.91 70 60649 12 1 6 5 29
TMR_00584 0.80 0.70 0.91 68 61000 10 3 4 3 29
TMR_00703 0.71 0.57 0.90 56 67466 6 2 4 0 43

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 1613
Total TN 3523826
Total FP 8227
Total FP CONTRA 1213
Total FP INCONS 6819
Total FP COMP 195
Total FN 5779
Total Scores
MCC 0.189
Average MCC ± 95% Confidence Intervals 0.259 ± 0.038
Sensitivity 0.218
Positive Predictive Value 0.167
Nr of predictions 116

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00083 0.22 0.25 0.20 27 62347 108 12 95 1 83
ASE_00185 0.17 0.19 0.16 24 73382 134 14 116 4 104
ASE_00190 0.24 0.28 0.21 25 45334 93 14 78 1 65
ASE_00221 0.20 0.21 0.18 25 64484 112 10 101 1 92
ASE_00228 0.05 0.06 0.04 5 46858 109 17 91 1 81
ASE_00229 0.12 0.14 0.11 12 42380 97 12 82 3 75
ASE_00231 0.37 0.42 0.33 40 47773 84 17 65 2 56
ASE_00234 0.25 0.27 0.23 35 75315 117 10 106 1 94
ASE_00238 0.26 0.30 0.23 34 64111 119 18 98 3 80
ASE_00248 0.13 0.15 0.12 17 62340 124 9 115 0 97
ASE_00254 0.15 0.16 0.14 13 36761 88 3 79 6 69
ASE_00255 0.11 0.12 0.10 16 74530 146 16 129 1 114
ASE_00257 0.37 0.42 0.33 41 50917 85 15 67 3 56
ASE_00267 0.00 0.00 0.00 0 45036 115 9 105 1 88
ASE_00270 0.19 0.21 0.17 27 72232 132 14 117 1 101
ASE_00274 0.00 0.00 0.00 0 55487 126 12 112 2 103
ASE_00277 0.10 0.12 0.08 11 48071 123 23 100 0 80
ASE_00279 0.14 0.16 0.13 16 53177 109 16 92 1 85
ASE_00285 0.34 0.36 0.31 44 68866 98 14 82 2 78
ASE_00287 0.15 0.17 0.14 17 54824 107 14 91 2 86
ASE_00294 0.12 0.13 0.12 23 114287 172 11 160 1 148
ASE_00298 0.09 0.10 0.08 11 67389 129 8 120 1 102
ASE_00318 0.03 0.04 0.03 5 80039 156 21 135 0 113
ASE_00321 0.11 0.14 0.09 12 54484 122 22 97 3 76
ASE_00335 0.10 0.12 0.09 14 75304 151 22 126 3 102
ASE_00340 0.06 0.07 0.05 6 45943 110 13 94 3 79
ASE_00367 0.08 0.09 0.07 8 42960 110 10 93 7 78
ASE_00370 0.03 0.04 0.03 3 41796 109 18 88 3 81
ASE_00372 0.17 0.20 0.15 20 51871 113 19 93 1 80
ASE_00376 0.00 0.00 0.00 0 56822 136 9 122 5 105
ASE_00377 0.25 0.27 0.23 29 57505 98 10 86 2 78
ASE_00386 0.14 0.16 0.13 15 50284 106 9 95 2 81
ASE_00393 0.31 0.35 0.27 33 47773 90 18 71 1 60
ASE_00412 0.15 0.17 0.14 18 58183 111 14 96 1 87
ASE_00413 0.08 0.10 0.07 8 44438 108 19 86 3 73
ASE_00416 0.39 0.43 0.35 54 77267 103 18 82 3 73
ASE_00422 0.28 0.31 0.25 29 46856 87 9 77 1 65
ASE_00427 0.06 0.09 0.04 4 40651 105 35 65 5 42
ASE_00428 0.24 0.26 0.22 33 76483 121 13 107 1 92
ASE_00430 0.16 0.18 0.14 17 46852 102 10 92 0 80
ASE_00437 0.00 0.00 0.00 0 79238 164 12 151 1 135
ASE_00451 0.20 0.22 0.19 27 70731 118 8 110 0 98
CRW_01535 0.20 0.24 0.17 9 7088 43 7 36 0 28
CRW_01539 0.10 0.11 0.10 4 7099 40 1 36 3 33
CRW_01563 0.11 0.14 0.11 5 6856 43 5 37 1 32
CRW_01583 0.36 0.41 0.33 15 7095 31 8 22 1 22
CRW_01590 -0.01 0.00 0.00 0 7093 50 7 40 3 37
CRW_01593 0.32 0.38 0.29 14 7091 36 5 30 1 23
CRW_01595 -0.01 0.00 0.00 0 8459 57 10 46 1 37
CRW_01603 0.41 0.46 0.38 17 7095 29 6 22 1 20
CRW_01617 0.20 0.24 0.17 9 7087 45 7 37 1 28
CRW_01625 0.43 0.49 0.38 18 7093 31 7 22 2 19
PDB_00070 0.25 0.29 0.23 7 2819 24 4 20 0 17
PDB_00571 -0.01 0.00 0.00 0 3286 36 6 29 1 25
PDB_00828 0.30 0.33 0.28 9 2453 23 0 23 0 18
PDB_01236 0.31 0.33 0.30 18 11568 43 3 39 1 36
RFA_00416 0.94 1.00 0.88 15 1468 6 2 0 4 0
RFA_00434 0.84 1.00 0.71 15 1410 8 6 0 2 0
RFA_00436 0.86 1.00 0.75 15 1411 7 5 0 2 0
RFA_00442 0.94 1.00 0.88 15 1414 4 2 0 2 0
RFA_00664 -0.01 0.00 0.00 0 978 13 0 12 1 14
RFA_00674 0.12 0.14 0.13 2 1112 15 0 14 1 12
RFA_00675 0.28 0.29 0.29 4 976 10 1 9 0 10
RFA_00677 0.62 0.64 0.60 9 975 10 0 6 4 5
RFA_00717 0.64 0.64 0.64 9 889 5 2 3 0 5
RFA_00730 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00763 0.38 0.42 0.36 5 889 10 3 6 1 7
RFA_00765 0.39 0.42 0.38 5 890 9 2 6 1 7
SPR_00066 0.47 0.57 0.39 12 2819 20 4 15 1 9
SPR_00314 0.47 0.57 0.39 12 2895 21 9 10 2 9
SPR_00333 0.25 0.33 0.21 7 2741 27 10 17 0 14
SPR_00353 0.64 0.81 0.52 17 2817 16 8 8 0 4
SPR_00452 0.27 0.33 0.23 7 2820 24 7 16 1 14
SPR_00456 0.49 0.57 0.43 12 2822 19 5 11 3 9
SPR_00476 0.47 0.52 0.42 11 2749 17 4 11 2 10
SPR_00496 0.47 0.57 0.40 12 2745 22 7 11 4 9
SPR_00497 0.27 0.33 0.23 7 2745 23 5 18 0 14
SPR_00502 0.27 0.33 0.23 7 2896 26 2 21 3 14
SPR_00509 0.23 0.30 0.19 6 2895 25 8 17 0 14
SPR_00512 0.61 0.75 0.50 15 2896 15 6 9 0 5
SPR_00557 0.42 0.52 0.34 11 2818 22 8 13 1 10
SPR_00868 0.49 0.57 0.43 12 2898 19 5 11 3 9
SPR_00957 0.63 0.76 0.53 16 2973 15 7 7 1 5
SPR_01040 0.15 0.19 0.13 4 2744 27 4 23 0 17
SRP_00038 0.23 0.27 0.20 9 5005 36 1 35 0 24
SRP_00046 0.33 0.38 0.29 12 4712 30 3 26 1 20
SRP_00103 0.00 0.00 0.00 0 45319 132 14 118 0 102
SRP_00170 0.29 0.36 0.25 9 4059 28 4 23 1 16
SRP_00173 0.00 0.00 0.00 0 38104 124 13 109 2 90
SRP_00194 0.27 0.33 0.23 7 3209 27 5 19 3 14
SRP_00200 0.41 0.47 0.35 17 6855 31 4 27 0 19
SRP_00233 0.25 0.30 0.22 10 5106 35 4 31 0 23
SRP_00234 0.21 0.24 0.18 21 36200 97 19 75 3 65
SRP_00243 0.47 0.55 0.40 17 5109 25 7 18 0 14
SRP_00244 0.38 0.44 0.34 17 6853 33 2 31 0 22
SRP_00266 0.55 0.66 0.46 19 4615 22 8 14 0 10
SRP_00269 0.46 0.52 0.43 17 5010 24 6 17 1 16
SRP_00273 0.36 0.40 0.33 16 6737 34 6 27 1 24
SRP_00303 0.36 0.41 0.33 13 6630 27 3 24 0 19
SRP_00308 0.14 0.17 0.12 15 36189 112 18 93 1 73
SRP_00317 0.14 0.16 0.12 16 45018 116 16 100 0 82
SRP_00333 0.20 0.24 0.19 8 5108 35 3 32 0 26
SRP_00337 0.42 0.46 0.38 42 35400 70 15 54 1 49
SRP_00338 0.52 0.56 0.49 20 5419 22 4 17 1 16
SRP_00342 0.20 0.24 0.17 6 3970 29 7 22 0 19
SRP_00358 0.32 0.37 0.28 13 6282 33 4 29 0 22
SRP_00383 -0.01 0.00 0.00 0 3205 35 9 26 0 24
TMR_00123 0.19 0.23 0.16 23 66283 126 20 104 2 78
TMR_00399 0.20 0.26 0.16 24 64833 130 38 85 7 70
TMR_00443 0.17 0.20 0.15 21 67018 126 24 98 4 83
TMR_00469 0.10 0.12 0.08 12 64469 145 27 112 6 88
TMR_00519 0.04 0.05 0.03 5 63046 141 33 106 2 90
TMR_00528 0.13 0.15 0.11 15 63051 131 20 104 7 82
TMR_00571 0.05 0.06 0.04 6 60577 149 19 124 6 93
TMR_00584 0.01 0.01 0.01 1 60922 159 33 119 7 96
TMR_00703 0.10 0.12 0.08 12 67376 140 26 114 0 87

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.