CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PETfold_pre2.0(seed) - scored higher in this pairwise comparison

  4. Performance of NanoFolder - scored lower in this pairwise comparison

  5. Compile and download dataset for PETfold_pre2.0(seed) & NanoFolder [.zip] - may take several seconds...


Overview

Metric PETfold_pre2.0(seed) NanoFolder
MCC 0.692 > 0.178
Average MCC ± 95% Confidence Intervals 0.691 ± 0.027 > 0.188 ± 0.030
Sensitivity 0.598 > 0.210
Positive Predictive Value 0.801 > 0.153
Total TP 5755 > 2022
Total TN 5922984 > 5916978
Total FP 2073 < 11571
Total FP CONTRA 428 < 2014
Total FP INCONS 1004 < 9157
Total FP COMP 641 > 400
Total FN 3863 < 7596
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of PETfold_pre2.0(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and NanoFolder).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and NanoFolder).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and NanoFolder).

^top





Performance of PETfold_pre2.0(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for PETfold_pre2.0(seed)

Total Base Pair Counts
Total TP 5755
Total TN 5922984
Total FP 2073
Total FP CONTRA 428
Total FP INCONS 1004
Total FP COMP 641
Total FN 3863
Total Scores
MCC 0.692
Average MCC ± 95% Confidence Intervals 0.691 ± 0.027
Sensitivity 0.598
Positive Predictive Value 0.801
Nr of predictions 97

^top



2. Individual counts for PETfold_pre2.0(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00017 0.64 0.61 0.67 39 50982 24 3 16 5 25
ASE_00083 0.76 0.65 0.88 72 62399 16 4 6 6 38
ASE_00122 0.85 0.82 0.89 72 43284 14 3 6 5 16
ASE_00185 0.69 0.54 0.87 69 73457 15 5 5 5 59
ASE_00190 0.76 0.69 0.85 62 45378 20 2 9 9 28
ASE_00196 0.74 0.69 0.80 52 31561 18 2 11 5 23
ASE_00221 0.85 0.81 0.90 95 64514 16 4 7 5 22
ASE_00227 0.59 0.50 0.69 67 78906 33 3 27 3 66
ASE_00228 0.83 0.76 0.90 65 46899 13 3 4 6 21
ASE_00229 0.87 0.80 0.93 70 42411 10 2 3 5 17
ASE_00231 0.72 0.63 0.83 60 47823 17 2 10 5 36
ASE_00234 0.66 0.53 0.82 68 75383 21 9 6 6 61
ASE_00238 0.66 0.56 0.78 64 64179 24 7 11 6 50
ASE_00248 0.88 0.85 0.92 97 62375 15 4 5 6 17
ASE_00250 0.78 0.70 0.86 92 78896 20 5 10 5 39
ASE_00254 0.89 0.83 0.96 68 36785 9 2 1 6 14
ASE_00255 0.66 0.59 0.75 77 74588 31 5 21 5 53
ASE_00257 0.76 0.69 0.83 67 50959 19 5 9 5 30
ASE_00267 0.85 0.82 0.89 72 45069 14 3 6 5 16
ASE_00270 0.59 0.45 0.77 57 72316 22 3 14 5 71
ASE_00274 0.71 0.63 0.80 65 55530 21 6 10 5 38
ASE_00277 0.75 0.68 0.82 62 48129 19 8 6 5 29
ASE_00279 0.77 0.69 0.86 70 53220 16 4 7 5 31
ASE_00285 0.72 0.59 0.88 72 68924 15 4 6 5 50
ASE_00287 0.70 0.62 0.78 64 54864 23 3 15 5 39
ASE_00294 0.71 0.57 0.90 97 114373 16 4 7 5 74
ASE_00295 0.78 0.69 0.87 91 73815 19 4 10 5 40
ASE_00298 0.75 0.64 0.88 72 67446 15 6 4 5 41
ASE_00318 0.81 0.74 0.90 87 80103 25 2 8 15 31
ASE_00335 0.85 0.79 0.90 92 75364 26 2 8 16 24
ASE_00339 0.52 0.47 0.57 56 80101 45 15 28 2 63
ASE_00340 0.80 0.73 0.89 62 45986 15 2 6 7 23
ASE_00343 0.31 0.27 0.36 31 83349 63 6 50 7 82
ASE_00367 0.87 0.85 0.89 73 42989 15 4 5 6 13
ASE_00370 0.85 0.83 0.88 70 41825 16 3 7 6 14
ASE_00372 0.79 0.71 0.88 71 51922 16 4 6 6 29
ASE_00375 0.77 0.66 0.89 73 60296 15 3 6 6 37
ASE_00376 0.76 0.68 0.87 71 56871 17 4 7 6 34
ASE_00377 0.74 0.65 0.84 70 57547 18 5 8 5 37
ASE_00386 0.82 0.76 0.89 73 50321 14 4 5 5 23
ASE_00393 0.77 0.72 0.82 67 47813 21 4 11 6 26
ASE_00412 0.64 0.57 0.72 60 58228 28 7 16 5 45
ASE_00413 0.80 0.79 0.81 64 44472 22 7 8 7 17
ASE_00416 0.78 0.72 0.84 91 77313 22 6 11 5 36
ASE_00417 0.77 0.68 0.87 68 54537 14 3 7 4 32
ASE_00422 0.79 0.71 0.88 67 46895 13 1 8 4 27
ASE_00428 0.78 0.73 0.84 91 76528 22 6 11 5 34
ASE_00430 0.81 0.71 0.92 69 46896 11 2 4 5 28
ASE_00434 0.74 0.71 0.78 78 68165 36 2 20 14 32
ASE_00437 0.60 0.53 0.68 72 79295 39 4 30 5 63
ASE_00451 0.84 0.78 0.91 98 70768 15 4 6 5 27
CRW_00671 0.52 0.32 0.84 37 62791 8 0 7 1 80
CRW_00672 0.52 0.32 0.84 36 72347 10 0 7 3 75
CRW_00674 0.25 0.15 0.45 18 83396 25 6 16 3 106
CRW_00676 0.29 0.16 0.53 18 93494 24 0 16 8 97
PDB_00571 0.85 0.80 0.91 20 3299 2 2 0 0 5
PDB_00828 0.84 0.78 0.91 21 2462 3 2 0 1 6
PDB_01009 0.93 0.90 0.95 19 2465 3 0 1 2 2
RFA_00770 0.63 0.56 0.71 10 2002 4 3 1 0 8
RFA_00773 0.63 0.56 0.71 10 1939 4 3 1 0 8
RFA_00808 0.71 0.56 0.90 9 2006 1 1 0 0 7
TMR_00036 0.60 0.50 0.74 50 90457 25 7 11 7 51
TMR_00123 0.72 0.61 0.84 62 66356 21 7 5 9 39
TMR_00180 0.73 0.65 0.83 64 67084 20 6 7 7 35
TMR_00240 0.53 0.49 0.57 42 63116 42 9 23 10 43
TMR_00352 0.48 0.40 0.59 30 95215 46 6 15 25 45
TMR_00353 0.42 0.35 0.51 37 84182 42 6 30 6 68
TMR_00357 0.52 0.45 0.60 41 82960 33 9 18 6 51
TMR_00374 0.60 0.52 0.69 48 81740 32 5 17 10 44
TMR_00384 0.70 0.57 0.85 62 65630 17 4 7 6 46
TMR_00395 0.72 0.60 0.88 65 64546 15 4 5 6 44
TMR_00398 0.71 0.59 0.85 63 65267 17 6 5 6 43
TMR_00399 0.66 0.59 0.74 55 64906 26 9 10 7 39
TMR_00430 0.58 0.47 0.71 51 82549 32 7 14 11 58
TMR_00431 0.35 0.31 0.40 17 51961 47 2 23 22 38
TMR_00443 0.72 0.62 0.84 64 67085 19 6 6 7 40
TMR_00457 0.58 0.48 0.70 49 78536 30 6 15 9 53
TMR_00469 0.71 0.61 0.84 61 64547 20 6 6 8 39
TMR_00502 0.50 0.44 0.57 41 75394 40 11 20 9 52
TMR_00503 0.73 0.60 0.88 67 67820 16 3 6 7 44
TMR_00519 0.64 0.57 0.72 54 63115 29 9 12 8 41
TMR_00528 0.64 0.56 0.74 54 63117 28 6 13 9 43
TMR_00541 0.70 0.59 0.83 62 61701 21 8 5 8 43
TMR_00547 0.70 0.60 0.81 60 65267 22 4 10 8 40
TMR_00562 0.72 0.58 0.91 59 72706 19 1 5 13 43
TMR_00566 0.71 0.61 0.84 61 60653 21 2 10 9 39
TMR_00571 0.78 0.67 0.92 66 60654 16 1 5 10 33
TMR_00584 0.77 0.65 0.91 63 61006 14 2 4 8 34
TMR_00594 0.53 0.43 0.65 45 79332 33 3 21 9 59
TMR_00605 0.57 0.43 0.77 46 74245 24 2 12 10 62
TMR_00609 0.61 0.50 0.75 57 63470 26 1 18 7 56
TMR_00624 0.73 0.60 0.88 65 62407 17 4 5 8 43
TMR_00683 0.59 0.50 0.70 50 82957 30 4 17 9 50
TMR_00698 0.57 0.48 0.68 50 63117 28 7 16 5 54
TMR_00703 0.68 0.60 0.79 59 67453 23 7 9 7 40
TMR_00706 0.61 0.50 0.76 51 66363 26 5 11 10 52
TMR_00711 0.63 0.55 0.71 54 67820 29 11 11 7 44

^top



Performance of NanoFolder - scored lower in this pairwise comparison

1. Total counts & total scores for NanoFolder

Total Base Pair Counts
Total TP 2022
Total TN 5916978
Total FP 11571
Total FP CONTRA 2014
Total FP INCONS 9157
Total FP COMP 400
Total FN 7596
Total Scores
MCC 0.178
Average MCC ± 95% Confidence Intervals 0.188 ± 0.030
Sensitivity 0.210
Positive Predictive Value 0.153
Nr of predictions 97

^top



2. Individual counts for NanoFolder [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
ASE_00017 0.02 0.03 0.02 2 50914 131 38 86 7 62
ASE_00083 0.22 0.25 0.20 27 62347 108 12 95 1 83
ASE_00122 0.27 0.30 0.24 26 43258 84 13 68 3 62
ASE_00185 0.17 0.19 0.16 24 73382 134 14 116 4 104
ASE_00190 0.24 0.28 0.21 25 45334 93 14 78 1 65
ASE_00196 0.49 0.57 0.42 43 31524 62 16 43 3 32
ASE_00221 0.20 0.21 0.18 25 64484 112 10 101 1 92
ASE_00227 0.28 0.31 0.25 41 78840 122 20 102 0 92
ASE_00228 0.05 0.06 0.04 5 46858 109 17 91 1 81
ASE_00229 0.12 0.14 0.11 12 42380 97 12 82 3 75
ASE_00231 0.37 0.42 0.33 40 47773 84 17 65 2 56
ASE_00234 0.25 0.27 0.23 35 75315 117 10 106 1 94
ASE_00238 0.26 0.30 0.23 34 64111 119 18 98 3 80
ASE_00248 0.13 0.15 0.12 17 62340 124 9 115 0 97
ASE_00250 0.25 0.29 0.23 38 78835 135 17 113 5 93
ASE_00254 0.15 0.16 0.14 13 36761 88 3 79 6 69
ASE_00255 0.11 0.12 0.10 16 74530 146 16 129 1 114
ASE_00257 0.37 0.42 0.33 41 50917 85 15 67 3 56
ASE_00267 0.00 0.00 0.00 0 45036 115 9 105 1 88
ASE_00270 0.19 0.21 0.17 27 72232 132 14 117 1 101
ASE_00274 0.00 0.00 0.00 0 55487 126 12 112 2 103
ASE_00277 0.10 0.12 0.08 11 48071 123 23 100 0 80
ASE_00279 0.14 0.16 0.13 16 53177 109 16 92 1 85
ASE_00285 0.34 0.36 0.31 44 68866 98 14 82 2 78
ASE_00287 0.15 0.17 0.14 17 54824 107 14 91 2 86
ASE_00294 0.12 0.13 0.12 23 114287 172 11 160 1 148
ASE_00295 0.14 0.16 0.13 21 73761 140 19 119 2 110
ASE_00298 0.09 0.10 0.08 11 67389 129 8 120 1 102
ASE_00318 0.03 0.04 0.03 5 80039 156 21 135 0 113
ASE_00335 0.10 0.12 0.09 14 75304 151 22 126 3 102
ASE_00339 0.25 0.29 0.22 34 80042 129 20 104 5 85
ASE_00340 0.06 0.07 0.05 6 45943 110 13 94 3 79
ASE_00343 0.14 0.17 0.12 19 83275 145 31 111 3 94
ASE_00367 0.08 0.09 0.07 8 42960 110 10 93 7 78
ASE_00370 0.03 0.04 0.03 3 41796 109 18 88 3 81
ASE_00372 0.17 0.20 0.15 20 51871 113 19 93 1 80
ASE_00375 0.12 0.14 0.11 15 60246 120 10 107 3 95
ASE_00376 0.00 0.00 0.00 0 56822 136 9 122 5 105
ASE_00377 0.25 0.27 0.23 29 57505 98 10 86 2 78
ASE_00386 0.14 0.16 0.13 15 50284 106 9 95 2 81
ASE_00393 0.31 0.35 0.27 33 47773 90 18 71 1 60
ASE_00412 0.15 0.17 0.14 18 58183 111 14 96 1 87
ASE_00413 0.08 0.10 0.07 8 44438 108 19 86 3 73
ASE_00416 0.39 0.43 0.35 54 77267 103 18 82 3 73
ASE_00417 0.32 0.36 0.28 36 54486 98 18 75 5 64
ASE_00422 0.28 0.31 0.25 29 46856 87 9 77 1 65
ASE_00428 0.24 0.26 0.22 33 76483 121 13 107 1 92
ASE_00430 0.16 0.18 0.14 17 46852 102 10 92 0 80
ASE_00434 0.29 0.34 0.25 37 68116 122 20 92 10 73
ASE_00437 0.00 0.00 0.00 0 79238 164 12 151 1 135
ASE_00451 0.20 0.22 0.19 27 70731 118 8 110 0 98
CRW_00671 0.18 0.21 0.16 24 62682 130 23 106 1 93
CRW_00672 0.36 0.43 0.31 48 72233 114 35 74 5 63
CRW_00674 0.16 0.19 0.14 23 83274 142 19 120 3 101
CRW_00676 0.11 0.14 0.09 16 93347 169 29 136 4 99
PDB_00571 -0.01 0.00 0.00 0 3286 36 6 29 1 25
PDB_00828 0.30 0.33 0.28 9 2453 23 0 23 0 18
PDB_01009 0.27 0.33 0.23 7 2455 25 3 20 2 14
RFA_00770 -0.01 0.00 0.00 0 1996 24 0 20 4 18
RFA_00773 0.93 1.00 0.86 18 1932 9 3 0 6 0
RFA_00808 0.83 1.00 0.70 16 1993 7 7 0 0 0
TMR_00036 0.22 0.29 0.17 29 90359 153 37 100 16 72
TMR_00123 0.19 0.23 0.16 23 66283 126 20 104 2 78
TMR_00180 0.12 0.16 0.10 16 66999 149 35 111 3 83
TMR_00240 0.23 0.31 0.17 26 63041 128 39 84 5 59
TMR_00352 0.07 0.09 0.05 7 95133 171 39 87 45 68
TMR_00353 0.41 0.51 0.32 54 84087 127 49 65 13 51
TMR_00357 0.26 0.34 0.19 31 82869 146 44 84 18 61
TMR_00374 0.09 0.13 0.07 12 81631 174 44 123 7 80
TMR_00384 0.17 0.20 0.14 22 65550 133 26 105 2 86
TMR_00395 0.27 0.31 0.24 34 64478 115 10 98 7 75
TMR_00398 0.05 0.07 0.05 7 65197 139 24 113 2 99
TMR_00399 0.20 0.26 0.16 24 64833 130 38 85 7 70
TMR_00430 0.18 0.23 0.15 25 82450 148 34 112 2 84
TMR_00431 0.24 0.33 0.17 18 51899 127 26 60 41 37
TMR_00443 0.17 0.20 0.15 21 67018 126 24 98 4 83
TMR_00457 0.19 0.25 0.15 25 78443 141 39 99 3 77
TMR_00469 0.10 0.12 0.08 12 64469 145 27 112 6 88
TMR_00502 0.14 0.19 0.11 18 75303 150 45 100 5 75
TMR_00503 0.11 0.14 0.10 15 67741 141 20 120 1 96
TMR_00519 0.04 0.05 0.03 5 63046 141 33 106 2 90
TMR_00528 0.13 0.15 0.11 15 63051 131 20 104 7 82
TMR_00541 0.12 0.14 0.10 15 61630 139 22 109 8 90
TMR_00547 0.28 0.35 0.23 35 65188 122 37 81 4 65
TMR_00562 0.09 0.12 0.08 12 72617 147 27 115 5 90
TMR_00566 0.13 0.16 0.11 16 60582 132 22 106 4 84
TMR_00571 0.05 0.06 0.04 6 60577 149 19 124 6 93
TMR_00584 0.01 0.01 0.01 1 60922 159 33 119 7 96
TMR_00594 0.08 0.10 0.06 10 79238 156 30 123 3 94
TMR_00605 0.28 0.35 0.23 38 74141 129 36 90 3 70
TMR_00609 0.53 0.62 0.45 70 63391 89 28 57 4 43
TMR_00624 0.36 0.44 0.30 47 62325 111 27 82 2 61
TMR_00683 0.01 0.02 0.01 2 82854 176 50 122 4 98
TMR_00698 0.20 0.24 0.17 25 63042 124 27 96 1 79
TMR_00703 0.10 0.12 0.08 12 67376 140 26 114 0 87
TMR_00706 0.14 0.17 0.11 17 66281 133 29 103 1 86
TMR_00711 0.18 0.22 0.14 22 67740 137 41 93 3 76

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.