CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of CRWrnafold - scored higher in this pairwise comparison

  4. Performance of Afold - scored lower in this pairwise comparison

  5. Compile and download dataset for CRWrnafold & Afold [.zip] - may take several seconds...


Overview

Metric CRWrnafold Afold
MCC 0.369 > 0.355
Average MCC ± 95% Confidence Intervals 0.361 ± 0.030 < 0.363 ± 0.027
Sensitivity 0.368 > 0.349
Positive Predictive Value 0.372 > 0.361
Total TP 12819 > 12172
Total TN 80099699 < 80100432
Total FP 25908 > 25863
Total FP CONTRA 3379 > 3194
Total FP INCONS 18290 < 18389
Total FP COMP 4239 < 4280
Total FN 22035 < 22682
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of CRWrnafold and Afold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CRWrnafold and Afold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CRWrnafold and Afold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for CRWrnafold and Afold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CRWrnafold and Afold).

^top





Performance of CRWrnafold - scored higher in this pairwise comparison

1. Total counts & total scores for CRWrnafold

Total Base Pair Counts
Total TP 12819
Total TN 80099699
Total FP 25908
Total FP CONTRA 3379
Total FP INCONS 18290
Total FP COMP 4239
Total FN 22035
Total Scores
MCC 0.369
Average MCC ± 95% Confidence Intervals 0.361 ± 0.030
Sensitivity 0.368
Positive Predictive Value 0.372
Nr of predictions 109

^top



2. Individual counts for CRWrnafold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00002 - 0.32 0.31 0.33 82 408808 203 23 147 33 182
CRW_00054 0.57 0.56 0.57 257 1107367 210 22 170 18 199
CRW_00068 0.57 0.57 0.58 259 1108856 209 20 170 19 198
CRW_00140 0.49 0.49 0.48 226 1135811 256 30 211 15 237
CRW_00173 0.38 0.38 0.39 168 1082220 282 31 237 14 277
CRW_00245 0.53 0.53 0.54 240 1105883 225 26 179 20 213
CRW_00270 0.49 0.49 0.50 225 1111832 246 27 202 17 230
CRW_00271 0.69 0.67 0.71 311 1122312 152 21 107 24 151
CRW_00273 0.41 0.41 0.41 181 1085155 279 39 226 14 265
CRW_00275 0.65 0.65 0.66 296 1114820 177 28 127 22 161
CRW_00277 0.44 0.43 0.44 196 1110351 263 35 213 15 257
CRW_00278 0.41 0.40 0.42 182 1129824 270 20 230 20 268
CRW_00280 0.67 0.66 0.68 299 1117818 160 18 125 17 154
CRW_00281 0.60 0.59 0.61 274 1144893 197 27 147 23 190
CRW_00283 0.38 0.38 0.37 164 1015587 296 41 233 22 268
CRW_00284 0.51 0.50 0.51 228 1131314 245 27 191 27 228
CRW_00285 0.68 0.67 0.69 305 1117819 160 25 111 24 153
CRW_00292 0.33 0.33 0.34 151 1123806 312 48 245 19 310
CRW_00338 - 0.11 0.12 0.10 47 1052969 417 74 336 7 348
CRW_00417 - 0.47 0.46 0.49 121 458154 138 19 109 10 141
CRW_00418 - 0.11 0.11 0.12 28 441094 219 29 179 11 232
CRW_00419 - 0.11 0.11 0.11 28 463922 224 26 190 8 237
CRW_00421 - 0.44 0.44 0.45 116 462946 156 22 119 15 150
CRW_00422 - 0.26 0.25 0.27 66 455293 196 40 136 20 197
CRW_00426 - 0.22 0.22 0.22 58 457187 213 31 170 12 207
CRW_00427 - 0.30 0.30 0.30 78 454321 200 36 146 18 183
CRW_00428 - 0.22 0.22 0.22 56 465839 213 30 170 13 204
CRW_00430 - 0.28 0.27 0.29 73 456241 183 22 154 7 193
CRW_00433 - 0.24 0.24 0.26 62 467785 187 23 158 6 201
CRW_00434 - 0.28 0.27 0.30 71 457208 176 24 143 9 192
CRW_00435 - 0.34 0.33 0.35 87 451479 166 32 127 7 178
CRW_00437 - 0.37 0.37 0.38 95 456241 166 25 129 12 165
CRW_00438 - 0.37 0.35 0.39 94 454341 154 16 130 8 174
CRW_00441 - 0.34 0.32 0.35 85 454341 166 23 132 11 177
CRW_00442 - 0.36 0.36 0.35 95 453360 184 38 135 11 166
CRW_00443 - 0.31 0.30 0.32 79 456243 194 27 141 26 181
CRW_00444 - 0.30 0.30 0.31 78 459108 186 25 150 11 185
CRW_00445 - 0.28 0.27 0.28 72 458149 187 34 148 5 193
CRW_00446 - 0.25 0.25 0.26 66 468741 200 28 161 11 199
CRW_00447 - 0.28 0.28 0.28 72 452417 197 30 157 10 188
CRW_00448 - 0.21 0.21 0.21 55 457190 217 34 167 16 206
CRW_00449 - 0.21 0.20 0.22 51 444861 190 28 156 6 209
CRW_00451 - 0.22 0.22 0.23 57 457196 202 24 169 9 207
CRW_00452 - 0.32 0.31 0.33 81 453380 179 30 137 12 181
CRW_00453 - 0.14 0.13 0.15 35 452439 210 30 172 8 226
CRW_00458 - 0.43 0.43 0.44 113 464872 153 31 114 8 151
CRW_00459 - 0.35 0.34 0.36 89 461993 180 16 143 21 174
CRW_00462 - 0.24 0.23 0.25 61 458160 190 33 149 8 200
CRW_00463 - 0.52 0.50 0.54 126 445807 130 25 82 23 128
CRW_00464 - 0.30 0.29 0.31 77 461995 176 26 143 7 187
CRW_00466 0.71 0.70 0.71 320 1116316 158 22 107 29 138
CRW_00530 - 0.23 0.25 0.21 67 534774 263 62 192 9 201
CRW_00535 - 0.33 0.35 0.30 116 898086 295 80 188 27 212
CRW_00538 - 0.23 0.25 0.21 76 811810 307 80 209 18 224
CRW_00589 0.35 0.34 0.36 153 1094035 290 40 232 18 292
CRW_00590 0.61 0.59 0.64 268 1119334 184 21 133 30 184
CRW_00591 0.56 0.56 0.56 249 1098461 218 30 163 25 197
CRW_00594 - 0.21 0.23 0.20 89 1028443 369 80 283 6 306
CRW_00596 - 0.23 0.24 0.21 92 1024167 349 55 282 12 294
CRW_00604 - 0.60 0.64 0.55 52 411684 224 14 28 182 29
CRW_00606 - 0.66 0.68 0.64 63 559055 272 9 26 237 29
CRW_00607 - 0.53 0.55 0.52 42 536049 294 8 31 255 35
CRW_00617 - 0.14 0.17 0.12 11 426336 250 28 51 171 53
CRW_00678 - 0.39 0.41 0.36 50 666297 318 15 73 230 71
CRW_00684 - 0.29 0.32 0.27 43 612013 300 32 83 185 92
CRW_00704 - 0.82 0.81 0.83 76 386668 201 0 16 185 18
CRW_00706 - 0.48 0.54 0.43 32 695536 339 10 32 297 27
CRW_00710 - 0.57 0.57 0.57 65 1040288 387 11 39 337 49
CRW_00712 - 0.27 0.33 0.22 36 952727 403 48 79 276 72
CRW_00713 - 0.58 0.58 0.58 59 731343 329 4 39 286 42
CRW_00766 - 0.18 0.18 0.18 43 440153 206 27 168 11 197
CRW_00768 - 0.38 0.37 0.38 96 471656 179 22 132 25 163
CRW_00769 - 0.43 0.42 0.44 106 451484 154 27 108 19 146
CRW_00772 0.19 0.18 0.19 74 945610 325 42 274 9 330
CRW_00774 - 0.18 0.18 0.18 47 488302 225 39 178 8 216
CRW_00778 0.42 0.41 0.42 159 1201646 291 25 195 71 225
CRW_00779 - 0.28 0.28 0.27 120 1031326 338 33 287 18 302
CRW_00780 - 0.13 0.13 0.13 34 475544 230 36 186 8 221
CRW_00783 - 0.20 0.20 0.20 51 453371 220 26 180 14 208
CRW_00784 - 0.26 0.26 0.27 67 456240 199 26 157 16 195
CRW_00785 - 0.30 0.29 0.30 74 452430 189 23 149 17 179
CRW_00787 - 0.28 0.29 0.28 88 646633 259 39 193 27 218
CRW_00789 0.51 0.50 0.52 224 1137356 236 32 174 30 223
CRW_00793 - 0.31 0.32 0.31 120 1048689 320 49 218 53 257
CRW_00794 - 0.25 0.25 0.25 62 434530 200 22 164 14 184
CRW_00796 - 0.13 0.13 0.14 30 384783 202 26 164 12 205
CRW_00797 - 0.25 0.26 0.24 89 692885 299 49 230 20 253
CRW_00801 - 0.43 0.44 0.43 109 438263 163 37 107 19 140
CRW_00802 - 0.30 0.30 0.31 75 460078 195 26 141 28 178
CRW_00849 0.47 0.47 0.47 222 1117790 269 18 230 21 248
CRW_01134 - 0.29 0.29 0.29 123 964928 311 41 263 7 304
CRW_01168 - 0.41 0.41 0.41 168 900747 251 32 206 13 243
CRW_01186 - 0.27 0.27 0.27 105 836184 289 42 240 7 281
CRW_01188 - 0.12 0.12 0.11 48 837445 375 55 317 3 345
CRW_01189 - 0.24 0.24 0.23 95 862232 324 57 257 10 297
CRW_01214 - 0.21 0.21 0.22 85 931905 318 31 274 13 320
CRW_01222 - 0.26 0.25 0.26 101 912892 291 46 237 8 296
CRW_01250 - 0.30 0.30 0.30 114 781500 271 38 223 10 264
CRW_01260 - 0.15 0.15 0.15 59 912890 348 33 294 21 334
CRW_01266 - 0.30 0.30 0.30 115 818176 283 35 234 14 266
CRW_01275 - 0.33 0.33 0.33 144 1082222 305 49 241 15 288
CRW_01283 - 0.39 0.39 0.39 161 963557 267 29 219 19 248
CRW_01314 - 0.28 0.27 0.28 110 937374 293 34 247 12 297
CRW_01343 0.37 0.36 0.38 158 977489 271 43 211 17 276
CRW_01344 - 0.23 0.23 0.23 83 791549 283 43 236 4 283
PDB_00703 0.65 0.61 0.69 304 1073402 166 18 121 27 191
PDB_00805 - 0.59 0.58 0.60 176 467736 139 17 99 23 126
PDB_01261 - 0.48 0.45 0.50 152 467726 150 4 146 0 184
PDB_01273 0.54 0.52 0.56 253 1117806 233 20 181 32 238

^top



Performance of Afold - scored lower in this pairwise comparison

1. Total counts & total scores for Afold

Total Base Pair Counts
Total TP 12172
Total TN 80100432
Total FP 25863
Total FP CONTRA 3194
Total FP INCONS 18389
Total FP COMP 4280
Total FN 22682
Total Scores
MCC 0.355
Average MCC ± 95% Confidence Intervals 0.363 ± 0.027
Sensitivity 0.349
Positive Predictive Value 0.361
Nr of predictions 109

^top



2. Individual counts for Afold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00002 - 0.55 0.55 0.55 145 408798 132 32 85 15 119
CRW_00054 0.46 0.46 0.46 208 1107368 260 31 209 20 248
CRW_00068 0.49 0.49 0.50 226 1108849 247 24 206 17 231
CRW_00140 0.64 0.63 0.64 291 1135826 182 20 141 21 172
CRW_00173 0.29 0.29 0.29 127 1082216 323 37 276 10 318
CRW_00245 0.38 0.38 0.38 172 1105880 289 44 232 13 281
CRW_00270 0.44 0.43 0.45 196 1111850 260 40 200 20 259
CRW_00271 0.51 0.50 0.51 231 1122302 237 43 175 19 231
CRW_00273 0.59 0.58 0.60 258 1085174 195 26 143 26 188
CRW_00275 0.47 0.47 0.48 213 1114826 258 28 204 26 244
CRW_00277 0.38 0.38 0.38 172 1110338 300 45 240 15 281
CRW_00278 0.41 0.40 0.42 180 1129825 274 33 218 23 270
CRW_00280 0.51 0.50 0.51 227 1117817 231 28 188 15 226
CRW_00281 0.30 0.30 0.29 138 1144872 347 41 290 16 326
CRW_00283 0.28 0.27 0.29 118 1015612 321 39 256 26 314
CRW_00284 0.28 0.27 0.28 124 1131316 337 30 290 17 332
CRW_00285 0.72 0.71 0.74 325 1117819 142 23 93 26 133
CRW_00292 0.31 0.30 0.32 138 1123814 315 46 252 17 323
CRW_00338 - 0.13 0.14 0.12 57 1052947 427 76 346 5 338
CRW_00417 - 0.38 0.37 0.39 97 458157 161 21 128 12 165
CRW_00418 - 0.24 0.22 0.26 56 441113 170 17 144 9 204
CRW_00419 - 0.19 0.17 0.20 45 463945 187 22 154 11 220
CRW_00421 - 0.38 0.36 0.41 96 462968 155 22 117 16 170
CRW_00422 - 0.36 0.35 0.38 92 455291 171 23 129 19 171
CRW_00426 - 0.30 0.29 0.31 78 457195 183 24 149 10 187
CRW_00427 - 0.18 0.18 0.19 46 454337 212 30 168 14 215
CRW_00428 - 0.39 0.38 0.40 98 465847 162 29 121 12 162
CRW_00430 - 0.18 0.18 0.19 47 456245 207 30 168 9 219
CRW_00433 - 0.35 0.32 0.38 85 467807 142 19 117 6 178
CRW_00434 - 0.38 0.35 0.40 92 457218 146 18 118 10 171
CRW_00435 - 0.36 0.33 0.39 88 451501 148 14 122 12 177
CRW_00437 - 0.21 0.21 0.22 54 456242 197 25 169 3 206
CRW_00438 - 0.35 0.34 0.36 90 454332 170 19 140 11 178
CRW_00441 - 0.17 0.16 0.19 42 454357 187 31 151 5 220
CRW_00442 - 0.37 0.36 0.39 93 453388 156 25 122 9 168
CRW_00443 - 0.39 0.36 0.42 93 456266 153 18 113 22 167
CRW_00444 - 0.27 0.26 0.28 68 459119 181 24 150 7 195
CRW_00445 - 0.28 0.27 0.30 71 458168 175 24 140 11 194
CRW_00446 - 0.46 0.43 0.48 115 468758 135 14 109 12 150
CRW_00447 - 0.27 0.27 0.28 69 452428 194 23 156 15 191
CRW_00448 - 0.34 0.32 0.36 83 457215 169 23 125 21 178
CRW_00449 - 0.22 0.21 0.23 54 444861 185 20 161 4 206
CRW_00451 - 0.21 0.20 0.21 52 457204 204 25 165 14 212
CRW_00452 - 0.31 0.29 0.34 77 453399 174 20 132 22 185
CRW_00453 - 0.34 0.31 0.36 81 452454 150 22 119 9 180
CRW_00458 - 0.32 0.31 0.32 83 464873 185 34 140 11 181
CRW_00459 - 0.29 0.27 0.31 72 462011 176 27 131 18 191
CRW_00462 - 0.28 0.27 0.30 70 458171 178 16 146 16 191
CRW_00463 - 0.34 0.33 0.35 85 445799 164 22 134 8 169
CRW_00464 - 0.39 0.36 0.42 95 462015 142 15 116 11 169
CRW_00466 0.60 0.59 0.62 268 1116330 196 21 146 29 190
CRW_00530 - 0.35 0.37 0.33 99 534792 223 55 149 19 169
CRW_00535 - 0.25 0.27 0.24 89 898092 316 73 216 27 239
CRW_00538 - 0.20 0.22 0.19 65 811827 304 77 206 21 235
CRW_00589 0.18 0.18 0.18 78 1094024 374 34 324 16 367
CRW_00590 0.36 0.36 0.37 161 1119326 290 30 239 21 291
CRW_00591 0.57 0.56 0.59 251 1098476 205 23 153 29 195
CRW_00594 - 0.22 0.24 0.20 93 1028441 382 52 309 21 302
CRW_00596 - 0.21 0.22 0.21 86 1024181 349 55 274 20 300
CRW_00604 - 0.52 0.57 0.48 46 411682 214 15 35 164 35
CRW_00606 - 0.66 0.64 0.68 59 559066 266 2 26 238 33
CRW_00607 - 0.55 0.53 0.56 41 536057 274 4 28 242 36
CRW_00617 - 0.66 0.64 0.68 41 426366 236 1 18 217 23
CRW_00678 - 0.66 0.64 0.67 78 666318 274 10 29 235 43
CRW_00684 - 0.34 0.39 0.29 53 611990 298 44 84 170 82
CRW_00704 - 0.81 0.80 0.82 75 386669 198 4 12 182 19
CRW_00706 - 0.45 0.51 0.39 30 695534 340 11 35 294 29
CRW_00710 - 0.66 0.65 0.67 74 1040293 375 8 28 339 40
CRW_00712 - 0.47 0.54 0.41 58 952747 370 30 55 285 50
CRW_00713 - 0.56 0.57 0.55 58 731339 313 11 37 265 43
CRW_00766 - 0.27 0.26 0.28 63 440168 177 20 140 17 177
CRW_00768 - 0.40 0.39 0.41 101 471658 164 22 125 17 158
CRW_00769 - 0.39 0.37 0.42 93 451502 148 20 110 18 159
CRW_00772 0.26 0.25 0.26 103 945610 313 37 250 26 301
CRW_00774 - 0.37 0.36 0.39 94 488322 170 17 133 20 169
CRW_00778 0.30 0.29 0.32 113 1201668 314 34 210 70 271
CRW_00779 - 0.30 0.30 0.30 126 1031350 307 35 255 17 296
CRW_00780 - 0.18 0.18 0.19 45 475569 201 21 165 15 210
CRW_00783 - 0.37 0.36 0.38 93 453382 165 15 138 12 166
CRW_00784 - 0.37 0.35 0.39 93 456254 166 21 122 23 169
CRW_00785 - 0.35 0.33 0.36 84 452443 162 27 122 13 169
CRW_00787 - 0.23 0.24 0.23 74 646626 267 36 217 14 232
CRW_00789 0.36 0.36 0.36 160 1137342 302 44 240 18 287
CRW_00793 - 0.33 0.34 0.33 127 1048686 318 55 208 55 250
CRW_00794 - 0.24 0.24 0.25 59 434543 196 15 161 20 187
CRW_00796 - 0.20 0.20 0.21 47 384777 193 27 152 14 188
CRW_00797 - 0.26 0.26 0.26 90 692907 280 33 223 24 252
CRW_00801 - 0.45 0.45 0.46 112 438270 149 29 105 15 137
CRW_00802 - 0.19 0.19 0.19 49 460062 218 33 176 9 204
CRW_00849 0.49 0.48 0.49 227 1117798 259 23 212 24 243
CRW_01134 - 0.25 0.26 0.25 109 964912 341 41 293 7 318
CRW_01168 - 0.39 0.39 0.39 160 900739 274 33 221 20 251
CRW_01186 - 0.27 0.27 0.27 103 836190 293 44 234 15 283
CRW_01188 - 0.17 0.17 0.17 68 837466 339 48 283 8 325
CRW_01189 - 0.21 0.21 0.21 82 862242 334 47 270 17 310
CRW_01214 - 0.29 0.29 0.29 116 931900 288 36 243 9 289
CRW_01222 - 0.23 0.23 0.23 92 912871 321 54 259 8 305
CRW_01250 - 0.31 0.30 0.32 115 781518 257 33 209 15 263
CRW_01260 - 0.15 0.15 0.15 60 912885 340 48 283 9 333
CRW_01266 - 0.28 0.28 0.28 108 818174 292 43 235 14 273
CRW_01275 - 0.30 0.30 0.30 128 1082230 316 45 253 18 304
CRW_01283 - 0.40 0.40 0.40 163 963560 265 37 206 22 246
CRW_01314 - 0.28 0.27 0.29 110 937386 284 37 232 15 297
CRW_01343 0.37 0.37 0.37 160 977470 284 42 229 13 274
CRW_01344 - 0.30 0.30 0.30 110 791545 267 40 216 11 256
PDB_00703 0.53 0.50 0.56 246 1073408 220 17 174 29 249
PDB_00805 - 0.64 0.61 0.67 184 467752 116 18 74 24 118
PDB_01261 - 0.50 0.45 0.55 152 467752 128 4 120 4 184
PDB_01273 0.51 0.48 0.53 237 1117812 242 21 190 31 254

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.