CompaRNA - on-line benchmarks of RNA structure prediction methods
Home

Methods
Datasets
Rankings
RNA 2D Atlas

Help
FAQ

Contact us
RSS feeds
Twitter

Table of contents:

  1. Overview

  2. Performance Plots

  3. Performance of PETfold_pre2.0(seed) - scored higher in this pairwise comparison

  4. Performance of Afold - scored lower in this pairwise comparison

  5. Compile and download dataset for PETfold_pre2.0(seed) & Afold [.zip] - may take several seconds...


Overview

Metric PETfold_pre2.0(seed) Afold
MCC 0.762 > 0.408
Average MCC ± 95% Confidence Intervals 0.764 ± 0.022 > 0.410 ± 0.029
Sensitivity 0.732 > 0.406
Positive Predictive Value 0.794 > 0.409
Total TP 27571 > 15319
Total TN 105964404 > 105961672
Total FP 11249 < 24845
Total FP CONTRA 1110 < 3511
Total FP INCONS 6043 < 18626
Total FP COMP 4096 > 2708
Total FN 10119 < 22371
P-value 3.56938820447e-08

^top




Performance plots


  1. Comparison of performance of PETfold_pre2.0(seed) and Afold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and Afold).

  2. Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and Afold).

  3. Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(seed) and Afold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and Afold).

^top





Performance of PETfold_pre2.0(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for PETfold_pre2.0(seed)

Total Base Pair Counts
Total TP 27571
Total TN 105964404
Total FP 11249
Total FP CONTRA 1110
Total FP INCONS 6043
Total FP COMP 4096
Total FN 10119
Total Scores
MCC 0.762
Average MCC ± 95% Confidence Intervals 0.764 ± 0.022
Sensitivity 0.732
Positive Predictive Value 0.794
Nr of predictions 79

^top



2. Individual counts for PETfold_pre2.0(seed) [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00024 0.89 0.88 0.90 405 1085153 80 10 33 37 53
CRW_00048 0.85 0.84 0.87 395 1170759 96 18 43 35 75
CRW_00054 0.86 0.86 0.87 390 1107367 95 16 43 36 66
CRW_00059 0.85 0.83 0.87 392 1172293 95 17 44 34 81
CRW_00061 0.84 0.82 0.86 388 1181504 96 16 45 35 88
CRW_00068 0.86 0.86 0.87 394 1108851 94 17 43 34 63
CRW_00096 0.86 0.84 0.88 392 1178434 98 11 43 44 73
CRW_00140 0.85 0.84 0.86 387 1135829 97 15 47 35 76
CRW_00141 0.85 0.82 0.87 392 1201573 98 16 44 38 84
CRW_00150 0.85 0.83 0.87 385 1152477 92 16 43 33 81
CRW_00152 0.84 0.82 0.86 386 1179969 96 16 45 35 85
CRW_00166 0.86 0.84 0.88 393 1190749 97 11 43 43 75
CRW_00173 0.80 0.79 0.81 353 1082222 118 17 64 37 92
CRW_00177 0.84 0.82 0.86 386 1178430 105 17 47 41 87
CRW_00196 0.83 0.81 0.86 384 1207786 102 15 50 37 91
CRW_00230 0.84 0.82 0.87 392 1198473 97 16 45 36 86
CRW_00245 0.86 0.86 0.87 388 1105880 95 18 42 35 65
CRW_00248 0.83 0.80 0.86 388 1218691 98 16 46 36 96
CRW_00250 0.84 0.82 0.87 391 1198474 97 16 45 36 87
CRW_00256 0.85 0.82 0.87 386 1192295 93 16 43 34 82
CRW_00270 0.87 0.86 0.87 391 1111838 94 15 42 37 64
CRW_00271 0.83 0.82 0.85 378 1122307 105 14 52 39 84
CRW_00273 0.86 0.85 0.86 380 1085161 98 14 46 38 66
CRW_00275 0.86 0.85 0.87 390 1114823 98 15 43 40 67
CRW_00277 0.84 0.83 0.85 377 1110354 101 16 48 37 76
CRW_00278 0.84 0.82 0.85 369 1129823 106 15 49 42 81
CRW_00279 0.85 0.83 0.86 385 1179970 103 16 45 42 79
CRW_00280 0.84 0.83 0.85 377 1117815 109 16 52 41 76
CRW_00281 0.85 0.84 0.87 388 1144894 97 16 43 38 76
CRW_00283 0.78 0.77 0.80 331 1015610 118 18 66 34 101
CRW_00284 0.83 0.82 0.84 372 1131319 111 16 53 42 84
CRW_00285 0.84 0.82 0.85 377 1117816 104 17 50 37 81
CRW_00289 0.82 0.80 0.84 382 1189200 109 16 55 38 94
CRW_00292 0.74 0.72 0.77 332 1123817 133 22 79 32 129
CRW_00298 0.66 0.63 0.69 310 1620452 212 16 122 74 184
CRW_00301 0.63 0.59 0.67 307 1762046 213 15 135 63 212
CRW_00310 0.63 0.59 0.68 305 1805503 209 14 128 67 216
CRW_00326 0.66 0.63 0.69 315 1627654 205 15 126 64 187
CRW_00330 0.64 0.59 0.69 307 1988567 210 13 128 69 210
CRW_00332 0.60 0.56 0.63 269 1769595 236 16 141 79 208
CRW_00333 0.58 0.52 0.64 269 1892065 217 19 132 66 245
CRW_00345 0.64 0.61 0.67 305 1586418 211 15 133 63 193
CRW_00348 0.65 0.62 0.68 312 1615046 210 15 130 65 191
CRW_00358 0.64 0.60 0.67 306 1674911 213 16 132 65 201
CRW_00366 0.65 0.62 0.68 300 1579311 211 15 127 69 182
CRW_00370 0.65 0.62 0.69 313 1640313 205 15 125 65 193
CRW_00371 0.68 0.65 0.71 322 1611455 199 13 120 66 174
CRW_00372 0.64 0.61 0.67 306 1625847 221 14 139 68 192
CRW_00379 0.60 0.54 0.66 286 2126521 223 18 128 77 248
CRW_00386 0.61 0.58 0.65 297 1750802 227 16 141 70 218
CRW_00403 0.68 0.66 0.70 306 1606090 215 15 117 83 158
CRW_00411 0.67 0.64 0.71 318 1615053 200 15 117 68 178
CRW_00414 0.64 0.61 0.67 303 1665770 223 20 132 71 192
CRW_00466 0.86 0.85 0.87 391 1116315 99 14 45 40 67
CRW_00589 0.87 0.84 0.89 376 1094037 99 6 41 52 69
CRW_00590 0.87 0.84 0.89 381 1119327 104 6 42 56 71
CRW_00591 0.87 0.86 0.89 382 1098473 103 6 42 55 64
CRW_00598 0.63 0.60 0.66 294 1665777 224 19 135 70 197
CRW_00758 0.64 0.61 0.67 294 1640327 224 15 130 79 189
CRW_00759 0.65 0.62 0.68 297 1636709 218 15 124 79 184
CRW_00760 0.63 0.59 0.66 287 1638522 227 17 129 81 199
CRW_00763 0.83 0.80 0.87 357 1215610 110 5 48 57 89
CRW_00772 0.79 0.76 0.81 309 945618 130 6 67 57 95
CRW_00778 0.84 0.83 0.85 317 1201654 139 6 48 85 67
CRW_00789 0.83 0.81 0.86 362 1137365 112 11 48 53 85
CRW_00795 0.79 0.76 0.82 299 1152557 159 6 59 94 95
CRW_00800 0.59 0.53 0.66 250 1401598 188 21 106 61 220
CRW_00849 0.79 0.76 0.81 359 1117819 116 16 66 34 111
CRW_01153 0.61 0.58 0.64 294 1606070 223 18 146 59 209
CRW_01171 0.63 0.60 0.67 304 1593548 214 15 138 61 202
CRW_01300 0.62 0.59 0.66 302 1693261 218 17 140 61 213
CRW_01343 0.87 0.85 0.89 367 977489 81 8 37 36 67
PDB_00703 0.79 0.73 0.86 361 1073423 107 9 52 46 134
PDB_00854 0.76 0.68 0.86 364 1500354 120 9 51 60 171
PDB_01088 0.75 0.67 0.85 365 1471040 119 5 60 54 182
PDB_01103 0.76 0.67 0.86 376 1606090 108 8 54 46 186
PDB_01105 0.77 0.70 0.86 376 1433534 108 3 58 47 164
PDB_01120 0.80 0.73 0.86 387 1204680 102 10 51 41 140
PDB_01273 0.80 0.75 0.86 368 1117832 112 8 52 52 123

^top



Performance of Afold - scored lower in this pairwise comparison

1. Total counts & total scores for Afold

Total Base Pair Counts
Total TP 15319
Total TN 105961672
Total FP 24845
Total FP CONTRA 3511
Total FP INCONS 18626
Total FP COMP 2708
Total FN 22371
Total Scores
MCC 0.408
Average MCC ± 95% Confidence Intervals 0.410 ± 0.029
Sensitivity 0.406
Positive Predictive Value 0.409
Nr of predictions 79

^top



2. Individual counts for Afold [ download as .csv ]

RNA Chain Rfam family MCC SENS PPV TP TN FP FP CONTRA FP INCONS FP COMP FN
CRW_00024 0.78 0.77 0.79 351 1085158 132 17 75 40 107
CRW_00048 0.49 0.49 0.50 228 1170758 252 20 209 23 242
CRW_00054 0.46 0.46 0.46 208 1107368 260 31 209 20 248
CRW_00059 0.52 0.51 0.53 240 1172289 233 26 191 16 233
CRW_00061 0.55 0.54 0.55 259 1181483 231 27 184 20 217
CRW_00068 0.49 0.49 0.50 226 1108849 247 24 206 17 231
CRW_00096 0.46 0.46 0.47 214 1178420 264 32 214 18 251
CRW_00140 0.64 0.63 0.64 291 1135826 182 20 141 21 172
CRW_00141 0.59 0.58 0.60 274 1201566 212 26 159 27 202
CRW_00150 0.42 0.41 0.42 193 1152465 282 35 228 19 273
CRW_00152 0.64 0.62 0.65 293 1179965 180 30 128 22 178
CRW_00166 0.54 0.53 0.55 248 1190745 226 28 175 23 220
CRW_00173 0.29 0.29 0.29 127 1082216 323 37 276 10 318
CRW_00177 0.44 0.43 0.45 203 1178433 259 22 222 15 270
CRW_00196 0.47 0.46 0.48 220 1207772 267 24 219 24 255
CRW_00230 0.57 0.56 0.58 266 1198467 216 21 172 23 212
CRW_00245 0.38 0.38 0.38 172 1105880 289 44 232 13 281
CRW_00248 0.57 0.57 0.58 274 1218666 234 21 180 33 210
CRW_00250 0.44 0.44 0.45 210 1198459 274 34 223 17 268
CRW_00256 0.41 0.40 0.42 188 1192291 278 39 222 17 280
CRW_00270 0.44 0.43 0.45 196 1111850 260 40 200 20 259
CRW_00271 0.51 0.50 0.51 231 1122302 237 43 175 19 231
CRW_00273 0.59 0.58 0.60 258 1085174 195 26 143 26 188
CRW_00275 0.47 0.47 0.48 213 1114826 258 28 204 26 244
CRW_00277 0.38 0.38 0.38 172 1110338 300 45 240 15 281
CRW_00278 0.41 0.40 0.42 180 1129825 274 33 218 23 270
CRW_00279 0.51 0.49 0.53 229 1179985 227 29 173 25 235
CRW_00280 0.51 0.50 0.51 227 1117817 231 28 188 15 226
CRW_00281 0.30 0.30 0.29 138 1144872 347 41 290 16 326
CRW_00283 0.28 0.27 0.29 118 1015612 321 39 256 26 314
CRW_00284 0.28 0.27 0.28 124 1131316 337 30 290 17 332
CRW_00285 0.72 0.71 0.74 325 1117819 142 23 93 26 133
CRW_00289 0.46 0.45 0.48 214 1189204 258 34 201 23 262
CRW_00292 0.31 0.30 0.32 138 1123814 315 46 252 17 323
CRW_00298 0.19 0.20 0.18 99 1620353 490 81 367 42 395
CRW_00301 0.30 0.31 0.30 162 1761957 428 86 298 44 357
CRW_00310 0.29 0.30 0.28 157 1805386 453 81 326 46 364
CRW_00326 0.41 0.41 0.42 204 1627621 339 47 238 54 298
CRW_00330 0.32 0.32 0.31 167 1988474 452 69 305 78 350
CRW_00332 0.18 0.20 0.17 94 1769479 492 105 343 44 383
CRW_00333 0.29 0.30 0.28 153 1891946 449 74 312 63 361
CRW_00345 0.34 0.35 0.33 175 1586340 403 71 285 47 323
CRW_00348 0.25 0.25 0.24 127 1614982 437 68 326 43 376
CRW_00358 0.36 0.36 0.35 183 1674845 391 61 276 54 324
CRW_00366 0.35 0.36 0.33 174 1579230 395 88 261 46 308
CRW_00370 0.28 0.28 0.28 143 1640248 420 77 298 45 363
CRW_00371 0.29 0.29 0.28 145 1611389 409 77 299 33 351
CRW_00372 0.36 0.36 0.35 181 1625787 378 59 279 40 317
CRW_00379 0.42 0.43 0.41 227 2126401 423 55 270 98 307
CRW_00386 0.24 0.25 0.23 129 1750683 478 83 361 34 386
CRW_00403 0.31 0.31 0.30 146 1606035 431 65 282 84 318
CRW_00411 0.34 0.34 0.34 169 1615006 381 62 266 53 327
CRW_00414 0.27 0.28 0.26 140 1665694 436 77 314 45 355
CRW_00466 0.60 0.59 0.62 268 1116330 196 21 146 29 190
CRW_00589 0.18 0.18 0.18 78 1094024 374 34 324 16 367
CRW_00590 0.36 0.36 0.37 161 1119326 290 30 239 21 291
CRW_00591 0.57 0.56 0.59 251 1098476 205 23 153 29 195
CRW_00598 0.31 0.33 0.29 162 1665675 440 88 300 52 329
CRW_00758 0.29 0.30 0.29 144 1640268 416 66 288 62 339
CRW_00759 0.22 0.22 0.21 107 1636633 462 78 327 57 374
CRW_00760 0.28 0.29 0.28 140 1638458 411 74 283 54 346
CRW_00763 0.38 0.37 0.40 165 1215603 304 24 228 52 281
CRW_00772 0.26 0.25 0.26 103 945610 313 37 250 26 301
CRW_00778 0.30 0.29 0.32 113 1201668 314 34 210 70 271
CRW_00789 0.36 0.36 0.36 160 1137342 302 44 240 18 287
CRW_00795 0.21 0.22 0.21 85 1152511 379 51 274 54 309
CRW_00800 0.55 0.53 0.57 251 1401532 269 24 168 77 219
CRW_00849 0.49 0.48 0.49 227 1117798 259 23 212 24 243
CRW_01153 0.32 0.32 0.32 160 1606024 390 54 290 46 343
CRW_01171 0.32 0.33 0.31 167 1593473 402 81 284 37 339
CRW_01300 0.36 0.37 0.35 189 1693186 392 71 274 47 326
CRW_01343 0.37 0.37 0.37 160 977470 284 42 229 13 274
PDB_00703 0.53 0.50 0.56 246 1073408 220 17 174 29 249
PDB_00854 0.55 0.53 0.56 284 1500271 276 22 201 53 251
PDB_01088 0.36 0.35 0.37 191 1470959 352 45 275 32 356
PDB_01103 0.56 0.54 0.58 306 1605999 254 32 191 31 256
PDB_01105 0.60 0.58 0.63 313 1433471 221 15 172 34 227
PDB_01120 0.46 0.43 0.49 228 1204659 250 31 210 9 299
PDB_01273 0.51 0.48 0.53 237 1117812 242 21 190 31 254

^top


Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.