CompaRNA

A server for continuous benchmarking of automated methods for RNA structure prediction

by Tomasz Puton, Kristian Rother, Łukasz Kozłowski, Janusz M. Bujnicki

https://iimcb.genesilico.pl/comparna/

What is CompaRNA

The CompaRNA web server benchmarks freely available web servers and standalone automated methods for RNA secondary structure prediction. The aim of CompaRNA is to assess the state of the art in the field, provide a detailed picture of what is possible with the available tools, where the progress is made and what major problems remain. The results of the benchmarks and the functionality of the CompaRNA web server have been described in a publication in Nucleic Acids Research.

CompaRNA uses both EF PDB and RNAstrand databases to prepare benchmarking datasets. Based on them, CompaRNA calculates a set of rankings for various methods to show their performance.

Reference RNA structures extracted weekly from the PDB database

CompaRNA monitors the Protein Data Bank for the release of new RNA structures solved experimentally. The latest release of new RNA structures in the PDB database was on March 16, 2016 (2 RNA chains from 1 PDB file were extracted). CompaRNA has extracted 1935 RNA structures from the PDB database since February 18, 2009.

CompaRNA changelog

---
Sept. 26, 2013
The two PDB datasets with pseudoknotted RNAs were updated (with standard and extended base pair definitions - the former contained two RNAs which did not have a pseudoknot properly annotated and the latter contained four such incorrect annotations).

---
Sept. 5, 2013
A bug in parsing RSpredict results was removed. RSpredict in many cases introduces a big gap at the beginning of the consensus sequences and its secondary structure, that was not handled correctly. Right now RCoffee is used to realign the consensus sequence and the query RNA, and the predicted structure is mapped onto the aligned query to get the prediction.

---
July 17, 2013
Added DAFS and DotKnot to the CompaRNA benchmarks, their performance has been already assessed on the RNAstrand dataset. But there are not enough predictions collected so far for the PDB dataset to conduct a statistically sound evaluation of their performance.

---
July 12, 2013
Changes in the layout of the presented rankings (much clearer desctiption of the methods' performance thanks to the CompaRNA users!).

---
June 12, 2013
Changes in how PETfold pre 2.0 is executed. PETfold pre 2.0 was run with incorrect type of input (sequence collections instead of sequence alignments) what caused the deterioration of PETfold pre 2.0 performance in the presented benchmarks (PETfold pre 2.0 *does not validate whether sequences are aligned*, now with the release of PETfold 2.0 this has changed). The rankings have been updated and the performance of PETfold improved significantly - it is now one of the best methods predicting RNA secondary structure tested in CompaRNA.

---
June 10, 2013
PETfold ver 2.0. was added to the CompaRNA benchmark.