CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(seed)		CMfinder(seed)
MCC	0.666	>	0.443
Average MCC ± 95% Confidence Intervals	0.661 ± 0.056	>	0.456 ± 0.186
Sensitivity	0.574	>	0.333
Positive Predictive Value	0.777	>	0.597
Total TP	143	>	83
Total TN	33752	<	33797
Total FP	43	<	57
Total FP CONTRA	17	>	3
Total FP INCONS	24	<	53
Total FP COMP	2	>	1
Total FN	106	<	166
P-value	9.16590079822e-09

Comparison of performance of PETfold_pre2.0(seed) and CMfinder(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and CMfinder(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and CMfinder(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(seed) and CMfinder(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and CMfinder(seed)).

Total Scores
MCC	0.666
Average MCC ± 95% Confidence Intervals	0.661 ± 0.056
Sensitivity	0.574
Positive Predictive Value	0.777
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.80	0.64	1.00	9	937	0	0	0	0	5
PDB_00716	HDV_ribozyme links to Rfam database...	0.68	0.57	0.81	13	2685	3	0	3	0	10
PDB_01092	glmS links to Rfam database...	0.68	0.56	0.83	29	10118	8	0	6	2	23
RFA_00632	HDV_ribozyme links to Rfam database...	0.69	0.64	0.75	18	4071	6	2	4	0	10
RFA_00636	HDV_ribozyme links to Rfam database...	0.73	0.68	0.79	19	3981	5	2	3	0	9
RFA_00769	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1939	4	3	1	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	2002	4	3	1	0	8
RFA_00773	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1939	4	3	1	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1939	4	3	1	0	8
RFA_00808	RydC links to Rfam database...	0.71	0.56	0.90	9	2006	1	1	0	0	7
RFA_00809	RydC links to Rfam database...	0.47	0.38	0.60	6	2135	4	0	4	0	10

Total Scores
MCC	0.443
Average MCC ± 95% Confidence Intervals	0.456 ± 0.186
Sensitivity	0.333
Positive Predictive Value	0.597
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.47	0.36	0.63	5	938	3	0	3	0	9
PDB_00716	HDV_ribozyme links to Rfam database...	-0.01	0.00	0.00	0	2688	13	0	13	0	23
PDB_01092	glmS links to Rfam database...	0.38	0.21	0.69	11	10137	6	0	5	1	41
RFA_00632	HDV_ribozyme links to Rfam database...	0.27	0.25	0.29	7	4071	17	0	17	0	21
RFA_00636	HDV_ribozyme links to Rfam database...	0.71	0.64	0.78	18	3982	5	2	3	0	10
RFA_00769	Corona_pk3 links to Rfam database...	0.71	0.50	1.00	9	1944	0	0	0	0	9
RFA_00770	Corona_pk3 links to Rfam database...	0.71	0.50	1.00	9	2007	0	0	0	0	9
RFA_00773	Corona_pk3 links to Rfam database...	0.00	0.00	0.00	0	1949	4	0	4	0	18
RFA_00779	Corona_pk3 links to Rfam database...	0.71	0.50	1.00	9	1944	0	0	0	0	9
RFA_00808	RydC links to Rfam database...	0.68	0.56	0.82	9	2005	2	0	2	0	7
RFA_00809	RydC links to Rfam database...	0.41	0.38	0.46	6	2132	7	1	6	0	10

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.