CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(seed)		RNASampler(seed)
MCC	0.661	>	0.655
Average MCC ± 95% Confidence Intervals	0.655 ± 0.046	>	0.639 ± 0.088
Sensitivity	0.572	=	0.572
Positive Predictive Value	0.769	>	0.755
Total TP	163	=	163
Total TN	37506	>	37502
Total FP	51	<	56
Total FP CONTRA	23	>	20
Total FP INCONS	26	<	33
Total FP COMP	2	<	3
Total FN	122	=	122
P-value	0.000309992208728

Comparison of performance of PETfold_pre2.0(seed) and RNASampler(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and RNASampler(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and RNASampler(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(seed) and RNASampler(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and RNASampler(seed)).

Total Scores
MCC	0.661
Average MCC ± 95% Confidence Intervals	0.655 ± 0.046
Sensitivity	0.572
Positive Predictive Value	0.769
Nr of predictions	13

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.80	0.64	1.00	9	937	0	0	0	0	5
PDB_00716	HDV_ribozyme links to Rfam database...	0.68	0.57	0.81	13	2685	3	0	3	0	10
PDB_01092	glmS links to Rfam database...	0.68	0.56	0.83	29	10118	8	0	6	2	23
RFA_00632	HDV_ribozyme links to Rfam database...	0.69	0.64	0.75	18	4071	6	2	4	0	10
RFA_00636	HDV_ribozyme links to Rfam database...	0.73	0.68	0.79	19	3981	5	2	3	0	9
RFA_00767	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1877	4	3	1	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1877	4	3	1	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1939	4	3	1	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	2002	4	3	1	0	8
RFA_00773	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1939	4	3	1	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1939	4	3	1	0	8
RFA_00808	RydC links to Rfam database...	0.71	0.56	0.90	9	2006	1	1	0	0	7
RFA_00809	RydC links to Rfam database...	0.47	0.38	0.60	6	2135	4	0	4	0	10

Total Scores
MCC	0.655
Average MCC ± 95% Confidence Intervals	0.639 ± 0.088
Sensitivity	0.572
Positive Predictive Value	0.755
Nr of predictions	13

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	0.50	0.39	0.64	9	2687	6	0	5	1	14
PDB_01092	glmS links to Rfam database...	0.72	0.62	0.84	32	10115	8	0	6	2	20
RFA_00632	HDV_ribozyme links to Rfam database...	0.78	0.68	0.90	19	4074	2	1	1	0	9
RFA_00636	HDV_ribozyme links to Rfam database...	0.85	0.75	0.95	21	3983	1	1	0	0	7
RFA_00767	Corona_pk3 links to Rfam database...	0.65	0.56	0.77	10	1878	3	3	0	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.65	0.56	0.77	10	1878	3	3	0	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.59	0.56	0.63	10	1937	6	1	5	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.52	0.39	0.70	7	2006	3	3	0	0	11
RFA_00773	Corona_pk3 links to Rfam database...	0.59	0.50	0.69	9	1940	4	4	0	0	9
RFA_00779	Corona_pk3 links to Rfam database...	0.65	0.56	0.77	10	1940	3	3	0	0	8
RFA_00808	RydC links to Rfam database...	0.58	0.56	0.60	9	2001	6	0	6	0	7
RFA_00809	RydC links to Rfam database...	0.36	0.38	0.35	6	2128	11	1	10	0	10

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.