CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PPfold(seed)		NanoFolder
MCC	0.599	>	0.368
Average MCC ± 95% Confidence Intervals	0.521 ± 0.284	>	0.432 ± 0.268
Sensitivity	0.383	<	0.429
Positive Predictive Value	0.939	>	0.322
Total TP	92	<	103
Total TN	48472	>	48250
Total FP	10	<	237
Total FP CONTRA	2	<	60
Total FP INCONS	4	<	157
Total FP COMP	4	<	20
Total FN	148	>	137
P-value	9.74569091494e-09

Comparison of performance of PPfold(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PPfold(seed) and NanoFolder).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PPfold(seed) and NanoFolder).
Comparison of average Matthews Correlation Coefficients (MCCs) for PPfold(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PPfold(seed) and NanoFolder).

Total Scores
MCC	0.599
Average MCC ± 95% Confidence Intervals	0.521 ± 0.284
Sensitivity	0.383
Positive Predictive Value	0.939
Nr of predictions	10

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00416	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1470	1	0	0	1	0
RFA_00664	Hammerhead_1 links to Rfam database...	0.38	0.14	1.00	2	988	1	0	0	1	12
RFA_00765	s2m links to Rfam database...	0.78	0.75	0.82	9	892	2	0	2	0	3
RFA_00770	Corona_pk3 links to Rfam database...	0.68	0.56	0.83	10	2004	2	1	1	0	8
RFA_00773	Corona_pk3 links to Rfam database...	0.68	0.56	0.83	10	1941	2	1	1	0	8
RFA_00808	RydC links to Rfam database...	0.75	0.56	1.00	9	2007	0	0	0	0	7
RFA_00816	R2_retro_el links to Rfam database...	0.95	0.90	1.00	37	23183	2	0	0	2	4
SRP_00273	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	6786	0	0	0	0	40
SRP_00285	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	3741	0	0	0	0	30
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5460	0	0	0	0	36

Total Scores
MCC	0.368
Average MCC ± 95% Confidence Intervals	0.432 ± 0.268
Sensitivity	0.429
Positive Predictive Value	0.322
Nr of predictions	10

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00416	Hammerhead_3 links to Rfam database...	0.94	1.00	0.88	15	1468	6	2	0	4	0
RFA_00664	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	978	13	0	12	1	14
RFA_00765	s2m links to Rfam database...	0.39	0.42	0.38	5	890	9	2	6	1	7
RFA_00770	Corona_pk3 links to Rfam database...	-0.01	0.00	0.00	0	1996	24	0	20	4	18
RFA_00773	Corona_pk3 links to Rfam database...	0.93	1.00	0.86	18	1932	9	3	0	6	0
RFA_00808	RydC links to Rfam database...	0.83	1.00	0.70	16	1993	7	7	0	0	0
RFA_00816	R2_retro_el links to Rfam database...	0.00	0.00	0.00	0	23133	89	30	57	2	41
SRP_00273	Bacteria_small_SRP links to Rfam database...	0.36	0.40	0.33	16	6737	34	6	27	1	24
SRP_00285	Bacteria_small_SRP links to Rfam database...	0.38	0.43	0.35	13	3704	24	6	18	0	17
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.52	0.56	0.49	20	5419	22	4	17	1	16

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.