CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	NanoFolder		Carnac(seed)
MCC	0.251	>	0.000
Average MCC ± 95% Confidence Intervals	0.269 ± 0.152	>	0.000 ± 0.000
Sensitivity	0.296	>	0.000
Positive Predictive Value	0.215	>	0.000
Total TP	190	>	0
Total TN	397546	<	398430
Total FP	715	>	0
Total FP CONTRA	140	>	0
Total FP INCONS	554	>	0
Total FP COMP	21	>	0
Total FN	452	<	642
P-value	0.0

Comparison of performance of NanoFolder and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for NanoFolder and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for NanoFolder and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for NanoFolder and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for NanoFolder and Carnac(seed)).

Total Scores
MCC	0.251
Average MCC ± 95% Confidence Intervals	0.269 ± 0.152
Sensitivity	0.296
Positive Predictive Value	0.215
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00196	RNaseP_arch links to Rfam database...	0.49	0.57	0.42	43	31524	62	16	43	3	32
ASE_00417	RNaseP_arch links to Rfam database...	0.32	0.36	0.28	36	54486	98	18	75	5	64
CRW_00671	Intron_gpI links to Rfam database...	0.18	0.21	0.16	24	62682	130	23	106	1	93
CRW_00672	Intron_gpI links to Rfam database...	0.36	0.43	0.31	48	72233	114	35	74	5	63
CRW_00674	Intron_gpI links to Rfam database...	0.16	0.19	0.14	23	83274	142	19	120	3	101
CRW_00676	Intron_gpI links to Rfam database...	0.11	0.14	0.09	16	93347	169	29	136	4	99

Total Scores
MCC	0.000
Average MCC ± 95% Confidence Intervals	0.000 ± 0.000
Sensitivity	0.000
Positive Predictive Value	0.000
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.