CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	CMfinder(seed)		NanoFolder
MCC	0.307	>	0.236
Average MCC ± 95% Confidence Intervals	0.353 ± 0.250	<	0.366 ± 0.341
Sensitivity	0.139	<	0.279
Positive Predictive Value	0.679	>	0.202
Total TP	72	<	145
Total TN	318068	>	317457
Total FP	34	<	595
Total FP CONTRA	1	<	116
Total FP INCONS	33	<	456
Total FP COMP	0	<	23
Total FN	447	>	374
P-value	0.0

Comparison of performance of CMfinder(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).
Comparison of average Matthews Correlation Coefficients (MCCs) for CMfinder(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CMfinder(seed) and NanoFolder).

Total Scores
MCC	0.307
Average MCC ± 95% Confidence Intervals	0.353 ± 0.250
Sensitivity	0.139
Positive Predictive Value	0.679
Nr of predictions	7

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00671	Intron_gpI links to Rfam database...	0.20	0.09	0.41	11	62808	16	0	16	0	106
CRW_00672	Intron_gpI links to Rfam database...	0.38	0.15	0.94	17	72372	1	0	1	0	94
CRW_00674	Intron_gpI links to Rfam database...	0.40	0.17	0.95	21	83414	1	0	1	0	103
CRW_00676	Intron_gpI links to Rfam database...	0.12	0.04	0.33	5	93513	10	1	9	0	110
RFA_00770	Corona_pk3 links to Rfam database...	0.71	0.50	1.00	9	2007	0	0	0	0	9
RFA_00773	Corona_pk3 links to Rfam database...	0.00	0.00	0.00	0	1949	4	0	4	0	18
RFA_00808	RydC links to Rfam database...	0.68	0.56	0.82	9	2005	2	0	2	0	7

Total Scores
MCC	0.236
Average MCC ± 95% Confidence Intervals	0.366 ± 0.341
Sensitivity	0.279
Positive Predictive Value	0.202
Nr of predictions	7

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00671	Intron_gpI links to Rfam database...	0.18	0.21	0.16	24	62682	130	23	106	1	93
CRW_00672	Intron_gpI links to Rfam database...	0.36	0.43	0.31	48	72233	114	35	74	5	63
CRW_00674	Intron_gpI links to Rfam database...	0.16	0.19	0.14	23	83274	142	19	120	3	101
CRW_00676	Intron_gpI links to Rfam database...	0.11	0.14	0.09	16	93347	169	29	136	4	99
RFA_00770	Corona_pk3 links to Rfam database...	-0.01	0.00	0.00	0	1996	24	0	20	4	18
RFA_00773	Corona_pk3 links to Rfam database...	0.93	1.00	0.86	18	1932	9	3	0	6	0
RFA_00808	RydC links to Rfam database...	0.83	1.00	0.70	16	1993	7	7	0	0	0

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.