CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	TurboFold(seed)		NanoFolder
MCC	0.834	>	0.297
Average MCC ± 95% Confidence Intervals	0.791 ± 0.165	>	0.382 ± 0.206
Sensitivity	0.797	>	0.355
Positive Predictive Value	0.874	>	0.254
Total TP	299	>	133
Total TN	96078	>	95897
Total FP	83	<	421
Total FP CONTRA	5	<	88
Total FP INCONS	38	<	302
Total FP COMP	40	>	31
Total FN	76	<	242
P-value	1.39551335374e-08

Comparison of performance of TurboFold(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for TurboFold(seed) and NanoFolder).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for TurboFold(seed) and NanoFolder).
Comparison of average Matthews Correlation Coefficients (MCCs) for TurboFold(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for TurboFold(seed) and NanoFolder).

Total Scores
MCC	0.834
Average MCC ± 95% Confidence Intervals	0.791 ± 0.165
Sensitivity	0.797
Positive Predictive Value	0.874
Nr of predictions	13

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_01535	5S_rRNA links to Rfam database...	0.94	0.89	1.00	33	7107	3	0	0	3	4
RFA_00416	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1470	2	0	0	2	0
RFA_00664	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	980	11	0	10	1	14
RFA_00703	Hammerhead_1 links to Rfam database...	0.67	0.64	0.69	9	4265	18	1	3	14	5
RFA_00765	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00770	Corona_pk3 links to Rfam database...	0.68	0.56	0.83	10	2004	5	0	2	3	8
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	2	1	0	1	8
RFA_00808	RydC links to Rfam database...	0.75	0.56	1.00	9	2007	0	0	0	0	7
RFA_00816	R2_retro_el links to Rfam database...	1.00	1.00	1.00	41	23179	14	0	0	14	0
SRP_00200	Bacteria_small_SRP links to Rfam database...	0.90	0.92	0.89	33	6866	4	1	3	0	3
SRP_00285	Bacteria_small_SRP links to Rfam database...	0.95	0.93	0.97	28	3712	1	0	1	0	2
SRP_00308	Bacteria_large_SRP links to Rfam database...	0.76	0.74	0.77	65	36231	20	2	17	1	23
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.94	0.94	0.94	34	5424	2	0	2	0	2

Total Scores
MCC	0.297
Average MCC ± 95% Confidence Intervals	0.382 ± 0.206
Sensitivity	0.355
Positive Predictive Value	0.254
Nr of predictions	13

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_01535	5S_rRNA links to Rfam database...	0.20	0.24	0.17	9	7088	43	7	36	0	28
RFA_00416	Hammerhead_3 links to Rfam database...	0.94	1.00	0.88	15	1468	6	2	0	4	0
RFA_00664	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	978	13	0	12	1	14
RFA_00703	Hammerhead_1 links to Rfam database...	0.26	0.36	0.19	5	4252	32	5	16	11	9
RFA_00765	s2m links to Rfam database...	0.39	0.42	0.38	5	890	9	2	6	1	7
RFA_00770	Corona_pk3 links to Rfam database...	-0.01	0.00	0.00	0	1996	24	0	20	4	18
RFA_00773	Corona_pk3 links to Rfam database...	0.93	1.00	0.86	18	1932	9	3	0	6	0
RFA_00808	RydC links to Rfam database...	0.83	1.00	0.70	16	1993	7	7	0	0	0
RFA_00816	R2_retro_el links to Rfam database...	0.00	0.00	0.00	0	23133	89	30	57	2	41
SRP_00200	Bacteria_small_SRP links to Rfam database...	0.41	0.47	0.35	17	6855	31	4	27	0	19
SRP_00285	Bacteria_small_SRP links to Rfam database...	0.38	0.43	0.35	13	3704	24	6	18	0	17
SRP_00308	Bacteria_large_SRP links to Rfam database...	0.14	0.17	0.12	15	36189	112	18	93	1	73
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.52	0.56	0.49	20	5419	22	4	17	1	16

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.