CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	TurboFold(seed)		NanoFolder
MCC	0.834	>	0.423
Average MCC ± 95% Confidence Intervals	0.776 ± 0.196	>	0.439 ± 0.228
Sensitivity	0.785	>	0.480
Positive Predictive Value	0.889	>	0.381
Total TP	193	>	118
Total TN	36668	>	36575
Total FP	49	<	220
Total FP CONTRA	3	<	40
Total FP INCONS	21	<	152
Total FP COMP	25	<	28
Total FN	53	<	128
P-value	1.43692248862e-08

Comparison of performance of TurboFold(seed) and NanoFolder. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for TurboFold(seed) and NanoFolder).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for TurboFold(seed) and NanoFolder).
Comparison of average Matthews Correlation Coefficients (MCCs) for TurboFold(seed) and NanoFolder. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for TurboFold(seed) and NanoFolder).

Total Scores
MCC	0.834
Average MCC ± 95% Confidence Intervals	0.776 ± 0.196
Sensitivity	0.785
Positive Predictive Value	0.889
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_01535	5S_rRNA links to Rfam database...	0.94	0.89	1.00	33	7107	3	0	0	3	4
RFA_00416	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1470	2	0	0	2	0
RFA_00664	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	980	11	0	10	1	14
RFA_00703	Hammerhead_1 links to Rfam database...	0.67	0.64	0.69	9	4265	18	1	3	14	5
RFA_00765	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00770	Corona_pk3 links to Rfam database...	0.68	0.56	0.83	10	2004	5	0	2	3	8
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	2	1	0	1	8
RFA_00808	RydC links to Rfam database...	0.75	0.56	1.00	9	2007	0	0	0	0	7
SRP_00200	Bacteria_small_SRP links to Rfam database...	0.90	0.92	0.89	33	6866	4	1	3	0	3
SRP_00285	Bacteria_small_SRP links to Rfam database...	0.95	0.93	0.97	28	3712	1	0	1	0	2
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.94	0.94	0.94	34	5424	2	0	2	0	2

Total Scores
MCC	0.423
Average MCC ± 95% Confidence Intervals	0.439 ± 0.228
Sensitivity	0.480
Positive Predictive Value	0.381
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_01535	5S_rRNA links to Rfam database...	0.20	0.24	0.17	9	7088	43	7	36	0	28
RFA_00416	Hammerhead_3 links to Rfam database...	0.94	1.00	0.88	15	1468	6	2	0	4	0
RFA_00664	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	978	13	0	12	1	14
RFA_00703	Hammerhead_1 links to Rfam database...	0.26	0.36	0.19	5	4252	32	5	16	11	9
RFA_00765	s2m links to Rfam database...	0.39	0.42	0.38	5	890	9	2	6	1	7
RFA_00770	Corona_pk3 links to Rfam database...	-0.01	0.00	0.00	0	1996	24	0	20	4	18
RFA_00773	Corona_pk3 links to Rfam database...	0.93	1.00	0.86	18	1932	9	3	0	6	0
RFA_00808	RydC links to Rfam database...	0.83	1.00	0.70	16	1993	7	7	0	0	0
SRP_00200	Bacteria_small_SRP links to Rfam database...	0.41	0.47	0.35	17	6855	31	4	27	0	19
SRP_00285	Bacteria_small_SRP links to Rfam database...	0.38	0.43	0.35	13	3704	24	6	18	0	17
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.52	0.56	0.49	20	5419	22	4	17	1	16

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.