CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(seed)		CMfinder(seed)
MCC	0.413	>	0.350
Average MCC ± 95% Confidence Intervals	0.414 ± 0.052	>	0.358 ± 0.060
Sensitivity	0.264	>	0.159
Positive Predictive Value	0.648	<	0.770
Total TP	388	>	234
Total TN	1036051	<	1036346
Total FP	254	>	77
Total FP CONTRA	17	>	3
Total FP INCONS	194	>	67
Total FP COMP	43	>	7
Total FN	1082	<	1236
P-value	2.34821445183e-08

Comparison of performance of PETfold_pre2.0(seed) and CMfinder(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and CMfinder(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and CMfinder(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(seed) and CMfinder(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and CMfinder(seed)).

Total Scores
MCC	0.413
Average MCC ± 95% Confidence Intervals	0.414 ± 0.052
Sensitivity	0.264
Positive Predictive Value	0.648
Nr of predictions	15

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00013	Intron_gpI links to Rfam database...	0.41	0.25	0.66	29	103696	18	0	15	3	86
CRW_00016	Intron_gpI links to Rfam database...	0.40	0.24	0.66	29	77377	17	0	15	2	91
CRW_00610	Intron_gpI links to Rfam database...	0.54	0.38	0.76	31	36274	12	1	9	2	50
CRW_00613	Intron_gpI links to Rfam database...	0.40	0.29	0.55	23	34938	22	1	18	3	55
CRW_00614	Intron_gpI links to Rfam database...	0.49	0.39	0.63	22	121736	16	2	11	3	35
CRW_00618	Intron_gpI links to Rfam database...	0.38	0.28	0.52	17	56247	16	2	14	0	44
CRW_00633	Intron_gpI links to Rfam database...	0.38	0.24	0.61	25	63505	16	2	14	0	81
CRW_00634	Intron_gpI links to Rfam database...	0.31	0.19	0.50	18	64584	24	1	17	6	76
CRW_00670	Intron_gpI links to Rfam database...	0.51	0.31	0.84	37	70081	8	0	7	1	83
CRW_00671	Intron_gpI links to Rfam database...	0.52	0.32	0.84	37	62791	8	0	7	1	80
CRW_00672	Intron_gpI links to Rfam database...	0.52	0.32	0.84	36	72347	10	0	7	3	75
CRW_00674	Intron_gpI links to Rfam database...	0.25	0.15	0.45	18	83396	25	6	16	3	106
CRW_00676	Intron_gpI links to Rfam database...	0.29	0.16	0.53	18	93494	24	0	16	8	97
CRW_00692	Intron_gpI links to Rfam database...	0.32	0.20	0.53	18	68601	24	0	16	8	72
PDB_00827	Intron_gpI links to Rfam database...	0.50	0.37	0.68	30	26984	14	2	12	0	51

Total Scores
MCC	0.350
Average MCC ± 95% Confidence Intervals	0.358 ± 0.060
Sensitivity	0.159
Positive Predictive Value	0.770
Nr of predictions	15

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00013	Intron_gpI links to Rfam database...	0.37	0.14	1.00	16	103724	0	0	0	0	99
CRW_00016	Intron_gpI links to Rfam database...	0.36	0.14	0.89	17	77402	2	0	2	0	103
CRW_00610	Intron_gpI links to Rfam database...	0.49	0.31	0.78	25	36283	7	1	6	0	56
CRW_00613	Intron_gpI links to Rfam database...	0.54	0.33	0.87	26	34950	4	0	4	0	52
CRW_00614	Intron_gpI links to Rfam database...	0.46	0.25	0.88	14	121755	9	0	2	7	43
CRW_00618	Intron_gpI links to Rfam database...	0.36	0.13	1.00	8	56272	0	0	0	0	53
CRW_00633	Intron_gpI links to Rfam database...	0.32	0.12	0.81	13	63530	3	1	2	0	93
CRW_00634	Intron_gpI links to Rfam database...	0.40	0.17	0.94	16	64603	1	0	1	0	78
CRW_00670	Intron_gpI links to Rfam database...	0.32	0.18	0.55	22	70085	18	0	18	0	98
CRW_00671	Intron_gpI links to Rfam database...	0.20	0.09	0.41	11	62808	16	0	16	0	106
CRW_00672	Intron_gpI links to Rfam database...	0.38	0.15	0.94	17	72372	1	0	1	0	94
CRW_00674	Intron_gpI links to Rfam database...	0.40	0.17	0.95	21	83414	1	0	1	0	103
CRW_00676	Intron_gpI links to Rfam database...	0.12	0.04	0.33	5	93513	10	1	9	0	110
CRW_00692	Intron_gpI links to Rfam database...	0.41	0.17	1.00	15	68620	0	0	0	0	75
PDB_00827	Intron_gpI links to Rfam database...	0.25	0.10	0.62	8	27015	5	0	5	0	73

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.