CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PPfold(20)		IPknot
MCC	0.746	>	0.540
Average MCC ± 95% Confidence Intervals	0.747 ± 0.127	>	0.543 ± 0.100
Sensitivity	0.681	>	0.464
Positive Predictive Value	0.818	>	0.628
Total TP	1989	>	1355
Total TN	7954946	<	7955221
Total FP	607	<	927
Total FP CONTRA	71	<	146
Total FP INCONS	371	<	655
Total FP COMP	165	>	126
Total FN	931	<	1565
P-value	0.0

Comparison of performance of PPfold(20) and IPknot. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PPfold(20) and IPknot).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PPfold(20) and IPknot).
Comparison of average Matthews Correlation Coefficients (MCCs) for PPfold(20) and IPknot. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PPfold(20) and IPknot).

Total Scores
MCC	0.746
Average MCC ± 95% Confidence Intervals	0.747 ± 0.127
Sensitivity	0.681
Positive Predictive Value	0.818
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.82	0.78	0.87	363	1192324	70	13	40	17	105
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.64	0.56	0.73	280	1627728	143	17	85	41	222
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.56	0.49	0.64	246	1674980	178	15	124	39	261
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.85	0.81	0.90	367	1119348	69	4	37	28	85
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.83	0.81	0.86	374	1135845	75	15	44	16	90
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.78	0.68	0.88	359	1204721	72	7	41	24	168

Total Scores
MCC	0.540
Average MCC ± 95% Confidence Intervals	0.543 ± 0.100
Sensitivity	0.464
Positive Predictive Value	0.628
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.56	0.44	0.71	207	1192448	104	18	67	19	261
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.53	0.43	0.66	216	1627783	139	27	84	28	286
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.39	0.34	0.43	174	1674963	268	35	193	40	333
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.53	0.48	0.59	217	1119386	165	27	126	12	235
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.68	0.62	0.75	289	1135893	113	17	79	17	175
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.56	0.48	0.66	252	1204748	138	22	106	10	275

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.