CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PPfold(20)		RNAalifold(20)
MCC	0.746	>	0.676
Average MCC ± 95% Confidence Intervals	0.747 ± 0.127	>	0.677 ± 0.215
Sensitivity	0.681	>	0.618
Positive Predictive Value	0.818	>	0.739
Total TP	1989	>	1806
Total TN	7954946	>	7954934
Total FP	607	<	746
Total FP CONTRA	71	<	115
Total FP INCONS	371	<	522
Total FP COMP	165	>	109
Total FN	931	<	1114
P-value	0.0

Comparison of performance of PPfold(20) and RNAalifold(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PPfold(20) and RNAalifold(20)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PPfold(20) and RNAalifold(20)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PPfold(20) and RNAalifold(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PPfold(20) and RNAalifold(20)).

Total Scores
MCC	0.746
Average MCC ± 95% Confidence Intervals	0.747 ± 0.127
Sensitivity	0.681
Positive Predictive Value	0.818
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.82	0.78	0.87	363	1192324	70	13	40	17	105
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.64	0.56	0.73	280	1627728	143	17	85	41	222
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.56	0.49	0.64	246	1674980	178	15	124	39	261
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.85	0.81	0.90	367	1119348	69	4	37	28	85
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.83	0.81	0.86	374	1135845	75	15	44	16	90
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.78	0.68	0.88	359	1204721	72	7	41	24	168

Total Scores
MCC	0.676
Average MCC ± 95% Confidence Intervals	0.677 ± 0.215
Sensitivity	0.618
Positive Predictive Value	0.739
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.82	0.78	0.87	363	1192322	68	14	41	13	105
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.41	0.35	0.47	178	1627730	219	36	166	17	324
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.43	0.38	0.49	192	1674972	227	34	167	26	315
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.84	0.79	0.89	358	1119354	70	6	38	26	94
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.84	0.81	0.88	374	1135854	61	12	38	11	90
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.72	0.65	0.80	341	1204702	101	13	72	16	186

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.