CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	Multilign(seed)		Afold
MCC	0.770	>	0.626
Average MCC ± 95% Confidence Intervals	0.795 ± 0.175	>	0.598 ± 0.211
Sensitivity	0.761	>	0.648
Positive Predictive Value	0.779	>	0.605
Total TP	385	>	328
Total TN	301271	>	301223
Total FP	185	<	293
Total FP CONTRA	23	<	63
Total FP INCONS	86	<	151
Total FP COMP	76	<	79
Total FN	121	<	178
P-value	0.0

Comparison of performance of Multilign(seed) and Afold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Multilign(seed) and Afold).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Multilign(seed) and Afold).
Comparison of average Matthews Correlation Coefficients (MCCs) for Multilign(seed) and Afold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Multilign(seed) and Afold).

Total Scores
MCC	0.770
Average MCC ± 95% Confidence Intervals	0.795 ± 0.175
Sensitivity	0.761
Positive Predictive Value	0.779
Nr of predictions	7

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00626	rne5 links to Rfam database...	0.90	0.85	0.95	74	56538	20	0	4	16	13
RFA_00628	rne5 links to Rfam database...	0.92	0.88	0.95	76	57211	18	0	4	14	10
RFA_00630	rne5 links to Rfam database...	0.83	0.80	0.86	70	56872	29	0	11	18	17
RFA_00816	R2_retro_el links to Rfam database...	1.00	1.00	1.00	41	23179	12	0	0	12	0
RFA_00818	R2_retro_el links to Rfam database...	0.85	0.80	0.89	33	20264	12	1	3	8	8
SRP_00241	Plant_SRP links to Rfam database...	0.48	0.50	0.46	41	45967	50	15	33	2	41
SRP_00340	Metazoa_SRP links to Rfam database...	0.59	0.61	0.57	50	41240	44	7	31	6	32

Total Scores
MCC	0.626
Average MCC ± 95% Confidence Intervals	0.598 ± 0.211
Sensitivity	0.648
Positive Predictive Value	0.605
Nr of predictions	7

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00626	rne5 links to Rfam database...	0.89	0.86	0.93	75	56535	31	2	4	25	12
RFA_00628	rne5 links to Rfam database...	0.83	0.80	0.85	69	57210	38	1	11	26	17
RFA_00630	rne5 links to Rfam database...	0.60	0.60	0.60	52	56866	46	8	27	11	35
RFA_00816	R2_retro_el links to Rfam database...	0.59	0.66	0.53	27	23169	33	13	11	9	14
RFA_00818	R2_retro_el links to Rfam database...	0.19	0.22	0.17	9	20247	48	19	26	3	32
SRP_00241	Plant_SRP links to Rfam database...	0.54	0.59	0.51	48	45961	50	11	36	3	34
SRP_00340	Metazoa_SRP links to Rfam database...	0.55	0.59	0.52	48	41235	47	9	36	2	34

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.