CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	Mastr(20)		Alterna
MCC	0.701	>	0.553
Average MCC ± 95% Confidence Intervals	0.722 ± 0.105	>	0.583 ± 0.168
Sensitivity	0.570	>	0.480
Positive Predictive Value	0.868	>	0.646
Total TP	184	>	155
Total TN	30002	>	29974
Total FP	34	<	93
Total FP CONTRA	7	<	11
Total FP INCONS	21	<	74
Total FP COMP	6	<	8
Total FN	139	<	168
P-value	1.34193807955e-08

Comparison of performance of Mastr(20) and Alterna. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Mastr(20) and Alterna).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Mastr(20) and Alterna).
Comparison of average Matthews Correlation Coefficients (MCCs) for Mastr(20) and Alterna. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Mastr(20) and Alterna).

Total Scores
MCC	0.701
Average MCC ± 95% Confidence Intervals	0.722 ± 0.105
Sensitivity	0.570
Positive Predictive Value	0.868
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KDQ_B	mir-TAR links to Rfam database...	0.95	0.91	1.00	10	396	0	0	0	0	1
2WRQ_Y	tRNA links to Rfam database...	0.57	0.59	0.56	10	2832	12	5	3	4	7
2XQD_Y	tRNA links to Rfam database...	0.88	0.78	1.00	21	2829	0	0	0	0	6
3A2K_C	tRNA links to Rfam database...	0.72	0.61	0.85	17	2906	3	0	3	0	11
3AMU_B	tRNA links to Rfam database...	0.77	0.59	1.00	16	2987	1	0	0	1	11
3GX2_A	SAM links to Rfam database...	0.39	0.28	0.55	11	4351	10	0	9	1	29
3IVN_B	Purine links to Rfam database...	0.78	0.65	0.95	20	2325	1	1	0	0	11
3JYV_7	tRNA links to Rfam database...	0.81	0.66	1.00	21	2829	0	0	0	0	11
3LA5_A	Purine links to Rfam database...	0.76	0.59	1.00	20	2465	0	0	0	0	14
3RKF_A	Purine links to Rfam database...	0.70	0.50	1.00	17	2194	0	0	0	0	17
3SD1_A	THF links to Rfam database...	0.61	0.50	0.75	21	3888	7	1	6	0	21

Total Scores
MCC	0.553
Average MCC ± 95% Confidence Intervals	0.583 ± 0.168
Sensitivity	0.480
Positive Predictive Value	0.646
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KDQ_B	mir-TAR links to Rfam database...	0.95	0.91	1.00	10	396	0	0	0	0	1
2WRQ_Y	tRNA links to Rfam database...	0.57	0.59	0.56	10	2832	13	5	3	5	7
2XQD_Y	tRNA links to Rfam database...	0.67	0.59	0.76	16	2829	5	0	5	0	11
3A2K_C	tRNA links to Rfam database...	0.44	0.43	0.46	12	2900	14	2	12	0	16
3AMU_B	tRNA links to Rfam database...	0.64	0.59	0.70	16	2980	9	0	7	2	11
3GX2_A	SAM links to Rfam database...	0.47	0.40	0.57	16	4343	13	1	11	1	24
3IVN_B	Purine links to Rfam database...	0.74	0.61	0.90	19	2325	2	1	1	0	12
3JYV_7	tRNA links to Rfam database...	-0.01	0.00	0.00	0	2828	22	0	22	0	32
3LA5_A	Purine links to Rfam database...	0.78	0.62	1.00	21	2464	0	0	0	0	13
3RKF_A	Purine links to Rfam database...	0.69	0.56	0.86	19	2189	3	1	2	0	15
3SD1_A	THF links to Rfam database...	0.46	0.38	0.57	16	3888	12	1	11	0	26

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.