CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	Afold		Cylofold
MCC	0.559	>	0.535
Average MCC ± 95% Confidence Intervals	0.556 ± 0.121	<	0.581 ± 0.098
Sensitivity	0.497	>	0.451
Positive Predictive Value	0.634	<	0.641
Total TP	383	>	348
Total TN	110453	<	110514
Total FP	246	>	212
Total FP CONTRA	17	>	12
Total FP INCONS	204	>	183
Total FP COMP	25	>	17
Total FN	388	<	423
P-value	6.07491858776e-08

Comparison of performance of Afold and Cylofold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Afold and Cylofold).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Afold and Cylofold).
Comparison of average Matthews Correlation Coefficients (MCCs) for Afold and Cylofold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Afold and Cylofold).

Total Scores
MCC	0.559
Average MCC ± 95% Confidence Intervals	0.556 ± 0.121
Sensitivity	0.497
Positive Predictive Value	0.634
Nr of predictions	22

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KX8_A	7SK links to Rfam database...	0.94	0.89	1.00	16	845	0	0	0	0	2
2LC8_A	GP_knot1 links to Rfam database...	0.61	0.55	0.69	11	1524	6	0	5	1	9
2LKR_A	-	0.93	0.87	1.00	34	6071	3	0	0	3	5
2M58_A	-	0.33	0.29	0.38	5	1640	8	1	7	0	12
3ADB_C	-	0.78	0.74	0.82	28	4152	6	0	6	0	10
3AKZ_H	tRNA links to Rfam database...	0.16	0.14	0.19	4	2680	17	2	15	0	24
3AM1_B	-	0.72	0.66	0.79	23	3211	6	0	6	0	12
3IZF_C	5S_rRNA links to Rfam database...	0.66	0.57	0.76	31	6862	10	1	9	0	23
3J0L_7	-	-0.01	0.00	0.00	0	1210	15	1	14	0	17
3J2C_O	-	0.58	0.49	0.69	31	10251	15	1	13	1	32
3NDB_M	-	0.84	0.74	0.96	45	9133	3	0	2	1	16
3NKB_B	-	0.59	0.54	0.67	14	1995	7	0	7	0	12
3O58_3	5_8S_rRNA links to Rfam database...	0.34	0.34	0.34	12	12368	37	2	21	14	23
3O58_2	5S_rRNA links to Rfam database...	0.66	0.66	0.66	25	7222	14	4	9	1	13
3RKF_A	Purine links to Rfam database...	0.76	0.59	1.00	20	2191	0	0	0	0	14
3U4M_B	-	0.38	0.32	0.46	12	3134	14	0	14	0	25
3W3S_B	tRNA-Sec links to Rfam database...	0.50	0.45	0.56	18	4721	15	1	13	1	22
3ZEX_D	5S_rRNA links to Rfam database...	0.72	0.61	0.86	30	6986	5	0	5	0	19
3ZEX_E	-	0.00	0.00	0.00	0	21892	55	2	51	2	77
4ATO_G	-	0.32	0.30	0.38	3	520	6	1	4	1	7
4ENB_A	crcB links to Rfam database...	0.67	0.58	0.79	11	1261	3	1	2	0	8
4FNJ_A	-	0.75	0.63	0.91	10	584	1	0	1	0	6

Total Scores
MCC	0.535
Average MCC ± 95% Confidence Intervals	0.581 ± 0.098
Sensitivity	0.451
Positive Predictive Value	0.641
Nr of predictions	22

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KX8_A	7SK links to Rfam database...	0.94	0.89	1.00	16	845	0	0	0	0	2
2LC8_A	GP_knot1 links to Rfam database...	0.61	0.55	0.69	11	1524	5	0	5	0	9
2LKR_A	-	0.50	0.44	0.59	17	6076	12	2	10	0	22
2M58_A	-	0.51	0.41	0.64	7	1642	4	1	3	0	10
3ADB_C	-	0.78	0.71	0.87	27	4155	4	0	4	0	11
3AKZ_H	tRNA links to Rfam database...	0.66	0.57	0.76	16	2680	6	0	5	1	12
3AM1_B	-	0.76	0.63	0.92	22	3216	2	0	2	0	13
3IZF_C	5S_rRNA links to Rfam database...	0.68	0.56	0.83	30	6867	6	0	6	0	24
3J0L_7	-	0.30	0.29	0.33	5	1210	10	0	10	0	12
3J2C_O	-	0.43	0.33	0.55	21	10258	18	0	17	1	42
3NDB_M	-	0.58	0.48	0.71	29	9139	13	1	11	1	32
3NKB_B	-	0.40	0.31	0.53	8	2001	7	0	7	0	18
3O58_3	5_8S_rRNA links to Rfam database...	0.36	0.34	0.38	12	12371	29	5	15	9	23
3O58_2	5S_rRNA links to Rfam database...	0.80	0.68	0.93	26	7232	3	0	2	1	12
3RKF_A	Purine links to Rfam database...	0.76	0.59	1.00	20	2191	0	0	0	0	14
3U4M_B	-	0.38	0.32	0.46	12	3134	14	0	14	0	25
3W3S_B	tRNA-Sec links to Rfam database...	0.44	0.38	0.52	15	4724	15	0	14	1	25
3ZEX_D	5S_rRNA links to Rfam database...	0.58	0.49	0.69	24	6986	11	0	11	0	25
3ZEX_E	-	0.00	0.00	0.00	0	21897	51	2	46	3	77
4ATO_G	-	0.73	0.70	0.78	7	519	2	1	1	0	3
4ENB_A	crcB links to Rfam database...	0.89	0.79	1.00	15	1260	0	0	0	0	4
4FNJ_A	-	0.70	0.50	1.00	8	587	0	0	0	0	8

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.