CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	IPknot		Afold
MCC	0.386	>	0.348
Average MCC ± 95% Confidence Intervals	0.454 ± 0.157	>	0.398 ± 0.140
Sensitivity	0.290	<	0.312
Positive Predictive Value	0.514	>	0.389
Total TP	329	<	354
Total TN	1166544	>	1166275
Total FP	330	<	599
Total FP CONTRA	25	<	58
Total FP INCONS	286	<	497
Total FP COMP	19	<	44
Total FN	806	>	781
P-value	3.45076534967e-08

Comparison of performance of IPknot and Afold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for IPknot and Afold).
Comparison of average Matthews Correlation Coefficients (MCCs) for IPknot and Afold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for IPknot and Afold).

Total Scores
MCC	0.386
Average MCC ± 95% Confidence Intervals	0.454 ± 0.157
Sensitivity	0.290
Positive Predictive Value	0.514
Nr of predictions	19

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2LC8_A	GP_knot1 links to Rfam database...	0.40	0.30	0.55	6	1529	5	1	4	0	14
2LKR_A	-	0.84	0.77	0.91	30	6072	4	0	3	1	9
2M58_A	-	0.60	0.41	0.88	7	1645	1	1	0	0	10
3J0L_7	-	-0.01	0.00	0.00	0	1218	7	0	7	0	17
3J2C_O	-	0.82	0.67	1.00	42	10254	1	0	0	1	21
3J3E_8	5_8S_rRNA links to Rfam database...	0.00	0.00	0.00	0	7484	25	2	17	6	33
3RKF_A	Purine links to Rfam database...	0.76	0.59	1.00	20	2191	0	0	0	0	14
3U4M_B	-	0.68	0.54	0.87	20	3137	3	1	2	0	17
3W3S_B	tRNA-Sec links to Rfam database...	0.85	0.75	0.97	30	4722	2	0	1	1	10
3ZEX_B	-	0.27	0.18	0.40	101	1072127	154	6	146	2	457
3ZEX_E	-	0.00	0.00	0.00	0	21894	53	4	47	2	77
3ZEX_D	5S_rRNA links to Rfam database...	0.72	0.63	0.82	31	6983	7	0	7	0	18
3ZEX_H	-	0.20	0.18	0.22	7	9013	25	4	21	0	31
3ZEX_F	-	0.00	0.00	0.00	0	2619	9	2	7	0	12
4A1C_2	5_8S_rRNA links to Rfam database...	0.17	0.15	0.19	5	11755	26	4	17	5	28
4ATO_G	-	0.41	0.40	0.44	4	519	6	0	5	1	6
4ENB_A	crcB links to Rfam database...	0.76	0.58	1.00	11	1264	0	0	0	0	8
4FNJ_A	-	0.79	0.63	1.00	10	585	0	0	0	0	6
4JRC_A	-	0.39	0.22	0.71	5	1533	2	0	2	0	18

Total Scores
MCC	0.348
Average MCC ± 95% Confidence Intervals	0.398 ± 0.140
Sensitivity	0.312
Positive Predictive Value	0.389
Nr of predictions	19

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2LC8_A	GP_knot1 links to Rfam database...	0.61	0.55	0.69	11	1524	6	0	5	1	9
2LKR_A	-	0.93	0.87	1.00	34	6071	3	0	0	3	5
2M58_A	-	0.33	0.29	0.38	5	1640	8	1	7	0	12
3J0L_7	-	-0.01	0.00	0.00	0	1210	15	1	14	0	17
3J2C_O	-	0.58	0.49	0.69	31	10251	15	1	13	1	32
3J3E_8	5_8S_rRNA links to Rfam database...	0.17	0.15	0.19	5	7477	31	2	19	10	28
3RKF_A	Purine links to Rfam database...	0.76	0.59	1.00	20	2191	0	0	0	0	14
3U4M_B	-	0.38	0.32	0.46	12	3134	14	0	14	0	25
3W3S_B	tRNA-Sec links to Rfam database...	0.50	0.45	0.56	18	4721	15	1	13	1	22
3ZEX_B	-	0.29	0.26	0.32	147	1071922	322	35	276	11	411
3ZEX_E	-	0.00	0.00	0.00	0	21892	55	2	51	2	77
3ZEX_D	5S_rRNA links to Rfam database...	0.72	0.61	0.86	30	6986	5	0	5	0	19
3ZEX_H	-	0.18	0.18	0.18	7	9006	33	6	26	1	31
3ZEX_F	-	0.00	0.00	0.00	0	2617	12	2	9	1	12
4A1C_2	5_8S_rRNA links to Rfam database...	0.14	0.15	0.14	5	11745	43	5	26	12	28
4ATO_G	-	0.32	0.30	0.38	3	520	6	1	4	1	7
4ENB_A	crcB links to Rfam database...	0.67	0.58	0.79	11	1261	3	1	2	0	8
4FNJ_A	-	0.75	0.63	0.91	10	584	1	0	1	0	6
4JRC_A	-	0.24	0.22	0.29	5	1523	12	0	12	0	18

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.