CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	Fold		Alterna
MCC	0.624	>	0.543
Average MCC ± 95% Confidence Intervals	0.547 ± 0.170	>	0.507 ± 0.113
Sensitivity	0.538	>	0.478
Positive Predictive Value	0.732	>	0.628
Total TP	240	>	213
Total TN	38915	>	38904
Total FP	91	<	130
Total FP CONTRA	8	<	15
Total FP INCONS	80	<	111
Total FP COMP	3	<	4
Total FN	206	<	233
P-value	3.02344513449e-08

Comparison of performance of Fold and Alterna. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Fold and Alterna).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Fold and Alterna).
Comparison of average Matthews Correlation Coefficients (MCCs) for Fold and Alterna. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Fold and Alterna).

Total Scores
MCC	0.624
Average MCC ± 95% Confidence Intervals	0.547 ± 0.170
Sensitivity	0.538
Positive Predictive Value	0.732
Nr of predictions	17

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KX8_A	7SK links to Rfam database...	0.94	0.89	1.00	16	845	0	0	0	0	2
2LA5_A	-	0.39	0.26	0.63	5	622	3	1	2	0	14
2LC8_A	GP_knot1 links to Rfam database...	-0.01	0.00	0.00	0	1525	15	2	13	0	20
2WWQ_V	tRNA links to Rfam database...	0.76	0.64	0.90	18	2906	3	0	2	1	10
2XQD_Y	tRNA links to Rfam database...	0.88	0.78	1.00	21	2829	0	0	0	0	6
3ADB_C	-	0.80	0.74	0.88	28	4154	4	0	4	0	10
3AKZ_H	tRNA links to Rfam database...	0.84	0.75	0.95	21	2679	1	1	0	0	7
3AM1_B	-	0.78	0.71	0.86	25	3211	4	0	4	0	10
3AMU_B	tRNA links to Rfam database...	0.64	0.59	0.70	16	2980	9	0	7	2	11
3J0L_7	-	-0.01	0.00	0.00	0	1213	12	0	12	0	17
3J0L_a	-	0.20	0.19	0.23	3	1115	10	2	8	0	13
3J0L_g	-	-0.01	0.00	0.00	0	461	4	1	3	0	4
3J16_L	tRNA links to Rfam database...	0.63	0.53	0.76	16	2754	5	0	5	0	14
3NKB_B	-	0.35	0.31	0.42	8	1997	11	0	11	0	18
3RKF_A	Purine links to Rfam database...	0.76	0.59	1.00	20	2191	0	0	0	0	14
3SD1_A	THF links to Rfam database...	0.57	0.48	0.69	20	3887	9	1	8	0	22
3UZL_B	tRNA links to Rfam database...	0.77	0.62	0.96	23	3546	1	0	1	0	14

Total Scores
MCC	0.543
Average MCC ± 95% Confidence Intervals	0.507 ± 0.113
Sensitivity	0.478
Positive Predictive Value	0.628
Nr of predictions	17

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KX8_A	7SK links to Rfam database...	0.88	0.83	0.94	15	845	1	0	1	0	3
2LA5_A	-	0.46	0.26	0.83	5	624	1	0	1	0	14
2LC8_A	GP_knot1 links to Rfam database...	0.46	0.40	0.53	8	1525	7	0	7	0	12
2WWQ_V	tRNA links to Rfam database...	0.76	0.68	0.86	19	2904	5	0	3	2	9
2XQD_Y	tRNA links to Rfam database...	0.67	0.59	0.76	16	2829	5	0	5	0	11
3ADB_C	-	0.59	0.55	0.64	21	4153	12	0	12	0	17
3AKZ_H	tRNA links to Rfam database...	0.40	0.39	0.42	11	2675	15	4	11	0	17
3AM1_B	-	0.61	0.57	0.67	20	3210	10	1	9	0	15
3AMU_B	tRNA links to Rfam database...	0.64	0.59	0.70	16	2980	9	0	7	2	11
3J0L_7	-	0.26	0.24	0.31	4	1212	9	0	9	0	13
3J0L_a	-	0.23	0.19	0.30	3	1118	7	1	6	0	13
3J0L_g	-	-0.01	0.00	0.00	0	456	9	4	5	0	4
3J16_L	tRNA links to Rfam database...	0.63	0.53	0.76	16	2754	5	1	4	0	14
3NKB_B	-	0.35	0.31	0.42	8	1997	11	0	11	0	18
3RKF_A	Purine links to Rfam database...	0.69	0.56	0.86	19	2189	3	1	2	0	15
3SD1_A	THF links to Rfam database...	0.46	0.38	0.57	16	3888	12	1	11	0	26
3UZL_B	tRNA links to Rfam database...	0.52	0.43	0.64	16	3545	9	2	7	0	21

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.