CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(20)		McQFold
MCC	0.622	>	0.541
Average MCC ± 95% Confidence Intervals	0.620 ± 0.196	>	0.586 ± 0.263
Sensitivity	0.597	>	0.592
Positive Predictive Value	0.654	>	0.502
Total TP	123	>	122
Total TN	22999	>	22944
Total FP	94	<	159
Total FP CONTRA	14	<	45
Total FP INCONS	51	<	76
Total FP COMP	29	<	38
Total FN	83	<	84
P-value	5.40200680498e-09

Comparison of performance of PETfold_pre2.0(20) and McQFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(20) and McQFold).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(20) and McQFold).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(20) and McQFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(20) and McQFold).

Total Scores
MCC	0.622
Average MCC ± 95% Confidence Intervals	0.620 ± 0.196
Sensitivity	0.597
Positive Predictive Value	0.654
Nr of predictions	10

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	0.95	0.95	0.95	20	1198	1	1	0	0	1
3J3D_C	tRNA links to Rfam database...	0.87	0.89	0.85	17	948	3	3	0	0	2
3J3E_8	5_8S_rRNA links to Rfam database...	0.48	0.47	0.50	7	2728	11	2	5	4	8
3J3F_8	5_8S_rRNA links to Rfam database...	0.34	0.37	0.32	7	4739	24	5	10	9	12
3ZEX_C	5_8S_rRNA links to Rfam database...	0.26	0.24	0.29	7	5350	20	1	16	3	22
4A1C_2	5_8S_rRNA links to Rfam database...	0.25	0.25	0.25	5	4496	25	2	13	10	15
4AOB_A	SAM links to Rfam database...	0.87	0.83	0.92	24	1411	4	0	2	2	5
4ENB_A	crcB links to Rfam database...	0.68	0.53	0.89	8	463	1	0	1	0	7
4ENC_A	crcB links to Rfam database...	0.61	0.53	0.73	8	485	3	0	3	0	7
4FRG_B	AdoCbl-variant links to Rfam database...	0.89	0.83	0.95	20	1181	2	0	1	1	4

Total Scores
MCC	0.541
Average MCC ± 95% Confidence Intervals	0.586 ± 0.263
Sensitivity	0.592
Positive Predictive Value	0.502
Nr of predictions	10

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	0.74	0.76	0.73	16	1197	7	3	3	1	5
3J3D_C	tRNA links to Rfam database...	0.90	0.95	0.86	18	947	3	3	0	0	1
3J3E_8	5_8S_rRNA links to Rfam database...	-0.01	0.00	0.00	0	2718	36	8	16	12	15
3J3F_8	5_8S_rRNA links to Rfam database...	0.31	0.42	0.23	8	4726	41	12	15	14	11
3ZEX_C	5_8S_rRNA links to Rfam database...	0.33	0.34	0.32	10	5343	25	4	17	4	19
4A1C_2	5_8S_rRNA links to Rfam database...	0.19	0.25	0.16	5	4484	33	11	16	6	15
4AOB_A	SAM links to Rfam database...	0.50	0.48	0.54	14	1411	13	3	9	1	15
4ENB_A	crcB links to Rfam database...	1.00	1.00	1.00	15	457	0	0	0	0	0
4ENC_A	crcB links to Rfam database...	0.97	1.00	0.94	15	480	1	1	0	0	0
4FRG_B	AdoCbl-variant links to Rfam database...	0.93	0.88	1.00	21	1181	0	0	0	0	3

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.