CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(20)		CMfinder(20)
MCC	0.681	>	0.521
Average MCC ± 95% Confidence Intervals	0.658 ± 0.133	>	0.514 ± 0.127
Sensitivity	0.564	>	0.366
Positive Predictive Value	0.826	>	0.748
Total TP	251	>	163
Total TN	67876	<	67962
Total FP	69	>	61
Total FP CONTRA	4	=	4
Total FP INCONS	49	<	51
Total FP COMP	16	>	6
Total FN	194	<	282
P-value	1.86109605532e-08

Comparison of performance of PETfold_pre2.0(20) and CMfinder(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(20) and CMfinder(20)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(20) and CMfinder(20)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(20) and CMfinder(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(20) and CMfinder(20)).

Total Scores
MCC	0.681
Average MCC ± 95% Confidence Intervals	0.658 ± 0.133
Sensitivity	0.564
Positive Predictive Value	0.826
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	0.84	0.70	1.00	21	2829	0	0	0	0	9
3J2L_3	5S_rRNA links to Rfam database...	0.78	0.62	0.97	33	7841	3	0	1	2	20
3J3D_C	tRNA links to Rfam database...	0.80	0.68	0.95	19	2755	1	0	1	0	9
3J3E_8	5_8S_rRNA links to Rfam database...	0.32	0.21	0.50	7	7489	11	0	7	4	26
3J3E_7	5S_rRNA links to Rfam database...	0.74	0.63	0.87	34	7101	5	1	4	0	20
3J3F_8	5_8S_rRNA links to Rfam database...	0.30	0.25	0.38	9	12222	22	0	15	7	27
3J3F_7	5S_rRNA links to Rfam database...	0.78	0.68	0.89	34	7222	5	1	3	1	16
3J3V_B	5S_rRNA links to Rfam database...	0.65	0.51	0.83	29	6986	6	1	5	0	28
3ZEX_D	5S_rRNA links to Rfam database...	0.81	0.71	0.92	35	6983	3	0	3	0	14
3ZND_W	tRNA links to Rfam database...	0.47	0.43	0.53	10	2984	11	0	9	2	13
4FRG_B	AdoCbl-variant links to Rfam database...	0.75	0.63	0.91	20	3464	2	1	1	0	12

Total Scores
MCC	0.521
Average MCC ± 95% Confidence Intervals	0.514 ± 0.127
Sensitivity	0.366
Positive Predictive Value	0.748
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	0.75	0.57	1.00	17	2833	1	0	0	1	13
3J2L_3	5S_rRNA links to Rfam database...	0.45	0.32	0.63	17	7848	11	1	9	1	36
3J3D_C	tRNA links to Rfam database...	0.76	0.61	0.94	17	2757	1	0	1	0	11
3J3E_8	5_8S_rRNA links to Rfam database...	0.10	0.06	0.17	2	7491	10	1	9	0	31
3J3E_7	5S_rRNA links to Rfam database...	0.62	0.46	0.83	25	7110	5	1	4	0	29
3J3F_8	5_8S_rRNA links to Rfam database...	0.45	0.33	0.60	12	12226	8	1	7	0	24
3J3F_7	5S_rRNA links to Rfam database...	0.66	0.52	0.84	26	7229	6	0	5	1	24
3J3V_B	5S_rRNA links to Rfam database...	0.38	0.21	0.71	12	7004	5	0	5	0	45
3ZEX_D	5S_rRNA links to Rfam database...	0.58	0.39	0.86	19	6999	3	0	3	0	30
3ZND_W	tRNA links to Rfam database...	0.45	0.39	0.53	9	2986	11	0	8	3	14
4FRG_B	AdoCbl-variant links to Rfam database...	0.47	0.22	1.00	7	3479	0	0	0	0	25

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.