CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(seed)		RNASampler(seed)
MCC	0.955	>	0.779
Average MCC ± 95% Confidence Intervals	0.955 ± 0.019	>	0.774 ± 0.092
Sensitivity	0.915	>	0.659
Positive Predictive Value	0.997	>	0.921
Total TP	614	>	442
Total TN	413626	<	413762
Total FP	78	<	105
Total FP CONTRA	0	<	9
Total FP INCONS	2	<	29
Total FP COMP	76	>	67
Total FN	57	<	229
P-value	6.70002765408e-09

Comparison of performance of PETfold_pre2.0(seed) and RNASampler(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and RNASampler(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and RNASampler(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(seed) and RNASampler(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and RNASampler(seed)).

Total Scores
MCC	0.955
Average MCC ± 95% Confidence Intervals	0.955 ± 0.019
Sensitivity	0.915
Positive Predictive Value	0.997
Nr of predictions	12

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00606	Telomerase-cil links to Rfam database...	0.91	0.82	1.00	32	21289	0	0	0	0	7
RFA_00620	Telomerase-cil links to Rfam database...	0.89	0.79	1.00	31	21914	0	0	0	0	8
RFA_00626	rne5 links to Rfam database...	0.96	0.92	1.00	80	56536	7	0	0	7	7
RFA_00627	rne5 links to Rfam database...	0.96	0.93	1.00	81	56872	12	0	0	12	6
RFA_00628	rne5 links to Rfam database...	0.96	0.93	1.00	80	57211	12	0	0	12	6
RFA_00630	rne5 links to Rfam database...	0.92	0.87	0.97	76	56875	17	0	2	15	11
RFA_00814	R2_retro_el links to Rfam database...	0.98	0.95	1.00	39	25161	5	0	0	5	2
RFA_00815	R2_retro_el links to Rfam database...	0.98	0.95	1.00	39	24492	6	0	0	6	2
RFA_00816	R2_retro_el links to Rfam database...	0.98	0.95	1.00	39	23181	6	0	0	6	2
RFA_00817	R2_retro_el links to Rfam database...	0.98	0.95	1.00	39	21906	4	0	0	4	2
RFA_00818	R2_retro_el links to Rfam database...	0.98	0.95	1.00	39	20262	3	0	0	3	2
RFA_00819	R2_retro_el links to Rfam database...	0.98	0.95	1.00	39	27927	6	0	0	6	2

Total Scores
MCC	0.779
Average MCC ± 95% Confidence Intervals	0.774 ± 0.092
Sensitivity	0.659
Positive Predictive Value	0.921
Nr of predictions	12

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00606	Telomerase-cil links to Rfam database...	0.44	0.36	0.54	14	21295	18	7	5	6	25
RFA_00620	Telomerase-cil links to Rfam database...	0.52	0.44	0.63	17	21918	15	2	8	5	22
RFA_00626	rne5 links to Rfam database...	0.80	0.64	1.00	56	56560	5	0	0	5	31
RFA_00627	rne5 links to Rfam database...	0.77	0.63	0.93	55	56894	8	0	4	4	32
RFA_00628	rne5 links to Rfam database...	0.78	0.64	0.95	55	57233	12	0	3	9	31
RFA_00630	rne5 links to Rfam database...	0.77	0.66	0.90	57	56890	15	0	6	9	30
RFA_00814	R2_retro_el links to Rfam database...	0.87	0.78	0.97	32	25167	6	0	1	5	9
RFA_00815	R2_retro_el links to Rfam database...	0.88	0.78	1.00	32	24499	8	0	0	8	9
RFA_00816	R2_retro_el links to Rfam database...	0.86	0.76	0.97	31	23188	6	0	1	5	10
RFA_00817	R2_retro_el links to Rfam database...	0.86	0.76	0.97	31	21913	1	0	1	0	10
RFA_00818	R2_retro_el links to Rfam database...	0.86	0.73	1.00	30	20271	6	0	0	6	11
RFA_00819	R2_retro_el links to Rfam database...	0.88	0.78	1.00	32	27934	5	0	0	5	9

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.