CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	Vsfold5		Carnac(seed)
MCC	0.289	>	0.000
Average MCC ± 95% Confidence Intervals	0.282 ± 0.040	>	0.000 ± 0.000
Sensitivity	0.276	>	0.000
Positive Predictive Value	0.306	>	0.000
Total TP	896	>	0
Total TN	1853404	<	1856331
Total FP	2136	>	0
Total FP CONTRA	254	>	0
Total FP INCONS	1777	>	0
Total FP COMP	105	>	0
Total FN	2352	<	3248
P-value	5.23657817852e-08

^top

Performance plots

Comparison of performance of Vsfold5 and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Vsfold5 and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Vsfold5 and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for Vsfold5 and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Vsfold5 and Carnac(seed)).

^top

Performance of Vsfold5 - scored higher in this pairwise comparison

1. Total counts & total scores for Vsfold5

Total Base Pair Counts
Total TP	896
Total TN	1853404
Total FP	2136
Total FP CONTRA	254
Total FP INCONS	1777
Total FP COMP	105
Total FN	2352

Total Scores
MCC	0.289
Average MCC ± 95% Confidence Intervals	0.282 ± 0.040
Sensitivity	0.276
Positive Predictive Value	0.306
Nr of predictions	35

^top

2. Individual counts for Vsfold5 [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.40	0.37	0.44	28	34127	39	6	30	3	47
ASE_00002	RNaseP_arch links to Rfam database...	0.48	0.42	0.53	31	35453	31	4	23	4	42
ASE_00009	RNaseP_arch links to Rfam database...	0.20	0.19	0.20	13	26042	52	0	51	1	54
ASE_00014	RNaseP_arch links to Rfam database...	0.44	0.44	0.45	47	54180	59	8	50	1	59
ASE_00116	RNaseP_arch links to Rfam database...	0.16	0.15	0.17	10	31566	51	5	45	1	56
ASE_00120	RNaseP_arch links to Rfam database...	0.20	0.19	0.21	18	46885	68	5	63	0	76
ASE_00121	RNaseP_arch links to Rfam database...	0.13	0.13	0.14	12	45065	75	9	64	2	80
ASE_00145	RNaseP_arch links to Rfam database...	0.31	0.29	0.33	32	70027	67	6	60	1	79
ASE_00154	RNaseP_arch links to Rfam database...	0.19	0.18	0.22	19	65253	70	4	65	1	88
ASE_00155	RNaseP_arch links to Rfam database...	0.31	0.30	0.34	42	93403	86	7	76	3	100
ASE_00187	RNaseP_arch links to Rfam database...	0.30	0.30	0.30	34	68523	78	11	67	0	81
ASE_00196	RNaseP_arch links to Rfam database...	0.30	0.28	0.32	21	31561	48	4	40	4	54
ASE_00205	RNaseP_arch links to Rfam database...	0.24	0.24	0.25	22	45967	68	10	57	1	68
ASE_00209	RNaseP_arch links to Rfam database...	0.16	0.15	0.17	13	42993	65	5	60	0	75
ASE_00210	RNaseP_arch links to Rfam database...	0.20	0.20	0.20	13	27197	53	8	43	2	52
ASE_00213	RNaseP_arch links to Rfam database...	0.22	0.21	0.23	14	27200	49	10	37	2	52
ASE_00346	RNaseP_arch links to Rfam database...	0.22	0.21	0.24	20	48120	66	3	62	1	77
ASE_00359	RNaseP_arch links to Rfam database...	0.18	0.19	0.18	15	42695	69	10	58	1	65
ASE_00360	RNaseP_arch links to Rfam database...	0.22	0.22	0.23	14	29343	46	10	36	0	51
ASE_00417	RNaseP_arch links to Rfam database...	0.37	0.37	0.37	37	54514	65	13	51	1	63
ASE_00418	RNaseP_arch links to Rfam database...	0.22	0.21	0.23	16	38712	58	1	52	5	59
CRW_00016	Intron_gpI links to Rfam database...	0.40	0.38	0.42	45	77315	67	4	57	6	75
CRW_00610	Intron_gpI links to Rfam database...	0.29	0.28	0.30	23	36238	57	6	48	3	58
CRW_00613	Intron_gpI links to Rfam database...	0.48	0.46	0.51	36	34909	41	2	33	6	42
CRW_00618	Intron_gpI links to Rfam database...	0.24	0.26	0.23	16	56210	76	9	45	22	45
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.32	0.33	0.32	31	64523	75	10	56	9	63
CRW_00670	Intron_gpI links to Rfam database...	0.40	0.38	0.43	45	70020	61	3	57	1	75
CRW_00671	Intron_gpI links to Rfam database...	0.29	0.26	0.33	31	62740	64	2	62	0	86
CRW_00672	Intron_gpI links to Rfam database...	0.39	0.40	0.39	44	72278	70	14	54	2	67
CRW_00674	Intron_gpI links to Rfam database...	0.33	0.31	0.36	38	83330	74	5	63	6	86
CRW_00676	Intron_gpI links to Rfam database...	0.34	0.36	0.33	41	93405	88	19	63	6	74
CRW_00692	Intron_gpI links to Rfam database...	0.36	0.38	0.34	34	68534	67	21	46	0	56
PDB_00827	Intron_gpI links to Rfam database...	0.51	0.42	0.62	34	26973	22	0	21	1	47
RFA_00598	Telomerase-vert links to Rfam database...	0.07	0.07	0.06	7	84557	111	20	82	9	94

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	0
Total TN	1856331
Total FP	0
Total FP CONTRA	0
Total FP INCONS	0
Total FP COMP	0
Total FN	3248

Total Scores
MCC	0.000
Average MCC ± 95% Confidence Intervals	0.000 ± 0.000
Sensitivity	0.000
Positive Predictive Value	0.000
Nr of predictions	35

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00155	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	142
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00205	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	90
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00360	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	29403	0	0	0	0	65
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00618	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	56280	0	0	0	0	61
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	64620	0	0	0	0	94
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
RFA_00598	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	84666	0	0	0	0	101

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of Vsfold5 - scored higher in this pairwise comparison

1. Total counts & total scores for Vsfold5

2. Individual counts for Vsfold5 [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]