CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	RNAshapes		Carnac(seed)
MCC	0.467	>	0.169
Average MCC ± 95% Confidence Intervals	0.460 ± 0.055	>	0.126 ± 0.083
Sensitivity	0.458	>	0.030
Positive Predictive Value	0.478	<	0.962
Total TP	1552	>	101
Total TN	1797277	<	1800416
Total FP	1881	>	5
Total FP CONTRA	214	>	3
Total FP INCONS	1478	>	1
Total FP COMP	189	>	1
Total FN	1839	<	3290
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of RNAshapes and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNAshapes and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNAshapes and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for RNAshapes and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNAshapes and Carnac(seed)).

^top

Performance of RNAshapes - scored higher in this pairwise comparison

1. Total counts & total scores for RNAshapes

Total Base Pair Counts
Total TP	1552
Total TN	1797277
Total FP	1881
Total FP CONTRA	214
Total FP INCONS	1478
Total FP COMP	189
Total FN	1839

Total Scores
MCC	0.467
Average MCC ± 95% Confidence Intervals	0.460 ± 0.055
Sensitivity	0.458
Positive Predictive Value	0.478
Nr of predictions	47

^top

2. Individual counts for RNAshapes [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.65	0.61	0.69	46	34124	28	3	18	7	29
ASE_00002	RNaseP_arch links to Rfam database...	0.47	0.45	0.49	33	35444	39	4	30	5	40
ASE_00009	RNaseP_arch links to Rfam database...	0.40	0.37	0.43	25	26048	35	2	31	2	42
ASE_00014	RNaseP_arch links to Rfam database...	0.12	0.12	0.13	13	54184	89	8	80	1	93
ASE_00116	RNaseP_arch links to Rfam database...	0.14	0.14	0.15	9	31567	53	5	45	3	57
ASE_00120	RNaseP_arch links to Rfam database...	0.30	0.29	0.31	27	46884	60	9	51	0	67
ASE_00121	RNaseP_arch links to Rfam database...	0.32	0.32	0.33	29	45063	60	8	50	2	63
ASE_00145	RNaseP_arch links to Rfam database...	0.36	0.35	0.38	39	70021	70	7	58	5	72
ASE_00154	RNaseP_arch links to Rfam database...	0.54	0.52	0.55	56	65239	49	6	40	3	51
ASE_00187	RNaseP_arch links to Rfam database...	0.41	0.40	0.43	46	68528	66	5	56	5	69
ASE_00196	RNaseP_arch links to Rfam database...	0.49	0.49	0.49	37	31551	41	2	36	3	38
ASE_00205	RNaseP_arch links to Rfam database...	0.46	0.47	0.45	42	45962	55	9	43	3	48
ASE_00209	RNaseP_arch links to Rfam database...	0.49	0.45	0.53	40	42995	36	3	33	0	48
ASE_00210	RNaseP_arch links to Rfam database...	0.55	0.55	0.55	36	27196	29	5	24	0	29
ASE_00213	RNaseP_arch links to Rfam database...	0.30	0.29	0.32	19	27201	43	2	39	2	47
ASE_00346	RNaseP_arch links to Rfam database...	0.35	0.34	0.37	33	48116	59	5	51	3	64
ASE_00359	RNaseP_arch links to Rfam database...	0.63	0.60	0.66	48	42705	30	1	24	5	32
ASE_00360	RNaseP_arch links to Rfam database...	0.29	0.29	0.29	19	29337	51	10	37	4	46
ASE_00417	RNaseP_arch links to Rfam database...	0.35	0.36	0.34	36	54510	73	4	65	4	64
ASE_00418	RNaseP_arch links to Rfam database...	0.33	0.33	0.32	25	38704	54	4	48	2	50
CRW_00016	Intron_gpI links to Rfam database...	0.49	0.48	0.51	57	77309	66	7	48	11	63
CRW_00610	Intron_gpI links to Rfam database...	0.55	0.53	0.57	43	36240	37	3	29	5	38
CRW_00613	Intron_gpI links to Rfam database...	0.64	0.58	0.70	45	34916	28	1	18	9	33
CRW_00618	Intron_gpI links to Rfam database...	0.17	0.18	0.16	11	56213	77	9	47	21	50
CRW_00633	Intron_gpI links to Rfam database...	0.55	0.54	0.56	57	63445	45	8	36	1	49
CRW_00634	Intron_gpI links to Rfam database...	0.33	0.34	0.33	32	64522	75	8	58	9	62
CRW_00670	Intron_gpI links to Rfam database...	0.74	0.72	0.76	86	70012	32	0	27	5	34
CRW_00671	Intron_gpI links to Rfam database...	0.65	0.62	0.67	73	62726	43	1	35	7	44
CRW_00672	Intron_gpI links to Rfam database...	0.66	0.66	0.66	73	72279	45	8	30	7	38
CRW_00674	Intron_gpI links to Rfam database...	0.68	0.67	0.69	83	83316	41	6	31	4	41
CRW_00676	Intron_gpI links to Rfam database...	0.58	0.62	0.55	71	93398	67	17	42	8	44
CRW_00692	Intron_gpI links to Rfam database...	0.43	0.46	0.40	41	68533	70	18	43	9	49
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	-0.01	0.00	0.00	0	2678	24	1	22	1	23
PDB_00827	Intron_gpI links to Rfam database...	0.64	0.56	0.74	45	26967	17	0	16	1	36
PDB_01092	glmS links to Rfam database...	0.54	0.50	0.58	26	10108	21	2	17	2	26
RFA_00598	Telomerase-vert links to Rfam database...	0.53	0.53	0.53	54	84565	71	7	40	24	47
RFA_00632	HDV_ribozyme links to Rfam database...	0.34	0.36	0.33	10	4065	20	2	18	0	18
RFA_00636	HDV_ribozyme links to Rfam database...	0.42	0.43	0.43	12	3977	16	2	14	0	16
RFA_00767	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1877	4	0	4	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.48	0.44	0.53	8	1876	7	1	6	0	10
RFA_00769	Corona_pk3 links to Rfam database...	0.54	0.56	0.53	10	1934	9	4	5	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.61	0.56	0.67	10	2001	8	0	5	3	8
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	4	1	0	3	8
RFA_00779	Corona_pk3 links to Rfam database...	0.55	0.56	0.56	10	1935	8	3	5	0	8
RFA_00808	RydC links to Rfam database...	-0.01	0.00	0.00	0	2001	15	2	13	0	16
RFA_00809	RydC links to Rfam database...	0.36	0.38	0.35	6	2128	11	1	10	0	10

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	101
Total TN	1800416
Total FP	5
Total FP CONTRA	3
Total FP INCONS	1
Total FP COMP	1
Total FN	3290

Total Scores
MCC	0.169
Average MCC ± 95% Confidence Intervals	0.126 ± 0.083
Sensitivity	0.030
Positive Predictive Value	0.962
Nr of predictions	47

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00205	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	90
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00360	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	29403	0	0	0	0	65
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00618	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	56280	0	0	0	0	61
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	64620	0	0	0	0	94
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	2701	0	0	0	0	23
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
PDB_01092	glmS links to Rfam database...	0.77	0.63	0.94	33	10118	3	1	1	1	19
RFA_00598	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	84666	0	0	0	0	101
RFA_00632	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4095	0	0	0	0	28
RFA_00636	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4005	0	0	0	0	28
RFA_00767	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.62	0.39	1.00	7	2009	0	0	0	0	11
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1943	0	0	0	0	8
RFA_00808	RydC links to Rfam database...	0.00	0.00	0.00	0	2016	0	0	0	0	16
RFA_00809	RydC links to Rfam database...	0.00	0.00	0.00	0	2145	0	0	0	0	16

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of RNAshapes - scored higher in this pairwise comparison

1. Total counts & total scores for RNAshapes

2. Individual counts for RNAshapes [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]