CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	Afold		Carnac(seed)
MCC	0.501	>	0.259
Average MCC ± 95% Confidence Intervals	0.469 ± 0.058	>	0.093 ± 0.067
Sensitivity	0.499	>	0.071
Positive Predictive Value	0.503	<	0.937
Total TP	2383	>	341
Total TN	4140237	<	4144613
Total FP	3040	>	40
Total FP CONTRA	431	>	9
Total FP INCONS	1926	>	14
Total FP COMP	683	>	17
Total FN	2388	<	4430
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of Afold and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Afold and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Afold and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for Afold and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Afold and Carnac(seed)).

^top

Performance of Afold - scored higher in this pairwise comparison

1. Total counts & total scores for Afold

Total Base Pair Counts
Total TP	2383
Total TN	4140237
Total FP	3040
Total FP CONTRA	431
Total FP INCONS	1926
Total FP COMP	683
Total FN	2388

Total Scores
MCC	0.501
Average MCC ± 95% Confidence Intervals	0.469 ± 0.058
Sensitivity	0.499
Positive Predictive Value	0.503
Nr of predictions	54

^top

2. Individual counts for Afold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.75	0.72	0.78	54	34122	23	3	12	8	21
ASE_00002	RNaseP_arch links to Rfam database...	0.62	0.62	0.63	45	35439	33	4	23	6	28
ASE_00009	RNaseP_arch links to Rfam database...	0.79	0.76	0.82	51	26044	13	0	11	2	16
ASE_00014	RNaseP_arch links to Rfam database...	0.24	0.25	0.24	26	54178	82	8	73	1	80
ASE_00116	RNaseP_arch links to Rfam database...	0.24	0.24	0.23	16	31557	54	8	45	1	50
ASE_00120	RNaseP_arch links to Rfam database...	0.37	0.35	0.38	33	46885	53	9	44	0	61
ASE_00121	RNaseP_arch links to Rfam database...	0.29	0.29	0.29	27	45058	65	8	57	0	65
ASE_00145	RNaseP_arch links to Rfam database...	0.40	0.40	0.40	44	70016	69	9	56	4	67
ASE_00148	RNaseP_arch links to Rfam database...	0.63	0.60	0.65	93	112433	53	4	45	4	61
ASE_00154	RNaseP_arch links to Rfam database...	0.57	0.55	0.58	59	65240	45	4	38	3	48
ASE_00155	RNaseP_arch links to Rfam database...	0.50	0.48	0.52	68	93396	69	10	54	5	74
ASE_00187	RNaseP_arch links to Rfam database...	0.50	0.48	0.51	55	68528	58	7	45	6	60
ASE_00196	RNaseP_arch links to Rfam database...	0.76	0.75	0.77	56	31553	21	2	15	4	19
ASE_00209	RNaseP_arch links to Rfam database...	0.29	0.28	0.30	25	42989	57	4	53	0	63
ASE_00210	RNaseP_arch links to Rfam database...	0.61	0.62	0.60	40	27194	28	6	21	1	25
ASE_00213	RNaseP_arch links to Rfam database...	0.72	0.71	0.73	47	27197	18	3	14	1	19
ASE_00222	RNaseP_arch links to Rfam database...	0.56	0.55	0.56	87	111947	68	8	59	1	70
ASE_00346	RNaseP_arch links to Rfam database...	0.31	0.30	0.32	29	48114	65	7	55	3	68
ASE_00359	RNaseP_arch links to Rfam database...	0.49	0.49	0.49	39	42698	49	3	38	8	41
ASE_00417	RNaseP_arch links to Rfam database...	0.36	0.37	0.35	37	54508	73	10	60	3	63
ASE_00418	RNaseP_arch links to Rfam database...	0.20	0.21	0.20	16	38701	67	12	52	3	59
CRW_00013	Intron_gpI links to Rfam database...	0.22	0.22	0.22	25	103625	113	13	77	23	90
CRW_00016	Intron_gpI links to Rfam database...	0.65	0.62	0.69	74	77313	47	5	29	13	46
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.78	0.77	0.79	351	1085158	132	17	75	40	107
CRW_00610	Intron_gpI links to Rfam database...	0.42	0.42	0.43	34	36235	46	6	40	0	47
CRW_00613	Intron_gpI links to Rfam database...	0.46	0.42	0.51	33	34915	41	2	30	9	45
CRW_00614	Intron_gpI links to Rfam database...	0.07	0.09	0.06	5	121689	141	24	53	64	52
CRW_00619	Intron_gpI links to Rfam database...	0.38	0.42	0.34	31	164933	138	25	36	77	42
CRW_00620	Intron_gpI links to Rfam database...	0.16	0.21	0.12	13	150869	161	32	61	68	50
CRW_00621	Intron_gpI links to Rfam database...	0.10	0.10	0.10	7	153108	157	13	53	91	64
CRW_00623	Intron_gpI links to Rfam database...	0.30	0.29	0.32	24	129719	123	11	41	71	59
CRW_00633	Intron_gpI links to Rfam database...	0.45	0.44	0.47	47	63445	58	10	44	4	59
CRW_00670	Intron_gpI links to Rfam database...	0.77	0.74	0.81	89	70015	26	0	21	5	31
CRW_00671	Intron_gpI links to Rfam database...	0.66	0.64	0.68	75	62725	41	8	27	6	42
CRW_00672	Intron_gpI links to Rfam database...	0.74	0.73	0.74	81	72281	36	10	18	8	30
CRW_00674	Intron_gpI links to Rfam database...	0.55	0.55	0.55	68	83312	61	8	48	5	56
CRW_00676	Intron_gpI links to Rfam database...	0.54	0.56	0.52	64	93406	75	17	41	17	51
CRW_00692	Intron_gpI links to Rfam database...	0.57	0.57	0.57	51	68545	54	15	24	15	39
PDB_00716	HDV_ribozyme links to Rfam database...	-0.01	0.00	0.00	0	2677	25	1	23	1	23
PDB_00827	Intron_gpI links to Rfam database...	0.67	0.59	0.76	48	26965	16	0	15	1	33
PDB_01092	glmS links to Rfam database...	0.69	0.60	0.79	31	10114	11	1	7	3	21
RFA_00597	Telomerase-vert links to Rfam database...	0.51	0.51	0.51	56	97794	82	5	48	29	53
RFA_00599	Telomerase-vert links to Rfam database...	0.67	0.70	0.64	78	101353	66	16	28	22	33
RFA_00601	Telomerase-vert links to Rfam database...	0.09	0.11	0.08	12	99092	139	35	96	8	102
RFA_00602	Telomerase-vert links to Rfam database...	0.50	0.53	0.47	62	101344	91	18	51	22	54
RFA_00632	HDV_ribozyme links to Rfam database...	0.40	0.39	0.41	11	4068	18	0	16	2	17
RFA_00636	HDV_ribozyme links to Rfam database...	0.42	0.43	0.43	12	3977	17	2	14	1	16
RFA_00767	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1877	5	0	4	1	8
RFA_00768	Corona_pk3 links to Rfam database...	0.48	0.44	0.53	8	1876	8	1	6	1	10
RFA_00769	Corona_pk3 links to Rfam database...	0.55	0.56	0.56	10	1935	10	3	5	2	8
RFA_00770	Corona_pk3 links to Rfam database...	0.68	0.56	0.83	10	2004	6	0	2	4	8
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	5	1	0	4	8
RFA_00808	RydC links to Rfam database...	-0.01	0.00	0.00	0	2001	16	2	13	1	16
RFA_00809	RydC links to Rfam database...	0.36	0.38	0.35	6	2128	12	1	10	1	10

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	341
Total TN	4144613
Total FP	40
Total FP CONTRA	9
Total FP INCONS	14
Total FP COMP	17
Total FN	4430

Total Scores
MCC	0.259
Average MCC ± 95% Confidence Intervals	0.093 ± 0.067
Sensitivity	0.071
Positive Predictive Value	0.937
Nr of predictions	54

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00148	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112575	0	0	0	0	154
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00155	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	142
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00222	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112101	0	0	0	0	157
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00013	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	103740	0	0	0	0	115
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.73	0.57	0.93	261	1085321	35	6	13	16	197
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00614	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	121771	0	0	0	0	57
CRW_00619	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	165025	0	0	0	0	73
CRW_00620	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	150975	0	0	0	0	63
CRW_00621	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	153181	0	0	0	0	71
CRW_00623	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	129795	0	0	0	0	83
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00716	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	2701	0	0	0	0	23
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
PDB_01092	glmS links to Rfam database...	0.77	0.63	0.94	33	10118	3	1	1	1	19
RFA_00597	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	97903	0	0	0	0	109
RFA_00599	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	111
RFA_00601	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	99235	0	0	0	0	114
RFA_00602	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	116
RFA_00632	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4095	0	0	0	0	28
RFA_00636	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4005	0	0	0	0	28
RFA_00767	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.62	0.39	1.00	7	2009	0	0	0	0	11
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00808	RydC links to Rfam database...	0.00	0.00	0.00	0	2016	0	0	0	0	16
RFA_00809	RydC links to Rfam database...	0.00	0.00	0.00	0	2145	0	0	0	0	16

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of Afold - scored higher in this pairwise comparison

1. Total counts & total scores for Afold

2. Individual counts for Afold [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]