CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	UNAFold		Carnac(seed)
MCC	0.511	>	0.253
Average MCC ± 95% Confidence Intervals	0.481 ± 0.053	>	0.107 ± 0.067
Sensitivity	0.508	>	0.068
Positive Predictive Value	0.514	<	0.940
Total TP	2700	>	362
Total TN	4543945	<	4548811
Total FP	3279	>	40
Total FP CONTRA	513	>	9
Total FP INCONS	2038	>	14
Total FP COMP	728	>	17
Total FN	2614	<	4952
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of UNAFold and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for UNAFold and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for UNAFold and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for UNAFold and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for UNAFold and Carnac(seed)).

^top

Performance of UNAFold - scored higher in this pairwise comparison

1. Total counts & total scores for UNAFold

Total Base Pair Counts
Total TP	2700
Total TN	4543945
Total FP	3279
Total FP CONTRA	513
Total FP INCONS	2038
Total FP COMP	728
Total FN	2614

Total Scores
MCC	0.511
Average MCC ± 95% Confidence Intervals	0.481 ± 0.053
Sensitivity	0.508
Positive Predictive Value	0.514
Nr of predictions	62

^top

2. Individual counts for UNAFold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.63	0.61	0.66	46	34121	30	2	22	6	29
ASE_00002	RNaseP_arch links to Rfam database...	0.66	0.62	0.70	45	35447	24	3	16	5	28
ASE_00009	RNaseP_arch links to Rfam database...	0.78	0.75	0.81	50	26044	15	0	12	3	17
ASE_00014	RNaseP_arch links to Rfam database...	0.24	0.25	0.24	26	54178	82	8	73	1	80
ASE_00116	RNaseP_arch links to Rfam database...	0.20	0.20	0.20	13	31561	55	2	50	3	53
ASE_00120	RNaseP_arch links to Rfam database...	0.51	0.49	0.53	46	46885	42	8	32	2	48
ASE_00121	RNaseP_arch links to Rfam database...	0.24	0.24	0.24	22	45060	68	5	63	0	70
ASE_00145	RNaseP_arch links to Rfam database...	0.46	0.45	0.48	50	70021	59	7	47	5	61
ASE_00148	RNaseP_arch links to Rfam database...	0.69	0.67	0.72	103	112431	44	4	37	3	51
ASE_00154	RNaseP_arch links to Rfam database...	0.64	0.63	0.65	67	65238	39	4	32	3	40
ASE_00155	RNaseP_arch links to Rfam database...	0.62	0.61	0.63	86	93392	57	8	42	7	56
ASE_00187	RNaseP_arch links to Rfam database...	0.47	0.44	0.49	51	68531	58	6	47	5	64
ASE_00196	RNaseP_arch links to Rfam database...	0.73	0.72	0.75	54	31554	21	2	16	3	21
ASE_00205	RNaseP_arch links to Rfam database...	0.43	0.43	0.43	39	45966	54	7	44	3	51
ASE_00209	RNaseP_arch links to Rfam database...	0.39	0.38	0.42	33	42992	46	5	41	0	55
ASE_00210	RNaseP_arch links to Rfam database...	0.59	0.60	0.59	39	27195	28	6	21	1	26
ASE_00213	RNaseP_arch links to Rfam database...	0.74	0.73	0.76	48	27198	16	3	12	1	18
ASE_00222	RNaseP_arch links to Rfam database...	0.59	0.59	0.61	92	111949	61	10	50	1	65
ASE_00346	RNaseP_arch links to Rfam database...	0.30	0.29	0.31	28	48115	64	7	55	2	69
ASE_00359	RNaseP_arch links to Rfam database...	0.49	0.49	0.49	39	42698	46	3	38	5	41
ASE_00360	RNaseP_arch links to Rfam database...	0.22	0.23	0.22	15	29335	57	10	43	4	50
ASE_00417	RNaseP_arch links to Rfam database...	0.36	0.37	0.35	37	54509	72	10	59	3	63
ASE_00418	RNaseP_arch links to Rfam database...	0.21	0.21	0.20	16	38702	65	12	51	2	59
CRW_00013	Intron_gpI links to Rfam database...	0.24	0.24	0.24	28	103624	107	18	70	19	87
CRW_00016	Intron_gpI links to Rfam database...	0.62	0.57	0.67	68	77320	45	5	28	12	52
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.77	0.76	0.78	347	1085158	132	19	77	36	111
CRW_00610	Intron_gpI links to Rfam database...	0.45	0.44	0.46	36	36236	43	6	37	0	45
CRW_00613	Intron_gpI links to Rfam database...	0.63	0.58	0.68	45	34914	29	1	20	8	33
CRW_00614	Intron_gpI links to Rfam database...	0.09	0.11	0.08	6	121694	140	20	51	69	51
CRW_00618	Intron_gpI links to Rfam database...	0.18	0.20	0.17	12	56210	83	10	48	25	49
CRW_00619	Intron_gpI links to Rfam database...	0.37	0.42	0.33	31	164931	138	28	35	75	42
CRW_00620	Intron_gpI links to Rfam database...	0.16	0.21	0.12	13	150870	160	31	61	68	50
CRW_00621	Intron_gpI links to Rfam database...	0.25	0.25	0.26	18	153111	144	9	43	92	53
CRW_00623	Intron_gpI links to Rfam database...	0.31	0.29	0.32	24	129721	119	11	39	69	59
CRW_00633	Intron_gpI links to Rfam database...	0.69	0.67	0.71	71	63446	33	8	21	4	35
CRW_00634	Intron_gpI links to Rfam database...	0.33	0.34	0.33	32	64522	70	14	52	4	62
CRW_00670	Intron_gpI links to Rfam database...	0.79	0.74	0.83	89	70018	21	6	12	3	31
CRW_00671	Intron_gpI links to Rfam database...	0.65	0.62	0.68	72	62729	39	7	27	5	45
CRW_00672	Intron_gpI links to Rfam database...	0.48	0.49	0.47	54	72275	65	19	42	4	57
CRW_00674	Intron_gpI links to Rfam database...	0.58	0.56	0.62	69	83324	45	7	36	2	55
CRW_00676	Intron_gpI links to Rfam database...	0.54	0.55	0.53	63	93410	69	15	40	14	52
CRW_00692	Intron_gpI links to Rfam database...	0.20	0.21	0.19	19	68537	84	24	55	5	71
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	-0.01	0.00	0.00	0	2677	24	5	19	0	23
PDB_00827	Intron_gpI links to Rfam database...	0.58	0.52	0.66	42	26964	23	1	21	1	39
PDB_01092	glmS links to Rfam database...	0.74	0.65	0.83	34	10112	8	1	6	1	18
RFA_00597	Telomerase-vert links to Rfam database...	0.50	0.50	0.50	55	97792	79	9	47	23	54
RFA_00598	Telomerase-vert links to Rfam database...	0.68	0.64	0.71	65	84575	52	6	20	26	36
RFA_00599	Telomerase-vert links to Rfam database...	0.67	0.70	0.63	78	101352	64	17	28	19	33
RFA_00600	Telomerase-vert links to Rfam database...	0.62	0.69	0.56	69	120172	85	29	25	31	31
RFA_00601	Telomerase-vert links to Rfam database...	0.47	0.49	0.44	56	99109	89	21	49	19	58
RFA_00602	Telomerase-vert links to Rfam database...	0.51	0.53	0.48	62	101347	86	18	48	20	54
RFA_00632	HDV_ribozyme links to Rfam database...	0.34	0.36	0.33	10	4065	20	2	18	0	18
RFA_00636	HDV_ribozyme links to Rfam database...	0.43	0.43	0.44	12	3978	15	2	13	0	16
RFA_00767	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1877	4	0	4	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.48	0.44	0.53	8	1876	7	1	6	0	10
RFA_00769	Corona_pk3 links to Rfam database...	0.54	0.56	0.53	10	1934	9	4	5	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.61	0.56	0.67	10	2001	8	0	5	3	8
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	4	1	0	3	8
RFA_00779	Corona_pk3 links to Rfam database...	0.55	0.56	0.56	10	1935	8	3	5	0	8
RFA_00808	RydC links to Rfam database...	-0.01	0.00	0.00	0	2001	15	2	13	0	16
RFA_00809	RydC links to Rfam database...	0.37	0.38	0.38	6	2129	10	1	9	0	10

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	362
Total TN	4548811
Total FP	40
Total FP CONTRA	9
Total FP INCONS	14
Total FP COMP	17
Total FN	4952

Total Scores
MCC	0.253
Average MCC ± 95% Confidence Intervals	0.107 ± 0.067
Sensitivity	0.068
Positive Predictive Value	0.940
Nr of predictions	62

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00148	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112575	0	0	0	0	154
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00155	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	142
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00205	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	90
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00222	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112101	0	0	0	0	157
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00360	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	29403	0	0	0	0	65
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00013	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	103740	0	0	0	0	115
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.73	0.57	0.93	261	1085321	35	6	13	16	197
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00614	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	121771	0	0	0	0	57
CRW_00618	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	56280	0	0	0	0	61
CRW_00619	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	165025	0	0	0	0	73
CRW_00620	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	150975	0	0	0	0	63
CRW_00621	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	153181	0	0	0	0	71
CRW_00623	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	129795	0	0	0	0	83
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	64620	0	0	0	0	94
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	2701	0	0	0	0	23
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
PDB_01092	glmS links to Rfam database...	0.77	0.63	0.94	33	10118	3	1	1	1	19
RFA_00597	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	97903	0	0	0	0	109
RFA_00598	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	84666	0	0	0	0	101
RFA_00599	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	111
RFA_00600	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	120295	0	0	0	0	100
RFA_00601	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	99235	0	0	0	0	114
RFA_00602	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	116
RFA_00632	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4095	0	0	0	0	28
RFA_00636	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4005	0	0	0	0	28
RFA_00767	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.62	0.39	1.00	7	2009	0	0	0	0	11
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1943	0	0	0	0	8
RFA_00808	RydC links to Rfam database...	0.00	0.00	0.00	0	2016	0	0	0	0	16
RFA_00809	RydC links to Rfam database...	0.00	0.00	0.00	0	2145	0	0	0	0	16

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of UNAFold - scored higher in this pairwise comparison

1. Total counts & total scores for UNAFold

2. Individual counts for UNAFold [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]