CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	RNAwolf		Carnac(seed)
MCC	0.298	>	0.141
Average MCC ± 95% Confidence Intervals	0.333 ± 0.046	>	0.097 ± 0.065
Sensitivity	0.312	>	0.021
Positive Predictive Value	0.287	<	0.962
Total TP	1513	>	101
Total TN	3458317	<	3463490
Total FP	4675	>	5
Total FP CONTRA	715	>	3
Total FP INCONS	3050	>	1
Total FP COMP	910	>	1
Total FN	3343	<	4755
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of RNAwolf and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNAwolf and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNAwolf and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for RNAwolf and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNAwolf and Carnac(seed)).

^top

Performance of RNAwolf - scored higher in this pairwise comparison

1. Total counts & total scores for RNAwolf

Total Base Pair Counts
Total TP	1513
Total TN	3458317
Total FP	4675
Total FP CONTRA	715
Total FP INCONS	3050
Total FP COMP	910
Total FN	3343

Total Scores
MCC	0.298
Average MCC ± 95% Confidence Intervals	0.333 ± 0.046
Sensitivity	0.312
Positive Predictive Value	0.287
Nr of predictions	61

^top

2. Individual counts for RNAwolf [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.76	0.73	0.80	55	34122	33	1	13	19	20
ASE_00002	RNaseP_arch links to Rfam database...	0.56	0.53	0.58	39	35444	45	2	26	17	34
ASE_00009	RNaseP_arch links to Rfam database...	0.39	0.39	0.40	26	26041	48	4	35	9	41
ASE_00014	RNaseP_arch links to Rfam database...	0.36	0.38	0.35	40	54171	80	16	58	6	66
ASE_00116	RNaseP_arch links to Rfam database...	0.05	0.06	0.05	4	31549	79	14	59	6	62
ASE_00120	RNaseP_arch links to Rfam database...	0.20	0.20	0.20	19	46877	79	15	60	4	75
ASE_00121	RNaseP_arch links to Rfam database...	0.15	0.15	0.14	14	45052	88	10	74	4	78
ASE_00145	RNaseP_arch links to Rfam database...	0.24	0.25	0.24	28	70006	101	8	83	10	83
ASE_00148	RNaseP_arch links to Rfam database...	0.36	0.36	0.36	56	112419	106	14	86	6	98
ASE_00154	RNaseP_arch links to Rfam database...	0.42	0.42	0.43	45	65236	73	8	52	13	62
ASE_00155	RNaseP_arch links to Rfam database...	0.41	0.42	0.41	59	93384	92	4	81	7	83
ASE_00187	RNaseP_arch links to Rfam database...	0.28	0.29	0.28	33	68517	96	13	72	11	82
ASE_00196	RNaseP_arch links to Rfam database...	0.57	0.59	0.55	44	31546	50	4	32	14	31
ASE_00205	RNaseP_arch links to Rfam database...	0.17	0.19	0.16	17	45949	90	17	73	0	73
ASE_00209	RNaseP_arch links to Rfam database...	0.30	0.30	0.31	26	42988	63	8	49	6	62
ASE_00210	RNaseP_arch links to Rfam database...	0.62	0.63	0.61	41	27194	39	9	17	13	24
ASE_00213	RNaseP_arch links to Rfam database...	0.68	0.65	0.70	43	27200	31	6	12	13	23
ASE_00222	RNaseP_arch links to Rfam database...	0.30	0.32	0.29	50	111929	125	20	102	3	107
ASE_00346	RNaseP_arch links to Rfam database...	0.26	0.26	0.27	25	48112	83	9	59	15	72
ASE_00359	RNaseP_arch links to Rfam database...	0.16	0.18	0.15	14	42682	87	15	67	5	66
ASE_00360	RNaseP_arch links to Rfam database...	0.25	0.28	0.23	18	29326	62	10	49	3	47
ASE_00417	RNaseP_arch links to Rfam database...	0.42	0.44	0.40	44	54506	75	9	56	10	56
ASE_00418	RNaseP_arch links to Rfam database...	0.17	0.19	0.16	14	38696	80	6	65	9	61
CRW_00013	Intron_gpI links to Rfam database...	0.03	0.03	0.03	4	103598	147	24	114	9	111
CRW_00016	Intron_gpI links to Rfam database...	0.24	0.23	0.24	28	77305	101	10	78	13	92
CRW_00610	Intron_gpI links to Rfam database...	0.52	0.52	0.52	42	36234	53	4	35	14	39
CRW_00613	Intron_gpI links to Rfam database...	0.60	0.62	0.59	48	34898	50	1	33	16	30
CRW_00614	Intron_gpI links to Rfam database...	0.09	0.12	0.07	7	121673	167	34	57	76	50
CRW_00618	Intron_gpI links to Rfam database...	0.01	0.02	0.01	1	56208	107	13	58	36	60
CRW_00619	Intron_gpI links to Rfam database...	0.27	0.32	0.24	23	164928	154	28	46	80	50
CRW_00620	Intron_gpI links to Rfam database...	0.31	0.33	0.28	21	150901	165	8	45	112	42
CRW_00621	Intron_gpI links to Rfam database...	0.26	0.30	0.23	21	153090	163	20	50	93	50
CRW_00623	Intron_gpI links to Rfam database...	0.11	0.13	0.10	11	129683	155	28	73	54	72
CRW_00633	Intron_gpI links to Rfam database...	0.29	0.29	0.30	31	63442	76	12	61	3	75
CRW_00634	Intron_gpI links to Rfam database...	0.19	0.22	0.17	21	64493	110	25	81	4	73
CRW_00670	Intron_gpI links to Rfam database...	0.35	0.35	0.35	42	70006	81	12	65	4	78
CRW_00671	Intron_gpI links to Rfam database...	0.13	0.13	0.13	15	62723	100	11	86	3	102
CRW_00672	Intron_gpI links to Rfam database...	0.06	0.06	0.06	7	72276	114	17	90	7	104
CRW_00674	Intron_gpI links to Rfam database...	0.28	0.28	0.28	35	83310	106	12	79	15	89
CRW_00676	Intron_gpI links to Rfam database...	0.16	0.17	0.16	19	93407	113	22	80	11	96
CRW_00692	Intron_gpI links to Rfam database...	0.18	0.19	0.18	17	68539	100	16	63	21	73
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.42	0.36	0.50	5	936	6	0	5	1	9
PDB_00716	HDV_ribozyme links to Rfam database...	0.68	0.57	0.81	13	2685	3	0	3	0	10
PDB_00827	Intron_gpI links to Rfam database...	0.58	0.58	0.57	47	26946	37	6	29	2	34
PDB_01092	glmS links to Rfam database...	0.43	0.38	0.49	20	10112	23	2	19	2	32
RFA_00597	Telomerase-vert links to Rfam database...	0.41	0.44	0.39	48	97780	99	18	57	24	61
RFA_00598	Telomerase-vert links to Rfam database...	0.18	0.20	0.17	20	84546	111	30	70	11	81
RFA_00599	Telomerase-vert links to Rfam database...	0.31	0.34	0.28	38	101339	111	33	65	13	73
RFA_00600	Telomerase-vert links to Rfam database...	0.06	0.08	0.05	8	120145	162	45	97	20	92
RFA_00601	Telomerase-vert links to Rfam database...	0.37	0.42	0.33	48	99088	112	28	71	13	66
RFA_00602	Telomerase-vert links to Rfam database...	0.32	0.35	0.30	41	101337	128	16	81	31	75
RFA_00632	HDV_ribozyme links to Rfam database...	0.25	0.25	0.27	7	4069	22	1	18	3	21
RFA_00636	HDV_ribozyme links to Rfam database...	0.40	0.39	0.41	11	3978	19	1	15	3	17
RFA_00767	Corona_pk3 links to Rfam database...	0.48	0.44	0.53	8	1876	8	1	6	1	10
RFA_00768	Corona_pk3 links to Rfam database...	0.50	0.44	0.57	8	1877	8	1	5	2	10
RFA_00769	Corona_pk3 links to Rfam database...	0.41	0.44	0.38	8	1932	13	3	10	0	10
RFA_00770	Corona_pk3 links to Rfam database...	0.51	0.50	0.53	9	1999	12	2	6	4	9
RFA_00773	Corona_pk3 links to Rfam database...	0.42	0.39	0.47	7	1938	12	1	7	4	11
RFA_00779	Corona_pk3 links to Rfam database...	0.26	0.28	0.26	5	1934	14	6	8	0	13
RFA_00808	RydC links to Rfam database...	0.68	0.56	0.82	9	2005	2	2	0	0	7
RFA_00809	RydC links to Rfam database...	0.47	0.38	0.60	6	2135	6	0	4	2	10

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	101
Total TN	3463490
Total FP	5
Total FP CONTRA	3
Total FP INCONS	1
Total FP COMP	1
Total FN	4755

Total Scores
MCC	0.141
Average MCC ± 95% Confidence Intervals	0.097 ± 0.065
Sensitivity	0.021
Positive Predictive Value	0.962
Nr of predictions	61

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00148	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112575	0	0	0	0	154
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00155	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	142
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00205	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	90
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00222	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112101	0	0	0	0	157
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00360	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	29403	0	0	0	0	65
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00013	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	103740	0	0	0	0	115
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00614	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	121771	0	0	0	0	57
CRW_00618	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	56280	0	0	0	0	61
CRW_00619	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	165025	0	0	0	0	73
CRW_00620	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	150975	0	0	0	0	63
CRW_00621	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	153181	0	0	0	0	71
CRW_00623	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	129795	0	0	0	0	83
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	64620	0	0	0	0	94
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	2701	0	0	0	0	23
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
PDB_01092	glmS links to Rfam database...	0.77	0.63	0.94	33	10118	3	1	1	1	19
RFA_00597	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	97903	0	0	0	0	109
RFA_00598	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	84666	0	0	0	0	101
RFA_00599	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	111
RFA_00600	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	120295	0	0	0	0	100
RFA_00601	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	99235	0	0	0	0	114
RFA_00602	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	116
RFA_00632	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4095	0	0	0	0	28
RFA_00636	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4005	0	0	0	0	28
RFA_00767	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.62	0.39	1.00	7	2009	0	0	0	0	11
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1943	0	0	0	0	8
RFA_00808	RydC links to Rfam database...	0.00	0.00	0.00	0	2016	0	0	0	0	16
RFA_00809	RydC links to Rfam database...	0.00	0.00	0.00	0	2145	0	0	0	0	16

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of RNAwolf - scored higher in this pairwise comparison

1. Total counts & total scores for RNAwolf

2. Individual counts for RNAwolf [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]