CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	PknotsRG		Carnac(seed)
MCC	0.487	>	0.000
Average MCC ± 95% Confidence Intervals	0.472 ± 0.056	>	0.000 ± 0.000
Sensitivity	0.493	>	0.000
Positive Predictive Value	0.483	>	0.000
Total TP	2254	>	0
Total TN	3421213	<	3425877
Total FP	3081	>	0
Total FP CONTRA	493	>	0
Total FP INCONS	1917	>	0
Total FP COMP	671	>	0
Total FN	2317	<	4571
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of PknotsRG and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PknotsRG and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PknotsRG and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PknotsRG and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PknotsRG and Carnac(seed)).

^top

Performance of PknotsRG - scored higher in this pairwise comparison

1. Total counts & total scores for PknotsRG

Total Base Pair Counts
Total TP	2254
Total TN	3421213
Total FP	3081
Total FP CONTRA	493
Total FP INCONS	1917
Total FP COMP	671
Total FN	2317

Total Scores
MCC	0.487
Average MCC ± 95% Confidence Intervals	0.472 ± 0.056
Sensitivity	0.493
Positive Predictive Value	0.483
Nr of predictions	48

^top

2. Individual counts for PknotsRG [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.63	0.61	0.64	46	34119	33	3	23	7	29
ASE_00002	RNaseP_arch links to Rfam database...	0.50	0.49	0.51	36	35440	40	5	30	5	37
ASE_00009	RNaseP_arch links to Rfam database...	0.78	0.75	0.82	50	26045	13	0	11	2	17
ASE_00014	RNaseP_arch links to Rfam database...	0.24	0.25	0.24	26	54178	82	8	73	1	80
ASE_00116	RNaseP_arch links to Rfam database...	0.19	0.20	0.19	13	31558	58	2	53	3	53
ASE_00120	RNaseP_arch links to Rfam database...	0.35	0.34	0.36	32	46883	56	9	47	0	62
ASE_00121	RNaseP_arch links to Rfam database...	0.25	0.25	0.26	23	45061	68	9	57	2	69
ASE_00145	RNaseP_arch links to Rfam database...	0.43	0.42	0.44	47	70019	64	6	53	5	64
ASE_00148	RNaseP_arch links to Rfam database...	0.70	0.67	0.73	103	112433	42	4	35	3	51
ASE_00154	RNaseP_arch links to Rfam database...	0.64	0.63	0.65	67	65238	39	4	32	3	40
ASE_00155	RNaseP_arch links to Rfam database...	0.51	0.50	0.52	71	93391	71	10	56	5	71
ASE_00187	RNaseP_arch links to Rfam database...	0.51	0.50	0.52	57	68526	57	7	45	5	58
ASE_00196	RNaseP_arch links to Rfam database...	0.70	0.69	0.71	52	31553	24	2	19	3	23
ASE_00205	RNaseP_arch links to Rfam database...	0.56	0.56	0.56	50	45967	44	7	32	5	40
ASE_00209	RNaseP_arch links to Rfam database...	0.28	0.27	0.29	24	42989	58	4	54	0	64
ASE_00210	RNaseP_arch links to Rfam database...	0.59	0.60	0.59	39	27195	28	6	21	1	26
ASE_00213	RNaseP_arch links to Rfam database...	0.73	0.73	0.73	48	27195	19	6	12	1	18
ASE_00222	RNaseP_arch links to Rfam database...	0.62	0.60	0.64	94	111954	55	9	44	2	63
ASE_00346	RNaseP_arch links to Rfam database...	0.30	0.29	0.31	28	48115	64	7	55	2	69
ASE_00359	RNaseP_arch links to Rfam database...	0.49	0.49	0.49	39	42698	46	3	38	5	41
ASE_00360	RNaseP_arch links to Rfam database...	0.30	0.31	0.30	20	29337	50	11	35	4	45
ASE_00417	RNaseP_arch links to Rfam database...	0.36	0.37	0.35	37	54508	73	10	60	3	63
ASE_00418	RNaseP_arch links to Rfam database...	0.21	0.21	0.21	16	38703	67	8	54	5	59
CRW_00013	Intron_gpI links to Rfam database...	0.16	0.17	0.15	19	103617	114	25	79	10	96
CRW_00016	Intron_gpI links to Rfam database...	0.64	0.62	0.66	74	77309	50	5	33	12	46
CRW_00610	Intron_gpI links to Rfam database...	0.49	0.49	0.49	40	36233	43	7	35	1	41
CRW_00613	Intron_gpI links to Rfam database...	0.64	0.59	0.69	46	34913	29	1	20	8	32
CRW_00614	Intron_gpI links to Rfam database...	0.09	0.11	0.07	6	121690	145	21	54	70	51
CRW_00618	Intron_gpI links to Rfam database...	0.11	0.11	0.10	7	56211	83	13	49	21	54
CRW_00619	Intron_gpI links to Rfam database...	0.38	0.44	0.34	32	164930	138	28	35	75	41
CRW_00620	Intron_gpI links to Rfam database...	0.16	0.21	0.13	13	150871	158	31	60	67	50
CRW_00621	Intron_gpI links to Rfam database...	0.10	0.10	0.10	7	153109	156	13	52	91	64
CRW_00623	Intron_gpI links to Rfam database...	0.30	0.29	0.32	24	129720	119	11	40	68	59
CRW_00633	Intron_gpI links to Rfam database...	0.50	0.48	0.52	51	63447	50	10	38	2	55
CRW_00634	Intron_gpI links to Rfam database...	0.56	0.60	0.52	56	64513	53	17	34	2	38
CRW_00670	Intron_gpI links to Rfam database...	0.75	0.74	0.77	89	70009	30	5	22	3	31
CRW_00671	Intron_gpI links to Rfam database...	0.77	0.76	0.78	89	62721	30	6	19	5	28
CRW_00672	Intron_gpI links to Rfam database...	0.53	0.55	0.52	61	72273	60	21	35	4	50
CRW_00674	Intron_gpI links to Rfam database...	0.52	0.52	0.52	65	83310	63	12	49	2	59
CRW_00676	Intron_gpI links to Rfam database...	0.56	0.60	0.52	69	93396	67	13	50	4	46
CRW_00692	Intron_gpI links to Rfam database...	0.43	0.46	0.41	41	68536	66	18	40	8	49
PDB_00827	Intron_gpI links to Rfam database...	0.50	0.46	0.55	37	26961	31	1	29	1	44
RFA_00597	Telomerase-vert links to Rfam database...	0.52	0.53	0.51	58	97789	83	8	48	27	51
RFA_00598	Telomerase-vert links to Rfam database...	0.68	0.66	0.69	67	84569	57	6	24	27	34
RFA_00599	Telomerase-vert links to Rfam database...	0.63	0.68	0.60	75	101349	70	18	33	19	36
RFA_00600	Telomerase-vert links to Rfam database...	0.63	0.70	0.56	70	120170	88	32	23	33	30
RFA_00601	Telomerase-vert links to Rfam database...	0.48	0.50	0.46	57	99111	86	21	46	19	57
RFA_00602	Telomerase-vert links to Rfam database...	0.69	0.72	0.67	83	101351	61	10	31	20	33

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	0
Total TN	3425877
Total FP	0
Total FP CONTRA	0
Total FP INCONS	0
Total FP COMP	0
Total FN	4571

Total Scores
MCC	0.000
Average MCC ± 95% Confidence Intervals	0.000 ± 0.000
Sensitivity	0.000
Positive Predictive Value	0.000
Nr of predictions	48

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00148	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112575	0	0	0	0	154
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00155	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	142
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00205	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	90
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00222	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112101	0	0	0	0	157
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00360	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	29403	0	0	0	0	65
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00013	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	103740	0	0	0	0	115
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00614	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	121771	0	0	0	0	57
CRW_00618	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	56280	0	0	0	0	61
CRW_00619	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	165025	0	0	0	0	73
CRW_00620	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	150975	0	0	0	0	63
CRW_00621	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	153181	0	0	0	0	71
CRW_00623	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	129795	0	0	0	0	83
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	64620	0	0	0	0	94
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
RFA_00597	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	97903	0	0	0	0	109
RFA_00598	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	84666	0	0	0	0	101
RFA_00599	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	111
RFA_00600	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	120295	0	0	0	0	100
RFA_00601	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	99235	0	0	0	0	114
RFA_00602	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	116

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of PknotsRG - scored higher in this pairwise comparison

1. Total counts & total scores for PknotsRG

2. Individual counts for PknotsRG [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]