CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	RNAsubopt		Carnac(seed)
MCC	0.487	>	0.146
Average MCC ± 95% Confidence Intervals	0.481 ± 0.050	>	0.104 ± 0.069
Sensitivity	0.495	>	0.022
Positive Predictive Value	0.481	<	0.962
Total TP	2258	>	101
Total TN	2859928	<	2864514
Total FP	2848	>	5
Total FP CONTRA	433	>	3
Total FP INCONS	2000	>	1
Total FP COMP	415	>	1
Total FN	2308	<	4465
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of RNAsubopt and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNAsubopt and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNAsubopt and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for RNAsubopt and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNAsubopt and Carnac(seed)).

^top

Performance of RNAsubopt - scored higher in this pairwise comparison

1. Total counts & total scores for RNAsubopt

Total Base Pair Counts
Total TP	2258
Total TN	2859928
Total FP	2848
Total FP CONTRA	433
Total FP INCONS	2000
Total FP COMP	415
Total FN	2308

Total Scores
MCC	0.487
Average MCC ± 95% Confidence Intervals	0.481 ± 0.050
Sensitivity	0.495
Positive Predictive Value	0.481
Nr of predictions	57

^top

2. Individual counts for RNAsubopt [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.74	0.73	0.74	55	34117	26	4	15	7	20
ASE_00002	RNaseP_arch links to Rfam database...	0.63	0.62	0.64	45	35441	30	4	21	5	28
ASE_00009	RNaseP_arch links to Rfam database...	0.71	0.70	0.72	47	26041	21	3	15	3	20
ASE_00014	RNaseP_arch links to Rfam database...	0.42	0.43	0.42	46	54175	67	10	54	3	60
ASE_00116	RNaseP_arch links to Rfam database...	0.25	0.26	0.25	17	31557	55	10	42	3	49
ASE_00120	RNaseP_arch links to Rfam database...	0.48	0.49	0.47	46	46874	51	7	44	0	48
ASE_00121	RNaseP_arch links to Rfam database...	0.31	0.33	0.31	30	45052	70	9	59	2	62
ASE_00145	RNaseP_arch links to Rfam database...	0.46	0.46	0.46	51	70015	66	7	52	7	60
ASE_00148	RNaseP_arch links to Rfam database...	0.60	0.60	0.61	92	112425	61	6	52	3	62
ASE_00154	RNaseP_arch links to Rfam database...	0.63	0.64	0.64	68	65234	42	4	35	3	39
ASE_00155	RNaseP_arch links to Rfam database...	0.54	0.54	0.54	76	93388	70	5	59	6	66
ASE_00187	RNaseP_arch links to Rfam database...	0.44	0.44	0.44	51	68520	69	8	56	5	64
ASE_00196	RNaseP_arch links to Rfam database...	0.74	0.73	0.75	55	31553	22	2	16	4	20
ASE_00205	RNaseP_arch links to Rfam database...	0.46	0.47	0.46	42	45964	54	8	42	4	48
ASE_00209	RNaseP_arch links to Rfam database...	0.40	0.39	0.41	34	42989	48	5	43	0	54
ASE_00210	RNaseP_arch links to Rfam database...	0.47	0.48	0.47	31	27195	36	7	28	1	34
ASE_00213	RNaseP_arch links to Rfam database...	0.73	0.73	0.73	48	27195	19	6	12	1	18
ASE_00222	RNaseP_arch links to Rfam database...	0.46	0.46	0.46	72	111946	83	12	71	0	85
ASE_00346	RNaseP_arch links to Rfam database...	0.42	0.41	0.42	40	48110	59	4	51	4	57
ASE_00359	RNaseP_arch links to Rfam database...	0.45	0.46	0.44	37	42693	55	3	45	7	43
ASE_00360	RNaseP_arch links to Rfam database...	0.31	0.32	0.31	21	29335	53	11	36	6	44
ASE_00417	RNaseP_arch links to Rfam database...	0.44	0.45	0.43	45	54510	64	9	51	4	55
ASE_00418	RNaseP_arch links to Rfam database...	0.33	0.33	0.32	25	38704	54	5	47	2	50
CRW_00013	Intron_gpI links to Rfam database...	0.21	0.22	0.21	25	103620	117	13	82	22	90
CRW_00016	Intron_gpI links to Rfam database...	0.60	0.58	0.63	70	77309	55	5	37	13	50
CRW_00610	Intron_gpI links to Rfam database...	0.49	0.51	0.48	41	36230	44	9	35	0	40
CRW_00613	Intron_gpI links to Rfam database...	0.66	0.62	0.71	48	34912	28	1	19	8	30
CRW_00614	Intron_gpI links to Rfam database...	0.16	0.19	0.13	11	121684	137	27	49	61	46
CRW_00618	Intron_gpI links to Rfam database...	0.07	0.08	0.07	5	56205	92	15	55	22	56
CRW_00633	Intron_gpI links to Rfam database...	0.40	0.42	0.39	44	63434	70	10	58	2	62
CRW_00634	Intron_gpI links to Rfam database...	0.50	0.53	0.48	50	64516	59	11	43	5	44
CRW_00670	Intron_gpI links to Rfam database...	0.62	0.61	0.63	73	70009	45	9	34	2	47
CRW_00671	Intron_gpI links to Rfam database...	0.65	0.63	0.67	74	62725	43	1	35	7	43
CRW_00672	Intron_gpI links to Rfam database...	0.74	0.75	0.74	83	72278	38	6	23	9	28
CRW_00674	Intron_gpI links to Rfam database...	0.60	0.61	0.59	76	83308	55	10	42	3	48
CRW_00676	Intron_gpI links to Rfam database...	0.56	0.58	0.54	67	93403	71	15	43	13	48
CRW_00692	Intron_gpI links to Rfam database...	0.34	0.37	0.32	33	68532	80	22	48	10	57
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	-0.01	0.00	0.00	0	2678	24	1	22	1	23
PDB_00827	Intron_gpI links to Rfam database...	0.59	0.53	0.65	43	26962	24	3	20	1	38
PDB_01092	glmS links to Rfam database...	0.66	0.60	0.74	31	10111	13	3	8	2	21
RFA_00597	Telomerase-vert links to Rfam database...	0.55	0.57	0.53	62	97786	78	17	38	23	47
RFA_00598	Telomerase-vert links to Rfam database...	0.27	0.28	0.26	28	84559	99	13	66	20	73
RFA_00599	Telomerase-vert links to Rfam database...	0.66	0.70	0.63	78	101351	67	17	29	21	33
RFA_00600	Telomerase-vert links to Rfam database...	0.52	0.58	0.46	58	120169	111	26	42	43	42
RFA_00601	Telomerase-vert links to Rfam database...	0.22	0.25	0.20	28	99098	124	19	90	15	86
RFA_00602	Telomerase-vert links to Rfam database...	0.49	0.52	0.47	60	101347	91	15	53	23	56
RFA_00632	HDV_ribozyme links to Rfam database...	0.31	0.32	0.30	9	4065	21	2	19	0	19
RFA_00636	HDV_ribozyme links to Rfam database...	0.42	0.43	0.43	12	3977	16	2	14	0	16
RFA_00767	Corona_pk3 links to Rfam database...	0.63	0.56	0.71	10	1877	4	0	4	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.48	0.44	0.53	8	1876	7	1	6	0	10
RFA_00769	Corona_pk3 links to Rfam database...	0.52	0.56	0.50	10	1933	10	4	6	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.61	0.56	0.67	10	2001	8	0	5	3	8
RFA_00773	Corona_pk3 links to Rfam database...	0.68	0.56	0.83	10	1941	5	1	1	3	8
RFA_00779	Corona_pk3 links to Rfam database...	0.55	0.56	0.56	10	1935	8	3	5	0	8
RFA_00808	RydC links to Rfam database...	-0.01	0.00	0.00	0	2001	18	2	13	3	16
RFA_00809	RydC links to Rfam database...	0.36	0.38	0.35	6	2128	11	1	10	0	10

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	101
Total TN	2864514
Total FP	5
Total FP CONTRA	3
Total FP INCONS	1
Total FP COMP	1
Total FN	4465

Total Scores
MCC	0.146
Average MCC ± 95% Confidence Intervals	0.104 ± 0.069
Sensitivity	0.022
Positive Predictive Value	0.962
Nr of predictions	57

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00148	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112575	0	0	0	0	154
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00155	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	142
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00205	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	90
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00222	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112101	0	0	0	0	157
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00360	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	29403	0	0	0	0	65
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00013	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	103740	0	0	0	0	115
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00614	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	121771	0	0	0	0	57
CRW_00618	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	56280	0	0	0	0	61
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	64620	0	0	0	0	94
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	2701	0	0	0	0	23
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
PDB_01092	glmS links to Rfam database...	0.77	0.63	0.94	33	10118	3	1	1	1	19
RFA_00597	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	97903	0	0	0	0	109
RFA_00598	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	84666	0	0	0	0	101
RFA_00599	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	111
RFA_00600	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	120295	0	0	0	0	100
RFA_00601	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	99235	0	0	0	0	114
RFA_00602	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	116
RFA_00632	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4095	0	0	0	0	28
RFA_00636	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4005	0	0	0	0	28
RFA_00767	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.62	0.39	1.00	7	2009	0	0	0	0	11
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1943	0	0	0	0	8
RFA_00808	RydC links to Rfam database...	0.00	0.00	0.00	0	2016	0	0	0	0	16
RFA_00809	RydC links to Rfam database...	0.00	0.00	0.00	0	2145	0	0	0	0	16

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of RNAsubopt - scored higher in this pairwise comparison

1. Total counts & total scores for RNAsubopt

2. Individual counts for RNAsubopt [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]