CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	CRWrnafold		Carnac(seed)
MCC	0.436	>	0.141
Average MCC ± 95% Confidence Intervals	0.430 ± 0.054	>	0.097 ± 0.065
Sensitivity	0.443	>	0.021
Positive Predictive Value	0.432	<	0.962
Total TP	2149	>	101
Total TN	3458620	<	3463490
Total FP	3482	>	5
Total FP CONTRA	522	>	3
Total FP INCONS	2304	>	1
Total FP COMP	656	>	1
Total FN	2707	<	4755
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of CRWrnafold and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for CRWrnafold and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for CRWrnafold and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for CRWrnafold and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for CRWrnafold and Carnac(seed)).

^top

Performance of CRWrnafold - scored higher in this pairwise comparison

1. Total counts & total scores for CRWrnafold

Total Base Pair Counts
Total TP	2149
Total TN	3458620
Total FP	3482
Total FP CONTRA	522
Total FP INCONS	2304
Total FP COMP	656
Total FN	2707

Total Scores
MCC	0.436
Average MCC ± 95% Confidence Intervals	0.430 ± 0.054
Sensitivity	0.443
Positive Predictive Value	0.432
Nr of predictions	61

^top

2. Individual counts for CRWrnafold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.61	0.57	0.64	43	34124	30	3	21	6	32
ASE_00002	RNaseP_arch links to Rfam database...	0.59	0.56	0.63	41	35446	31	2	22	7	32
ASE_00009	RNaseP_arch links to Rfam database...	0.68	0.67	0.69	45	26041	20	5	15	0	22
ASE_00014	RNaseP_arch links to Rfam database...	0.47	0.47	0.47	50	54179	57	7	49	1	56
ASE_00116	RNaseP_arch links to Rfam database...	0.04	0.05	0.05	3	31562	66	6	55	5	63
ASE_00120	RNaseP_arch links to Rfam database...	0.58	0.55	0.60	52	46885	34	4	30	0	42
ASE_00121	RNaseP_arch links to Rfam database...	0.43	0.42	0.43	39	45059	57	6	46	5	53
ASE_00145	RNaseP_arch links to Rfam database...	0.36	0.35	0.36	39	70018	73	5	63	5	72
ASE_00148	RNaseP_arch links to Rfam database...	0.48	0.47	0.50	72	112432	71	10	61	0	82
ASE_00154	RNaseP_arch links to Rfam database...	0.44	0.43	0.45	46	65239	60	4	52	4	61
ASE_00155	RNaseP_arch links to Rfam database...	0.40	0.40	0.41	57	93388	86	10	73	3	85
ASE_00187	RNaseP_arch links to Rfam database...	0.33	0.31	0.35	36	68531	71	6	62	3	79
ASE_00196	RNaseP_arch links to Rfam database...	0.80	0.79	0.81	59	31553	20	2	12	6	16
ASE_00205	RNaseP_arch links to Rfam database...	0.45	0.46	0.44	41	45963	55	8	44	3	49
ASE_00209	RNaseP_arch links to Rfam database...	0.37	0.35	0.39	31	42992	50	4	44	2	57
ASE_00210	RNaseP_arch links to Rfam database...	0.48	0.48	0.49	31	27198	38	3	29	6	34
ASE_00213	RNaseP_arch links to Rfam database...	0.60	0.61	0.60	40	27194	28	6	21	1	26
ASE_00222	RNaseP_arch links to Rfam database...	0.62	0.61	0.63	96	111949	60	4	52	4	61
ASE_00346	RNaseP_arch links to Rfam database...	0.45	0.44	0.47	43	48113	53	5	44	4	54
ASE_00359	RNaseP_arch links to Rfam database...	0.27	0.28	0.27	22	42695	61	8	53	0	58
ASE_00360	RNaseP_arch links to Rfam database...	0.30	0.29	0.30	19	29340	46	10	34	2	46
ASE_00417	RNaseP_arch links to Rfam database...	0.48	0.48	0.48	48	54515	54	8	44	2	52
ASE_00418	RNaseP_arch links to Rfam database...	0.18	0.19	0.17	14	38699	70	7	61	2	61
CRW_00013	Intron_gpI links to Rfam database...	0.24	0.25	0.24	29	103617	106	20	74	12	86
CRW_00016	Intron_gpI links to Rfam database...	0.59	0.56	0.61	67	77312	55	6	36	13	53
CRW_00610	Intron_gpI links to Rfam database...	0.46	0.44	0.48	36	36240	39	7	32	0	45
CRW_00613	Intron_gpI links to Rfam database...	0.79	0.74	0.84	58	34911	19	0	11	8	20
CRW_00614	Intron_gpI links to Rfam database...	0.09	0.11	0.08	6	121691	140	26	48	66	51
CRW_00618	Intron_gpI links to Rfam database...	0.20	0.21	0.19	13	56212	77	12	43	22	48
CRW_00619	Intron_gpI links to Rfam database...	0.23	0.26	0.20	19	164932	156	20	54	82	54
CRW_00620	Intron_gpI links to Rfam database...	0.17	0.21	0.14	13	150882	154	25	55	74	50
CRW_00621	Intron_gpI links to Rfam database...	0.17	0.17	0.17	12	153109	150	13	47	90	59
CRW_00623	Intron_gpI links to Rfam database...	0.13	0.14	0.11	12	129686	140	24	73	43	71
CRW_00633	Intron_gpI links to Rfam database...	0.39	0.42	0.38	44	63429	73	15	58	0	62
CRW_00634	Intron_gpI links to Rfam database...	0.25	0.28	0.22	26	64504	92	23	67	2	68
CRW_00670	Intron_gpI links to Rfam database...	0.45	0.47	0.44	56	69999	70	9	61	0	64
CRW_00671	Intron_gpI links to Rfam database...	0.61	0.58	0.64	68	62728	44	7	32	5	49
CRW_00672	Intron_gpI links to Rfam database...	0.76	0.77	0.75	85	72276	39	6	23	10	26
CRW_00674	Intron_gpI links to Rfam database...	0.55	0.55	0.56	68	83314	58	12	42	4	56
CRW_00676	Intron_gpI links to Rfam database...	0.52	0.57	0.47	65	93390	78	24	49	5	50
CRW_00692	Intron_gpI links to Rfam database...	0.39	0.40	0.39	36	68542	75	10	47	18	54
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	-0.01	0.00	0.00	0	2680	22	0	21	1	23
PDB_00827	Intron_gpI links to Rfam database...	0.69	0.63	0.76	51	26961	18	0	16	2	30
PDB_01092	glmS links to Rfam database...	0.68	0.62	0.76	32	10111	12	1	9	2	20
RFA_00597	Telomerase-vert links to Rfam database...	0.23	0.26	0.21	28	97771	118	29	75	14	81
RFA_00598	Telomerase-vert links to Rfam database...	0.35	0.37	0.34	37	84557	94	11	61	22	64
RFA_00599	Telomerase-vert links to Rfam database...	0.69	0.72	0.67	80	101355	60	17	23	20	31
RFA_00600	Telomerase-vert links to Rfam database...	0.26	0.30	0.23	30	120165	121	30	70	21	70
RFA_00601	Telomerase-vert links to Rfam database...	0.51	0.54	0.50	61	99112	81	10	52	19	53
RFA_00602	Telomerase-vert links to Rfam database...	0.50	0.53	0.47	61	101346	90	13	55	22	55
RFA_00632	HDV_ribozyme links to Rfam database...	0.37	0.36	0.38	10	4069	19	0	16	3	18
RFA_00636	HDV_ribozyme links to Rfam database...	0.51	0.50	0.52	14	3978	13	3	10	0	14
RFA_00767	Corona_pk3 links to Rfam database...	0.59	0.56	0.63	10	1875	6	3	3	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.47	0.44	0.50	8	1875	8	1	7	0	10
RFA_00769	Corona_pk3 links to Rfam database...	0.54	0.56	0.53	10	1934	9	4	5	0	8
RFA_00770	Corona_pk3 links to Rfam database...	-0.01	0.00	0.00	0	2002	17	1	13	3	18
RFA_00773	Corona_pk3 links to Rfam database...	0.59	0.56	0.63	10	1937	6	1	5	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.59	0.56	0.63	10	1937	6	3	3	0	8
RFA_00808	RydC links to Rfam database...	-0.01	0.00	0.00	0	2000	18	2	14	2	16
RFA_00809	RydC links to Rfam database...	0.40	0.38	0.43	6	2131	8	1	7	0	10

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	101
Total TN	3463490
Total FP	5
Total FP CONTRA	3
Total FP INCONS	1
Total FP COMP	1
Total FN	4755

Total Scores
MCC	0.141
Average MCC ± 95% Confidence Intervals	0.097 ± 0.065
Sensitivity	0.021
Positive Predictive Value	0.962
Nr of predictions	61

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
ASE_00001	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	34191	0	0	0	0	75
ASE_00002	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	35511	0	0	0	0	73
ASE_00009	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	26106	0	0	0	0	67
ASE_00014	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54285	0	0	0	0	106
ASE_00116	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	66
ASE_00120	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46971	0	0	0	0	94
ASE_00121	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	45150	0	0	0	0	92
ASE_00145	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	111
ASE_00148	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112575	0	0	0	0	154
ASE_00154	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	65341	0	0	0	0	107
ASE_00155	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	142
ASE_00187	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	115
ASE_00196	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	31626	0	0	0	0	75
ASE_00205	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	90
ASE_00209	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	43071	0	0	0	0	88
ASE_00210	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	65
ASE_00213	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	27261	0	0	0	0	66
ASE_00222	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	112101	0	0	0	0	157
ASE_00346	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	48205	0	0	0	0	97
ASE_00359	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	42778	0	0	0	0	80
ASE_00360	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	29403	0	0	0	0	65
ASE_00417	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	54615	0	0	0	0	100
ASE_00418	RNaseP_arch links to Rfam database...	0.00	0.00	0.00	0	38781	0	0	0	0	75
CRW_00013	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	103740	0	0	0	0	115
CRW_00016	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	77421	0	0	0	0	120
CRW_00610	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	81
CRW_00613	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	34980	0	0	0	0	78
CRW_00614	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	121771	0	0	0	0	57
CRW_00618	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	56280	0	0	0	0	61
CRW_00619	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	165025	0	0	0	0	73
CRW_00620	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	150975	0	0	0	0	63
CRW_00621	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	153181	0	0	0	0	71
CRW_00623	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	129795	0	0	0	0	83
CRW_00633	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	63546	0	0	0	0	106
CRW_00634	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	64620	0	0	0	0	94
CRW_00670	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	70125	0	0	0	0	120
CRW_00671	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	62835	0	0	0	0	117
CRW_00672	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	72390	0	0	0	0	111
CRW_00674	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	83436	0	0	0	0	124
CRW_00676	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	93528	0	0	0	0	115
CRW_00692	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	68635	0	0	0	0	90
PDB_00005	Tymo_tRNA-like links to Rfam database...	0.88	0.79	1.00	11	935	0	0	0	0	3
PDB_00716	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	2701	0	0	0	0	23
PDB_00827	Intron_gpI links to Rfam database...	0.00	0.00	0.00	0	27028	0	0	0	0	81
PDB_01092	glmS links to Rfam database...	0.77	0.63	0.94	33	10118	3	1	1	1	19
RFA_00597	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	97903	0	0	0	0	109
RFA_00598	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	84666	0	0	0	0	101
RFA_00599	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	111
RFA_00600	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	120295	0	0	0	0	100
RFA_00601	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	99235	0	0	0	0	114
RFA_00602	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	116
RFA_00632	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4095	0	0	0	0	28
RFA_00636	HDV_ribozyme links to Rfam database...	0.00	0.00	0.00	0	4005	0	0	0	0	28
RFA_00767	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00768	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1881	0	0	0	0	8
RFA_00769	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00770	Corona_pk3 links to Rfam database...	0.62	0.39	1.00	7	2009	0	0	0	0	11
RFA_00773	Corona_pk3 links to Rfam database...	0.71	0.56	0.91	10	1942	1	1	0	0	8
RFA_00779	Corona_pk3 links to Rfam database...	0.74	0.56	1.00	10	1943	0	0	0	0	8
RFA_00808	RydC links to Rfam database...	0.00	0.00	0.00	0	2016	0	0	0	0	16
RFA_00809	RydC links to Rfam database...	0.00	0.00	0.00	0	2145	0	0	0	0	16

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of CRWrnafold - scored higher in this pairwise comparison

1. Total counts & total scores for CRWrnafold

2. Individual counts for CRWrnafold [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]