CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	Murlet(seed)		RNAshapes
MCC	0.544	>	0.542
Average MCC ± 95% Confidence Intervals	0.533 ± 0.096	<	0.538 ± 0.072
Sensitivity	0.354	<	0.549
Positive Predictive Value	0.836	>	0.537
Total TP	782	<	1211
Total TN	1143390	>	1142068
Total FP	201	<	1243
Total FP CONTRA	9	<	177
Total FP INCONS	144	<	869
Total FP COMP	48	<	197
Total FN	1425	>	996
P-value	0.239390405605

^top

Performance plots

Comparison of performance of Murlet(seed) and RNAshapes. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Murlet(seed) and RNAshapes).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Murlet(seed) and RNAshapes).
Comparison of average Matthews Correlation Coefficients (MCCs) for Murlet(seed) and RNAshapes. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Murlet(seed) and RNAshapes).

^top

Performance of Murlet(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

Total Base Pair Counts
Total TP	782
Total TN	1143390
Total FP	201
Total FP CONTRA	9
Total FP INCONS	144
Total FP COMP	48
Total FN	1425

Total Scores
MCC	0.544
Average MCC ± 95% Confidence Intervals	0.533 ± 0.096
Sensitivity	0.354
Positive Predictive Value	0.836
Nr of predictions	28

^top

2. Individual counts for Murlet(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00606	Telomerase-cil links to Rfam database...	0.00	0.00	0.00	0	21321	0	0	0	0	39
RFA_00620	Telomerase-cil links to Rfam database...	0.00	0.00	0.00	0	21945	0	0	0	0	39
RFA_00626	rne5 links to Rfam database...	0.81	0.66	1.00	57	56559	4	0	0	4	30
RFA_00627	rne5 links to Rfam database...	0.84	0.70	1.00	61	56892	5	0	0	5	26
RFA_00628	rne5 links to Rfam database...	0.83	0.69	1.00	59	57232	8	0	0	8	27
RFA_00639	7SK links to Rfam database...	0.21	0.11	0.40	10	54590	15	4	11	0	77
RFA_00814	R2_retro_el links to Rfam database...	0.75	0.63	0.90	26	25171	4	0	3	1	15
RFA_00815	R2_retro_el links to Rfam database...	0.83	0.68	1.00	28	24503	1	0	0	1	13
RFA_00816	R2_retro_el links to Rfam database...	0.75	0.63	0.90	26	23191	4	0	3	1	15
RFA_00817	R2_retro_el links to Rfam database...	0.83	0.68	1.00	28	21917	1	0	0	1	13
RFA_00818	R2_retro_el links to Rfam database...	0.83	0.68	1.00	28	20273	1	0	0	1	13
RFA_00819	R2_retro_el links to Rfam database...	0.83	0.68	1.00	28	27938	0	0	0	0	13
SRP_00079	Plant_SRP links to Rfam database...	0.54	0.33	0.88	29	41872	6	0	4	2	59
SRP_00124	Metazoa_SRP links to Rfam database...	0.15	0.09	0.24	8	37094	27	3	23	1	77
SRP_00182	Plant_SRP links to Rfam database...	0.59	0.35	1.00	35	46021	1	0	0	1	66
SRP_00197	Archaea_SRP links to Rfam database...	0.47	0.28	0.80	28	49420	8	0	7	1	73
SRP_00198	Archaea_SRP links to Rfam database...	0.47	0.27	0.82	28	51006	7	0	6	1	75
SRP_00199	Archaea_SRP links to Rfam database...	0.47	0.24	0.93	25	51013	3	0	2	1	80
SRP_00202	Archaea_SRP links to Rfam database...	0.48	0.25	0.94	29	54254	2	0	2	0	88
SRP_00241	Plant_SRP links to Rfam database...	0.51	0.33	0.79	27	46022	9	0	7	2	55
SRP_00255	Archaea_SRP links to Rfam database...	0.65	0.43	1.00	40	47546	2	0	0	2	54
SRP_00260	Metazoa_SRP links to Rfam database...	0.21	0.14	0.33	14	47852	29	1	28	0	86
SRP_00278	Archaea_SRP links to Rfam database...	0.53	0.30	0.94	31	46023	3	0	2	1	74
SRP_00322	Archaea_SRP links to Rfam database...	0.44	0.25	0.79	27	48482	8	0	7	1	82
SRP_00328	Metazoa_SRP links to Rfam database...	0.49	0.34	0.71	29	39580	16	1	11	4	56
SRP_00329	Metazoa_SRP links to Rfam database...	0.50	0.34	0.73	29	39863	16	0	11	5	56
SRP_00339	Archaea_SRP links to Rfam database...	0.49	0.32	0.77	30	44512	9	0	9	0	64
SRP_00340	Metazoa_SRP links to Rfam database...	0.44	0.27	0.73	22	41298	12	0	8	4	60

^top

Performance of RNAshapes - scored lower in this pairwise comparison

1. Total counts & total scores for RNAshapes

Total Base Pair Counts
Total TP	1211
Total TN	1142068
Total FP	1243
Total FP CONTRA	177
Total FP INCONS	869
Total FP COMP	197
Total FN	996

Total Scores
MCC	0.542
Average MCC ± 95% Confidence Intervals	0.538 ± 0.072
Sensitivity	0.549
Positive Predictive Value	0.537
Nr of predictions	28

^top

2. Individual counts for RNAshapes [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00606	Telomerase-cil links to Rfam database...	0.67	0.67	0.67	26	21282	32	1	12	19	13
RFA_00620	Telomerase-cil links to Rfam database...	0.44	0.46	0.42	18	21902	36	7	18	11	21
RFA_00626	rne5 links to Rfam database...	0.45	0.45	0.46	39	56531	61	9	37	15	48
RFA_00627	rne5 links to Rfam database...	0.63	0.61	0.66	53	56873	42	4	23	15	34
RFA_00628	rne5 links to Rfam database...	0.82	0.80	0.84	69	57209	33	1	12	20	17
RFA_00639	7SK links to Rfam database...	0.22	0.24	0.21	21	54515	82	20	59	3	66
RFA_00814	R2_retro_el links to Rfam database...	0.62	0.63	0.60	26	25157	26	6	11	9	15
RFA_00815	R2_retro_el links to Rfam database...	0.67	0.71	0.63	29	24485	31	1	16	14	12
RFA_00816	R2_retro_el links to Rfam database...	0.21	0.24	0.18	10	23164	47	18	28	1	31
RFA_00817	R2_retro_el links to Rfam database...	0.10	0.12	0.08	5	21881	60	21	38	1	36
RFA_00818	R2_retro_el links to Rfam database...	0.52	0.56	0.48	23	20253	31	9	16	6	18
RFA_00819	R2_retro_el links to Rfam database...	0.84	0.80	0.87	33	27928	36	0	5	31	8
SRP_00079	Plant_SRP links to Rfam database...	0.53	0.52	0.53	46	41819	52	1	39	12	42
SRP_00124	Metazoa_SRP links to Rfam database...	0.42	0.41	0.42	35	37045	51	5	43	3	50
SRP_00182	Plant_SRP links to Rfam database...	0.59	0.56	0.61	57	45963	37	5	31	1	44
SRP_00197	Archaea_SRP links to Rfam database...	0.31	0.31	0.32	31	49358	67	4	62	1	70
SRP_00198	Archaea_SRP links to Rfam database...	0.46	0.45	0.47	46	50943	56	3	48	5	57
SRP_00199	Archaea_SRP links to Rfam database...	0.31	0.30	0.33	31	50945	67	3	61	3	74
SRP_00202	Archaea_SRP links to Rfam database...	0.78	0.79	0.78	92	54167	30	3	23	4	25
SRP_00241	Plant_SRP links to Rfam database...	0.55	0.59	0.52	48	45963	47	13	32	2	34
SRP_00255	Archaea_SRP links to Rfam database...	0.59	0.60	0.58	56	47489	47	2	39	6	38
SRP_00260	Metazoa_SRP links to Rfam database...	0.75	0.77	0.73	77	47789	31	6	23	2	23
SRP_00278	Archaea_SRP links to Rfam database...	0.70	0.69	0.72	72	45956	30	3	25	2	33
SRP_00322	Archaea_SRP links to Rfam database...	0.57	0.55	0.58	60	48413	44	3	40	1	49
SRP_00328	Metazoa_SRP links to Rfam database...	0.64	0.65	0.63	55	39533	38	8	25	5	30
SRP_00329	Metazoa_SRP links to Rfam database...	0.67	0.66	0.67	56	39820	28	4	23	1	29
SRP_00339	Archaea_SRP links to Rfam database...	0.51	0.53	0.49	50	44449	54	8	44	2	44
SRP_00340	Metazoa_SRP links to Rfam database...	0.54	0.57	0.51	47	41236	47	9	36	2	35

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of Murlet(seed) - scored higher in this pairwise comparison

1. Total counts & total scores for Murlet(seed)

2. Individual counts for Murlet(seed) [ download as .csv ]

Performance of RNAshapes - scored lower in this pairwise comparison

1. Total counts & total scores for RNAshapes

2. Individual counts for RNAshapes [ download as .csv ]