CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	Carnac(20)		Mastr(20)
MCC	0.445	>	0.000
Average MCC ± 95% Confidence Intervals	0.445 ± 0.019	>	0.000 ± 0.000
Sensitivity	0.241	>	0.000
Positive Predictive Value	0.824	>	0.000
Total TP	3217	>	0
Total TN	44604823	<	44608726
Total FP	1342	>	0
Total FP CONTRA	38	>	0
Total FP INCONS	648	>	0
Total FP COMP	656	>	0
Total FN	10143	<	13360
P-value	5.23657817852e-08

^top

Performance plots

Comparison of performance of Carnac(20) and Mastr(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Carnac(20) and Mastr(20)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Carnac(20) and Mastr(20)).
Comparison of average Matthews Correlation Coefficients (MCCs) for Carnac(20) and Mastr(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Carnac(20) and Mastr(20)).

^top

Performance of Carnac(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Carnac(20)

Total Base Pair Counts
Total TP	3217
Total TN	44604823
Total FP	1342
Total FP CONTRA	38
Total FP INCONS	648
Total FP COMP	656
Total FN	10143

Total Scores
MCC	0.445
Average MCC ± 95% Confidence Intervals	0.445 ± 0.019
Sensitivity	0.241
Positive Predictive Value	0.824
Nr of predictions	27

^top

2. Individual counts for Carnac(20) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00298	SSU_rRNA_eukarya links to Rfam database...	0.41	0.19	0.86	95	1620790	37	0	15	22	399
CRW_00301	SSU_rRNA_eukarya links to Rfam database...	0.46	0.25	0.83	131	1762345	56	0	27	29	388
CRW_00306	SSU_rRNA_eukarya links to Rfam database...	0.55	0.32	0.95	146	1445697	38	0	7	31	316
CRW_00310	SSU_rRNA_eukarya links to Rfam database...	0.42	0.22	0.81	116	1805806	46	5	23	18	405
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.45	0.23	0.92	113	1627987	32	1	9	22	389
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.34	0.17	0.67	89	1988883	69	3	40	26	428
CRW_00332	SSU_rRNA_eukarya links to Rfam database...	0.39	0.20	0.76	97	1769893	46	1	30	15	380
CRW_00333	SSU_rRNA_eukarya links to Rfam database...	0.35	0.17	0.70	88	1892359	50	4	34	12	426
CRW_00345	SSU_rRNA_eukarya links to Rfam database...	0.48	0.29	0.81	145	1586691	63	2	33	28	353
CRW_00346	SSU_rRNA_eukarya links to Rfam database...	0.47	0.28	0.79	138	1584915	66	1	36	29	363
CRW_00348	SSU_rRNA_eukarya links to Rfam database...	0.40	0.23	0.72	114	1615344	78	4	41	33	389
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.49	0.25	0.96	129	1675231	29	0	5	24	378
CRW_00366	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.87	122	1579613	37	1	17	19	360
CRW_00370	SSU_rRNA_eukarya links to Rfam database...	0.47	0.27	0.84	135	1640605	54	1	25	28	371
CRW_00371	SSU_rRNA_eukarya links to Rfam database...	0.48	0.29	0.81	143	1611733	59	1	33	25	353
CRW_00372	SSU_rRNA_eukarya links to Rfam database...	0.44	0.23	0.84	117	1626166	46	2	21	23	381
CRW_00379	SSU_rRNA_eukarya links to Rfam database...	0.49	0.26	0.93	140	2126802	50	1	10	39	394
CRW_00403	SSU_rRNA_eukarya links to Rfam database...	0.48	0.27	0.83	127	1606375	60	0	26	34	337
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.39	0.19	0.81	94	1615387	42	0	22	20	402
CRW_00598	SSU_rRNA_eukarya links to Rfam database...	0.42	0.20	0.85	100	1666107	41	0	18	23	391
CRW_00758	SSU_rRNA_eukarya links to Rfam database...	0.46	0.26	0.79	127	1640605	57	4	30	23	356
CRW_00759	SSU_rRNA_eukarya links to Rfam database...	0.39	0.19	0.79	93	1637028	51	3	21	27	388
CRW_00760	SSU_rRNA_eukarya links to Rfam database...	0.45	0.24	0.84	117	1638815	55	0	23	32	369
CRW_00772	SSU_rRNA_bacteria links to Rfam database...	0.48	0.30	0.77	122	945842	42	3	33	6	282
CRW_01153	SSU_rRNA_eukarya links to Rfam database...	0.44	0.25	0.76	126	1606363	56	1	38	17	377
CRW_01171	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.90	125	1593866	44	0	14	30	381
CRW_01300	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.88	128	1693575	38	0	17	21	387

^top

Performance of Mastr(20) - scored lower in this pairwise comparison

1. Total counts & total scores for Mastr(20)

Total Base Pair Counts
Total TP	0
Total TN	44608726
Total FP	0
Total FP CONTRA	0
Total FP INCONS	0
Total FP COMP	0
Total FN	13360

Total Scores
MCC	0.000
Average MCC ± 95% Confidence Intervals	0.000 ± 0.000
Sensitivity	0.000
Positive Predictive Value	0.000
Nr of predictions	27

^top

2. Individual counts for Mastr(20) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00298	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1620900	0	0	0	0	494
CRW_00301	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1762503	0	0	0	0	519
CRW_00306	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1445850	0	0	0	0	462
CRW_00310	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1805950	0	0	0	0	521
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1628110	0	0	0	0	502
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1989015	0	0	0	0	517
CRW_00332	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1770021	0	0	0	0	477
CRW_00333	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1892485	0	0	0	0	514
CRW_00345	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1586871	0	0	0	0	498
CRW_00346	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1585090	0	0	0	0	501
CRW_00348	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1615503	0	0	0	0	503
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1675365	0	0	0	0	507
CRW_00366	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1579753	0	0	0	0	482
CRW_00370	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1640766	0	0	0	0	506
CRW_00371	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1611910	0	0	0	0	496
CRW_00372	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1626306	0	0	0	0	498
CRW_00379	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	2126953	0	0	0	0	534
CRW_00403	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1606528	0	0	0	0	464
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1615503	0	0	0	0	496
CRW_00598	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1666225	0	0	0	0	491
CRW_00758	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1640766	0	0	0	0	483
CRW_00759	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1637145	0	0	0	0	481
CRW_00760	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1638955	0	0	0	0	486
CRW_00772	SSU_rRNA_bacteria links to Rfam database...	0.00	0.00	0.00	0	946000	0	0	0	0	404
CRW_01153	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1606528	0	0	0	0	503
CRW_01171	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1594005	0	0	0	0	506
CRW_01300	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1693720	0	0	0	0	515

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of Carnac(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Carnac(20)

2. Individual counts for Carnac(20) [ download as .csv ]

Performance of Mastr(20) - scored lower in this pairwise comparison

1. Total counts & total scores for Mastr(20)

2. Individual counts for Mastr(20) [ download as .csv ]