CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	PETfold_pre2.0(20)		Carnac(seed)
MCC	0.692	>	0.431
Average MCC ± 95% Confidence Intervals	0.643 ± 0.097	>	0.373 ± 0.095
Sensitivity	0.533	>	0.206
Positive Predictive Value	0.904	<	0.907
Total TP	752	>	291
Total TN	185416	<	185927
Total FP	133	>	70
Total FP CONTRA	0	<	4
Total FP INCONS	80	>	26
Total FP COMP	53	>	40
Total FN	659	<	1120
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of PETfold_pre2.0(20) and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(20) and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(20) and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(20) and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(20) and Carnac(seed)).

^top

Performance of PETfold_pre2.0(20) - scored higher in this pairwise comparison

1. Total counts & total scores for PETfold_pre2.0(20)

Total Base Pair Counts
Total TP	752
Total TN	185416
Total FP	133
Total FP CONTRA	0
Total FP INCONS	80
Total FP COMP	53
Total FN	659

Total Scores
MCC	0.692
Average MCC ± 95% Confidence Intervals	0.643 ± 0.097
Sensitivity	0.533
Positive Predictive Value	0.904
Nr of predictions	68

^top

2. Individual counts for PETfold_pre2.0(20) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_01484	5S_rRNA links to Rfam database...	0.92	0.87	0.97	34	7468	3	0	1	2	5
CRW_01499	5S_rRNA links to Rfam database...	0.72	0.66	0.79	27	7967	7	0	7	0	14
CRW_01603	5S_rRNA links to Rfam database...	0.89	0.84	0.94	31	7107	5	0	2	3	6
PDB_00012	mir-TAR links to Rfam database...	1.00	1.00	1.00	7	399	2	0	0	2	0
PDB_00213	Gammaretro_CES links to Rfam database...	0.85	0.73	1.00	30	5020	0	0	0	0	11
PDB_00553	mir-TAR links to Rfam database...	1.00	1.00	1.00	11	454	0	0	0	0	0
PDB_00810	s2m links to Rfam database...	0.84	0.71	1.00	12	1069	0	0	0	0	5
PDB_01050	UnaL2 links to Rfam database...	0.78	0.62	1.00	8	622	2	0	0	2	5
PDB_01152	HIV_FS2 links to Rfam database...	0.80	0.71	0.91	10	550	1	0	1	0	4
RFA_00389	Hammerhead_3 links to Rfam database...	0.97	0.93	1.00	14	1417	2	0	0	2	1
RFA_00390	Hammerhead_3 links to Rfam database...	0.97	0.93	1.00	14	1417	2	0	0	2	1
RFA_00391	Hammerhead_3 links to Rfam database...	0.97	0.93	1.00	14	1417	2	0	0	2	1
RFA_00409	Hammerhead_3 links to Rfam database...	0.97	0.93	1.00	14	1417	2	0	0	2	1
RFA_00416	Hammerhead_3 links to Rfam database...	0.97	0.93	1.00	14	1471	2	0	0	2	1
RFA_00433	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00434	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00436	Hammerhead_3 links to Rfam database...	0.97	0.93	1.00	14	1417	1	0	0	1	1
RFA_00440	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1470	2	0	0	2	0
RFA_00442	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00587	Y_RNA links to Rfam database...	0.93	0.93	0.93	14	4836	9	0	1	8	1
RFA_00658	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	1126	3	0	2	1	14
RFA_00664	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	986	4	0	4	0	14
RFA_00667	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	985	5	0	5	0	14
RFA_00668	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	986	4	0	4	0	14
RFA_00672	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	898	5	0	5	0	13
RFA_00673	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	1123	5	0	5	0	14
RFA_00674	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	1124	4	0	4	0	14
RFA_00675	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	985	5	0	5	0	14
RFA_00677	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	984	6	0	6	0	14
RFA_00678	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	941	5	0	5	0	14
RFA_00680	Hammerhead_1 links to Rfam database...	0.15	0.07	0.33	1	1125	3	0	2	1	13
RFA_00704	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	988	3	0	2	1	14
RFA_00707	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	1029	6	0	6	0	14
RFA_00715	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	942	4	0	4	0	14
RFA_00717	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	902	1	0	1	0	14
RFA_00730	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00731	s2m links to Rfam database...	1.00	1.00	1.00	12	891	0	0	0	0	0
RFA_00733	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00734	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00736	s2m links to Rfam database...	1.00	1.00	1.00	12	891	0	0	0	0	0
RFA_00737	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00745	s2m links to Rfam database...	1.00	1.00	1.00	12	934	1	0	0	1	0
RFA_00749	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00758	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00762	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00763	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00764	s2m links to Rfam database...	1.00	1.00	1.00	12	891	1	0	0	1	0
RFA_00765	s2m links to Rfam database...	1.00	1.00	1.00	12	891	0	0	0	0	0
RFA_00786	Gammaretro_CES links to Rfam database...	0.97	0.94	1.00	30	5020	0	0	0	0	2
RFA_00791	Gammaretro_CES links to Rfam database...	0.97	0.94	1.00	30	5121	0	0	0	0	2
RFA_00792	Gammaretro_CES links to Rfam database...	0.94	0.91	0.97	29	5020	1	0	1	0	3
RFA_00801	Gammaretro_CES links to Rfam database...	0.97	0.94	1.00	30	5020	0	0	0	0	2
SPR_00273	tRNA links to Rfam database...	0.95	0.95	0.95	20	2829	1	0	1	0	1
SPR_00721	tRNA links to Rfam database...	1.00	1.00	1.00	21	2905	0	0	0	0	0
SRP_00020	Bacteria_small_SRP links to Rfam database...	0.58	0.33	1.00	11	5554	0	0	0	0	22
SRP_00058	Bacteria_small_SRP links to Rfam database...	0.54	0.29	1.00	10	5768	0	0	0	0	24
SRP_00084	Bacteria_small_SRP links to Rfam database...	0.45	0.21	1.00	7	5453	0	0	0	0	27
SRP_00105	Bacteria_small_SRP links to Rfam database...	0.45	0.21	1.00	7	5558	0	0	0	0	27
SRP_00134	Bacteria_large_SRP links to Rfam database...	0.41	0.28	0.62	8	5982	6	0	5	1	21
SRP_00141	Bacteria_small_SRP links to Rfam database...	0.38	0.15	1.00	6	6322	1	0	0	1	35
SRP_00200	Bacteria_small_SRP links to Rfam database...	0.41	0.17	1.00	6	6897	1	0	0	1	30
SRP_00231	Bacteria_small_SRP links to Rfam database...	0.42	0.21	0.88	7	5452	1	0	1	0	27
SRP_00273	Bacteria_small_SRP links to Rfam database...	0.52	0.28	1.00	11	6775	0	0	0	0	29
SRP_00274	Bacteria_small_SRP links to Rfam database...	0.43	0.18	1.00	7	6896	0	0	0	0	31
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.44	0.19	1.00	7	5453	0	0	0	0	29
SRP_00341	Bacteria_small_SRP links to Rfam database...	0.47	0.23	1.00	7	4746	0	0	0	0	24
SRP_00357	Bacteria_small_SRP links to Rfam database...	0.44	0.19	1.00	7	5349	0	0	0	0	29
SRP_00367	Bacteria_small_SRP links to Rfam database...	0.41	0.17	1.00	6	6780	1	0	0	1	29

^top

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

Total Base Pair Counts
Total TP	291
Total TN	185927
Total FP	70
Total FP CONTRA	4
Total FP INCONS	26
Total FP COMP	40
Total FN	1120

Total Scores
MCC	0.431
Average MCC ± 95% Confidence Intervals	0.373 ± 0.095
Sensitivity	0.206
Positive Predictive Value	0.907
Nr of predictions	68

^top

2. Individual counts for Carnac(seed) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_01484	5S_rRNA links to Rfam database...	0.00	0.00	0.00	0	7503	0	0	0	0	39
CRW_01499	5S_rRNA links to Rfam database...	0.00	0.00	0.00	0	8001	0	0	0	0	41
CRW_01603	5S_rRNA links to Rfam database...	0.00	0.00	0.00	0	7140	0	0	0	0	37
PDB_00012	mir-TAR links to Rfam database...	1.00	1.00	1.00	7	399	3	0	0	3	0
PDB_00213	Gammaretro_CES links to Rfam database...	0.58	0.34	1.00	14	5036	0	0	0	0	27
PDB_00553	mir-TAR links to Rfam database...	0.79	0.64	1.00	7	458	0	0	0	0	4
PDB_00810	s2m links to Rfam database...	0.64	0.41	1.00	7	1074	0	0	0	0	10
PDB_01050	UnaL2 links to Rfam database...	0.62	0.38	1.00	5	625	1	0	0	1	8
PDB_01152	HIV_FS2 links to Rfam database...	0.88	0.79	1.00	11	550	0	0	0	0	3
RFA_00389	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00390	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00391	Hammerhead_3 links to Rfam database...	0.93	0.87	1.00	13	1418	2	0	0	2	2
RFA_00409	Hammerhead_3 links to Rfam database...	0.77	0.60	1.00	9	1422	1	0	0	1	6
RFA_00416	Hammerhead_3 links to Rfam database...	0.93	0.93	0.93	14	1470	4	0	1	3	1
RFA_00433	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00434	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00436	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00440	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1470	2	0	0	2	0
RFA_00442	Hammerhead_3 links to Rfam database...	1.00	1.00	1.00	15	1416	2	0	0	2	0
RFA_00587	Y_RNA links to Rfam database...	0.00	0.00	0.00	0	4851	0	0	0	0	15
RFA_00658	Hammerhead_1 links to Rfam database...	0.40	0.29	0.57	4	1121	5	0	3	2	10
RFA_00664	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	990	0	0	0	0	14
RFA_00667	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	985	5	1	4	0	14
RFA_00668	Hammerhead_1 links to Rfam database...	0.50	0.43	0.60	6	980	4	0	4	0	8
RFA_00672	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	903	0	0	0	0	13
RFA_00673	Hammerhead_1 links to Rfam database...	0.47	0.36	0.63	5	1120	4	0	3	1	9
RFA_00674	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	1128	0	0	0	0	14
RFA_00675	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	990	0	0	0	0	14
RFA_00677	Hammerhead_1 links to Rfam database...	0.53	0.43	0.67	6	981	4	0	3	1	8
RFA_00678	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	946	0	0	0	0	14
RFA_00680	Hammerhead_1 links to Rfam database...	0.84	0.71	1.00	10	1118	2	0	0	2	4
RFA_00704	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	990	0	0	0	0	14
RFA_00707	Hammerhead_1 links to Rfam database...	0.53	0.29	1.00	4	1031	0	0	0	0	10
RFA_00715	Hammerhead_1 links to Rfam database...	-0.01	0.00	0.00	0	943	3	1	2	0	14
RFA_00717	Hammerhead_1 links to Rfam database...	0.00	0.00	0.00	0	903	0	0	0	0	14
RFA_00730	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00731	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00733	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00734	s2m links to Rfam database...	0.64	0.42	1.00	5	898	0	0	0	0	7
RFA_00736	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00737	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00745	s2m links to Rfam database...	0.64	0.42	1.00	5	941	1	0	0	1	7
RFA_00749	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00758	s2m links to Rfam database...	0.00	0.00	0.00	0	903	0	0	0	0	12
RFA_00762	s2m links to Rfam database...	0.64	0.42	1.00	5	898	0	0	0	0	7
RFA_00763	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00764	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00765	s2m links to Rfam database...	0.64	0.42	1.00	5	898	1	0	0	1	7
RFA_00786	Gammaretro_CES links to Rfam database...	0.25	0.13	0.50	4	5042	4	1	3	0	28
RFA_00791	Gammaretro_CES links to Rfam database...	0.00	0.00	0.00	0	5151	0	0	0	0	32
RFA_00792	Gammaretro_CES links to Rfam database...	0.00	0.00	0.00	0	5046	4	1	3	0	32
RFA_00801	Gammaretro_CES links to Rfam database...	0.00	0.00	0.00	0	5050	0	0	0	0	32
SPR_00273	tRNA links to Rfam database...	0.00	0.00	0.00	0	2850	0	0	0	0	21
SPR_00721	tRNA links to Rfam database...	0.00	0.00	0.00	0	2926	0	0	0	0	21
SRP_00020	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5565	0	0	0	0	33
SRP_00058	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5778	0	0	0	0	34
SRP_00084	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5460	0	0	0	0	34
SRP_00105	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5565	0	0	0	0	34
SRP_00134	Bacteria_large_SRP links to Rfam database...	0.00	0.00	0.00	0	5995	0	0	0	0	29
SRP_00141	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	6328	0	0	0	0	41
SRP_00200	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	6903	0	0	0	0	36
SRP_00231	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5460	0	0	0	0	34
SRP_00273	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	6786	0	0	0	0	40
SRP_00274	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	6903	0	0	0	0	38
SRP_00338	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5460	0	0	0	0	36
SRP_00341	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	4753	0	0	0	0	31
SRP_00357	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	5356	0	0	0	0	36
SRP_00367	Bacteria_small_SRP links to Rfam database...	0.00	0.00	0.00	0	6786	0	0	0	0	35

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of PETfold_pre2.0(20) - scored higher in this pairwise comparison

1. Total counts & total scores for PETfold_pre2.0(20)

2. Individual counts for PETfold_pre2.0(20) [ download as .csv ]

Performance of Carnac(seed) - scored lower in this pairwise comparison

1. Total counts & total scores for Carnac(seed)

2. Individual counts for Carnac(seed) [ download as .csv ]