CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	Carnac(20)		Sfold
MCC	0.558	>	0.453
Average MCC ± 95% Confidence Intervals	0.553 ± 0.023	>	0.455 ± 0.031
Sensitivity	0.348	<	0.403
Positive Predictive Value	0.895	>	0.510
Total TP	12872	<	14880
Total TN	104356083	>	104341300
Total FP	2613	<	16483
Total FP CONTRA	170	<	2320
Total FP INCONS	1341	<	11966
Total FP COMP	1102	<	2197
Total FN	24092	>	22084
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of Carnac(20) and Sfold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Carnac(20) and Sfold).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Carnac(20) and Sfold).
Comparison of average Matthews Correlation Coefficients (MCCs) for Carnac(20) and Sfold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Carnac(20) and Sfold).

^top

Performance of Carnac(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Carnac(20)

Total Base Pair Counts
Total TP	12872
Total TN	104356083
Total FP	2613
Total FP CONTRA	170
Total FP INCONS	1341
Total FP COMP	1102
Total FN	24092

Total Scores
MCC	0.558
Average MCC ± 95% Confidence Intervals	0.553 ± 0.023
Sensitivity	0.348
Positive Predictive Value	0.895
Nr of predictions	77

^top

2. Individual counts for Carnac(20) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.74	0.59	0.94	270	1085313	35	5	13	17	188
CRW_00034	SSU_rRNA_bacteria links to Rfam database...	0.56	0.35	0.90	166	1142131	28	4	15	9	308
CRW_00048	SSU_rRNA_bacteria links to Rfam database...	0.62	0.43	0.91	202	1170992	29	3	18	8	268
CRW_00054	SSU_rRNA_bacteria links to Rfam database...	0.62	0.42	0.91	191	1107607	23	4	14	5	265
CRW_00059	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.95	204	1172532	17	3	7	7	269
CRW_00061	SSU_rRNA_bacteria links to Rfam database...	0.63	0.42	0.94	199	1181742	19	2	10	7	277
CRW_00068	SSU_rRNA_bacteria links to Rfam database...	0.69	0.51	0.94	234	1109056	28	2	13	13	223
CRW_00096	SSU_rRNA_bacteria links to Rfam database...	0.62	0.43	0.92	198	1178664	29	0	18	11	267
CRW_00140	SSU_rRNA_bacteria links to Rfam database...	0.71	0.53	0.95	246	1136018	21	2	12	7	217
CRW_00141	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.93	207	1201802	26	4	12	10	269
CRW_00150	SSU_rRNA_bacteria links to Rfam database...	0.66	0.46	0.95	216	1152694	16	3	8	5	250
CRW_00152	SSU_rRNA_bacteria links to Rfam database...	0.69	0.52	0.92	244	1180151	26	4	17	5	227
CRW_00166	SSU_rRNA_bacteria links to Rfam database...	0.64	0.44	0.93	207	1190974	24	0	15	9	261
CRW_00173	SSU_rRNA_bacteria links to Rfam database...	0.56	0.36	0.89	158	1082478	26	2	18	6	287
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.52	0.32	0.87	149	1178708	29	3	20	6	324
CRW_00196	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.96	204	1208022	12	2	7	3	271
CRW_00230	SSU_rRNA_bacteria links to Rfam database...	0.66	0.47	0.93	227	1198681	28	3	15	10	251
CRW_00245	SSU_rRNA_bacteria links to Rfam database...	0.65	0.45	0.93	205	1106108	20	3	12	5	248
CRW_00248	SSU_rRNA_bacteria links to Rfam database...	0.56	0.35	0.91	168	1218956	27	4	13	10	316
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.68	0.50	0.91	241	1198660	33	4	21	8	237
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.97	200	1192533	11	3	4	4	268
CRW_00270	SSU_rRNA_bacteria links to Rfam database...	0.66	0.47	0.93	212	1112057	22	4	13	5	243
CRW_00271	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.94	191	1122548	17	3	9	5	271
CRW_00273	SSU_rRNA_bacteria links to Rfam database...	0.57	0.35	0.92	156	1085431	16	4	10	2	290
CRW_00275	SSU_rRNA_bacteria links to Rfam database...	0.60	0.37	0.95	171	1115091	13	2	7	4	286
CRW_00277	SSU_rRNA_bacteria links to Rfam database...	0.64	0.44	0.93	201	1110578	20	3	13	4	252
CRW_00278	SSU_rRNA_bacteria links to Rfam database...	0.52	0.29	0.92	130	1130115	15	4	7	4	320
CRW_00279	SSU_rRNA_bacteria links to Rfam database...	0.68	0.51	0.92	237	1180158	31	3	18	10	227
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.66	0.46	0.96	209	1118042	12	1	8	3	244
CRW_00281	SSU_rRNA_bacteria links to Rfam database...	0.65	0.45	0.95	209	1145120	17	2	10	5	255
CRW_00284	SSU_rRNA_bacteria links to Rfam database...	0.53	0.33	0.87	150	1131587	27	3	20	4	306
CRW_00285	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.93	188	1118058	18	2	12	4	270
CRW_00289	SSU_rRNA_bacteria links to Rfam database...	0.58	0.41	0.84	194	1189422	41	5	32	4	282
CRW_00298	SSU_rRNA_eukarya links to Rfam database...	0.41	0.19	0.86	95	1620790	37	0	15	22	399
CRW_00301	SSU_rRNA_eukarya links to Rfam database...	0.46	0.25	0.83	131	1762345	56	0	27	29	388
CRW_00306	SSU_rRNA_eukarya links to Rfam database...	0.55	0.32	0.95	146	1445697	38	0	7	31	316
CRW_00310	SSU_rRNA_eukarya links to Rfam database...	0.42	0.22	0.81	116	1805806	46	5	23	18	405
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.45	0.23	0.92	113	1627987	32	1	9	22	389
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.34	0.17	0.67	89	1988883	69	3	40	26	428
CRW_00332	SSU_rRNA_eukarya links to Rfam database...	0.39	0.20	0.76	97	1769893	46	1	30	15	380
CRW_00333	SSU_rRNA_eukarya links to Rfam database...	0.35	0.17	0.70	88	1892359	50	4	34	12	426
CRW_00345	SSU_rRNA_eukarya links to Rfam database...	0.48	0.29	0.81	145	1586691	63	2	33	28	353
CRW_00346	SSU_rRNA_eukarya links to Rfam database...	0.47	0.28	0.79	138	1584915	66	1	36	29	363
CRW_00348	SSU_rRNA_eukarya links to Rfam database...	0.40	0.23	0.72	114	1615344	78	4	41	33	389
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.49	0.25	0.96	129	1675231	29	0	5	24	378
CRW_00366	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.87	122	1579613	37	1	17	19	360
CRW_00370	SSU_rRNA_eukarya links to Rfam database...	0.47	0.27	0.84	135	1640605	54	1	25	28	371
CRW_00371	SSU_rRNA_eukarya links to Rfam database...	0.48	0.29	0.81	143	1611733	59	1	33	25	353
CRW_00372	SSU_rRNA_eukarya links to Rfam database...	0.44	0.23	0.84	117	1626166	46	2	21	23	381
CRW_00379	SSU_rRNA_eukarya links to Rfam database...	0.49	0.26	0.93	140	2126802	50	1	10	39	394
CRW_00386	SSU_rRNA_eukarya links to Rfam database...	0.30	0.13	0.71	67	1751161	53	2	26	25	448
CRW_00403	SSU_rRNA_eukarya links to Rfam database...	0.48	0.27	0.83	127	1606375	60	0	26	34	337
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.39	0.19	0.81	94	1615387	42	0	22	20	402
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.38	0.19	0.74	96	1666095	60	1	33	26	399
CRW_00466	SSU_rRNA_bacteria links to Rfam database...	0.62	0.42	0.91	192	1116554	28	4	15	9	266
CRW_00589	SSU_rRNA_bacteria links to Rfam database...	0.58	0.35	0.97	156	1094299	10	0	5	5	289
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.55	0.33	0.93	148	1119597	15	1	10	4	304
CRW_00591	SSU_rRNA_bacteria links to Rfam database...	0.66	0.46	0.93	206	1098682	26	1	14	11	240
CRW_00598	SSU_rRNA_eukarya links to Rfam database...	0.42	0.20	0.85	100	1666107	41	0	18	23	391
CRW_00758	SSU_rRNA_eukarya links to Rfam database...	0.46	0.26	0.79	127	1640605	57	4	30	23	356
CRW_00759	SSU_rRNA_eukarya links to Rfam database...	0.39	0.19	0.79	93	1637028	51	3	21	27	388
CRW_00760	SSU_rRNA_eukarya links to Rfam database...	0.45	0.24	0.84	117	1638815	55	0	23	32	369
CRW_00772	SSU_rRNA_bacteria links to Rfam database...	0.48	0.30	0.77	122	945842	42	3	33	6	282
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.61	0.43	0.88	191	1137570	34	1	24	9	256
CRW_00795	SSU_rRNA_bacteria links to Rfam database...	0.59	0.38	0.90	150	1152755	40	0	16	24	244
CRW_00849	SSU_rRNA_bacteria links to Rfam database...	0.62	0.42	0.92	196	1118046	27	3	15	9	274
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.60	0.39	0.93	181	1136084	19	3	10	6	283
CRW_01153	SSU_rRNA_eukarya links to Rfam database...	0.44	0.25	0.76	126	1606363	56	1	38	17	377
CRW_01171	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.90	125	1593866	44	0	14	30	381
CRW_01300	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.88	128	1693575	38	0	17	21	387
PDB_00703	SSU_rRNA_bacteria links to Rfam database...	0.60	0.40	0.92	196	1073633	28	3	13	12	299
PDB_00854	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.95	218	1500548	29	3	9	17	317
PDB_01088	SSU_rRNA_bacteria links to Rfam database...	0.59	0.38	0.92	206	1471246	35	2	16	17	341
PDB_01103	SSU_rRNA_bacteria links to Rfam database...	0.62	0.40	0.96	226	1606292	20	3	7	10	336
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.58	0.36	0.94	192	1433767	24	1	11	12	348
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.93	216	1204895	30	4	13	13	311
PDB_01273	SSU_rRNA_bacteria links to Rfam database...	0.66	0.48	0.92	234	1118007	37	4	15	18	257

^top

Performance of Sfold - scored lower in this pairwise comparison

1. Total counts & total scores for Sfold

Total Base Pair Counts
Total TP	14880
Total TN	104341300
Total FP	16483
Total FP CONTRA	2320
Total FP INCONS	11966
Total FP COMP	2197
Total FN	22084

Total Scores
MCC	0.453
Average MCC ± 95% Confidence Intervals	0.455 ± 0.031
Sensitivity	0.403
Positive Predictive Value	0.510
Nr of predictions	77

^top

2. Individual counts for Sfold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.80	0.77	0.83	352	1085176	112	13	60	39	106
CRW_00034	SSU_rRNA_bacteria links to Rfam database...	0.65	0.64	0.66	304	1141856	180	27	129	24	170
CRW_00048	SSU_rRNA_bacteria links to Rfam database...	0.50	0.45	0.54	213	1170824	198	17	161	20	257
CRW_00054	SSU_rRNA_bacteria links to Rfam database...	0.48	0.43	0.54	195	1107453	185	23	145	17	261
CRW_00059	SSU_rRNA_bacteria links to Rfam database...	0.54	0.49	0.60	232	1172361	169	19	134	16	241
CRW_00061	SSU_rRNA_bacteria links to Rfam database...	0.49	0.45	0.54	215	1181555	193	31	152	10	261
CRW_00068	SSU_rRNA_bacteria links to Rfam database...	0.48	0.43	0.53	196	1108938	184	19	152	13	261
CRW_00096	SSU_rRNA_bacteria links to Rfam database...	0.59	0.55	0.63	258	1178469	180	16	137	27	207
CRW_00140	SSU_rRNA_bacteria links to Rfam database...	0.66	0.60	0.73	280	1135893	125	10	95	20	183
CRW_00141	SSU_rRNA_bacteria links to Rfam database...	0.63	0.57	0.70	272	1201636	139	16	101	22	204
CRW_00150	SSU_rRNA_bacteria links to Rfam database...	0.40	0.37	0.44	171	1152531	233	30	189	14	295
CRW_00152	SSU_rRNA_bacteria links to Rfam database...	0.67	0.61	0.74	287	1180029	118	21	79	18	184
CRW_00166	SSU_rRNA_bacteria links to Rfam database...	0.54	0.47	0.61	221	1190832	160	15	128	17	247
CRW_00173	SSU_rRNA_bacteria links to Rfam database...	0.36	0.32	0.39	144	1082291	233	16	205	12	301
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.45	0.41	0.50	195	1178491	206	27	167	12	278
CRW_00196	SSU_rRNA_bacteria links to Rfam database...	0.47	0.44	0.50	210	1207811	232	23	191	18	265
CRW_00230	SSU_rRNA_bacteria links to Rfam database...	0.61	0.53	0.70	254	1198565	129	7	100	22	224
CRW_00245	SSU_rRNA_bacteria links to Rfam database...	0.54	0.45	0.66	202	1106023	115	16	87	12	251
CRW_00248	SSU_rRNA_bacteria links to Rfam database...	0.58	0.57	0.59	274	1218677	223	21	169	33	210
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.50	0.46	0.54	221	1198516	206	28	161	17	257
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.37	0.33	0.42	155	1192370	232	35	180	17	313
CRW_00270	SSU_rRNA_bacteria links to Rfam database...	0.48	0.39	0.59	178	1111985	137	19	104	14	277
CRW_00271	SSU_rRNA_bacteria links to Rfam database...	0.49	0.43	0.57	198	1122402	170	32	119	19	264
CRW_00273	SSU_rRNA_bacteria links to Rfam database...	0.55	0.50	0.61	224	1085232	162	22	123	17	222
CRW_00275	SSU_rRNA_bacteria links to Rfam database...	0.58	0.47	0.73	213	1114978	98	14	66	18	244
CRW_00277	SSU_rRNA_bacteria links to Rfam database...	0.40	0.30	0.54	136	1110544	127	18	97	12	317
CRW_00278	SSU_rRNA_bacteria links to Rfam database...	0.51	0.37	0.70	166	1130018	88	6	66	16	284
CRW_00279	SSU_rRNA_bacteria links to Rfam database...	0.61	0.54	0.69	252	1180052	131	20	92	19	212
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.60	0.55	0.66	250	1117881	149	21	108	20	203
CRW_00281	SSU_rRNA_bacteria links to Rfam database...	0.68	0.60	0.77	279	1144977	109	11	74	24	185
CRW_00284	SSU_rRNA_bacteria links to Rfam database...	0.42	0.33	0.53	149	1131480	147	20	111	16	307
CRW_00285	SSU_rRNA_bacteria links to Rfam database...	0.77	0.71	0.83	326	1117866	94	16	52	26	132
CRW_00289	SSU_rRNA_bacteria links to Rfam database...	0.59	0.51	0.68	241	1189298	141	19	95	27	235
CRW_00298	SSU_rRNA_eukarya links to Rfam database...	0.19	0.17	0.21	84	1620496	355	58	262	35	410
CRW_00301	SSU_rRNA_eukarya links to Rfam database...	0.33	0.30	0.36	156	1762064	326	45	238	43	363
CRW_00306	SSU_rRNA_eukarya links to Rfam database...	0.21	0.19	0.24	88	1445476	332	57	229	46	374
CRW_00310	SSU_rRNA_eukarya links to Rfam database...	0.30	0.26	0.35	138	1805554	298	56	202	40	383
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.46	0.37	0.56	187	1627778	175	32	113	30	315
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.34	0.28	0.41	145	1988661	279	50	159	70	372
CRW_00332	SSU_rRNA_eukarya links to Rfam database...	0.22	0.21	0.24	102	1769589	370	75	255	40	375
CRW_00333	SSU_rRNA_eukarya links to Rfam database...	0.30	0.30	0.31	153	1891989	400	68	275	57	361
CRW_00345	SSU_rRNA_eukarya links to Rfam database...	0.40	0.35	0.45	173	1586487	244	36	175	33	325
CRW_00346	SSU_rRNA_eukarya links to Rfam database...	0.34	0.32	0.36	161	1584648	319	58	223	38	340
CRW_00348	SSU_rRNA_eukarya links to Rfam database...	0.32	0.26	0.40	129	1615184	223	34	156	33	374
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.38	0.33	0.43	166	1674980	241	44	175	22	341
CRW_00366	SSU_rRNA_eukarya links to Rfam database...	0.36	0.35	0.36	169	1579286	334	74	224	36	313
CRW_00370	SSU_rRNA_eukarya links to Rfam database...	0.34	0.32	0.36	160	1640327	325	54	225	46	346
CRW_00371	SSU_rRNA_eukarya links to Rfam database...	0.38	0.34	0.42	171	1611504	271	50	185	36	325
CRW_00372	SSU_rRNA_eukarya links to Rfam database...	0.39	0.35	0.45	172	1625925	245	34	175	36	326
CRW_00379	SSU_rRNA_eukarya links to Rfam database...	0.44	0.34	0.58	183	2126636	205	17	117	71	351
CRW_00386	SSU_rRNA_eukarya links to Rfam database...	0.29	0.27	0.31	138	1750806	345	60	252	33	377
CRW_00403	SSU_rRNA_eukarya links to Rfam database...	0.31	0.30	0.32	137	1606104	362	58	229	75	327
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.39	0.32	0.47	157	1615171	217	28	147	42	339
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.33	0.29	0.38	146	1665840	278	40	199	39	349
CRW_00466	SSU_rRNA_bacteria links to Rfam database...	0.57	0.54	0.61	247	1116358	185	18	142	25	211
CRW_00589	SSU_rRNA_bacteria links to Rfam database...	0.31	0.26	0.36	116	1094139	220	19	186	15	329
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.58	0.40	0.84	179	1119542	50	5	30	15	273
CRW_00591	SSU_rRNA_bacteria links to Rfam database...	0.60	0.56	0.65	248	1098523	153	22	110	21	198
CRW_00598	SSU_rRNA_eukarya links to Rfam database...	0.35	0.34	0.37	165	1665773	333	62	225	46	326
CRW_00758	SSU_rRNA_eukarya links to Rfam database...	0.34	0.30	0.38	147	1640380	287	42	197	48	336
CRW_00759	SSU_rRNA_eukarya links to Rfam database...	0.23	0.22	0.25	105	1636722	354	42	276	36	376
CRW_00760	SSU_rRNA_eukarya links to Rfam database...	0.33	0.27	0.39	133	1638613	257	32	177	48	353
CRW_00772	SSU_rRNA_bacteria links to Rfam database...	0.30	0.27	0.34	108	945681	233	27	184	22	296
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.40	0.34	0.46	152	1137458	193	30	146	17	295
CRW_00795	SSU_rRNA_bacteria links to Rfam database...	0.19	0.17	0.20	68	1152586	311	44	223	44	326
CRW_00849	SSU_rRNA_bacteria links to Rfam database...	0.50	0.48	0.52	227	1117826	230	18	189	23	243
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.59	0.52	0.66	241	1135913	144	19	105	20	223
CRW_01153	SSU_rRNA_eukarya links to Rfam database...	0.37	0.30	0.45	152	1606193	221	35	148	38	351
CRW_01171	SSU_rRNA_eukarya links to Rfam database...	0.33	0.29	0.38	149	1593608	285	55	193	37	357
CRW_01300	SSU_rRNA_eukarya links to Rfam database...	0.39	0.34	0.45	175	1693327	254	42	176	36	340
PDB_00703	SSU_rRNA_bacteria links to Rfam database...	0.52	0.46	0.58	229	1073451	189	12	153	24	266
PDB_00854	SSU_rRNA_bacteria links to Rfam database...	0.54	0.49	0.60	263	1500337	220	22	156	42	272
PDB_01088	SSU_rRNA_bacteria links to Rfam database...	0.38	0.32	0.47	173	1471098	222	33	166	23	374
PDB_01103	SSU_rRNA_bacteria links to Rfam database...	0.56	0.50	0.63	280	1606082	191	20	146	25	282
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.60	0.53	0.67	288	1433540	169	14	129	26	252
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.55	0.44	0.70	230	1204799	110	12	87	11	297
PDB_01273	SSU_rRNA_bacteria links to Rfam database...	0.50	0.46	0.54	227	1117836	223	19	178	26	264

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of Carnac(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Carnac(20)

2. Individual counts for Carnac(20) [ download as .csv ]

Performance of Sfold - scored lower in this pairwise comparison

1. Total counts & total scores for Sfold

2. Individual counts for Sfold [ download as .csv ]