CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	Carnac(20)		Fold
MCC	0.558	>	0.385
Average MCC ± 95% Confidence Intervals	0.553 ± 0.023	>	0.388 ± 0.024
Sensitivity	0.348	<	0.390
Positive Predictive Value	0.895	>	0.381
Total TP	12872	<	14429
Total TN	104356083	>	104332585
Total FP	2613	<	26119
Total FP CONTRA	170	<	3646
Total FP INCONS	1341	<	19806
Total FP COMP	1102	<	2667
Total FN	24092	>	22535
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of Carnac(20) and Fold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Carnac(20) and Fold).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Carnac(20) and Fold).
Comparison of average Matthews Correlation Coefficients (MCCs) for Carnac(20) and Fold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Carnac(20) and Fold).

^top

Performance of Carnac(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Carnac(20)

Total Base Pair Counts
Total TP	12872
Total TN	104356083
Total FP	2613
Total FP CONTRA	170
Total FP INCONS	1341
Total FP COMP	1102
Total FN	24092

Total Scores
MCC	0.558
Average MCC ± 95% Confidence Intervals	0.553 ± 0.023
Sensitivity	0.348
Positive Predictive Value	0.895
Nr of predictions	77

^top

2. Individual counts for Carnac(20) [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.74	0.59	0.94	270	1085313	35	5	13	17	188
CRW_00034	SSU_rRNA_bacteria links to Rfam database...	0.56	0.35	0.90	166	1142131	28	4	15	9	308
CRW_00048	SSU_rRNA_bacteria links to Rfam database...	0.62	0.43	0.91	202	1170992	29	3	18	8	268
CRW_00054	SSU_rRNA_bacteria links to Rfam database...	0.62	0.42	0.91	191	1107607	23	4	14	5	265
CRW_00059	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.95	204	1172532	17	3	7	7	269
CRW_00061	SSU_rRNA_bacteria links to Rfam database...	0.63	0.42	0.94	199	1181742	19	2	10	7	277
CRW_00068	SSU_rRNA_bacteria links to Rfam database...	0.69	0.51	0.94	234	1109056	28	2	13	13	223
CRW_00096	SSU_rRNA_bacteria links to Rfam database...	0.62	0.43	0.92	198	1178664	29	0	18	11	267
CRW_00140	SSU_rRNA_bacteria links to Rfam database...	0.71	0.53	0.95	246	1136018	21	2	12	7	217
CRW_00141	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.93	207	1201802	26	4	12	10	269
CRW_00150	SSU_rRNA_bacteria links to Rfam database...	0.66	0.46	0.95	216	1152694	16	3	8	5	250
CRW_00152	SSU_rRNA_bacteria links to Rfam database...	0.69	0.52	0.92	244	1180151	26	4	17	5	227
CRW_00166	SSU_rRNA_bacteria links to Rfam database...	0.64	0.44	0.93	207	1190974	24	0	15	9	261
CRW_00173	SSU_rRNA_bacteria links to Rfam database...	0.56	0.36	0.89	158	1082478	26	2	18	6	287
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.52	0.32	0.87	149	1178708	29	3	20	6	324
CRW_00196	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.96	204	1208022	12	2	7	3	271
CRW_00230	SSU_rRNA_bacteria links to Rfam database...	0.66	0.47	0.93	227	1198681	28	3	15	10	251
CRW_00245	SSU_rRNA_bacteria links to Rfam database...	0.65	0.45	0.93	205	1106108	20	3	12	5	248
CRW_00248	SSU_rRNA_bacteria links to Rfam database...	0.56	0.35	0.91	168	1218956	27	4	13	10	316
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.68	0.50	0.91	241	1198660	33	4	21	8	237
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.64	0.43	0.97	200	1192533	11	3	4	4	268
CRW_00270	SSU_rRNA_bacteria links to Rfam database...	0.66	0.47	0.93	212	1112057	22	4	13	5	243
CRW_00271	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.94	191	1122548	17	3	9	5	271
CRW_00273	SSU_rRNA_bacteria links to Rfam database...	0.57	0.35	0.92	156	1085431	16	4	10	2	290
CRW_00275	SSU_rRNA_bacteria links to Rfam database...	0.60	0.37	0.95	171	1115091	13	2	7	4	286
CRW_00277	SSU_rRNA_bacteria links to Rfam database...	0.64	0.44	0.93	201	1110578	20	3	13	4	252
CRW_00278	SSU_rRNA_bacteria links to Rfam database...	0.52	0.29	0.92	130	1130115	15	4	7	4	320
CRW_00279	SSU_rRNA_bacteria links to Rfam database...	0.68	0.51	0.92	237	1180158	31	3	18	10	227
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.66	0.46	0.96	209	1118042	12	1	8	3	244
CRW_00281	SSU_rRNA_bacteria links to Rfam database...	0.65	0.45	0.95	209	1145120	17	2	10	5	255
CRW_00284	SSU_rRNA_bacteria links to Rfam database...	0.53	0.33	0.87	150	1131587	27	3	20	4	306
CRW_00285	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.93	188	1118058	18	2	12	4	270
CRW_00289	SSU_rRNA_bacteria links to Rfam database...	0.58	0.41	0.84	194	1189422	41	5	32	4	282
CRW_00298	SSU_rRNA_eukarya links to Rfam database...	0.41	0.19	0.86	95	1620790	37	0	15	22	399
CRW_00301	SSU_rRNA_eukarya links to Rfam database...	0.46	0.25	0.83	131	1762345	56	0	27	29	388
CRW_00306	SSU_rRNA_eukarya links to Rfam database...	0.55	0.32	0.95	146	1445697	38	0	7	31	316
CRW_00310	SSU_rRNA_eukarya links to Rfam database...	0.42	0.22	0.81	116	1805806	46	5	23	18	405
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.45	0.23	0.92	113	1627987	32	1	9	22	389
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.34	0.17	0.67	89	1988883	69	3	40	26	428
CRW_00332	SSU_rRNA_eukarya links to Rfam database...	0.39	0.20	0.76	97	1769893	46	1	30	15	380
CRW_00333	SSU_rRNA_eukarya links to Rfam database...	0.35	0.17	0.70	88	1892359	50	4	34	12	426
CRW_00345	SSU_rRNA_eukarya links to Rfam database...	0.48	0.29	0.81	145	1586691	63	2	33	28	353
CRW_00346	SSU_rRNA_eukarya links to Rfam database...	0.47	0.28	0.79	138	1584915	66	1	36	29	363
CRW_00348	SSU_rRNA_eukarya links to Rfam database...	0.40	0.23	0.72	114	1615344	78	4	41	33	389
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.49	0.25	0.96	129	1675231	29	0	5	24	378
CRW_00366	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.87	122	1579613	37	1	17	19	360
CRW_00370	SSU_rRNA_eukarya links to Rfam database...	0.47	0.27	0.84	135	1640605	54	1	25	28	371
CRW_00371	SSU_rRNA_eukarya links to Rfam database...	0.48	0.29	0.81	143	1611733	59	1	33	25	353
CRW_00372	SSU_rRNA_eukarya links to Rfam database...	0.44	0.23	0.84	117	1626166	46	2	21	23	381
CRW_00379	SSU_rRNA_eukarya links to Rfam database...	0.49	0.26	0.93	140	2126802	50	1	10	39	394
CRW_00386	SSU_rRNA_eukarya links to Rfam database...	0.30	0.13	0.71	67	1751161	53	2	26	25	448
CRW_00403	SSU_rRNA_eukarya links to Rfam database...	0.48	0.27	0.83	127	1606375	60	0	26	34	337
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.39	0.19	0.81	94	1615387	42	0	22	20	402
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.38	0.19	0.74	96	1666095	60	1	33	26	399
CRW_00466	SSU_rRNA_bacteria links to Rfam database...	0.62	0.42	0.91	192	1116554	28	4	15	9	266
CRW_00589	SSU_rRNA_bacteria links to Rfam database...	0.58	0.35	0.97	156	1094299	10	0	5	5	289
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.55	0.33	0.93	148	1119597	15	1	10	4	304
CRW_00591	SSU_rRNA_bacteria links to Rfam database...	0.66	0.46	0.93	206	1098682	26	1	14	11	240
CRW_00598	SSU_rRNA_eukarya links to Rfam database...	0.42	0.20	0.85	100	1666107	41	0	18	23	391
CRW_00758	SSU_rRNA_eukarya links to Rfam database...	0.46	0.26	0.79	127	1640605	57	4	30	23	356
CRW_00759	SSU_rRNA_eukarya links to Rfam database...	0.39	0.19	0.79	93	1637028	51	3	21	27	388
CRW_00760	SSU_rRNA_eukarya links to Rfam database...	0.45	0.24	0.84	117	1638815	55	0	23	32	369
CRW_00772	SSU_rRNA_bacteria links to Rfam database...	0.48	0.30	0.77	122	945842	42	3	33	6	282
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.61	0.43	0.88	191	1137570	34	1	24	9	256
CRW_00795	SSU_rRNA_bacteria links to Rfam database...	0.59	0.38	0.90	150	1152755	40	0	16	24	244
CRW_00849	SSU_rRNA_bacteria links to Rfam database...	0.62	0.42	0.92	196	1118046	27	3	15	9	274
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.60	0.39	0.93	181	1136084	19	3	10	6	283
CRW_01153	SSU_rRNA_eukarya links to Rfam database...	0.44	0.25	0.76	126	1606363	56	1	38	17	377
CRW_01171	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.90	125	1593866	44	0	14	30	381
CRW_01300	SSU_rRNA_eukarya links to Rfam database...	0.47	0.25	0.88	128	1693575	38	0	17	21	387
PDB_00703	SSU_rRNA_bacteria links to Rfam database...	0.60	0.40	0.92	196	1073633	28	3	13	12	299
PDB_00854	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.95	218	1500548	29	3	9	17	317
PDB_01088	SSU_rRNA_bacteria links to Rfam database...	0.59	0.38	0.92	206	1471246	35	2	16	17	341
PDB_01103	SSU_rRNA_bacteria links to Rfam database...	0.62	0.40	0.96	226	1606292	20	3	7	10	336
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.58	0.36	0.94	192	1433767	24	1	11	12	348
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.62	0.41	0.93	216	1204895	30	4	13	13	311
PDB_01273	SSU_rRNA_bacteria links to Rfam database...	0.66	0.48	0.92	234	1118007	37	4	15	18	257

^top

Performance of Fold - scored lower in this pairwise comparison

1. Total counts & total scores for Fold

Total Base Pair Counts
Total TP	14429
Total TN	104332585
Total FP	26119
Total FP CONTRA	3646
Total FP INCONS	19806
Total FP COMP	2667
Total FN	22535

Total Scores
MCC	0.385
Average MCC ± 95% Confidence Intervals	0.388 ± 0.024
Sensitivity	0.390
Positive Predictive Value	0.381
Nr of predictions	77

^top

2. Individual counts for Fold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00024	SSU_rRNA_archaea links to Rfam database...	0.56	0.57	0.56	259	1085140	238	22	180	36	199
CRW_00034	SSU_rRNA_bacteria links to Rfam database...	0.62	0.63	0.61	299	1141829	219	35	153	31	175
CRW_00048	SSU_rRNA_bacteria links to Rfam database...	0.51	0.51	0.52	239	1170753	241	24	199	18	231
CRW_00054	SSU_rRNA_bacteria links to Rfam database...	0.40	0.41	0.40	185	1107355	286	30	246	10	271
CRW_00059	SSU_rRNA_bacteria links to Rfam database...	0.44	0.44	0.44	207	1172278	279	31	230	18	266
CRW_00061	SSU_rRNA_bacteria links to Rfam database...	0.54	0.54	0.54	259	1181475	236	33	186	17	217
CRW_00068	SSU_rRNA_bacteria links to Rfam database...	0.43	0.43	0.43	198	1108845	279	32	230	17	259
CRW_00096	SSU_rRNA_bacteria links to Rfam database...	0.42	0.43	0.42	200	1178403	299	28	249	22	265
CRW_00140	SSU_rRNA_bacteria links to Rfam database...	0.38	0.38	0.38	176	1135818	299	27	257	15	287
CRW_00141	SSU_rRNA_bacteria links to Rfam database...	0.53	0.53	0.54	253	1201553	246	34	185	27	223
CRW_00150	SSU_rRNA_bacteria links to Rfam database...	0.45	0.44	0.45	204	1152472	262	26	219	17	262
CRW_00152	SSU_rRNA_bacteria links to Rfam database...	0.53	0.52	0.54	246	1179962	231	33	175	23	225
CRW_00166	SSU_rRNA_bacteria links to Rfam database...	0.40	0.40	0.39	188	1190720	309	32	256	21	280
CRW_00173	SSU_rRNA_bacteria links to Rfam database...	0.28	0.28	0.28	125	1082214	333	31	286	16	320
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.49	0.48	0.50	226	1178426	251	26	202	23	247
CRW_00196	SSU_rRNA_bacteria links to Rfam database...	0.45	0.46	0.44	219	1207740	296	42	234	20	256
CRW_00230	SSU_rRNA_bacteria links to Rfam database...	0.53	0.52	0.54	247	1198465	241	32	182	27	231
CRW_00245	SSU_rRNA_bacteria links to Rfam database...	0.37	0.37	0.37	167	1105882	297	32	247	18	286
CRW_00248	SSU_rRNA_bacteria links to Rfam database...	0.49	0.49	0.48	238	1218645	286	36	222	28	246
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.37	0.38	0.37	181	1198432	321	48	265	8	297
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.39	0.39	0.40	183	1192281	294	41	235	18	285
CRW_00270	SSU_rRNA_bacteria links to Rfam database...	0.54	0.54	0.54	245	1111832	232	27	182	23	210
CRW_00271	SSU_rRNA_bacteria links to Rfam database...	0.36	0.36	0.37	168	1122293	309	37	253	19	294
CRW_00273	SSU_rRNA_bacteria links to Rfam database...	0.52	0.52	0.52	233	1085151	238	41	176	21	213
CRW_00275	SSU_rRNA_bacteria links to Rfam database...	0.59	0.59	0.59	271	1114809	212	28	163	21	186
CRW_00277	SSU_rRNA_bacteria links to Rfam database...	0.32	0.32	0.32	144	1110348	313	38	265	10	309
CRW_00278	SSU_rRNA_bacteria links to Rfam database...	0.36	0.35	0.37	159	1129828	293	31	238	24	291
CRW_00279	SSU_rRNA_bacteria links to Rfam database...	0.58	0.56	0.59	262	1179969	207	27	158	22	202
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.33	0.33	0.32	151	1117794	327	43	272	12	302
CRW_00281	SSU_rRNA_bacteria links to Rfam database...	0.53	0.52	0.53	243	1144885	238	27	186	25	221
CRW_00284	SSU_rRNA_bacteria links to Rfam database...	0.29	0.29	0.29	130	1131315	331	40	275	16	326
CRW_00285	SSU_rRNA_bacteria links to Rfam database...	0.48	0.49	0.48	225	1117789	265	38	208	19	233
CRW_00289	SSU_rRNA_bacteria links to Rfam database...	0.51	0.50	0.52	237	1189193	251	38	185	28	239
CRW_00298	SSU_rRNA_eukarya links to Rfam database...	0.22	0.24	0.21	118	1620334	479	100	348	31	376
CRW_00301	SSU_rRNA_eukarya links to Rfam database...	0.29	0.30	0.29	156	1761959	461	80	308	73	363
CRW_00306	SSU_rRNA_eukarya links to Rfam database...	0.09	0.09	0.08	43	1445337	532	80	390	62	419
CRW_00310	SSU_rRNA_eukarya links to Rfam database...	0.28	0.29	0.27	153	1805378	473	77	342	54	368
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.40	0.40	0.39	203	1627589	370	67	251	52	299
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.31	0.32	0.30	166	1988453	469	82	314	73	351
CRW_00332	SSU_rRNA_eukarya links to Rfam database...	0.19	0.21	0.18	99	1769473	516	102	347	67	378
CRW_00333	SSU_rRNA_eukarya links to Rfam database...	0.32	0.33	0.31	169	1891947	449	66	303	80	345
CRW_00345	SSU_rRNA_eukarya links to Rfam database...	0.33	0.34	0.33	171	1586345	407	61	294	52	327
CRW_00346	SSU_rRNA_eukarya links to Rfam database...	0.33	0.35	0.32	175	1584535	423	75	305	43	326
CRW_00348	SSU_rRNA_eukarya links to Rfam database...	0.28	0.28	0.27	142	1614977	433	60	324	49	361
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.36	0.37	0.35	186	1674831	392	68	280	44	321
CRW_00366	SSU_rRNA_eukarya links to Rfam database...	0.27	0.29	0.25	138	1579211	437	87	317	33	344
CRW_00370	SSU_rRNA_eukarya links to Rfam database...	0.28	0.29	0.27	148	1640226	440	80	312	48	358
CRW_00371	SSU_rRNA_eukarya links to Rfam database...	0.34	0.35	0.33	173	1611383	414	61	293	60	323
CRW_00372	SSU_rRNA_eukarya links to Rfam database...	0.26	0.28	0.25	137	1625762	453	84	323	46	361
CRW_00379	SSU_rRNA_eukarya links to Rfam database...	0.44	0.45	0.44	241	2126401	411	60	251	100	293
CRW_00386	SSU_rRNA_eukarya links to Rfam database...	0.29	0.30	0.28	155	1750699	443	74	328	41	360
CRW_00403	SSU_rRNA_eukarya links to Rfam database...	0.33	0.34	0.33	157	1606045	425	57	269	99	307
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.35	0.36	0.34	177	1614982	392	60	284	48	319
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.29	0.31	0.28	152	1665688	448	68	317	63	343
CRW_00466	SSU_rRNA_bacteria links to Rfam database...	0.57	0.56	0.57	257	1116315	221	28	165	28	201
CRW_00589	SSU_rRNA_bacteria links to Rfam database...	0.30	0.30	0.31	133	1094029	317	35	263	19	312
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.42	0.42	0.42	191	1119306	279	34	225	20	261
CRW_00591	SSU_rRNA_bacteria links to Rfam database...	0.50	0.50	0.50	222	1098458	256	20	203	33	224
CRW_00598	SSU_rRNA_eukarya links to Rfam database...	0.35	0.37	0.34	181	1665691	415	68	285	62	310
CRW_00758	SSU_rRNA_eukarya links to Rfam database...	0.25	0.26	0.24	127	1640244	444	62	333	49	356
CRW_00759	SSU_rRNA_eukarya links to Rfam database...	0.31	0.32	0.29	156	1636612	443	67	310	66	325
CRW_00760	SSU_rRNA_eukarya links to Rfam database...	0.27	0.29	0.26	140	1638417	454	75	323	56	346
CRW_00772	SSU_rRNA_bacteria links to Rfam database...	0.25	0.25	0.25	100	945594	319	37	269	13	304
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.39	0.38	0.39	172	1137342	303	31	241	31	275
CRW_00795	SSU_rRNA_bacteria links to Rfam database...	0.32	0.33	0.30	130	1152494	342	43	254	45	264
CRW_00849	SSU_rRNA_bacteria links to Rfam database...	0.53	0.53	0.52	250	1117779	255	29	202	24	220
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.37	0.38	0.37	175	1135806	315	41	256	18	289
CRW_01153	SSU_rRNA_eukarya links to Rfam database...	0.36	0.37	0.35	187	1605990	391	60	291	40	316
CRW_01171	SSU_rRNA_eukarya links to Rfam database...	0.28	0.28	0.27	143	1593476	430	75	311	44	363
CRW_01300	SSU_rRNA_eukarya links to Rfam database...	0.28	0.29	0.27	150	1693165	443	63	342	38	365
PDB_00703	SSU_rRNA_bacteria links to Rfam database...	0.48	0.46	0.50	229	1073385	256	28	203	25	266
PDB_00854	SSU_rRNA_bacteria links to Rfam database...	0.46	0.46	0.46	246	1500244	334	36	252	46	289
PDB_01088	SSU_rRNA_bacteria links to Rfam database...	0.34	0.33	0.34	180	1470946	368	36	308	24	367
PDB_01103	SSU_rRNA_bacteria links to Rfam database...	0.44	0.44	0.44	245	1605966	343	42	275	26	317
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.46	0.45	0.47	245	1433450	306	32	244	30	295
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.31	0.30	0.33	158	1204643	339	37	290	12	369
PDB_01273	SSU_rRNA_bacteria links to Rfam database...	0.38	0.38	0.39	186	1117784	320	28	262	30	305

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of Carnac(20) - scored higher in this pairwise comparison

1. Total counts & total scores for Carnac(20)

2. Individual counts for Carnac(20) [ download as .csv ]

Performance of Fold - scored lower in this pairwise comparison

1. Total counts & total scores for Fold

2. Individual counts for Fold [ download as .csv ]