CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	Sfold		UNAFold
MCC	0.521	>	0.496
Average MCC ± 95% Confidence Intervals	0.594 ± 0.077	>	0.573 ± 0.083
Sensitivity	0.524	<	0.530
Positive Predictive Value	0.519	>	0.464
Total TP	2005	<	2027
Total TN	5850269	>	5849768
Total FP	2379	<	2881
Total FP CONTRA	507	<	674
Total FP INCONS	1354	<	1666
Total FP COMP	518	<	541
Total FN	1820	>	1798
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of Sfold and UNAFold. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Sfold and UNAFold).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Sfold and UNAFold).
Comparison of average Matthews Correlation Coefficients (MCCs) for Sfold and UNAFold. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Sfold and UNAFold).

^top

Performance of Sfold - scored higher in this pairwise comparison

1. Total counts & total scores for Sfold

Total Base Pair Counts
Total TP	2005
Total TN	5850269
Total FP	2379
Total FP CONTRA	507
Total FP INCONS	1354
Total FP COMP	518
Total FN	1820

Total Scores
MCC	0.521
Average MCC ± 95% Confidence Intervals	0.594 ± 0.077
Sensitivity	0.524
Positive Predictive Value	0.519
Nr of predictions	46

^top

2. Individual counts for Sfold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KFC_A	-	-0.03	0.00	0.00	0	229	7	1	6	0	8
2KRL_A	-	0.91	0.87	0.95	20	2003	9	1	0	8	3
2LC8_A	GP_knot1 links to Rfam database...	0.46	0.39	0.58	7	516	5	0	5	0	11
2M58_A	-	0.34	0.25	0.50	3	538	3	1	2	0	9
2RP0_A	-	0.84	0.71	1.00	5	111	0	0	0	0	2
2WDL_A	-	0.50	0.50	0.50	392	1444543	429	62	325	42	396
2ZZN_D	tRNA links to Rfam database...	0.93	0.95	0.91	21	961	3	2	0	1	1
3A2K_C	tRNA links to Rfam database...	0.47	0.50	0.46	11	1084	13	3	10	0	11
3A3A_A	tRNA-Sec links to Rfam database...	0.97	0.93	1.00	28	1472	0	0	0	0	2
3ADB_C	-	0.67	0.64	0.72	21	1790	9	0	8	1	12
3GCA_A	-	0.84	0.71	1.00	5	153	0	0	0	0	2
3GX2_A	SAM links to Rfam database...	0.86	0.79	0.96	22	1426	2	1	0	1	6
3IVN_B	Purine links to Rfam database...	0.91	0.83	1.00	19	884	0	0	0	0	4
3IWN_A	c-di-GMP-I links to Rfam database...	0.69	0.68	0.70	19	1445	9	1	7	1	9
3IYQ_A	tmRNA links to Rfam database...	0.25	0.33	0.19	17	22350	86	39	34	13	34
3IZ4_A	tmRNA links to Rfam database...	0.59	0.56	0.63	53	25452	34	14	17	3	42
3J0L_a	-	0.55	0.64	0.50	7	397	8	5	2	1	4
3J20_2	SSU_rRNA_archaea links to Rfam database...	0.57	0.58	0.56	240	421940	241	31	157	53	172
3J20_0	tRNA links to Rfam database...	0.51	0.57	0.48	12	1194	14	3	10	1	9
3J3D_C	tRNA links to Rfam database...	0.44	0.37	0.54	7	955	6	1	5	0	12
3J3E_8	5_8S_rRNA links to Rfam database...	0.10	0.13	0.08	2	2718	32	6	16	10	13
3J3F_8	5_8S_rRNA links to Rfam database...	0.39	0.47	0.33	9	4734	37	8	10	19	10
3JYV_7	tRNA links to Rfam database...	-0.02	0.00	0.00	0	1095	16	2	14	0	20
3JYX_5	-	0.42	0.46	0.38	223	1968418	588	172	189	227	258
3KIY_A	-	0.53	0.51	0.56	403	1485155	373	56	266	51	383
3LA5_A	Purine links to Rfam database...	0.91	0.84	1.00	21	933	0	0	0	0	4
3NKB_B	-	0.75	0.74	0.78	14	717	4	0	4	0	5
3NPB_A	SAM links to Rfam database...	0.85	0.78	0.94	29	2247	7	0	2	5	8
3O58_3	5_8S_rRNA links to Rfam database...	0.51	0.50	0.52	11	4743	18	2	8	8	11
3PDR_A	ykoK links to Rfam database...	0.82	0.80	0.85	40	4793	9	2	5	2	10
3RKF_A	Purine links to Rfam database...	0.91	0.83	1.00	20	846	0	0	0	0	4
3SD1_A	THF links to Rfam database...	0.78	0.72	0.84	21	1508	4	2	2	0	8
3U4M_B	-	0.49	0.55	0.46	12	1250	14	3	11	0	10
3W1K_J	tRNA-Sec links to Rfam database...	0.97	0.97	0.97	30	1647	1	1	0	0	1
3W3S_B	tRNA-Sec links to Rfam database...	0.63	0.64	0.64	21	1956	13	4	8	1	12
3ZEX_B	-	0.37	0.39	0.35	136	420960	296	65	191	40	210
3ZEX_C	5_8S_rRNA links to Rfam database...	0.41	0.34	0.50	10	5354	27	2	8	17	19
4A1C_2	5_8S_rRNA links to Rfam database...	0.21	0.25	0.19	5	4489	34	8	14	12	15
4AOB_A	SAM links to Rfam database...	0.60	0.59	0.63	17	1410	11	4	6	1	12
4ATO_G	-	0.37	0.14	1.00	1	219	0	0	0	0	6
4ENB_A	crcB links to Rfam database...	0.85	0.73	1.00	11	461	0	0	0	0	4
4ENC_A	crcB links to Rfam database...	0.57	0.33	1.00	5	491	0	0	0	0	10
4FRG_B	AdoCbl-variant links to Rfam database...	0.67	0.58	0.78	14	1184	4	3	1	0	10
4FRN_A	AdoCbl-variant links to Rfam database...	0.71	0.57	0.89	16	1830	2	2	0	0	12
4JF2_A	preQ1-II links to Rfam database...	0.89	0.79	1.00	19	1063	0	0	0	0	5
4JRC_A	-	0.34	0.35	0.35	6	605	11	0	11	0	11

^top

Performance of UNAFold - scored lower in this pairwise comparison

1. Total counts & total scores for UNAFold

Total Base Pair Counts
Total TP	2027
Total TN	5849768
Total FP	2881
Total FP CONTRA	674
Total FP INCONS	1666
Total FP COMP	541
Total FN	1798

Total Scores
MCC	0.496
Average MCC ± 95% Confidence Intervals	0.573 ± 0.083
Sensitivity	0.530
Positive Predictive Value	0.464
Nr of predictions	46

^top

2. Individual counts for UNAFold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KFC_A	-	-0.03	0.00	0.00	0	229	7	1	6	0	8
2KRL_A	-	0.91	0.87	0.95	20	2003	9	1	0	8	3
2LC8_A	GP_knot1 links to Rfam database...	-0.03	0.00	0.00	0	513	15	3	12	0	18
2M58_A	-	0.60	0.58	0.64	7	533	4	1	3	0	5
2RP0_A	-	0.84	0.71	1.00	5	111	0	0	0	0	2
2WDL_A	-	0.49	0.50	0.47	396	1444479	488	87	360	41	392
2ZZN_D	tRNA links to Rfam database...	0.93	0.95	0.91	21	961	2	2	0	0	1
3A2K_C	tRNA links to Rfam database...	0.47	0.50	0.46	11	1084	13	3	10	0	11
3A3A_A	tRNA-Sec links to Rfam database...	0.97	0.93	1.00	28	1472	0	0	0	0	2
3ADB_C	-	0.86	0.85	0.88	28	1787	4	0	4	0	5
3GCA_A	-	0.84	0.71	1.00	5	153	0	0	0	0	2
3GX2_A	SAM links to Rfam database...	0.81	0.79	0.85	22	1423	5	2	2	1	6
3IVN_B	Purine links to Rfam database...	0.91	0.83	1.00	19	884	0	0	0	0	4
3IWN_A	c-di-GMP-I links to Rfam database...	0.80	0.79	0.81	22	1445	6	1	4	1	6
3IYQ_A	tmRNA links to Rfam database...	0.23	0.33	0.17	17	22339	96	46	38	12	34
3IZ4_A	tmRNA links to Rfam database...	0.58	0.60	0.57	57	25436	53	12	31	10	38
3J0L_a	-	0.55	0.64	0.50	7	397	8	5	2	1	4
3J20_2	SSU_rRNA_archaea links to Rfam database...	0.57	0.58	0.55	240	421931	250	36	161	53	172
3J20_0	tRNA links to Rfam database...	0.51	0.57	0.48	12	1194	14	3	10	1	9
3J3D_C	tRNA links to Rfam database...	0.33	0.37	0.32	7	946	15	3	12	0	12
3J3E_8	5_8S_rRNA links to Rfam database...	0.15	0.20	0.13	3	2718	32	6	15	11	12
3J3F_8	5_8S_rRNA links to Rfam database...	0.32	0.42	0.25	8	4729	44	12	12	20	11
3JYV_7	tRNA links to Rfam database...	0.24	0.25	0.25	5	1091	16	7	8	1	15
3JYX_5	-	0.36	0.45	0.29	216	1968262	763	245	279	239	265
3KIY_A	-	0.50	0.52	0.48	410	1485034	486	79	357	50	376
3LA5_A	Purine links to Rfam database...	0.91	0.84	1.00	21	933	0	0	0	0	4
3NKB_B	-	0.69	0.74	0.67	14	714	7	0	7	0	5
3NPB_A	SAM links to Rfam database...	0.85	0.78	0.94	29	2247	6	0	2	4	8
3O58_3	5_8S_rRNA links to Rfam database...	0.42	0.50	0.35	11	4733	34	5	15	14	11
3PDR_A	ykoK links to Rfam database...	0.93	0.92	0.94	46	4791	5	1	2	2	4
3RKF_A	Purine links to Rfam database...	0.91	0.83	1.00	20	846	0	0	0	0	4
3SD1_A	THF links to Rfam database...	0.77	0.76	0.79	22	1505	6	4	2	0	7
3U4M_B	-	0.49	0.55	0.46	12	1250	14	3	11	0	10
3W1K_J	tRNA-Sec links to Rfam database...	0.97	0.97	0.97	30	1647	1	1	0	0	1
3W3S_B	tRNA-Sec links to Rfam database...	0.61	0.61	0.63	20	1957	13	4	8	1	13
3ZEX_B	-	0.36	0.40	0.33	138	420935	317	72	207	38	208
3ZEX_C	5_8S_rRNA links to Rfam database...	0.30	0.34	0.26	10	5336	43	5	23	15	19
4A1C_2	5_8S_rRNA links to Rfam database...	0.19	0.25	0.15	5	4483	42	11	17	14	15
4AOB_A	SAM links to Rfam database...	0.60	0.59	0.63	17	1410	11	4	6	1	12
4ATO_G	-	0.38	0.43	0.38	3	212	5	5	0	0	4
4ENB_A	crcB links to Rfam database...	0.85	0.73	1.00	11	461	2	0	0	2	4
4ENC_A	crcB links to Rfam database...	0.37	0.33	0.45	5	485	7	0	6	1	10
4FRG_B	AdoCbl-variant links to Rfam database...	0.36	0.38	0.38	9	1178	15	2	13	0	15
4FRN_A	AdoCbl-variant links to Rfam database...	0.48	0.46	0.52	13	1823	12	2	10	0	15
4JF2_A	preQ1-II links to Rfam database...	0.89	0.79	1.00	19	1063	0	0	0	0	5
4JRC_A	-	0.34	0.35	0.35	6	605	11	0	11	0	11

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of Sfold - scored higher in this pairwise comparison

1. Total counts & total scores for Sfold

2. Individual counts for Sfold [ download as .csv ]

Performance of UNAFold - scored lower in this pairwise comparison

1. Total counts & total scores for UNAFold

2. Individual counts for UNAFold [ download as .csv ]