CompaRNA - on-line benchmarks of RNA structure prediction methods

Laboratory of Structural Bioinformatics @ Adam Mickiewicz University in Poznan

Home

Methods

Datasets

Rankings

RNA 2D Atlas

Help

FAQ

Contact us

RSS feeds

Twitter

Overview

Metric	McQFold		RNAsubopt
MCC	0.625	>	0.581
Average MCC ± 95% Confidence Intervals	0.654 ± 0.102	>	0.566 ± 0.090
Sensitivity	0.641	>	0.612
Positive Predictive Value	0.616	>	0.560
Total TP	649	>	619
Total TN	109157	>	109105
Total FP	485	<	601
Total FP CONTRA	155	<	175
Total FP INCONS	250	<	312
Total FP COMP	80	<	114
Total FN	363	<	393
P-value	3.56938820447e-08

^top

Performance plots

Comparison of performance of McQFold and RNAsubopt. Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for McQFold and RNAsubopt).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for McQFold and RNAsubopt).
Comparison of average Matthews Correlation Coefficients (MCCs) for McQFold and RNAsubopt. The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for McQFold and RNAsubopt).

^top

Performance of McQFold - scored higher in this pairwise comparison

1. Total counts & total scores for McQFold

Total Base Pair Counts
Total TP	649
Total TN	109157
Total FP	485
Total FP CONTRA	155
Total FP INCONS	250
Total FP COMP	80
Total FN	363

Total Scores
MCC	0.625
Average MCC ± 95% Confidence Intervals	0.654 ± 0.102
Sensitivity	0.641
Positive Predictive Value	0.616
Nr of predictions	41

^top

2. Individual counts for McQFold [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KFC_A	-	0.79	0.63	1.00	5	231	0	0	0	0	3
2KRL_A	-	0.75	0.83	0.68	19	1996	15	7	2	6	4
2LC8_A	GP_knot1 links to Rfam database...	0.64	0.61	0.69	11	512	5	1	4	0	7
2M58_A	-	0.60	0.58	0.64	7	533	5	1	3	1	5
2RP0_A	-	0.84	0.71	1.00	5	111	0	0	0	0	2
2ZZN_D	tRNA links to Rfam database...	0.93	0.95	0.91	21	961	2	2	0	0	1
3A2K_C	tRNA links to Rfam database...	0.98	0.95	1.00	21	1087	0	0	0	0	1
3A3A_A	tRNA-Sec links to Rfam database...	0.97	0.93	1.00	28	1472	0	0	0	0	2
3ADB_C	-	0.98	0.97	1.00	32	1787	1	0	0	1	1
3GCA_A	-	0.84	0.71	1.00	5	153	0	0	0	0	2
3GX2_A	SAM links to Rfam database...	0.56	0.57	0.57	16	1421	13	4	8	1	12
3IVN_B	Purine links to Rfam database...	0.91	0.83	1.00	19	884	0	0	0	0	4
3IWN_A	c-di-GMP-I links to Rfam database...	0.21	0.21	0.24	6	1447	19	2	17	0	22
3IYQ_A	tmRNA links to Rfam database...	0.22	0.31	0.15	16	22335	100	45	44	11	35
3IZ4_A	tmRNA links to Rfam database...	0.46	0.46	0.47	44	25442	56	14	36	6	51
3J0L_a	-	0.21	0.18	0.29	2	404	6	3	2	1	9
3J20_0	tRNA links to Rfam database...	0.74	0.76	0.73	16	1197	7	3	3	1	5
3J3D_C	tRNA links to Rfam database...	0.90	0.95	0.86	18	947	3	3	0	0	1
3J3E_8	5_8S_rRNA links to Rfam database...	-0.01	0.00	0.00	0	2718	36	8	16	12	15
3J3F_8	5_8S_rRNA links to Rfam database...	0.31	0.42	0.23	8	4726	41	12	15	14	11
3JYV_7	tRNA links to Rfam database...	-0.02	0.00	0.00	0	1091	20	4	16	0	20
3LA5_A	Purine links to Rfam database...	0.91	0.84	1.00	21	933	0	0	0	0	4
3NKB_B	-	0.42	0.42	0.44	8	717	10	2	8	0	11
3NPB_A	SAM links to Rfam database...	0.90	0.81	1.00	30	2248	5	0	0	5	7
3O58_3	5_8S_rRNA links to Rfam database...	0.30	0.36	0.25	8	4732	27	15	9	3	14
3PDR_A	ykoK links to Rfam database...	0.83	0.80	0.87	40	4794	9	2	4	3	10
3RKF_A	Purine links to Rfam database...	0.91	0.83	1.00	20	846	0	0	0	0	4
3SD1_A	THF links to Rfam database...	0.82	0.83	0.83	24	1504	5	1	4	0	5
3U4M_B	-	0.95	0.91	1.00	20	1256	2	0	0	2	2
3W1K_J	tRNA-Sec links to Rfam database...	0.86	0.77	0.96	24	1653	1	1	0	0	7
3W3S_B	tRNA-Sec links to Rfam database...	0.49	0.45	0.54	15	1961	14	1	12	1	18
3ZEX_C	5_8S_rRNA links to Rfam database...	0.33	0.34	0.32	10	5343	25	4	17	4	19
4A1C_2	5_8S_rRNA links to Rfam database...	0.19	0.25	0.16	5	4484	33	11	16	6	15
4AOB_A	SAM links to Rfam database...	0.50	0.48	0.54	14	1411	13	3	9	1	15
4ATO_G	-	-0.03	0.00	0.00	0	215	5	0	5	0	7
4ENB_A	crcB links to Rfam database...	1.00	1.00	1.00	15	457	0	0	0	0	0
4ENC_A	crcB links to Rfam database...	0.97	1.00	0.94	15	480	1	1	0	0	0
4FRG_B	AdoCbl-variant links to Rfam database...	0.93	0.88	1.00	21	1181	0	0	0	0	3
4FRN_A	AdoCbl-variant links to Rfam database...	0.82	0.71	0.95	20	1827	1	1	0	0	8
4JF2_A	preQ1-II links to Rfam database...	0.92	1.00	0.86	24	1054	5	4	0	1	0
4JRC_A	-	0.97	0.94	1.00	16	606	0	0	0	0	1

^top

Performance of RNAsubopt - scored lower in this pairwise comparison

1. Total counts & total scores for RNAsubopt

Total Base Pair Counts
Total TP	619
Total TN	109105
Total FP	601
Total FP CONTRA	175
Total FP INCONS	312
Total FP COMP	114
Total FN	393

Total Scores
MCC	0.581
Average MCC ± 95% Confidence Intervals	0.566 ± 0.090
Sensitivity	0.612
Positive Predictive Value	0.560
Nr of predictions	41

^top

2. Individual counts for RNAsubopt [ download as .csv ]

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
2KFC_A	-	-0.03	0.00	0.00	0	229	7	1	6	0	8
2KRL_A	-	0.62	0.61	0.64	14	2002	15	5	3	7	9
2LC8_A	GP_knot1 links to Rfam database...	-0.03	0.00	0.00	0	513	15	3	12	0	18
2M58_A	-	0.60	0.58	0.64	7	533	4	1	3	0	5
2RP0_A	-	0.84	0.71	1.00	5	111	0	0	0	0	2
2ZZN_D	tRNA links to Rfam database...	0.52	0.55	0.52	12	961	11	3	8	0	10
3A2K_C	tRNA links to Rfam database...	0.47	0.50	0.46	11	1084	13	3	10	0	11
3A3A_A	tRNA-Sec links to Rfam database...	0.97	0.93	1.00	28	1472	0	0	0	0	2
3ADB_C	-	0.98	0.97	1.00	32	1787	0	0	0	0	1
3GCA_A	-	0.84	0.71	1.00	5	153	0	0	0	0	2
3GX2_A	SAM links to Rfam database...	0.53	0.54	0.54	15	1421	14	4	9	1	13
3IVN_B	Purine links to Rfam database...	0.88	0.78	1.00	18	885	0	0	0	0	5
3IWN_A	c-di-GMP-I links to Rfam database...	0.69	0.68	0.70	19	1445	9	1	7	1	9
3IYQ_A	tmRNA links to Rfam database...	0.28	0.39	0.20	20	22342	96	41	37	18	31
3IZ4_A	tmRNA links to Rfam database...	0.56	0.60	0.53	57	25429	54	26	24	4	38
3J0L_a	-	0.35	0.36	0.36	4	400	8	3	4	1	7
3J20_0	tRNA links to Rfam database...	0.51	0.57	0.48	12	1194	14	2	11	1	9
3J3D_C	tRNA links to Rfam database...	0.65	0.74	0.58	14	944	10	5	5	0	5
3J3E_8	5_8S_rRNA links to Rfam database...	0.25	0.33	0.20	5	2717	32	7	13	12	10
3J3F_8	5_8S_rRNA links to Rfam database...	0.35	0.47	0.26	9	4726	42	13	13	16	10
3JYV_7	tRNA links to Rfam database...	-0.02	0.00	0.00	0	1090	21	4	17	0	20
3LA5_A	Purine links to Rfam database...	0.91	0.84	1.00	21	933	0	0	0	0	4
3NKB_B	-	0.66	0.68	0.65	13	715	7	0	7	0	6
3NPB_A	SAM links to Rfam database...	0.84	0.78	0.91	29	2246	8	0	3	5	8
3O58_3	5_8S_rRNA links to Rfam database...	0.41	0.50	0.34	11	4732	35	6	15	14	11
3PDR_A	ykoK links to Rfam database...	0.90	0.90	0.90	45	4790	7	2	3	2	5
3RKF_A	Purine links to Rfam database...	0.91	0.83	1.00	20	846	0	0	0	0	4
3SD1_A	THF links to Rfam database...	0.73	0.72	0.75	21	1505	7	4	3	0	8
3U4M_B	-	0.49	0.55	0.46	12	1250	14	3	11	0	10
3W1K_J	tRNA-Sec links to Rfam database...	0.97	0.97	0.97	30	1647	1	1	0	0	1
3W3S_B	tRNA-Sec links to Rfam database...	0.95	0.94	0.97	31	1957	2	0	1	1	2
3ZEX_C	5_8S_rRNA links to Rfam database...	0.30	0.34	0.26	10	5336	43	5	23	15	19
4A1C_2	5_8S_rRNA links to Rfam database...	0.18	0.25	0.14	5	4481	43	13	17	13	15
4AOB_A	SAM links to Rfam database...	0.62	0.62	0.64	18	1409	11	4	6	1	11
4ATO_G	-	0.36	0.43	0.33	3	211	6	6	0	0	4
4ENB_A	crcB links to Rfam database...	0.85	0.73	1.00	11	461	2	0	0	2	4
4ENC_A	crcB links to Rfam database...	0.36	0.33	0.42	5	484	7	0	7	0	10
4FRG_B	AdoCbl-variant links to Rfam database...	0.36	0.38	0.38	9	1178	15	2	13	0	15
4FRN_A	AdoCbl-variant links to Rfam database...	0.58	0.57	0.59	16	1821	11	3	8	0	12
4JF2_A	preQ1-II links to Rfam database...	0.69	0.67	0.73	16	1060	6	4	2	0	8
4JRC_A	-	0.34	0.35	0.35	6	605	11	0	11	0	11

^top

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.

Table of contents:

Overview

Performance plots

Performance of McQFold - scored higher in this pairwise comparison

1. Total counts & total scores for McQFold

2. Individual counts for McQFold [ download as .csv ]

Performance of RNAsubopt - scored lower in this pairwise comparison

1. Total counts & total scores for RNAsubopt

2. Individual counts for RNAsubopt [ download as .csv ]