CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(20)		Murlet(20)
MCC	0.622	>	0.513
Average MCC ± 95% Confidence Intervals	0.620 ± 0.196	>	0.516 ± 0.165
Sensitivity	0.597	>	0.461
Positive Predictive Value	0.654	>	0.579
Total TP	123	>	95
Total TN	22999	<	23023
Total FP	94	>	90
Total FP CONTRA	14	<	16
Total FP INCONS	51	<	53
Total FP COMP	29	>	21
Total FN	83	<	111
P-value	6.08519761914e-09

Comparison of performance of PETfold_pre2.0(20) and Murlet(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(20) and Murlet(20)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(20) and Murlet(20)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(20) and Murlet(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(20) and Murlet(20)).

Total Scores
MCC	0.622
Average MCC ± 95% Confidence Intervals	0.620 ± 0.196
Sensitivity	0.597
Positive Predictive Value	0.654
Nr of predictions	10

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	0.95	0.95	0.95	20	1198	1	1	0	0	1
3J3D_C	tRNA links to Rfam database...	0.87	0.89	0.85	17	948	3	3	0	0	2
3J3E_8	5_8S_rRNA links to Rfam database...	0.48	0.47	0.50	7	2728	11	2	5	4	8
3J3F_8	5_8S_rRNA links to Rfam database...	0.34	0.37	0.32	7	4739	24	5	10	9	12
3ZEX_C	5_8S_rRNA links to Rfam database...	0.26	0.24	0.29	7	5350	20	1	16	3	22
4A1C_2	5_8S_rRNA links to Rfam database...	0.25	0.25	0.25	5	4496	25	2	13	10	15
4AOB_A	SAM links to Rfam database...	0.87	0.83	0.92	24	1411	4	0	2	2	5
4ENB_A	crcB links to Rfam database...	0.68	0.53	0.89	8	463	1	0	1	0	7
4ENC_A	crcB links to Rfam database...	0.61	0.53	0.73	8	485	3	0	3	0	7
4FRG_B	AdoCbl-variant links to Rfam database...	0.89	0.83	0.95	20	1181	2	0	1	1	4

Total Scores
MCC	0.513
Average MCC ± 95% Confidence Intervals	0.516 ± 0.165
Sensitivity	0.461
Positive Predictive Value	0.579
Nr of predictions	10

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	0.65	0.62	0.68	13	1200	6	1	5	0	8
3J3D_C	tRNA links to Rfam database...	0.84	0.84	0.84	16	949	3	3	0	0	3
3J3E_8	5_8S_rRNA links to Rfam database...	0.35	0.33	0.38	5	2729	13	2	6	5	10
3J3F_8	5_8S_rRNA links to Rfam database...	0.54	0.53	0.56	10	4743	13	2	6	5	9
3ZEX_C	5_8S_rRNA links to Rfam database...	0.42	0.38	0.48	11	5351	16	2	10	4	18
4A1C_2	5_8S_rRNA links to Rfam database...	0.22	0.25	0.20	5	4491	24	5	15	4	15
4AOB_A	SAM links to Rfam database...	0.87	0.76	1.00	22	1415	3	0	0	3	7
4ENB_A	crcB links to Rfam database...	0.51	0.27	1.00	4	468	0	0	0	0	11
4ENC_A	crcB links to Rfam database...	0.57	0.33	1.00	5	491	0	0	0	0	10
4FRG_B	AdoCbl-variant links to Rfam database...	0.19	0.17	0.25	4	1186	12	1	11	0	20

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.