CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	RNAalifold(20)		Murlet(20)
MCC	0.672	>	0.621
Average MCC ± 95% Confidence Intervals	0.675 ± 0.157	>	0.620 ± 0.161
Sensitivity	0.619	>	0.544
Positive Predictive Value	0.730	>	0.708
Total TP	2692	>	2368
Total TN	12465136	<	12465477
Total FP	1183	>	1159
Total FP CONTRA	170	>	149
Total FP INCONS	824	<	828
Total FP COMP	189	>	182
Total FN	1657	<	1981
P-value	0.0

Comparison of performance of RNAalifold(20) and Murlet(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for RNAalifold(20) and Murlet(20)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for RNAalifold(20) and Murlet(20)).
Comparison of average Matthews Correlation Coefficients (MCCs) for RNAalifold(20) and Murlet(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for RNAalifold(20) and Murlet(20)).

Total Scores
MCC	0.672
Average MCC ± 95% Confidence Intervals	0.675 ± 0.157
Sensitivity	0.619
Positive Predictive Value	0.730
Nr of predictions	9

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.80	0.76	0.84	358	1178455	78	13	54	11	115
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.83	0.78	0.88	373	1198502	61	12	39	10	105
CRW_00278	SSU_rRNA_bacteria links to Rfam database...	0.82	0.78	0.86	349	1129851	72	10	46	16	101
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.83	0.79	0.87	360	1117845	71	12	43	16	93
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.40	0.35	0.47	182	1988624	243	35	174	34	335
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.43	0.39	0.48	191	1615105	227	43	164	20	305
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.38	0.35	0.41	171	1665811	266	36	207	23	324
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.82	0.78	0.87	348	1137388	80	6	44	30	99
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.76	0.67	0.87	360	1433555	85	3	53	29	180

Total Scores
MCC	0.621
Average MCC ± 95% Confidence Intervals	0.620 ± 0.161
Sensitivity	0.544
Positive Predictive Value	0.708
Nr of predictions	9

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.75	0.70	0.81	330	1178473	93	14	63	16	143
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.75	0.68	0.84	325	1198537	73	14	50	9	153
CRW_00278	SSU_rRNA_bacteria links to Rfam database...	0.75	0.67	0.83	302	1129893	75	12	49	14	148
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.78	0.72	0.85	328	1117874	75	11	47	17	125
CRW_00330	SSU_rRNA_eukarya links to Rfam database...	0.39	0.31	0.49	160	1988690	189	26	139	24	357
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.31	0.25	0.40	122	1615195	206	23	163	20	374
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.33	0.29	0.36	146	1665818	288	38	223	27	349
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.78	0.71	0.86	318	1137416	79	8	44	27	129
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.73	0.62	0.86	337	1433581	81	3	50	28	203

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.