CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	Murlet(20)		RSpredict(seed)
MCC	0.636	>	0.006
Average MCC ± 95% Confidence Intervals	0.635 ± 0.200	>	0.006 ± 0.010
Sensitivity	0.564	>	0.005
Positive Predictive Value	0.717	>	0.007
Total TP	1906	>	18
Total TN	9346894	<	9346965
Total FP	895	<	2625
Total FP CONTRA	111	<	363
Total FP INCONS	640	<	2205
Total FP COMP	144	>	57
Total FN	1476	<	3364
P-value	0.0

Comparison of performance of Murlet(20) and RSpredict(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for Murlet(20) and RSpredict(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for Murlet(20) and RSpredict(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for Murlet(20) and RSpredict(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for Murlet(20) and RSpredict(seed)).

Total Scores
MCC	0.636
Average MCC ± 95% Confidence Intervals	0.635 ± 0.200
Sensitivity	0.564
Positive Predictive Value	0.717
Nr of predictions	7

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.75	0.70	0.81	330	1178473	93	14	63	16	143
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.75	0.68	0.84	325	1198537	73	14	50	9	153
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.78	0.72	0.85	328	1117874	75	11	47	17	125
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.31	0.25	0.40	122	1615195	206	23	163	20	374
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.33	0.29	0.36	146	1665818	288	38	223	27	349
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.78	0.71	0.86	318	1137416	79	8	44	27	129
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.73	0.62	0.86	337	1433581	81	3	50	28	203

Total Scores
MCC	0.006
Average MCC ± 95% Confidence Intervals	0.006 ± 0.010
Sensitivity	0.005
Positive Predictive Value	0.007
Nr of predictions	7

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00177	SSU_rRNA_bacteria links to Rfam database...	0.00	0.00	0.00	0	1178475	411	54	351	6	473
CRW_00250	SSU_rRNA_bacteria links to Rfam database...	0.00	0.00	0.00	0	1198517	412	51	358	3	478
CRW_00280	SSU_rRNA_bacteria links to Rfam database...	0.02	0.02	0.02	9	1117885	368	54	312	2	444
CRW_00411	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1615206	309	45	252	12	496
CRW_00414	SSU_rRNA_eukarya links to Rfam database...	0.00	0.00	0.00	0	1665933	312	53	239	20	495
CRW_00789	SSU_rRNA_bacteria links to Rfam database...	0.02	0.02	0.02	9	1137386	398	62	329	7	438
PDB_01105	SSU_rRNA_bacteria links to Rfam database...	0.00	0.00	0.00	0	1433563	415	44	364	7	540

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.