CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(seed)		PPfold(20)
MCC	0.775	>	0.746
Average MCC ± 95% Confidence Intervals	0.777 ± 0.108	>	0.747 ± 0.127
Sensitivity	0.743	>	0.681
Positive Predictive Value	0.808	<	0.818
Total TP	2170	>	1989
Total TN	7954690	<	7954946
Total FP	810	>	607
Total FP CONTRA	80	>	71
Total FP INCONS	437	>	371
Total FP COMP	293	>	165
Total FN	750	<	931
P-value	0.0

Comparison of performance of PETfold_pre2.0(seed) and PPfold(20). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and PPfold(20)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and PPfold(20)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(seed) and PPfold(20). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(seed) and PPfold(20)).

Total Scores
MCC	0.775
Average MCC ± 95% Confidence Intervals	0.777 ± 0.108
Sensitivity	0.743
Positive Predictive Value	0.808
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.85	0.82	0.87	386	1192295	93	16	43	34	82
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.66	0.63	0.69	315	1627654	205	15	126	64	187
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.64	0.60	0.67	306	1674911	213	16	132	65	201
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.87	0.84	0.89	381	1119327	104	6	42	56	71
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.86	0.85	0.87	395	1135823	93	17	43	33	69
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.80	0.73	0.86	387	1204680	102	10	51	41	140

Total Scores
MCC	0.746
Average MCC ± 95% Confidence Intervals	0.747 ± 0.127
Sensitivity	0.681
Positive Predictive Value	0.818
Nr of predictions	6

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
CRW_00256	SSU_rRNA_bacteria links to Rfam database...	0.82	0.78	0.87	363	1192324	70	13	40	17	105
CRW_00326	SSU_rRNA_eukarya links to Rfam database...	0.64	0.56	0.73	280	1627728	143	17	85	41	222
CRW_00358	SSU_rRNA_eukarya links to Rfam database...	0.56	0.49	0.64	246	1674980	178	15	124	39	261
CRW_00590	SSU_rRNA_bacteria links to Rfam database...	0.85	0.81	0.90	367	1119348	69	4	37	28	85
CRW_00997	SSU_rRNA_bacteria links to Rfam database...	0.83	0.81	0.86	374	1135845	75	15	44	16	90
PDB_01120	SSU_rRNA_bacteria links to Rfam database...	0.78	0.68	0.88	359	1204721	72	7	41	24	168

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.