CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PETfold_pre2.0(20)		RSpredict(seed)
MCC	0.718	>	0.471
Average MCC ± 95% Confidence Intervals	0.637 ± 0.179	>	0.184 ± 0.149
Sensitivity	0.682	>	0.321
Positive Predictive Value	0.758	>	0.694
Total TP	304	>	143
Total TN	183468	<	183663
Total FP	163	>	90
Total FP CONTRA	26	>	17
Total FP INCONS	71	>	46
Total FP COMP	66	>	27
Total FN	142	<	303
P-value	1.50834591464e-08

Comparison of performance of PETfold_pre2.0(20) and RSpredict(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PETfold_pre2.0(20) and RSpredict(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PETfold_pre2.0(20) and RSpredict(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PETfold_pre2.0(20) and RSpredict(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PETfold_pre2.0(20) and RSpredict(seed)).

Total Scores
MCC	0.718
Average MCC ± 95% Confidence Intervals	0.637 ± 0.179
Sensitivity	0.682
Positive Predictive Value	0.758
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	0.95	0.95	0.95	20	1198	1	1	0	0	1
3J2C_N	SSU_rRNA_bacteria links to Rfam database...	0.80	0.75	0.85	181	160469	69	12	20	37	59
3J3D_C	tRNA links to Rfam database...	0.87	0.89	0.85	17	948	3	3	0	0	2
3J3E_8	5_8S_rRNA links to Rfam database...	0.48	0.47	0.50	7	2728	11	2	5	4	8
3J3F_8	5_8S_rRNA links to Rfam database...	0.34	0.37	0.32	7	4739	24	5	10	9	12
3ZEX_C	5_8S_rRNA links to Rfam database...	0.26	0.24	0.29	7	5350	20	1	16	3	22
4A1C_2	5_8S_rRNA links to Rfam database...	0.25	0.25	0.25	5	4496	25	2	13	10	15
4AOB_A	SAM links to Rfam database...	0.87	0.83	0.92	24	1411	4	0	2	2	5
4ENB_A	crcB links to Rfam database...	0.68	0.53	0.89	8	463	1	0	1	0	7
4ENC_A	crcB links to Rfam database...	0.61	0.53	0.73	8	485	3	0	3	0	7
4FRG_B	AdoCbl-variant links to Rfam database...	0.89	0.83	0.95	20	1181	2	0	1	1	4

Total Scores
MCC	0.471
Average MCC ± 95% Confidence Intervals	0.184 ± 0.149
Sensitivity	0.321
Positive Predictive Value	0.694
Nr of predictions	11

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
3J20_0	tRNA links to Rfam database...	-0.01	0.00	0.00	0	1215	4	0	4	0	21
3J2C_N	SSU_rRNA_bacteria links to Rfam database...	0.65	0.53	0.79	127	160521	59	9	25	25	113
3J3D_C	tRNA links to Rfam database...	-0.01	0.00	0.00	0	964	4	1	3	0	19
3J3E_8	5_8S_rRNA links to Rfam database...	0.00	0.00	0.00	0	2742	0	0	0	0	15
3J3F_8	5_8S_rRNA links to Rfam database...	0.19	0.11	0.33	2	4755	5	3	1	1	17
3ZEX_C	5_8S_rRNA links to Rfam database...	0.13	0.07	0.25	2	5366	6	2	4	0	27
4A1C_2	5_8S_rRNA links to Rfam database...	0.00	0.00	0.00	0	4512	5	2	2	1	20
4AOB_A	SAM links to Rfam database...	0.42	0.21	0.86	6	1430	1	0	1	0	23
4ENB_A	crcB links to Rfam database...	0.34	0.20	0.60	3	467	2	0	2	0	12
4ENC_A	crcB links to Rfam database...	0.34	0.20	0.60	3	491	2	0	2	0	12
4FRG_B	AdoCbl-variant links to Rfam database...	-0.01	0.00	0.00	0	1200	2	0	2	0	24

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.