CompaRNA - on-line benchmarks of RNA structure prediction methods

FAQ

Metric	PPfold(20)		Carnac(seed)
MCC	0.803	>	0.000
Average MCC ± 95% Confidence Intervals	0.786 ± 0.101	>	0.000 ± 0.000
Sensitivity	0.725	>	0.000
Positive Predictive Value	0.890	>	0.000
Total TP	876	>	0
Total TN	635616	<	636600
Total FP	207	>	0
Total FP CONTRA	4	>	0
Total FP INCONS	104	>	0
Total FP COMP	99	>	0
Total FN	332	<	1208
P-value	1.80908287419e-08

Comparison of performance of PPfold(20) and Carnac(seed). Positive Predictive Value (PPV) is plotted against sensitivity. Each dot represents a single test of each method. See tables below for raw data (individual counts for PPfold(20) and Carnac(seed)).
Average Matthews Correlation Coefficients (MCC) with 95% confidence intervals (CIs) were plotted for different RNA families, for which at least 3 members were present in the benchmarking dataset. 'n' denotes the number of MCCs used to calculate the average and CI. See tables below for raw data (individual counts for PPfold(20) and Carnac(seed)).
Comparison of average Matthews Correlation Coefficients (MCCs) for PPfold(20) and Carnac(seed). The whiskers correspond to 95% confidence intervals (CIs). 'n' denotes the number of MCCs used to calculate average MCCs and CIs. See tables below for raw data (individual counts for PPfold(20) and Carnac(seed)).

Total Scores
MCC	0.803
Average MCC ± 95% Confidence Intervals	0.786 ± 0.101
Sensitivity	0.725
Positive Predictive Value	0.890
Nr of predictions	12

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00599	Telomerase-vert links to Rfam database...	0.86	0.80	0.93	89	101379	16	3	4	9	22
RFA_00601	Telomerase-vert links to Rfam database...	0.77	0.68	0.89	77	99148	20	1	9	10	37
SRP_00011	Plant_SRP links to Rfam database...	0.78	0.73	0.84	76	46269	22	0	15	7	28
SRP_00015	Plant_SRP links to Rfam database...	0.81	0.75	0.87	74	46275	21	0	11	10	25
SRP_00152	Plant_SRP links to Rfam database...	0.85	0.80	0.90	82	45662	20	0	9	11	21
SRP_00178	Plant_SRP links to Rfam database...	0.85	0.78	0.93	80	45667	17	0	6	11	22
SRP_00179	Plant_SRP links to Rfam database...	0.86	0.80	0.92	84	45965	15	0	7	8	21
SRP_00181	Plant_SRP links to Rfam database...	0.80	0.74	0.86	75	45969	21	0	12	9	27
SRP_00182	Plant_SRP links to Rfam database...	0.89	0.84	0.93	85	45965	13	0	6	7	16
SRP_00184	Plant_SRP links to Rfam database...	0.83	0.79	0.87	79	45965	24	0	12	12	21
SRP_00188	Plant_SRP links to Rfam database...	0.85	0.81	0.89	64	31053	13	0	8	5	15
SRP_00308	Bacteria_large_SRP links to Rfam database...	0.29	0.13	0.69	11	36299	5	0	5	0	77

Total Scores
MCC	0.000
Average MCC ± 95% Confidence Intervals	0.000 ± 0.000
Sensitivity	0.000
Positive Predictive Value	0.000
Nr of predictions	12

RNA Chain	Rfam family	MCC	SENS	PPV	TP	TN	FP	FP CONTRA	FP INCONS	FP COMP	FN
RFA_00599	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	101475	0	0	0	0	111
RFA_00601	Telomerase-vert links to Rfam database...	0.00	0.00	0.00	0	99235	0	0	0	0	114
SRP_00011	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	46360	0	0	0	0	104
SRP_00015	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	46360	0	0	0	0	99
SRP_00152	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	45753	0	0	0	0	103
SRP_00178	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	45753	0	0	0	0	102
SRP_00179	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	105
SRP_00181	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	102
SRP_00182	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	101
SRP_00184	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	46056	0	0	0	0	100
SRP_00188	Plant_SRP links to Rfam database...	0.00	0.00	0.00	0	31125	0	0	0	0	79
SRP_00308	Bacteria_large_SRP links to Rfam database...	0.00	0.00	0.00	0	36315	0	0	0	0	88

Matthews Correlation Coeffient, Sensitivity and Positive Predictive Value have been calculated based on the paper by Gardener & Giegerich, 2004.