title: Work efficiency: a new criterion for comprehensive comparison and evaluation of statistical methods in large-scale identification of differentially expressed genes.
abstract: Receiver operating characteristic (ROC) has been widely used to evaluate statistical methods, but a fatal problem is that ROC cannot evaluate estimation of the false discovery rate (FDR) of a statistical method and hence the area under of curve as a criterion cannot tell us if a statistical method is conservative. To address this issue, we propose an alternative criterion, work efficiency. Work efficiency is defined as the product of the power and degree of conservativeness of a statistical method. We conducted large-scale simulation comparisons among the optimizing discovery procedure (ODP), the Bonferroni (B-) procedure, Local FDR (Localfdr), ranking analysis of the F-statistics (RAF), the Benjamini-Hochberg (BH-) procedure, and significance analysis of microarray data (SAM). The results show that ODP, SAM, and the B-procedure perform with low efficiencies while the BH-procedure, RAF, and Localfdr work with higher efficiency. ODP and SAM have the same ROC curves but their efficiencies are significantly different.
authors: College of Life Science, Hunan Normal University, Changsha, P.R. China. tanyuande@gmail.com
authors: Yuan-De
authors: YD
authors: Tan
mesh_terms: D000465:Algorithms; D019540:Area Under Curve; D003198:Computer Simulation; D003627:Data Interpretation, Statistical; D020869:Gene Expression Profiling / Q000379:methods*; D005786:Gene Expression Regulation; D015233:Models, Statistical; D020411:Oligonucleotide Array Sequence Analysis; D012372:ROC Curve