title: A Brief Survey of Modern Optimization for Statisticians.
abstract: Modern computational statistics is turning more and more to high-dimensional optimization to handle the deluge of big data. Once a model is formulated, its parameters can be estimated by optimization. Because model parsimony is important, models routinely include nondifferentiable penalty terms such as the lasso. This sober reality complicates minimization and maximization. Our broad survey stresses a few important principles in algorithm design. Rather than view these principles in isolation, it is more productive to mix and match them. A few well chosen examples illustrate this point. Algorithm derivation is also emphasized, and theory is downplayed, particularly the abstractions of the convex calculus. Thus, our survey should be useful and accessible to a broad audience.
authors.affiliation: Departments of Biomathematics, Human Genetics, and Statistics University of California Los Angeles, CA 90095-1766 klange@ucla.edu.
authors.forename: Kenneth
authors.initials: K
authors.lastname: Lange
authors.affiliation: Department of Human Genetics University of California Los Angeles, CA 90095 ecchi@ucla.edu.
authors.forename: Eric C
authors.initials: EC
authors.lastname: Chi
authors.affiliation: Department of Statistics North Carolina State University Raleigh, NC 27695-8203 hua_zhou@ncsu.edu.
authors.forename: Hua
authors.initials: H
authors.lastname: Zhou
keywords: Block relaxation; MM algorithm; Newton’s Method; acceleration; augmented Lagrangian; penalization