title: A LASSO FOR HIERARCHICAL INTERACTIONS.
abstract: We add a set of convex constraints to the lasso to produce sparse interaction models that honor the hierarchy restriction that an interaction only be included in a model if one or both variables are marginally important. We give a precise characterization of the effect of this hierarchy constraint, prove that hierarchy holds with probability one and derive an unbiased estimate for the degrees of freedom of our estimator. A bound on this estimate reveals the amount of fitting "saved" by the hierarchy constraint. We distinguish betweenparameter sparsity-the number of nonzero coefficients-andpractical sparsity-the number of raw variables one mustmeasureto make a new prediction. Hierarchy focuses on the latter, which is more closely tied to important data collection concerns such as cost, time and effort. We develop an algorithm, available in the R package hierNet, and perform an empirical study of our method.
authors.affiliation: Cornell University, Stanford University and Stanford University.
authors.forename: Jacob
authors.initials: J
authors.lastname: Bien
authors.affiliation: Cornell University, Stanford University and Stanford University.
authors.forename: Jonathan
authors.initials: J
authors.lastname: Taylor
authors.affiliation: Cornell University, Stanford University and Stanford University.
authors.forename: Robert
authors.initials: R
authors.lastname: Tibshirani
keywords: Regularized regression; convexity; hierarchical sparsity; interactions; lasso