title: Effective degrees of freedom: a flawed metaphor.
abstract: To most applied statisticians, a fitting procedure's degrees of freedom is synonymous with its model complexity, or its capacity for overfitting to data. In particular, it is often used to parameterize the bias-variance tradeoff in model selection. We argue that, on the contrary, model complexity and degrees of freedom may correspond very poorly. We exhibit and theoretically explore various fitting procedures for which degrees of freedom is not monotonic in the model complexity parameter, and can exceed the total dimension of the ambient space even in very simple settings. We show that the degrees of freedom for any non-convex projection method can be unbounded.
authors.affiliation: Department of Statistics, Sequoia Hall, 390 Serra Mall, Stanford University, Stanford, California 94305-4065, U.S.A.
authors.forename: Lucas
authors.initials: L
authors.lastname: Janson
authors.affiliation: Department of Statistics, Sequoia Hall, 390 Serra Mall, Stanford University, Stanford, California 94305-4065, U.S.A.
authors.forename: William
authors.initials: W
authors.lastname: Fithian
authors.affiliation: Department of Statistics, Sequoia Hall, 390 Serra Mall, Stanford University, Stanford, California 94305-4065, U.S.A.
authors.forename: Trevor J
authors.initials: TJ
authors.lastname: Hastie
keywords: Model complexity; Number of parameters; Optimism