title: Confidence Intervals for Random Forests: The Jackknife and the Infinitesimal Jackknife.
abstract: We study the variability of predictions made by bagged learners and random forests, and show how to estimate standard errors for these methods. Our work builds on variance estimates for bagging proposed by Efron (1992, 2013) that are based on the jackknife and the infinitesimal jackknife (IJ). In practice, bagged predictors are computed using a finite numberBof bootstrap replicates, and working with a largeBcan be computationally expensive. Direct applications of jackknife and IJ estimators to bagging requireB= Θ(n1.5) bootstrap replicates to converge, wherenis the size of the training set. We propose improved versions that only requireB= Θ(n) replicates. Moreover, we show that the IJ estimator requires 1.7 times less bootstrap replicates than the jackknife to achieve a given accuracy. Finally, we study the sampling distributions of the jackknife and IJ variance estimates themselves. We illustrate our findings with multiple experiments and simulation studies.
authors.affiliation: Department of Statistics, Stanford University, Stanford, CA 94305, USA.
authors.forename: Stefan
authors.initials: S
authors.lastname: Wager
authors.affiliation: Department of Statistics, Stanford University, Stanford, CA 94305, USA.
authors.forename: Trevor
authors.initials: T
authors.lastname: Hastie
authors.affiliation: Department of Statistics, Stanford University, Stanford, CA 94305, USA.
authors.forename: Bradley
authors.initials: B
authors.lastname: Efron
keywords: Monte Carlo noise; bagging; jackknife methods; variance estimation