title: Prediction-based Termination Rule for Greedy Learning with Massive Data.
abstract: The appearance of massive data has become increasingly common in contemporary scientific research. When sample sizenis huge, classical learning methods become computationally costly for the regression purpose. Recently, the orthogonal greedy algorithm (OGA) has been revitalized as an efficient alternative in the context of kernel-based statistical learning. In a learning problem, accurate and fast prediction is often of interest. This makes an appropriate termination crucial for OGA. In this paper, we propose a new termination rule for OGA via investigating its predictive performance. The proposed rule is conceptually simple and convenient for implementation, which suggests an [Formula: see text] number of essential updates in an OGA process. It therefore provides an appealing route to conduct efficient learning for massive data. With a sample dependent kernel dictionary, we show that the proposed method is strongly consistent with an [Formula: see text] convergence rate to the oracle prediction. The promising performance of the method is supported by both simulation and real data examples.
authors.affiliation: The Pennsylvania State University.
authors.forename: Chen
authors.initials: C
authors.lastname: Xu
authors.affiliation: Xi'an Jiaotong University.
authors.forename: Shaobo
authors.initials: S
authors.lastname: Lin
authors.affiliation: Xi'an Jiaotong University.
authors.forename: Jian
authors.initials: J
authors.lastname: Fang
authors.affiliation: The Pennsylvania State University.
authors.forename: Runze
authors.initials: R
authors.lastname: Li
keywords: Forward regression; Greedy algorithms; Kernel methods; Massive data; Nonparametric regression; Sparse modeling