On the approximation rate of hierarchical mixtures-of-experts for generalized linear models.
We investigate a class of hierarchical mixtures-of-experts (HME) models where generalized linear models with nonlinear mean functions of the form psi (alpha + xT beta) are mixed. Here psi (.) is the inverse link function. It is shown that mixtures of such mean functions can approximate a class of smooth functions of the form psi (h(x)), where h(.) epsilon W2;K infinity (a Sobolev class over [0, 1]s), as the number of experts m in the network increases. An upper bound of the approximation rate is given as O(m-2/s) in Lp norm. This rate can be achieved within the family of HME structures with no more than s-layers, where s is the dimension of the predictor x.
Department of Statistics, Northwestern University, Evanston, IL 60208, USA.
W
W
Jiang
M A
MA
Tanner
D000465:Algorithms; D008962:Models, Theoretical; D009415:Nerve Net