Laplace Variational Approximation for Semiparametric Regression in the Presence of Heteroskedastic Errors.
Variational approximations provide fast, deterministic alternatives to Markov Chain Monte Carlo for Bayesian inference on the parameters of complex, hierarchical models. Variational approximations are often limited in practicality in the absence of conjugate posterior distributions. Recent work has focused on the application of variational methods to models with only partial conjugacy, such as in semiparametric regression with heteroskedastic errors. Here, both the mean and log variance functions are modeled as smooth functions of covariates. For this problem, we derive a mean field variational approximation with an embedded Laplace approximation to account for the non-conjugate structure. Empirical results with simulated and real data show that our approximate method has significant computational advantages over traditional Markov Chain Monte Carlo; in this case, a delayed rejection adaptive Metropolis algorithm. The variational approximation is much faster and eliminates the need for tuning parameter selection, achieves good fits for both the mean and log variance functions, and reasonably reflects the posterior uncertainty. We apply the methods to log-intensity data from a small angle X-ray scattering experiment, in which properly accounting for the smooth heteroskedasticity leads to significant improvements in posterior inference for key physical characteristics of an organic molecule.
Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, 77030 ( bdbugbee@mdanderson.org ).
Bruce D
BD
Bugbee
Department of Statistics, Colorado State University, Fort Collins, CO, 80523 ( jbreidt@stat.colostate.edu ).
F Jay
FJ
Breidt
Department of Biochemistry & Molecular Biology, Colorado State University, Fort Collins, CO, 80523 ( Mark.van der Woerd@colostate.edu ) October 12, 2014.
Mark J
MJ
van der Woerd
mean field variational Bayes; penalized splines; small angle X-ray scattering
Q Rev Biophys. 2007 Aug;40(3):191-285
18078545
J Time Ser Anal. 2012 Sep 1;33(5):704-717
23355752
Annu Rev Biochem. 1979;48:159-91
382983