Spherical Hamiltonian Monte Carlo for Constrained Target Distributions.
Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit models, many copula models, and Latent Dirichlet Allocation (LDA) models. Bayesian inference involving probability distributions confined to constrained domains could be quite challenging for commonly used sampling algorithms. For such problems, we propose a novel Markov Chain Monte Carlo (MCMC) method that provides a general and computationally efficient framework for handling boundary conditions. Our method first maps theD-dimensional constrained domain of parameters to the unit ball [Formula: see text], then augments it to aD-dimensional sphereSDsuch that the original boundary corresponds to the equator ofSD. This way, our method handles the constraints implicitly by moving freely on the sphere generating proposals that remain within boundaries when mapped back to the original space. To improve the computational efficiency of our algorithm, we divide the dynamics into several parts such that the resulting split dynamics has a partial analytical solution as a geodesic flow on the sphere. We apply our method to several examples including truncated Gaussian, Bayesian Lasso, Bayesian bridge regression, and a copula model for identifying synchrony among multiple neurons. Our results show that the proposed method can provide a natural and efficient framework for handling several types of constraints on target distributions.
Department of Statistics, University of California, Irvine, CA 92697, USA.
Shiwei
S
Lan
Department of Statistics, University of California, Irvine, CA 92697, USA.
Bo
B
Zhou
Department of Statistics, University of California, Irvine, CA 92697, USA.
Babak
B
Shahbaba