title: Bayesian Prior Choice in IRT Estimation Using MCMC and Variational Bayes.
abstract: This study investigated the impact of three prior distributions: matched, standard vague, and hierarchical in Bayesian estimation parameter recovery in two and one parameter models. Two Bayesian estimation methods were utilized: Markov chain Monte Carlo (MCMC) and the relatively new, Variational Bayesian (VB). Conditional (CML) and Marginal Maximum Likelihood (MML) estimates were used as baseline methods for comparison. Vague priors produced large errors or convergence issues and are not recommended. For both MCMC and VB, the hierarchical and matched priors showed the lowest root mean squared errors (RMSEs) for ability estimates; RMSEs of difficulty estimates were similar across estimation methods. For the standard errors (SEs), MCMC-hierarchical displayed the largest values across most conditions. SEs from the VB estimation were among the lowest in all but one case. Overall, VB-hierarchical, VB-matched, and MCMC-matched performed best. VB with hierarchical priors are recommended in terms of their accuracy, and cost and (subsequently) time effectiveness.
authors.affiliation: Department of Educational Psychology, University of North Texas Denton, TX, USA.
authors.forename: Prathiba
authors.initials: P
authors.lastname: Natesan
authors.affiliation: School of Education, University of Delaware Newark, DE, USA.
authors.forename: Ratna
authors.initials: R
authors.lastname: Nandakumar
authors.affiliation: Microsoft Research Cambridge Cambridge, UK.
authors.forename: Tom
authors.initials: T
authors.lastname: Minka
authors.affiliation: National Board of Medical Examiners Philadelphia, PA, USA.
authors.forename: Jonathan D
authors.initials: JD
authors.lastname: Rubright
keywords: Bayesian; Markov chain Monte Carlo; item response theory; marginal maximum likelihood; variational Bayesian