A novel method for expediting the development of patient-reported outcome measures and an evaluation across several populations.
Item response theory (IRT) models provide an appropriate alternative to the classical ordinal confirmatory factor analysis (CFA) during the development of patient-reported outcome measures (PROMs). Current literature has identified the assessment of IRT model fit as both challenging and underdeveloped (Sinharay & Johnson, 2003; Sinharay, Johnson, & Stern, 2006). This study evaluates the performance of Ordinal Bayesian Instrument Development (OBID), a Bayesian IRT model with a probit link function approach, through applications in two breast cancer-related instrument development studies. The primary focus is to investigate an appropriate method for comparing Bayesian IRT models in PROMs development. An exact Bayesian leave-one-out cross-validation (LOO-CV) approach (Vehtari & Lampinen, 2002) is implemented to assess prior selection for the item discrimination parameter in the IRT model and subject content experts' bias (in a statistical sense and not to be confused with psychometric bias as in differential item functioning) toward the estimation of item-to-domain correlations. Results support the utilization of content subject experts' information in establishing evidence for construct validity when sample size is small. However, the incorporation of subject experts' content information in the OBID approach can be sensitive to the level of expertise of the recruited experts. More stringent efforts need to be invested in the appropriate selection of subject experts to efficiently use the OBID approach and reduce potential bias during PROMs development.
Division of Biometrics III, OB/OTS/CDER, U.S. Food and Drug Administration, Silver Spring, MD 20993.
Lili
L
Garrard
College of Education, Texas State University, San Marcos, TX 78666, USA.
Larry R
LR
Price
University of Kansas School of Nursing, Mail Stop 4043, 3901 Rainbow Blvd., Kansas City, KS 66160, USA.
Marjorie J
MJ
Bott
Department of Biostatistics, University of Kansas Medical Center, Mail Stop 1026, 3901 Rainbow Blvd., Kansas City, KS 66160, USA; University of Kansas School of Nursing, Mail Stop 4043, 3901 Rainbow Blvd., Kansas City, KS 66160, USA.
Byron J
BJ
Gajewski
Bayesian IRT; Bayesian leave-one-out cross-validation; Bayesian model comparison; OBID; PROMs; patient-reported outcome measures
Res Nurs Health. 2006 Oct;29(5):489-97
16977646
Res Nurs Health. 1997 Jun;20(3):269-74
9179180
BMC Womens Health. 2010 Dec 15;10:34
21159197
BMC Med Res Methodol. 2015 Sep 29;15:77
26419748
J Cancer Educ. 2016 Sep;31(3):493-9
25952941
Appl Psychol Meas. 2014 Jun;38(4):296-310
24882893
Prev Chronic Dis. 2013 Jul 03;10:E109
23823698
Neural Comput. 2002 Oct;14(10):2439-68
12396570
Psychometrika. 2016 Dec;81(4):1142-1163
26400070
Nurs Res. 1986 Nov-Dec;35(6):382-5
3640358
Health Qual Life Outcomes. 2006 Oct 11;4:79
17034633
Br J Math Stat Psychol. 2006 May;59(Pt 1):173-94
16709285