Inter-rater reliability of nursing home surveys: a Bayesian latent class approach.
In the U.S., federal and state governments perform routine inspections of nursing homes. Results of the inspections allow government to generate fines for findings of non-compliance as well as allow consumers to rank facilities. The purpose of this study is to investigate the inter-rater reliability of the nursing home survey process. In general, the survey data involves 191 binary deficiency variables interpreted as 'deficient' or 'non-deficient'. To reduce the dimensionality of the problem, our proposed method involves two steps. First, we reduce the deficiency categories to sub-categories using previous nursing home studies. Second, looking at the State of Kansas specifically, we take the deficiency data from 1 year, and use Bayesian latent class analysis (LCA) to collapse the sub-categories to a binary variable. We evaluate inter-rater agreement using deficiency data from two separate survey teams on one facility, a matched-pair design. We evaluate the agreement of the two raters on binary data using the weights from the LCA. This allows a two-by-two contingency analysis using a Bayesian beta-binomial model. We elicit informative prior distributions from the nursing home providers. Together, with the experimental data, this provides a posterior distribution of the kappa agreement of the raters for nursing home deficiency citation data.
Gajewski
Byron J
BJ
Schools of Allied Health and Nursing, University of Kansas, Kansas City, KS 66160, USA. bgajewski@kumc.edu
Thompson
Sarah
S
Dunton
Nancy
N
Becker
Annette
A
Wrona
Marcia
M
D001499:Bayes Theorem; D003625:Data Collection / Q000379:methods; D003627:Data Interpretation, Statistical; D006801:Humans; D009735:Nursing Homes / Q000592:standards*; D015588:Observer Variation; D011787:Quality of Health Care / Q000592:standards*