Reliability of the nursing home survey process: a simultaneous survey approach.
OBJECTIVE
We designed this study to examine the reliability of the nursing home survey process in the state of Kansas using regular and simultaneous survey teams. In particular, the study examined how two survey teams exposed to the same information at the same time differed in their interpretations.


METHODS
The protocol for simultaneous surveys consists of having one in-region and one out-of-region team survey a facility together.


RESULTS
The regular and simultaneous survey teams generally agreed about the number of deficiencies. The intraclass correlation coefficient was 0.87 for total deficiencies and 0.76 for deficiencies with scores of G or higher. But in a substantial number of instances the teams did not agree about the scope and severity of the deficiency or about what regulation the nursing home had breached.


CONCLUSIONS
The survey process is reliable when assessing aggregate results, but it is only moderately reliable when examining individual citations. Stakeholders (i.e., consumers, policy makers, nursing home administrators) should be aware of the limitations of the survey process. It needs to be modified to reduce variability.
Lee
Robert H
RH
Department of Health Policy and Management, University of Kansas Medical Center, Mail Stop 3044, 3901 Rainbow Boulevard, Kansas City, KS 66160, USA. rlee2@kumc.edu
Gajewski
Byron J
BJ
Thompson
Sarah
S
D003625:Data Collection / Q000379:methods; D003627:Data Interpretation, Statistical; D006801:Humans; D007615:Kansas; D009735:Nursing Homes / Q000592:standards*; D015588:Observer Variation; D011787:Quality of Health Care / Q000592:standards*; D015203:Reproducibility of Results