Equivalence of mail and telephone responses to the CAHPS Hospital Survey.
OBJECTIVE
To estimate the effect of survey mode (mail versus telephone) on reports and ratings of hospital care.


METHODS
The total sample included 20,826 patients discharged from a group of 24 distinct hospitals in three states (Arizona, Maryland, New York). We collected CAHPS data in 2003 by mail and telephone from 9,504 patients, of whom 39 percent responded by telephone and 61 percent by mail.


METHODS
We estimated mode effects in an observational design, using both propensity score blocking and (ordered) logistic regression on covariates. We used variables derived from administrative data (either included as covariates in the regression function or used in estimating the propensity score) grouped in three categories: individual characteristics, characteristics of the stay and hospital, and survey administration variables.


METHODS
We mailed a 66-item questionnaire to everyone in the sample and followed up by telephone with those who did not respond.


RESULTS
We found significant (p<.01) mode effects for 13 of the 21 questions examined in this study. The maximum magnitude of the survey mode effect was an 11 percentage-point difference in the probability of a "yes" response to one of the survey questions. Telephone respondents were more likely to rate care positively and health status negatively, compared with mail respondents. Standard regression-based case-mix adjustment captured much of the mode effects detected by propensity score techniques in this application.


CONCLUSIONS
Telephone mode increases the propensity for more favorable evaluations of care for more than half of the items examined. This suggests that mode of administration should be standardized or carefully adjusted for. Alternatively, further item development may minimize the sensitivity of items to mode of data collection.
de Vries
Han
H
RAND Corporation, Santa Monica, CA 90407-2138, USA.
Elliott
Marc N
MN
Hepner
Kimberly A
KA
Keller
San D
SD
Hays
Ron D
RD
D003625:Data Collection / Q000706:statistics & numerical data*; D005260:Female; D019538:Health Care Surveys / Q000706:statistics & numerical data*; D006761:Hospitals / Q000706:statistics & numerical data*; D006801:Humans; D008297:Male; D008875:Middle Aged; D017060:Patient Satisfaction; D011177:Postal Service; D011787:Quality of Health Care / Q000706:statistics & numerical data*; D013689:Telephone