title: Methodological challenges associated with patient responses to follow-up longitudinal surveys regarding quality of care.
abstract: OBJECTIVE
  To illustrate, using empirical data, methodological challenges associated with patient responses to longitudinal surveys regarding the quality of process of care and health status, including overall response rate, differential response rate, and stability of responses with time.
  
  
  METHODS
  Primary patient self-report data were collected from 30,308 patients in 1996 and 13,438 patients in 1998 as part of a two-year longitudinal study of quality of care and health status of patients receiving care delivered by 63 physician organizations (physician groups) across three West Coast states.
  
  
  METHODS
  We analyzed longitudinal, observational data collected by Pacific Business Group on Health (PBGH) from patients aged 18-70 using a four-page survey in 1996 and a similar survey in 1998 to assess health status, satisfaction, use of services, and self-reported process of care. A subset of patients with self-reported chronic disease in the 1996 study received an enriched survey in 1998 to more fully detail processes of care for patients with chronic disease.
  
  
  METHODS
  We measured response rate overall and separately for patients with chronic disease. Logistic regression was used to assess the impact of 1996 predictors on response to the follow-up 1998 survey. We compared process of care scores without and with nonresponse weights. Additionally, we measured stability of patient responses over time using percent agreement and kappa statistics, and examined rates of gender inconsistencies reported across the 1996 and 1998 surveys.
  
  
  RESULTS
  In 1998, response rates were 54 percent overall and 63 percent for patients with chronic disease. Patient demographics, health status, use of services, and satisfaction with care in 1996 were all significant predictors of response in 1998, highlighting the importance of analytic strategies (i.e., application of nonresponse weights) to minimize bias in estimates of care and outcomes associated with longitudinal quality of care and health outcome analyses. Process of care scores weighted for nonresponse differed from unweighted scores (p<.001). Stability of responses across time was moderate, but varied by survey item from fair to excellent.
  
  
  CONCLUSIONS
  Longitudinal analyses involving the collection of data from the same patients at two points in time provide opportunities for analysis of relationships between process and outcomes of care that cannot occur with cross-sectional data. We present empirical results documenting the scope of the problems and discuss options for responding to these challenges. With increasing emphasis in the United States on quality reporting and use of financial incentives for quality in the health care market, it is important to identify and address methodological challenges that potentially threaten the validity of quality-of-care assessments.
authors.affiliation: UCLA School of Medicine, Division of General Internal Medicine and Health Services Research, Los Angeles, CA 90095-1736, USA.
authors.forename: Katherine L
authors.initials: KL
authors.lastname: Kahn
authors.forename: Honghu
authors.initials: H
authors.lastname: Liu
authors.forename: John L
authors.initials: JL
authors.lastname: Adams
authors.forename: Wen-Pin
authors.initials: WP
authors.lastname: Chen
authors.forename: Diana M
authors.initials: DM
authors.lastname: Tisnado
authors.forename: David M
authors.initials: DM
authors.lastname: Carlisle
authors.forename: Ron D
authors.initials: RD
authors.lastname: Hays
authors.forename: Carol M
authors.initials: CM
authors.lastname: Mangione
authors.forename: Cheryl L
authors.initials: CL
authors.lastname: Damberg
mesh_terms: D000328:Adult; D000368:Aged; D002908:Chronic Disease / Q000628:therapy; D005260:Female; D005500:Follow-Up Studies; D019538:Health Care Surveys / Q000379:methods; D006306:Health Surveys; D006801:Humans; D008137:Longitudinal Studies; D008297:Male; D008875:Middle Aged; D010043:Outcome and Process Assessment, Health Care / Q000379:methods* / Q000706:statistics & numerical data; D019601:Pacific States / Q000453:epidemiology; D017060:Patient Satisfaction / Q000706:statistics & numerical data*; D011320:Primary Health Care / Q000592:standards*; D011795:Surveys and Questionnaires