title: Measuring the quality of physician practice by using clinical vignettes: a prospective validation study.
abstract: BACKGROUND
  Worldwide efforts are under way to improve the quality of clinical practice. Most quality measurements, however, are poorly validated, expensive, and difficult to compare among sites.
  
  
  OBJECTIVE
  To validate whether vignettes accurately measure the quality of clinical practice by using a comparison with standardized patients (the gold standard method), and to determine whether vignettes are a more or less accurate method than medical record abstraction.
  
  
  METHODS
  Prospective, multisite study.
  
  
  METHODS
  Outpatient primary care clinics in 2 Veterans Affairs medical centers and 2 large, private medical centers.
  
  
  METHODS
  144 of 163 eligible physicians agreed to participate, and, of these, 116 were randomly selected to see standardized patients, to complete vignettes, or both.
  
  
  METHODS
  Scores, expressed as the percentage of explicit quality criteria correctly completed, were obtained by using 3 methods.
  
  
  RESULTS
  Among all physicians, the quality of clinical practice as measured by the standardized patients was 73% correct (95% CI, 72.1% to 73.4%). By using exactly the same criteria, physicians scored 68% (CI, 67.9% to 68.9%) when measured by the vignettes but only 63% (CI, 62.7% to 64.0%) when assessed by medical record abstraction. These findings were consistent across all diseases and were independent of case complexity or physician training level. Vignettes also accurately measured unnecessary care. Finally, vignettes seem to capture the range in the quality of clinical practice among physicians within a site.
  
  
  CONCLUSIONS
  Despite finding variation in the quality of clinical practice, we did not determine whether poorer quality translated into worse health status for patients. In addition, the quality scores are based on measurements from 1 patient-provider interaction. As with all other scoring criteria, vignette criteria must be regularly updated.
  
  
  CONCLUSIONS
  Vignettes are a valid tool for measuring the quality of clinical practice. They can be used for diverse clinical settings, diseases, physician types, and situations in which case-mix variation is a concern. They are inexpensive and easy to use. Vignettes are particularly useful for comparing quality among and within sites and may be useful for longitudinal evaluations of interventions intended to change clinical practice.
authors.affiliation: San Francisco Veterans Affairs Medical Center, Institute for Global Health, University of California, San Francisco 94105, USA. peabody@psg.ucsf.edu
authors.forename: John W
authors.initials: JW
authors.lastname: Peabody
authors.forename: Jeff
authors.initials: J
authors.lastname: Luck
authors.forename: Peter
authors.initials: P
authors.lastname: Glassman
authors.forename: Sharad
authors.initials: S
authors.lastname: Jain
authors.forename: Joyce
authors.initials: J
authors.lastname: Hansen
authors.forename: Maureen
authors.initials: M
authors.lastname: Spell
authors.forename: Martin
authors.initials: M
authors.lastname: Lee
mesh_terms: D000553:Ambulatory Care / Q000592:standards; D019985:Benchmarking / Q000379:methods*; D002983:Clinical Competence / Q000592:standards*; D003201:Computers; D006801:Humans; D016544:Patient Simulation; D011320:Primary Health Care / Q000592:standards; D011446:Prospective Studies; D011787:Quality of Health Care