title: Using standardised patients to measure physicians' practice: validation study using audio recordings.
abstract: OBJECTIVE
  To assess the validity of standardised patients to measure the quality of physicians' practice.
  
  
  METHODS
  Validation study of standardised patients' assessments. Physicians saw unannounced standardised patients presenting with common outpatient conditions. The standardised patients covertly tape recorded their visit and completed a checklist of quality criteria immediately afterwards. Their assessments were compared against independent assessments of the recordings by a trained medical records abstractor.
  
  
  METHODS
  Four general internal medicine primary care clinics in California.
  
  
  METHODS
  144 randomly selected consenting physicians.
  
  
  METHODS
  Rates of agreement between the patients' assessments and independent assessment.
  
  
  RESULTS
  40 visits, one per standardised patient, were recorded. The overall rate of agreement between the standardised patients' checklists and the independent assessment of the audio transcripts was 91% (kappa=0.81). Disaggregating the data by medical condition, site, level of physicians' training, and domain (stage of the consultation) gave similar rates of agreement. Sensitivity of the standardised patients' assessments was 95%, and specificity was 85%. The area under the receiver operator characteristic curve was 90%.
  
  
  CONCLUSIONS
  Standardised patients' assessments seem to be a valid measure of the quality of physicians' care for a variety of common medical conditions in actual outpatient settings. Properly trained standardised patients compare well with independent assessment of recordings of the consultations and may justify their use as a "gold standard" in comparing the quality of care across sites or evaluating data obtained from other sources, such as medical records and clinical vignettes.
mesh_terms: D002140:California; D002983:Clinical Competence; D006801:Humans; D017060:Patient Satisfaction; D016544:Patient Simulation; D012680:Sensitivity and Specificity; D013637:Tape Recording