title: Evaluating health worker performance in Benin using the simulated client method with real children.
abstract: BACKGROUND
  The simulated client (SC) method for evaluating health worker performance utilizes surveyors who pose as patients to make surreptitious observations during consultations. Compared to conspicuous observation (CO) by surveyors, which is commonly done in developing countries, SC data better reflect usual health worker practices. This information is important because CO can cause performance to be better than usual. Despite this advantage of SCs, the method's full potential has not been realized for evaluating performance for pediatric illnesses because real children have not been utilized as SCs. Previous SC studies used scenarios of ill children that were not actually brought to health workers. During a trial that evaluated a quality improvement intervention in Benin (the Integrated Management of Childhood Illness [IMCI] strategy), we conducted an SC survey with adult caretakers as surveyors and real children to evaluate the feasibility of this approach and used the results to assess the validity of CO.
  
  
  METHODS
  We conducted an SC survey and a CO survey (one right after the other) of health workers in the same 55 health facilities. A detailed description of the SC survey process was produced. Results of the two surveys were compared for 27 performance indicators using logistic regression modeling.
  
  
  RESULTS
  SC and CO surveyors observed 54 and 185 consultations, respectively. No serious problems occurred during the SC survey. Performance levels measured by CO were moderately higher than those measured by SCs (median CO - SC difference = 16.4 percentage-points). Survey differences were sometimes much greater for IMCI-trained health workers (median difference = 29.7 percentage-points) than for workers without IMCI training (median difference = 3.1 percentage-points).
  
  
  CONCLUSIONS
  SC surveys can be done safely with real children if appropriate precautions are taken. CO can introduce moderately large positive biases, and these biases might be greater for health workers exposed to quality improvement interventions.
  
  
  BACKGROUND
  http://clinicaltrials.gov Identifier NCT00510679.
authors.affiliation: Malaria Branch, Division of Parasitic Diseases and Malaria, Center for Global Health, Centers for Disease Control and Prevention, Mailstop A06, 1600 Clifton Road NE, Atlanta, GA 30333, USA. axr9@cdc.gov
authors.forename: Alexander K
authors.initials: AK
authors.lastname: Rowe
authors.forename: Faustin
authors.initials: F
authors.lastname: Onikpo
authors.forename: Marcel
authors.initials: M
authors.lastname: Lama
authors.forename: Michael S
authors.initials: MS
authors.lastname: Deming
mesh_terms: D000328:Adult; D001541:Benin; D002675:Child, Preschool; D002983:Clinical Competence; D003430:Cross-Sectional Studies; D004650:Employee Performance Appraisal / Q000379:methods*; D005260:Female; D006282:Health Personnel / Q000592:standards*; D006801:Humans; D007223:Infant; D008297:Male; D016544:Patient Simulation; D011787:Quality of Health Care / Q000592:standards