title: Do Standardized Patients Have Concerns About Students Not Captured by Traditional Assessment Forms?
abstract: Construct: Traditionally, standardized patients (SPs) assess students' clinical skills principally through numerical rating forms-an approach that may not fully capture SPs' concerns. SPs are students' closest approximation to real patients. To maximally benefit students' clinical training and evaluation it is important to find ways to give voice to the totality of SPs' perspectives.
  
  
  BACKGROUND
  SPs have been shown to be a reliable and valid means to assess medical students' clinical skills in clinical skills examinations. We noticed, however, that SPs often express "off the record" concerns about students, which they do not include on traditional assessment forms.
  
  
  METHODS
  To explore these "off the record" concerns, we designed a Concerns item and added it to the traditional assessment form for an end-of-3rd-year clinical skills examination shared by three medical schools. We asked SPs to use this Concerns item to identify students about whom they had any "gut-level" concerns and provided them with a narrative opportunity to explain why. SPs were informed that the purpose of the item was to help students with difficulties and was not part of the student's grade.
  
  
  RESULTS
  We analyzed the concerns data using quantitative and qualitative methods. Of 551 students at three schools, 223 (∼40%) had concerns recorded. Seventy students received two or more concerns. Qualitative analysis of SPs' comments revealed 3 major categories of concern: communication and interpersonal skills, history taking, and physical exam. Grouped under each were several subcategories. More than half of the written comments from the SPs related to the communication/interpersonal skills category and included subcategories commonly addressed in communications courses: lack of empathy, good listening skills, and lack of connection to the patient. They also included subcategories that in our experience are less commonly addressed: odd or off-putting mannerisms, lack of confidence, unprofessional behavior, domineering behavior, and biased behavior. Another 47% of concerns identified deficiencies in history taking and physical examination. Of the students with concerns noted by two or more SPs, SPs' narrative comments on 84%, 42%, and 48% of the students in the domains of communications, history, and physical exam respectively indicated potential problems not identified by scores on the traditional assessment form.
  
  
  CONCLUSIONS
  The Concerns item is a narrative assessment method that may add value to traditional quantitative scoring by identifying and characterizing problematic student performance not captured by the traditional assessment form. It may thus contribute to giving fuller voice to the totality of SPs' perspective.
authors.affiliation: a Department of Medicine and The Clinical Learning & Simulation Skills (CLASS) Center, George Washington School of Medicine and Health Sciences , Washington , DC , USA.
authors.forename: Benjamin
authors.initials: B
authors.lastname: Blatt
authors.affiliation: b Physical Therapy and Health Care Sciences, George Washington School of Medicine and Health Sciences , Washington , DC , USA.
authors.forename: Margaret
authors.initials: M
authors.lastname: Plack
authors.affiliation: c Department of Epidemiology and Biostatistics , Milken Institute School of Public Health, George Washington University , Washington , DC , USA.
authors.forename: Samuel
authors.initials: S
authors.lastname: Simmens
authors.affiliation: d Department of Pediatrics and Val G. Hemming Simulation Center, Uniformed Services University of the Health Sciences , Bethesda , Maryland , USA.
authors.forename: Joseph
authors.initials: J
authors.lastname: Lopreiato
authors.affiliation: e Department of Medicine , The Sidney Kimmel Medical College, Jefferson University , Philadelphia , Pennsylvania , USA.
authors.forename: Katherine
authors.initials: K
authors.lastname: Berg
authors.affiliation: f Dr. Robert and Dorothy Rector Clinical Skills and Simulation Center, The Sidney Kimmel Medical College, Jefferson University , Philadelphia , Pennsylvania , USA.
authors.forename: Jacqueline
authors.initials: J
authors.lastname: Klevan
authors.affiliation: g The Clinical Learning & Simulation Skills (CLASS) Center, George Washington School of Medicine and Health Sciences , Washington , DC , USA.
authors.forename: Karen
authors.initials: K
authors.lastname: Lewis
mesh_terms: D002983:Clinical Competence; D003142:Communication; D004645:Empathy; D006801:Humans; D016544:Patient Simulation; D010817:Physician-Patient Relations; D013337:Students, Medical
keywords: Doctor–patient communication; Evaluation of medical students; Standardized patients