title: A Pilot Study to Gauge Effectiveness of Standardized Patient Scenarios in Assessing General Surgery Milestones.
abstract: OBJECTIVE
  Some General Surgery Milestones can be difficult to assess in traditional clinical settings and especially difficult to assess in junior residents. The purpose of this pilot study was to METHODS: A total of 9 categorical interns participated in a comprehensive, 4-module, SP scenario designed to evaluate and manage right upper quadrant pain. SP checklist scores (SP%) were converted to Milestone-equivalent scores for direct comparison (SP-C). Milestone scores were analyzed from 3 different sources: SP, faculty (FAC), and CCC. Interns completed course evaluations at the end of each session. Spearman's rho was used to determine correlations. Wilcoxon signed rank tests were used to test for differences between scores from different sources.
  
  
  RESULTS
  Individual intern Milestone scores from the 3 sources (SP-C, FAC, and CCC) did not correlate. All 7 mean Milestone scores from SPs were significantly higher than from FAC and CCC. FAC and CCC scores were statistically equivalent except for Systems-Based Practice 1 (SBP1) and Patient Care 3 (PC3) where CCC scores were significantly higher than FAC. Mean SP% scores for PC1 were significantly lower than for PROF1, MK1, MK2, and ICS1 (p < 0.05). Interns felt the modules were moderately to very useful.
  
  
  CONCLUSIONS
  Developing an SP scenario for Milestones evaluation is feasible. SPs, faculty observers, and CCC each use different data to provide a unique source of Milestone assessment. SP scenarios may be ideally suited to assess specific resident strengths and weaknesses and provide individualized feedback, thus augmenting traditional evaluations. Additional SP scenarios, assessing a broader range of skills and Milestones, are advisable for more reliable estimates of resident performance.
authors.affiliation: Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts. Electronic address: dgee@mgh.harvard.edu.
authors.forename: Denise W
authors.initials: DW
authors.lastname: Gee
authors.affiliation: Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts.
authors.forename: Roy
authors.initials: R
authors.lastname: Phitayakorn
authors.affiliation: Laboratory of Computer Science, Massachusetts General Hospital, Boston, Massachusetts.
authors.forename: Avni
authors.initials: A
authors.lastname: Khatri
authors.affiliation: Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts.
authors.forename: Kathryn
authors.initials: K
authors.lastname: Butler
authors.affiliation: Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts.
authors.forename: John T
authors.initials: JT
authors.lastname: Mullen
authors.affiliation: Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts.
authors.forename: Emil R
authors.initials: ER
authors.lastname: Petrusa
mesh_terms: D015746:Abdominal Pain / Q000175:diagnosis / Q000601:surgery; D057189:Checklist; D002983:Clinical Competence / Q000592:standards*; D003162:Competency-Based Education / Q000592:standards*; D004503:Education, Medical, Graduate / Q000458:organization & administration*; D004521:Educational Measurement; D005260:Female; D013502:General Surgery / Q000193:education*; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D008297:Male; D010865:Pilot Projects; D015397:Program Evaluation; D000066908:Simulation Training / Q000592:standards*; D014481:United States
keywords: Interpersonal and Communication Skills; Medical Knowledge; Patient Care; Professionalism; Systems-Based Practice; assessment; milestones; patient satisfaction; resident education; standardized patient