title: Cognitive and Psychomotor Entrustable Professional Activities: Can Simulators Help Assess Competency in Trainees?
abstract: BACKGROUND
  An entrustable professional activity describes a professional task that postgraduate residents must master during their training. The use of simulation to assess performance of entrustable professional activities requires further investigation.
  
  
  OBJECTIVE
  (1) Is simulation-based assessment of resident performance of entrustable professional activities reliable? (2) Is there evidence of important differences between Postgraduate Year (PGY)-1 and PGY-4 residents when performing simulated entrustable professional activities?
  
  
  METHODS
  Three entrustable professional activities were chosen from a list of competencies: management of the patient for total knee arthroplasty (TKA); management of the patient with an intertrochanteric hip fracture; and management of the patient with an ankle fracture. Each assessment of entrustable professional activity was 40 minutes long with three components: preoperative management of a patient (history-taking, examination, image interpretation); performance of a technical procedure on a sawbones model; and postoperative management of a patient (postoperative orders, management of complications). Residents were assessed by six faculty members who used checklists based on a modified Delphi technique, an overall global rating scale as well as a previously validated global rating scale for the technical procedure component of each activity. Nine PGY-1 and nine PGY-4 residents participated in our simulated assessment. We assessed reliability by calculating the internal consistency of the mean global rating for each activity as well as the interrater reliability between the faculty assessment and blinded review of videotaped encounters. We sought evidence of a difference in performance between PGY-1 and PGY-4 residents on the overall global rating scale for each station of each entrustable professional activity.
  
  
  RESULTS
  The reliability (Cronbach's α) for the hip fracture activity was 0.88, it was 0.89 for the ankle fracture activity, and it was 0.84 for the TKA activity. A strong correlation was seen between blinded observer video review and faculty scores (mean 0.87 [0.07], p < 0.001). For the hip fracture entrustable professional activity, the PGY-4 group had a higher mean global rating scale than the PGY-1 group for preoperative management (3.56 [0.5] versus 2.33 [0.5], p < 0.001), postoperative management (3.67 [0.5] versus 2.22 [0.7], p < 0.001), and technical procedures (3.11 [0.3] versus 3.67 [0.5], p = 0.015). For the TKA activity, the PGY-4 group scored higher for postoperative management (3.5 [0.8] versus 2.67 [0.5], p = 0.016) and technical procedures (3.22 [0.9] versus 2.22 [0.9], p = 0.04) than the PGY-1 group, but no difference for preoperative management with the numbers available (PGY-4, 3.44 [0.7] versus PGY-1 2.89 [0.8], p = 0.14). For the ankle fracture activity, the PGY-4 group scored higher for postoperative management (3.22 [0.8] versus 2.33 [0.7], p = 0.18) and technical procedures (3.22 [1.2] versus 2.0 [0.7], p = 0.018) than the PGY-1 groups, but no difference for preoperative management with the numbers available (PGY-4, 3.22 [0.8] versus PGY-1, 2.78 [0.7], p = 0.23).
  
  
  CONCLUSIONS
  The results of our study show that simulated assessment of entrustable professional activities may be used to determine the ability of a resident to perform professional tasks that are critical components of medical training. In this manner, educators can ensure that competent performance of these skills in the simulated setting occurs before actual practice with patients in the clinical setting.
authors.affiliation: Women's College and Mt Sinai Hospital, University of Toronto, 76 Grenville Street, Toronto, M5S 1B1, ON, Canada. tim.dwyer@wchospital.ca.
authors.forename: Tim
authors.initials: T
authors.lastname: Dwyer
authors.affiliation: Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada.
authors.forename: Veronica
authors.initials: V
authors.lastname: Wadey
authors.affiliation: University of Ottawa, Ottawa, ON, Canada.
authors.forename: Douglas
authors.initials: D
authors.lastname: Archibald
authors.affiliation: Toronto East General Hospital, University of Toronto, Toronto, ON, Canada.
authors.forename: William
authors.initials: W
authors.lastname: Kraemer
authors.affiliation: University of Toronto, Toronto, ON, Canada.
authors.forename: Jesse Slade
authors.initials: JS
authors.lastname: Shantz
authors.affiliation: University of Toronto, Toronto, ON, Canada.
authors.forename: John
authors.initials: J
authors.lastname: Townley
authors.affiliation: Women's College Hospital, University of Toronto, Toronto, ON, Canada.
authors.forename: Darrell
authors.initials: D
authors.lastname: Ogilvie-Harris
authors.affiliation: University of Toronto, Toronto, ON, Canada.
authors.forename: Massimo
authors.initials: M
authors.lastname: Petrera
authors.affiliation: Mt Sinai Hospital, University of Toronto, Toronto, ON, Canada.
authors.forename: Peter
authors.initials: P
authors.lastname: Ferguson
authors.affiliation: Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada.
authors.forename: Markku
authors.initials: M
authors.lastname: Nousiainen
mesh_terms: D064386:Ankle Fractures / Q000601:surgery; D019645:Arthroplasty, Replacement, Knee / Q000193:education; D057189:Checklist; D002983:Clinical Competence; D003071:Cognition; D003198:Computer Simulation; D003194:Computer-Assisted Instruction / Q000379:methods*; D003479:Curriculum; D004503:Education, Medical, Graduate / Q000379:methods*; D004522:Educational Status; D005592:Fracture Fixation / Q000193:education; D006620:Hip Fractures / Q000601:surgery; D006801:Humans; D008953:Models, Anatomic; D019637:Orthopedic Procedures / Q000193:education*; D015397:Program Evaluation; D011597:Psychomotor Performance; D015203:Reproducibility of Results; D013647:Task Performance and Analysis; D013663:Teaching / Q000379:methods*; D013997:Time Factors; D014741:Video Recording