title: Assessment of multiple physician competencies in postgraduate training: utility of the structured oral examination.
abstract: Competency based medical education involves assessing physicians-in-training in multiple roles. Training programs are challenged by the need to introduce appropriate yet feasible assessment methods. We therefore examined the utility of a structured oral examination (SOE) in the assessment of the 7 CanMEDS roles (Medical Expert, Communicator, Collaborator, Manager, Health Advocate, Scholar and Professional) in a Neonatal-Perinatal Medicine subspecialty training program. Between 2004 and 2008, 68 trainees participated in an annual SOE. Each SOE consisted of 4 1st year and 4 2nd year clinical scenarios with standardized questions written by neonatologists that addressed all 7 CanMEDS roles. Examiners assigned a checklist score and global rating scores for knowledge and organization. A satisfaction survey was completed at the end of 3 examinations. Across the 5 SOEs, questions related to each competency were tabulated and an overall checklist score for each competency calculated. Inter-station reliability was determined for each CanMEDS role. Values for Cronbach's alpha were 0.62 ± 0.2 (Medical Expert), 0.43 (Communicator), 0.40 ± 0.34 (Collaborator), 0.19 ± 0.17 (Manager), 0.46 ± 0.32 (Advocate), 0.67 ± 0.18 (Scholar) and 0.79 (Professional). Inter-rater reliability, determined for the first examination when there were 2 examiners per case, was moderate to substantial for 67% of cases. Participant satisfaction was high. Electronic communication facilitated exam development and costs were minimal. The SOE demonstrates psychometric properties suitable for a formative, in-training assessment as well as low cost, ease of administration and acceptability. It may be a useful way to assess physician competencies in training programs.
authors.affiliation: Department of Paediatrics, Mount Sinai Hospital, University of Toronto, ON, Canada. ajefferies@mtsinai.on.ca
authors.forename: Ann
authors.initials: A
authors.lastname: Jefferies
authors.forename: Brian
authors.initials: B
authors.lastname: Simmons
authors.forename: Eugene
authors.initials: E
authors.lastname: Ng
authors.forename: Martin
authors.initials: M
authors.lastname: Skidmore
mesh_terms: D000328:Adult; D002983:Clinical Competence; D003142:Communication; D003162:Competency-Based Education; D004503:Education, Medical, Graduate; D004521:Educational Measurement / Q000379:methods*; D005260:Female; D006801:Humans; D008297:Male; D009359:Neonatology / Q000193:education; D010500:Perinatology / Q000193:education; D010820:Physicians; D015203:Reproducibility of Results