title: How to assess communication, professionalism, collaboration and the other intrinsic CanMEDS roles in orthopedic residents: use of an objective structured clinical examination (OSCE).
abstract: BACKGROUND
  Assessing residents' understanding and application of the 6 intrinsic CanMEDS roles (communicator, professional, manager, collaborator, health advocate, scholar) is challenging for postgraduate medical educators. We hypothesized that an objective structured clinical examination (OSCE) designed to assess multiple intrinsic CanMEDS roles would be sufficiently reliable and valid.
  
  
  METHODS
  The OSCE comprised 6 10-minute stations, each testing 2 intrinsic roles using case-based scenarios (with or without the use of standardized patients). Residents were evaluated using 5-point scales and an overall performance rating at each station. Concurrent validity was sought by correlation with in-training evaluation reports (ITERs) from the last 12 months and an ordinal ranking created by program directors (PDs).
  
  
  RESULTS
  Twenty-five residents from postgraduate years (PGY) 0, 3 and 5 participated. The interstation reliability for total test scores (percent) was 0.87, while reliability for each of the communicator, collaborator, manager and professional roles was greater than 0.8. Total test scores, individual station scores and individual CanMEDS role scores all showed a significant effect by PGY level. Analysis of the PD rankings of intrinsic roles demonstrated a high correlation with the OSCE role scores. A correlation was seen between ITER and OSCE for the communicator role, while the ITER medical expert and total scores highly correlated with the communicator, manager and professional OSCE scores.
  
  
  CONCLUSIONS
  An OSCE designed to assess the intrinsic CanMEDS roles was sufficiently valid and reliable for regular use in an orthopedic residency program.
authors: The Women's College Hospital, Mt. Sinai Hospital, Toronto, Ont.
authors: Tim
authors: T
authors: Dwyer
authors: The Postgraduate Medicine Office, University of Toronto, Toronto, Ont.
authors: Susan
authors: S
authors: Glover Takahashi
authors: The Postgraduate Medicine Office, University of Toronto, Toronto, Ont.
authors: Melissa
authors: M
authors: Kennedy Hynes
authors: The University of Toronto, Toronto, Ont.
authors: Jodi
authors: J
authors: Herold
authors: The University of Toronto, Toronto, Ont.
authors: David
authors: D
authors: Wasserstein
authors: The Holland Orthopaedic and Arthritic Centre, Toronto, Ont.
authors: Markku
authors: M
authors: Nousiainen
authors: The Mt. Sinai Hospital, Toronto, Ont.
authors: Peter
authors: P
authors: Ferguson
authors: The Sunnybrook Health Sciences Centre, Toronto, Ont.
authors: Veronica
authors: V
authors: Wadey
authors: The Women's College Hospital, University of Toronto Orthopaedics Sports Medicine, and the Hospital for Sick Children, Toronto, Ont.
authors: M Lucas
authors: ML
authors: Murnaghan
authors: The University of Toronto, Toronto, Ont.
authors: Tim
authors: T
authors: Leroux
authors: The Women's College Hospital, Toronto, Ont.
authors: John
authors: J
authors: Semple
authors: The Toronto General Hospital, Toronto, Ont.
authors: Brian
authors: B
authors: Hodges
authors: The Women's College Hospital, The Hospital for Sick Children, and Toronto Western Hospital, Toronto, Ont.
authors: Darrell
authors: D
authors: Ogilvie-Harris
mesh_terms: D002170:Canada; D002983:Clinical Competence; D003142:Communication; D003299:Cooperative Behavior; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement / Q000379:methods*; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D009985:Orthopedics / Q000193:education* / Q000592:standards; D010819:Physician's Role; D015203:Reproducibility of Results