title: Validity and reliability of an in-training evaluation report to measure the CanMEDS roles in emergency medicine residents.
abstract: BACKGROUND
  There is a question of whether a single assessment tool can assess the key competencies of residents as mandated by the Royal College of Physicians and Surgeons of Canada CanMEDS roles framework.
  
  
  OBJECTIVE
  The objective of the present study was to investigate the reliability and validity of an emergency medicine (EM) in-training evaluation report (ITER).
  
  
  METHODS
  ITER data from 2009 to 2011 were combined for residents across the 5 years of the EM residency training program. An exploratory factor analysis with varimax rotation was used to explore the construct validity of the ITER. A total of 172 ITERs were completed on residents across their first to fifth year of training.
  
  
  RESULTS
  A combined, 24-item ITER yielded a five-factor solution measuring the CanMEDs role Medical Expert/Scholar, Communicator/Collaborator, Professional, Health Advocate and Manager subscales. The factor solution accounted for 79% of the variance, and reliability coefficients (Cronbach alpha) ranged from α  =  0.90 to 0.95 for each subscale and α  =  0.97 overall. The combined, 24-item ITER used to assess residents' competencies in the EM residency program showed strong reliability and evidence of construct validity for assessment of the CanMEDS roles.
  
  
  CONCLUSIONS
  Further research is needed to develop and test ITER items that will differentiate each CanMEDS role exclusively.
authors.forename: Aliya
authors.initials: A
authors.lastname: Kassam
authors.forename: Tyrone
authors.initials: T
authors.lastname: Donnon
authors.forename: Ian
authors.initials: I
authors.lastname: Rigby
mesh_terms: D000328:Adult; D002170:Canada; D002983:Clinical Competence; D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement; D004635:Emergency Medicine / Q000193:education*; D006801:Humans; D007396:Internship and Residency; D015203:Reproducibility of Results; D012189:Retrospective Studies