title: Using "standardized narratives" to explore new ways to represent faculty opinions of resident performance.
abstract: OBJECTIVE
  Most efforts to develop reliable evaluations of clinical competence have been oriented toward deconstructing the requisite competencies into separate scales. However, many are questioning the value of this approach on theoretical and empirical bases. This study uses "standardized narratives" to explore a different approach to assessing resident performance.
  
  
  METHODS
  In 2009, based on interviews with 19 experienced clinical faculty from two institutions, 16 narrative profiles were created to represent the range of resident competence that clinical faculty might encounter during supervision. Fourteen clinicians from three institutions independently grouped the profiles into as many categories as necessary to reflect various levels of performance, described their categories, then ranked the individual profiles within each category. Then, in groups of three or four, participants negotiated a final ranking and grouping of the 16 profiles.
  
  
  RESULTS
  Despite interesting idiosyncracies in the factors some participants identified as guiding their rankings, there was strong consistency across the 14 clinicians regarding the rankings (single-rater intraclass correlation [ICC] = 0.86) and groupings (single-rater ICC = 0.81) of the profiles. Similarly, across institutions, the four groups were highly consistent in their final negotiated rankings (single-group ICC = 0.91) and groupings (single-group ICC = 0.87) of the profiles.
  
  
  CONCLUSIONS
  Faculty showed more consistency in their decisions of what constitutes excellent, competent, and problematic performance in residents than implied by current assessment techniques that require deconstruction of resident competencies. This use of standardized narratives points to interesting opportunities for more authentically codifying faculty opinions of residents.
authors.affiliation: Department of Surgery, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada.
authors.forename: Glenn
authors.initials: G
authors.lastname: Regehr
authors.forename: Shiphra
authors.initials: S
authors.lastname: Ginsburg
authors.forename: Jodi
authors.initials: J
authors.lastname: Herold
authors.forename: Rose
authors.initials: R
authors.lastname: Hatala
authors.forename: Kevin
authors.initials: K
authors.lastname: Eva
authors.forename: Olga
authors.initials: O
authors.lastname: Oulanova
mesh_terms: D001291:Attitude of Health Personnel; D002170:Canada; D002983:Clinical Competence / Q000592:standards*; D005180:Faculty, Medical; D006801:Humans; D007396:Internship and Residency / Q000592:standards*; D033262:Narration; D015588:Observer Variation