title: Involving users in the refinement of the competency-based achievement system: an innovative approach to competency-based assessment.
abstract: BACKGROUND
  Competency-based assessment innovations are being implemented to address concerns about the effectiveness of traditional approaches to medical training and the assessment of competence.
  
  
  OBJECTIVE
  Integrating intended users' perspectives during the piloting and refinement process of an innovation is necessary to ensure the innovation meets users' needs. Failure to do so results in no opportunity for users to influence the innovation, nor for developers to assess why an innovation works or does not work in different contexts.
  
  
  METHODS
  A qualitative participatory action research approach was used. Sixteen first-year residents participated in three focus groups and two interviews during piloting. Verbatim transcripts were analyzed individually and then across all transcripts using a constant comparison approach.
  
  
  RESULTS
  The analysis revealed three key characteristics related to the impact on the residents' acceptance of the innovation as being a worthwhile investment of time and effort: access to frequent, timely, and specific feedback from preceptors. Findings were used to refine the innovation further.
  
  
  CONCLUSIONS
  This study highlights the necessary conditions for assessing the success of implementation of educational innovations. Reciprocal communication between users and developers is vital. This reflects the approaches recommended in the Ottawa Consensus Statement on research in assessment published in Medical Teacher in March 2011.
authors.affiliation: Department of Family Medicine, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Canada. sross@ualberta.ca
authors.forename: Shelley
authors.initials: S
authors.lastname: Ross
authors.forename: Cheryl-Anne
authors.initials: CA
authors.lastname: Poth
authors.forename: Michel G
authors.initials: MG
authors.lastname: Donoff
authors.forename: Chiara
authors.initials: C
authors.lastname: Papile
authors.forename: Paul
authors.initials: P
authors.lastname: Humphries
authors.forename: Samantha
authors.initials: S
authors.lastname: Stasiuk
authors.forename: Rebecca
authors.initials: R
authors.lastname: Georgis
mesh_terms: D003162:Competency-Based Education / Q000379:methods / Q000592:standards*; D004521:Educational Measurement / Q000379:methods; D017144:Focus Groups; D006801:Humans; D007396:Internship and Residency / Q000379:methods / Q000592:standards*; D007407:Interviews as Topic; D036301:Qualitative Research