Multi-rater feedback with gap analysis: an innovative means to assess communication skill and self-insight.
OBJECTIVE
Multi-rater assessment with gap analysis is a powerful method for assessing communication skills and self-insight, and enhancing self-reflection. We demonstrate the use of this methodology.


METHODS
The Program for the Approach to Complex Encounters (PACE) is an interdisciplinary simulation-based communication skills program. Encounters are assessed using an expanded Kalamazoo Consensus Statement Essential Elements Checklist adapted for multi-rater feedback and gap analysis. Data from a representative conversation were analyzed.


RESULTS
Likert and forced-choice data with gap analysis are used to assess performance. Participants were strong in Demonstrating Empathy and Providing Closure, and needed to improve Relationship Building, Gathering Information, and understanding the Patient's/Family's Perspective. Participants under-appraised their abilities in Relationship Building, Providing Closure, and Demonstrating Empathy, as well as their overall performance. The conversion of these results into verbal feedback is discussed.


CONCLUSIONS
We describe an evaluation methodology using multi-rater assessment with gap analysis to assess communication skills and self-insight. This methodology enables faculty to identify undervalued skills and perceptual blind spots, provide comprehensive, data driven, feedback, and encourage reflection.


CONCLUSIONS
Implementation of graphical feedback forms coupled with one-on-one discussion using the above methodology has the potential to enhance trainee self-awareness and reflection, improving the impact of educational programs.
Calhoun
Aaron W
AW
Division of Critical Care Medicine, University of Louisville, Louisville, KY 40202, USA. aaron.calhoun@lousville.edu
Rider
Elizabeth A
EA
Peterson
Eleanor
E
Meyer
Elaine C
EC
D002983:Clinical Competence; D003142:Communication; D004521:Educational Measurement / Q000379:methods; D005246:Feedback; D005260:Female; D006801:Humans; D015278:Intensive Care Units, Pediatric; D007396:Internship and Residency / Q000379:methods; D007629:Kentucky; D008297:Male; D015588:Observer Variation; D010348:Patient Care Team; D016544:Patient Simulation; D011368:Professional-Family Relations; D015397:Program Evaluation