Assessing change in clinical teaching skills: are we up for the challenge?
BACKGROUND
The faculty development community has been challenged to more rigorously assess program impact and move beyond traditional outcomes of knowledge tests and self ratings.


OBJECTIVE
The purpose was to (a) assess our ability to measure supervisors' feedback skills as demonstrated in a clinical setting and (b) compare the results with traditional outcome measures of faculty development interventions.


METHODS
A pre-post study design was used. Resident and expert ratings of supervisors' demonstrated feedback skills were compared with traditional outcomes, including a knowledge test and participant self-evaluation.


RESULTS
Pre-post knowledge increased significantly (pre = 61%, post = 85%; p < .001) as did participant's self-evaluation scores (pre = 4.13, post = 4.79; p < .001). Participants' self-evaluations were moderately to poorly correlated with resident (pre r = .20, post r = .08) and expert ratings (pre r = .43, post r = -.52). Residents and experts would need to evaluate 110 and 200 participants, respectively, to reach significance.


CONCLUSIONS
It is possible to measure feedback skills in a clinical setting. Although traditional outcome measures show a significant effect, demonstrating change in teaching behaviors used in practice will require larger scale studies than typically undertaken currently.
Marks
Meridith B
MB
Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. meridith.marks@uottawa.ca
Wood
Timothy J
TJ
Nuth
Janet
J
Touchie
Claire
C
O'Brien
Heather
H
Dugan
Alison
A
D004503:Education, Medical, Graduate / Q000592:standards*; D004521:Educational Measurement; D005246:Feedback; D006801:Humans; D015397:Program Evaluation; D012107:Research Design; D013192:Staff Development; D013663:Teaching / Q000592:standards*