The retrospective pre-post: a practical method to evaluate learning from an educational program.
OBJECTIVE
Program evaluation remains a critical but underutilized step in medical education. This study compared traditional and retrospective pre-post self-assessment methods to objective learning measures to assess which correlated better to actual learning.


METHODS
Forty-seven medical students participated in a 4-hour pediatric resuscitation course. They completed pre and post self-assessments on pediatric resuscitation and two distracter topics. Postcourse, students also retrospectively rated their understanding as it was precourse (the "retrospective pre" instrument). Changes in traditional and retrospective pre- to postcourse self-assessment measures were compared to an objectives-based multiple-choice exam.


RESULTS
The traditional pre to post self-assessment means showed an increase from 1.9 of 5 to 3.7 of 5 (p < 0.001); the retrospective pre to post scores also increased from 1.9 of 5 to 3.7 of 5 (p < 0.001). Although the group means were the same, individual participants demonstrated a response shift by either increasing or decreasing their traditional pre to retrospective pre scores. Scores on the 22-item objective multiple choice test also increased, from a median score of 13.0 to 18.0 (p < 0.001). There was no correlation between the change in self-assessments and objective measures as demonstrated by a Spearman correlation of -0.02 and -0.13 for the traditional and retrospective pre-post methods, respectively. Students reported fewer changes on the two distracters using the retrospective pre-post versus the traditional method (11 vs. 29).


CONCLUSIONS
Students were able to accurately identify, but not quantify, learning using either traditional or retrospective pre-post "self-assessment" measures. Retrospective pre-post self-assessment was more accurate in excluding perceived change in understanding of subject matter that was not taught.
Montreal Children's Hospital, McGill University, Montreal, Canada.
Farhan
F
Bhanji
Ronald
R
Gottesman
Willem
W
de Grave
Yvonne
Y
Steinert
Laura R
LR
Winer
D002983:Clinical Competence; D004504:Education, Medical, Undergraduate / Q000379:methods*; D004521:Educational Measurement; D004635:Emergency Medicine / Q000193:education*; D005260:Female; D006801:Humans; D008297:Male; D010372:Pediatrics / Q000193:education*; D015397:Program Evaluation; D012151:Resuscitation / Q000193:education*; D018709:Statistics, Nonparametric