title: Influence of the wording of evaluation items on outcome-based evaluation results for large-group teaching in anatomy, biochemistry and legal medicine.
abstract: Student learning outcome is an important dimension of teaching quality in undergraduate medical education. Measuring an increase in knowledge during teaching requires repetitive objective testing which is usually not feasible. As an alternative, student learning outcome can be calculated from student self-ratings. Comparative self-assessment (CSA) gain reflects the performance difference before and after teaching, adjusted for initial knowledge. It has been shown to be a valid proxy measure of actual learning outcome derived from objective tests. However, student self-ratings are prone to a number of confounding factors. In the context of outcome-based evaluation, the wording of self-rating items is crucial to the validity of evaluation results. This randomized trial assessed whether including qualifiers in these statements impacts on student ratings and CSA gain. First-year medical students self-rated their initial (then-test) and final (post-test) knowledge for lectures in anatomy, biochemistry and legal medicine, respectively, and 659 questionnaires were retrieved. Six-point scales were used for self-ratings with 1 being the most positive option. Qualifier use did not affect then-test ratings but was associated with slightly less favorable post-test ratings. Consecutively, mean CSA gain was smaller for items containing qualifiers than for items lacking qualifiers (50.6±15.0% vs. 56.3±14.6%, p=0.079). The effect was more pronounced (Cohen's d=0.82) for items related to anatomy. In order to increase fairness of outcome-based evaluation and increase the comparability of CSA gain data across subjects, medical educators should agree on a consistent approach (qualifiers for all items or no qualifiers at all) when drafting self-rating statements for outcome-based evaluation.
authors: Dept. of Legal Medicine, University Medical Center Hamburg-Eppendorf, Butenfeld 34, 22529 Hamburg, Germany. Electronic address: s.anders@uke.uni-hamburg.de.
authors: Sven
authors: S
authors: Anders
authors: Dept. of Legal Medicine, University Medical Center Hamburg-Eppendorf, Butenfeld 34, 22529 Hamburg, Germany.
authors: Katharina
authors: K
authors: Pyka
authors: Dept. of Legal Medicine, University Medical Center Hamburg-Eppendorf, Butenfeld 34, 22529 Hamburg, Germany.
authors: Tjark
authors: T
authors: Mueller
authors: Institute of Medical Psychology and Medical Sociology, Georg-August-University Göttingen, Waldweg 37, 37075 Göttingen, Germany.
authors: Nicole
authors: N
authors: von Streinbuechel
authors: Division of Medical Education Research and Curriculum Development, Göttingen University Medical Centre, Robert-Koch-Straße 40, 37075 Göttingen, Germany; Health Behaviour Research Centre, University College London, 1-19 Torrington Place, London WC1E 7HB, United Kingdom.
authors: Tobias
authors: T
authors: Raupach
mesh_terms: D000328:Adult; D000715:Anatomy / Q000193:education*; D001671:Biochemistry / Q000193:education*; D004504:Education, Medical, Undergraduate / Q000706:statistics & numerical data; D004521:Educational Measurement / Q000379:methods / Q000706:statistics & numerical data*; D005260:Female; D005554:Forensic Medicine / Q000193:education*; D005858:Germany; D006801:Humans; D008297:Male; D012660:Semantics; D013337:Students, Medical / Q000706:statistics & numerical data*; D009626:Terminology as Topic; D055815:Young Adult
keywords: Anatomy teaching; Evaluation; Medical education