title: METACOGNITIVE SCAFFOLDS IMPROVE SELF-JUDGMENTS OF ACCURACY IN A MEDICAL INTELLIGENT TUTORING SYSTEM.
abstract: In this study, we examined the effect of two metacognitive scaffolds on the accuracy of confidence judgments made while diagnosing dermatopathology slides in SlideTutor. Thirty-one (N= 31) first- to fourth-year pathology and dermatology residents were randomly assigned to one of the two scaffolding conditions. The cases used in this study were selected from the domain of Nodular and Diffuse Dermatitides. Both groups worked with a version of SlideTutor that provided immediate feedback on their actions for two hours before proceeding to solve cases in either theConsidering AlternativesorPlaybackcondition. No immediate feedback was provided on actions performed by participants in the scaffolding mode. Measurements included learning gains (pre-test and post-test), as well as metacognitive performance, including Goodman-Kruskal Gamma correlation, bias, and discrimination. Results showed that participants in both conditions improved significantly in terms of their diagnostic scores from pre-test to post-test. More importantly, participants in theConsidering Alternativescondition outperformed those in thePlaybackcondition in the accuracy of their confidence judgments and the discrimination of the correctness of their assertions while solving cases. The results suggested that presenting participants with their diagnostic decision paths and highlighting correct and incorrect paths helps them to become more metacognitively accurate in their confidence judgments.
authors.affiliation: Department of Educational and Counselling Psychology, McGill University, Montréal, Québec.
authors.forename: Reza
authors.initials: R
authors.lastname: Feyzi-Behnagh
authors.affiliation: Department of Educational and Counselling Psychology, McGill University, Montréal, Québec.
authors.forename: Roger
authors.initials: R
authors.lastname: Azevedo
authors.affiliation: Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA.
authors.forename: Elizabeth
authors.initials: E
authors.lastname: Legowski
authors.affiliation: Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA.
authors.forename: Kayse
authors.initials: K
authors.lastname: Reitmeyer
authors.affiliation: Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA.
authors.forename: Eugene
authors.initials: E
authors.lastname: Tseytlin
authors.affiliation: Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA ; Department of Pathology, University of Pittsburgh, Pittsburgh, PA.
authors.forename: Rebecca S
authors.initials: RS
authors.lastname: Crowley
keywords: Cognitive debiasing; Computer based education; Diagnostic errors; Diagnostic reasoning; Intelligent tutoring systems; Metacognition; Metacognitive judgments; Pathology