title: A Big Data and Learning Analytics Approach to Process-Level Feedback in Cognitive Simulations.
abstract: Collecting and analyzing large amounts of process data for the purposes of education can be considered a big data/learning analytics (BD/LA) approach to improving learning. However, in the education of health care professionals, the application of BD/LA is limited to date. The authors discuss the potential advantages of the BD/LA approach for the process of learning via cognitive simulations. Using the lens of a cognitive model of radiograph interpretation with four phases (orientation, searching/scanning, feature detection, and decision making), they reanalyzed process data from a cognitive simulation of pediatric ankle radiography where 46 practitioners from three expertise levels classified 234 cases online. To illustrate the big data component, they highlight the data available in a digital environment (time-stamped, click-level process data). Learning analytics were illustrated using algorithmic computer-enabled approaches to process-level feedback.For each phase, the authors were able to identify examples of potentially useful BD/LA measures. For orientation, the trackable behavior of re-reviewing the clinical history was associated with increased diagnostic accuracy. For searching/scanning, evidence of skipping views was associated with an increased false-negative rate. For feature detection, heat maps overlaid on the radiograph can provide a metacognitive visualization of common novice errors. For decision making, the measured influence of sequence effects can reflect susceptibility to bias, whereas computer-generated path maps can provide insights into learners' diagnostic strategies.In conclusion, the augmented collection and dynamic analysis of learning process data within a cognitive simulation can improve feedback and prompt more precise reflection on a novice clinician's skill development.
authors.affiliation: M. Pecaric is lead consultant, Contrail Consulting Services, Toronto, Ontario, Canada. K. Boutis is associate professor and pediatric emergency physician, Department of Pediatrics, Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada. J. Beckstead is associate professor of nursing, University of South Florida College of Nursing, Tampa, Florida. M. Pusic is assistant professor of emergency medicine and director, Division of Learning Analytics, New York University School of Medicine, New York, New York.
authors.forename: Martin
authors.initials: M
authors.lastname: Pecaric
authors.forename: Kathy
authors.initials: K
authors.lastname: Boutis
authors.forename: Jason
authors.initials: J
authors.lastname: Beckstead
authors.forename: Martin
authors.initials: M
authors.lastname: Pusic
mesh_terms: D000842:Ankle / Q000000981:diagnostic imaging*; D035843:Biomedical Research / Q000379:methods*; D002648:Child; D002675:Child, Preschool; D003071:Cognition; D003625:Data Collection / Q000379:methods*; D005260:Female; D006801:Humans; D007223:Infant; D007858:Learning; D008297:Male; D011859:Radiography