title: The effect of multimedia replacing text in resident clinical decision-making assessment.
abstract: Multimedia in assessing clinical decision-making skills (CDMS) has been poorly studied, particularly in comparison to traditional text-based assessments. The literature suggests multimedia is more difficult for trainees. We hypothesize that pediatric residents score lower in diagnostic skill when clinical vignettes use multimedia rather than text for patient findings. A standardized method was developed to write text-based questions from 60 high-resolution, quality multimedia; a series of expert panels selected 40 questions with both a multimedia and text-based counterpart, and two online tests were developed. Each test featured 40 identical questions with reciprocal and alternating modality (multimedia vs. text). Pediatric residents and rising 4th year medical students (MS-IV) at a single residency were randomized to complete either test stratified by postgraduate training year (PGY). A mixed between-within subjects ANOVA analyzed differences in score due to modality and PGY. Secondary analyses ascertained modality effect in dermatology and respiratory questions using Mann-Whitney U tests, and correlations on test performance to In-service Training Exam (ITE) scores using Spearman rank. Eighty-eight residents and rising interns completed the study. Overall multimedia scores were lower than text-based scores (p = 0.047, ηp2= 0.04), with highest disparity in rising interns (MS-IV); however, PGY had a greater effect on scores (p = 0.001, ηp2= 0.16). Respiratory questions were not significantly lower with multimedia (n = 9, median 0.71 vs. 0.86, p = 0.09) nor dermatology questions (n = 13, p = 0.41). ITEs correlated significantly with text-based scores (ρ = 0.23-0.25, p = 0.04-0.06) but not with multimedia scores. In physician trainees with less clinical experience, multimedia-based case vignettes are associated with significantly lower scores. These results help shed light on the role of multimedia versus text-based information in CDMS, particularly in less experienced clinicians.
authors.affiliation: Division of Emergency Medicine and Transport, Children's Hospital Los Angeles and University of Southern California Keck School of Medicine, Los Angeles, CA, USA. dr.toddchang@gmail.com.
authors.forename: Todd P
authors.identifier: http://orcid.org/0000-0002-4508-2551
authors.initials: TP
authors.lastname: Chang
authors.affiliation: Division of Hospital Medicine, Children's Hospital Los Angeles and University of Southern California Keck School of Medicine, Los Angeles, CA, USA.
authors.forename: Sheree M
authors.initials: SM
authors.lastname: Schrager
authors.affiliation: Department of Critical Care and Anesthesiology, Children's Hospital Los Angeles and University of Southern California Keck School of Medicine, Los Angeles, CA, USA.
authors.forename: Alyssa J
authors.initials: AJ
authors.lastname: Rake
authors.affiliation: Division of Emergency Medicine, Ann and Robert H. Lurie Children's Hospital of Chicago and Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
authors.forename: Michael W
authors.initials: MW
authors.lastname: Chan
authors.affiliation: Division of Emergency Medicine and Transport, Children's Hospital Los Angeles and University of Southern California Keck School of Medicine, Los Angeles, CA, USA.|Division of Behavioral and Organizational Sciences, Claremont Graduate University, Claremont, CA, USA.
authors.forename: Phung K
authors.initials: PK
authors.lastname: Pham
authors.affiliation: Division of Hospital Medicine, Children's Hospital Los Angeles and University of Southern California Keck School of Medicine, Los Angeles, CA, USA.
authors.forename: Grant
authors.initials: G
authors.lastname: Christman
mesh_terms: D002983:Clinical Competence; D000066491:Clinical Decision-Making / Q000379:methods*; D003880:Dermatology / Q000193:education; D004521:Educational Measurement / Q000379:methods*; D006776:Hospitals, Pediatric; D006801:Humans; D007396:Internship and Residency / Q000379:methods*; D019212:Multimedia; D010372:Pediatrics / Q000193:education*; D012014:Reference Books, Medical
keywords: Assessment; Clinical decision making; E-learning; Graduate medical education; Multimedia; Pediatrics; Residency; Screen-based simulation