title: Caries Removal by First-Year Dental Students: A Multisource Competency Assessment Strategy for Reflective Practice.
abstract: Multisource assessment (MSA) uses multiple assessors to provide feedback. Little is known about the validity of using MSA feedback for improving students' ability to self-assess in a preclinical environment. Therefore, the aim of this study was to measure the validity of using a defined reflective process involving an MSA tool for building skill in dental students' self-evaluation of caries excavation on extracted teeth. As part of this process, 104 first-year students at one U.S. dental school used a self-generated study plan (SGSP) for structured reflection on MSA feedback during the 2013-14 academic year. Interrater agreement, determined through calculation of percentage-agreements in scoring, was measured among three assessor groups (self-, peer, and expert assessors) in formative assessment and between two assessor groups (self- and expert assessors) in summative assessment two weeks apart, allowing for reflective practice and completion of an SGSP between assessments. Validity for improving self-assessment was determined by measuring significance in positive shifts of agreement between self- and expert assessors. The results showed that interrater agreement between the self- and expert assessors increased significantly: from a 28% agreement in formative assessment to a 60% agreement in summative assessment. Significance in percentage shifts between assessments was demonstrated with a McNemar score of 0.26 (p<0.001). These results suggest that the described MSA tool and reflective process in an SGSP may be valid methods for improving skill in student self-evaluation of competence in caries excavation on extracted teeth.
authors.affiliation: Dr. de Peralta is Clinical Associate Professor, Department of Cariology, Restorative Sciences, Endodontics, and Director of Curriculum and Assessment Integration, School of Dentistry, University of Michigan; Dr. Ramaswamy is Associate Director of Curriculum and Program Evaluation, Office of Academic Affairs, School of Dentistry, University of Michigan; Dr. Karl is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. Van Tubergen is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. McLean is Clinical Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; and Dr. Fitzgerald is Associate Chair and Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan. tdeperal@umich.edu.
authors.forename: Tracy L
authors.initials: TL
authors.lastname: de Peralta
authors.affiliation: Dr. de Peralta is Clinical Associate Professor, Department of Cariology, Restorative Sciences, Endodontics, and Director of Curriculum and Assessment Integration, School of Dentistry, University of Michigan; Dr. Ramaswamy is Associate Director of Curriculum and Program Evaluation, Office of Academic Affairs, School of Dentistry, University of Michigan; Dr. Karl is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. Van Tubergen is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. McLean is Clinical Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; and Dr. Fitzgerald is Associate Chair and Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan.
authors.forename: Vidya
authors.initials: V
authors.lastname: Ramaswamy
authors.affiliation: Dr. de Peralta is Clinical Associate Professor, Department of Cariology, Restorative Sciences, Endodontics, and Director of Curriculum and Assessment Integration, School of Dentistry, University of Michigan; Dr. Ramaswamy is Associate Director of Curriculum and Program Evaluation, Office of Academic Affairs, School of Dentistry, University of Michigan; Dr. Karl is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. Van Tubergen is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. McLean is Clinical Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; and Dr. Fitzgerald is Associate Chair and Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan.
authors.forename: Elisabeta
authors.initials: E
authors.lastname: Karl
authors.affiliation: Dr. de Peralta is Clinical Associate Professor, Department of Cariology, Restorative Sciences, Endodontics, and Director of Curriculum and Assessment Integration, School of Dentistry, University of Michigan; Dr. Ramaswamy is Associate Director of Curriculum and Program Evaluation, Office of Academic Affairs, School of Dentistry, University of Michigan; Dr. Karl is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. Van Tubergen is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. McLean is Clinical Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; and Dr. Fitzgerald is Associate Chair and Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan.
authors.forename: Elizabeth
authors.initials: E
authors.lastname: Van Tubergen
authors.affiliation: Dr. de Peralta is Clinical Associate Professor, Department of Cariology, Restorative Sciences, Endodontics, and Director of Curriculum and Assessment Integration, School of Dentistry, University of Michigan; Dr. Ramaswamy is Associate Director of Curriculum and Program Evaluation, Office of Academic Affairs, School of Dentistry, University of Michigan; Dr. Karl is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. Van Tubergen is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. McLean is Clinical Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; and Dr. Fitzgerald is Associate Chair and Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan.
authors.forename: Mary Ellen
authors.initials: ME
authors.lastname: McLean
authors.affiliation: Dr. de Peralta is Clinical Associate Professor, Department of Cariology, Restorative Sciences, Endodontics, and Director of Curriculum and Assessment Integration, School of Dentistry, University of Michigan; Dr. Ramaswamy is Associate Director of Curriculum and Program Evaluation, Office of Academic Affairs, School of Dentistry, University of Michigan; Dr. Karl is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. Van Tubergen is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. McLean is Clinical Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; and Dr. Fitzgerald is Associate Chair and Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan.
authors.forename: Mark
authors.initials: M
authors.lastname: Fitzgerald
mesh_terms: D002983:Clinical Competence / Q000592:standards*; D003731:Dental Caries / Q000601:surgery*; D003737:Dental Cavity Preparation / Q000592:standards; D004497:Education, Dental / Q000592:standards*; D004521:Educational Measurement / Q000592:standards*; D006801:Humans; D015588:Observer Variation; D015203:Reproducibility of Results; D012647:Self-Assessment; D013335:Students, Dental
keywords: assessment; dental caries; dental education; operative dentistry; preclinical education; self-assessment