title: Constructing and evaluating a validity argument for the final-year ward simulation exercise.
abstract: The authors report final-year ward simulation data from the University of Dundee Medical School. Faculty who designed this assessment intend for the final score to represent an individual senior medical student's level of clinical performance. The results are included in each student's portfolio as one source of evidence of the student's capability as a practitioner, professional, and scholar. Our purpose in conducting this study was to illustrate how assessment designers who are creating assessments to evaluate clinical performance might develop propositions and then collect and examine various sources of evidence to construct and evaluate a validity argument. The data were from all 154 medical students who were in their final year of study at the University of Dundee Medical School in the 2010-2011 academic year. To the best of our knowledge, this is the first report on an analysis of senior medical students' clinical performance while they were taking responsibility for the management of a simulated ward. Using multi-facet Rasch measurement and a generalizability theory approach, we examined various sources of validity evidence that the medical school faculty have gathered for a set of six propositions needed to support their use of scores as measures of students' clinical ability. Based on our analysis of the evidence, we would conclude that, by and large, the propositions appear to be sound, and the evidence seems to support their proposed score interpretation. Given the body of evidence collected thus far, their intended interpretation seems defensible.
authors: Centre for Medical Education, School of Medicine, University of Dundee, Dundee, UK. hettietill@gmail.com.|, 11 Van Riebeeck Street, Franschhoek, 7690, South Africa. hettietill@gmail.com.
authors: Hettie
authors: H
authors: Till
authors: School of Medicine, University of Dundee, Dundee, UK.
authors: Jean
authors: J
authors: Ker
authors: Department of Educational Psychology, College of Education, University of Illinois at Chicago, Chicago, IL, USA.
authors: Carol
authors: C
authors: Myford
authors: Clinical Skills Centre, School of Medicine, University of Dundee, Dundee, UK.
authors: Kevin
authors: K
authors: Stirling
authors: School of Medicine, University of Dundee, Dundee, UK.
authors: Gary
authors: G
authors: Mires
mesh_terms: D002983:Clinical Competence; D003142:Communication; D004521:Educational Measurement / Q000379:methods* / Q000592:standards*; D005260:Female; D006801:Humans; D007400:Interprofessional Relations; D008297:Male; D061214:Patient Safety; D016544:Patient Simulation; D010817:Physician-Patient Relations; D000068616:Professionalism; D011897:Random Allocation; D015203:Reproducibility of Results
keywords: Clinical ability; Clinical performance; Evaluating medical students’ fitness to practice; Generalizability theory; Multi-facet Rasch measurement; Score interpretation; Simulated ward; Simulation in assessment of clinical performance; Validity argument; Validity evidence; Validity propositions