title: Embracing the Complexity of Valid Assessments of Clinicians' Performance: A Call for In-Depth Examination of Methodological and Statistical Contexts That Affect the Measurement of Change.
abstract: Assessments of clinicians' professional performance have become more entrenched in clinical practice globally. Systems and tools have been developed and implemented, and factors that impact performance in response to assessments have been studied. The validity and reliability of data yielded by assessment tools have been studied extensively. However, there are important methodological and statistical issues that can impact the assessment of performance and change that are often omitted or ignored by research and practice. In this article, the authors aim to address five of these issues and show how they can impact the validity of performance and change assessments, using empirical illustrations based on longitudinal data of clinicians' teaching performance. Specifically, the authors address the following: characteristics of a measurement scale that affect the performance data yielded by an assessment tool; different summary statistics of the same data that lead to opposing conclusions about performance and performance change; performance at the item level that does not easily translate to overall performance; how estimating performance change from two time-indexed measurements and assessing change retrospectively yield different results; and the context that affects performance and performance assessments. The authors explain how these issues affect the validity of performance assessments and offer suggestions for how to correct these issues.
authors.affiliation: B.C.M. Boerebach is employed by the Strategy and Information Department, University of Amsterdam. At the time this article was written, he was a researcher, Professional Performance Research Group, Center for Evidence-Based Education, Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands. O.A. Arah is professor of epidemiology and public health, Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles (UCLA), and faculty associate, UCLA Center for Health Policy Research, Los Angeles, California. M.J. Heineman is professor of obstetrics and gynecology and member, Board of Directors, Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands. K.M.J.M.H. Lombarts is professor of professional performance, Professional Performance Research Group, Center for Evidence-Based Education, Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands.
authors.forename: Benjamin C M
authors.initials: BC
authors.lastname: Boerebach
authors.forename: Onyebuchi A
authors.initials: OA
authors.lastname: Arah
authors.forename: Maas Jan
authors.initials: MJ
authors.lastname: Heineman
authors.forename: Kiki M J M H
authors.initials: KM
authors.lastname: Lombarts
mesh_terms: D002983:Clinical Competence / Q000592:standards / Q000706:statistics & numerical data*; D004503:Education, Medical, Graduate / Q000592:standards / Q000706:statistics & numerical data*; D004521:Educational Measurement; D006801:Humans; D015203:Reproducibility of Results