title: The construct and criterion validity of the multi-source feedback process to assess physician performance: a meta-analysis.
abstract: BACKGROUND
  The purpose of this study was to conduct a meta-analysis on the construct and criterion validity of multi-source feedback (MSF) to assess physicians and surgeons in practice.
  
  
  METHODS
  In this study, we followed the guidelines for the reporting of observational studies included in a meta-analysis. In addition to PubMed and MEDLINE databases, the CINAHL, EMBASE, and PsycINFO databases were searched from January 1975 to November 2012. All articles listed in the references of the MSF studies were reviewed to ensure that all relevant publications were identified. All 35 articles were independently coded by two authors (AA, TD), and any discrepancies (eg, effect size calculations) were reviewed by the other authors (KA, AD, CV).
  
  
  RESULTS
  Physician/surgeon performance measures from 35 studies were identified. A random-effects model of weighted mean effect size differences (d) resulted in: construct validity coefficients for the MSF system on physician/surgeon performance across different levels in practice ranged from d=0.14 (95% confidence interval [CI] 0.40-0.69) to d=1.78 (95% CI 1.20-2.30); construct validity coefficients for the MSF on physician/surgeon performance on two different occasions ranged from d=0.23 (95% CI 0.13-0.33) to d=0.90 (95% CI 0.74-1.10); concurrent validity coefficients for the MSF based on differences in assessor group ratings ranged from d=0.50 (95% CI 0.47-0.52) to d=0.57 (95% CI 0.55-0.60); and predictive validity coefficients for the MSF on physician/surgeon performance across different standardized measures ranged from d=1.28 (95% CI 1.16-1.41) to d=1.43 (95% CI 0.87-2.00).
  
  
  CONCLUSIONS
  The construct and criterion validity of the MSF system is supported by small to large effect size differences based on the MSF process and physician/surgeon performance across different clinical and nonclinical domain measures.
authors.affiliation: Department of General Surgery, Bahrain Defense Force Hospital, Riffa, Kingdom of Bahrain.
authors.forename: Ahmed
authors.initials: A
authors.lastname: Al Ansari
authors.affiliation: Medical Education and Research Unit, Department of Community Health Sciences, Faculty of Medicine, University of Calgary, AB, Canada.
authors.forename: Tyrone
authors.initials: T
authors.lastname: Donnon
authors.affiliation: Department of General Surgery, Bahrain Defense Force Hospital, Riffa, Kingdom of Bahrain.
authors.forename: Khalid
authors.initials: K
authors.lastname: Al Khalifa
authors.affiliation: Department of Pathology, Bahrain Defense Force Hospital, Riffa, Kingdom of Bahrain.
authors.forename: Abdulla
authors.initials: A
authors.lastname: Darwish
authors.affiliation: Department of Medical Education, Faculty of Medicine, University Ambrosiana, Milan, Italy.
authors.forename: Claudio
authors.initials: C
authors.lastname: Violato
keywords: clinical performance; construct validity; criterion validity; meta-analysis; multi-source feedback system