title: A comparison of alternative item weighting strategies on the data gathering component of a clinical skills performance assessment.
abstract: BACKGROUND
  Checklist scores used to produce the data gathering score on the Step 2 CS examination are currently weighted using an algorithm based on expert judgment about the importance of the item. The present research was designed to compare this approach with alternative weighting strategies.
  
  
  METHODS
  Scores from 21,140 examinees who took the United States Medical Licensing Examination Step 2 between May 2006 and February 2007 were subjected to five weighting models: (1) a regression weights model, (2) a factor loading weights model, (3) a standardized response model, (4) an equal weights model, and (5) the operational expert-judgment weights model.
  
  
  RESULTS
  Alternative weighting procedures may have a significant impact on the reliability and validity of checklist scores.
  
  
  CONCLUSIONS
  The results suggest that the current weighting procedure is useful, and the regression-based model holds promise for practical application. The regression-based model produces scores that are more reliable than those produced by the current procedure and more strongly related to the external criteria.
authors.affiliation: nkahraman@nbme.org
authors.forename: Nilufer
authors.initials: N
authors.lastname: Kahraman
authors.forename: Brian E
authors.initials: BE
authors.lastname: Clauser
authors.forename: Melissa J
authors.initials: MJ
authors.lastname: Margolis
mesh_terms: D000465:Algorithms; D002983:Clinical Competence / Q000706:statistics & numerical data*; D015331:Cohort Studies; D005163:Factor Analysis, Statistical; D006801:Humans; D007600:Judgment; D008007:Licensure, Medical; D015233:Models, Statistical; D011594:Psychometrics; D015203:Reproducibility of Results; D012189:Retrospective Studies; D014481:United States