title: Evaluation of an automated procedure for scoring patient notes as part of a clinical skills examination.
abstract: OBJECTIVE
  The purpose of the present study was to examine the extent to which an automated scoring procedure that emulates expert ratings with latent semantic analysis could be used to score the written patient note component of the proposed clinical skills examination (CSE).
  
  
  METHODS
  Human ratings for four CSE cases collected in 2002 were compared to automated holistic scores and to regression-based scores based on automated holistic and component scores.
  
  
  CONCLUSIONS
  Regression-based scores account for approximately half of the variance in the human ratings and are more highly correlated with the ratings than the scores produced from the automated algorithm. Implications of this study and suggestions for follow-up research are discussed.
authors.affiliation: Measurement Consulting Services, NBME, Philadelphia, PA 19104, USA. kswygert@nbme.org
authors.forename: Kimberly
authors.initials: K
authors.lastname: Swygert
authors.forename: Melissa
authors.initials: M
authors.lastname: Margolis
authors.forename: Ann
authors.initials: A
authors.lastname: King
authors.forename: Time
authors.initials: T
authors.lastname: Siftar
authors.forename: Stephen
authors.initials: S
authors.lastname: Clyman
authors.forename: Richard
authors.initials: R
authors.lastname: Hawkins
authors.forename: Brian
authors.initials: B
authors.lastname: Clauser
mesh_terms: D000465:Algorithms; D002983:Clinical Competence; D004521:Educational Measurement / Q000706:statistics & numerical data*; D006801:Humans; D008007:Licensure, Medical; D016014:Linear Models; D008487:Medical History Taking; D010808:Physical Examination; D012984:Software; D014481:United States