title: Is the evaluation of the personal statement a reliable component of the general surgery residency application?
abstract: BACKGROUND
  Each year, fourth-year medical students spend considerable time writing and rewriting their personal statements. However, there is little evidence of what role the personal statement plays in deciding which applicants will be invited for an interview.
  
  
  OBJECTIVE
  To evaluate the inter-rater reliability of a surgical selection committee's ratings for both the personal statement and the application summary parts of the residency application.
  
  
  METHODS
  We completed a retrospective analysis of the 2007-2008 Scott & White surgical residency application pool. From a total pool of 174 residency applications, we selected 8 (5%) applications randomly to be evaluated by 4 experienced members of the selection committee. The 4 committee members rated each personal statement on a 7-point scale, from "negative-would not invite for an interview" to "positive-will invite for an interview." They rated respective application summaries separately on a similar 7-point scale. Committee members also listed their top three reasons for assigning their scores.
  
  
  METHODS
  Rating scores for the personal statements and the applications were analyzed for inter-rater correlation. The qualitative data (ie, reasons for the scores) were reviewed to help the investigators profile the reasons given for very positive and very negative scores.
  
  
  RESULTS
  For the application summaries, the correlations between each pair of raters ranged from 0.79 to 0.94 with an overall average of 0.88. For the personal statements, inter-rater correlations ranged from -0.83 to 0.63 with an overall average of -0.09.
  
  
  CONCLUSIONS
  These results demonstrate that the personal statements lacked objective criteria for evaluation.
authors.affiliation: The Office of Faculty Development, Texas A&M Health Science Center, College of Medicine, Temple, Texas 76508, USA. bawhite@medicine.tamhsc.edu
authors.forename: Bobbie Ann Adair
authors.initials: BA
authors.lastname: White
authors.forename: Mark
authors.initials: M
authors.lastname: Sadoski
authors.forename: Scott
authors.initials: S
authors.lastname: Thomas
authors.forename: Mohsen
authors.initials: M
authors.lastname: Shabahang
mesh_terms: D000328:Adult; D004503:Education, Medical, Graduate / Q000379:methods; D005069:Evaluation Studies as Topic; D005260:Female; D013502:General Surgery / Q000193:education*; D006801:Humans; D007586:Job Application; D008297:Male; D010560:Personnel Selection; D012189:Retrospective Studies; D012570:School Admission Criteria; D012650:Self Disclosure; D013337:Students, Medical / Q000706:statistics & numerical data; D014481:United States; D014956:Writing