title: Ranking Surgical Residency Programs: Reputation Survey or Outcomes Measures?
abstract: OBJECTIVE
  The release of general surgery residency program rankings by Doximity and U.S. News & World Report accentuates the need to define and establish measurable standards of program quality. This study evaluated the extent to which program rankings based solely on peer nominations correlated with familiar program outcomes measures.
  
  
  METHODS
  Publicly available data were collected for all 254 general surgery residency programs. To generate a rudimentary outcomes-based program ranking, surgery programs were rank-ordered according to an average percentile rank that was calculated using board pass rates and the prevalence of alumni publications. A Kendall τ-b rank correlation computed the linear association between program rankings based on reputation alone and those derived from outcomes measures to validate whether reputation was a reasonable surrogate for globally judging program quality.
  
  
  RESULTS
  For the 218 programs with complete data eligible for analysis, the mean board pass rate was 72% with a standard deviation of 14%. A total of 60 programs were placed in the 75th percentile or above for the number of publications authored by program alumni. The correlational analysis reported a significant correlation of 0.428, indicating only a moderate association between programs ranked by outcomes measures and those ranked according to reputation. Seventeen programs that were ranked in the top 30 according to reputation were also ranked in the top 30 based on outcomes measures.
  
  
  CONCLUSIONS
  This study suggests that reputation alone does not fully capture a representative snapshot of a program's quality. Rather, the use of multiple quantifiable indicators and attributes unique to programs ought to be given more consideration when assigning ranks to denote program quality. It is advised that the interpretation and subsequent use of program rankings be met with caution until further studies can rigorously demonstrate best practices for awarding program standings.
authors.affiliation: Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana. Electronic address: wilsoadb@iupui.edu.
authors.forename: Adam B
authors.initials: AB
authors.lastname: Wilson
authors.affiliation: Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana.
authors.forename: Laura J
authors.initials: LJ
authors.lastname: Torbeck
authors.affiliation: Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana.
authors.forename: Gary L
authors.initials: GL
authors.lastname: Dunnington
mesh_terms: D013502:General Surgery / Q000193:education*; D007396:Internship and Residency / Q000145:classification*; D011795:Surveys and Questionnaires; D014481:United States
keywords: outcomes; program quality; program rankings; reputation; surgery rankings