title: Reliability of surgeon-specific reporting of complications after colectomy.
abstract: OBJECTIVE
  We sought to determine the reliability of surgeon-specific postoperative complication rates after colectomy.
  
  
  BACKGROUND
  Conventional measures of surgeon-specific performance fail to acknowledge variation attributed to statistical noise, risking unreliable assessment of quality.
  
  
  METHODS
  We examined all patients who underwent segmental colectomy with anastomosis from 2008 through 2010 participating in the Michigan Surgical Quality Collaborative Colectomy Project. Surgeon-specific complication rates were risk-adjusted according to patient characteristics with multiple logistic regression. Hierarchical modeling techniques were used to determine the reliability of surgeon-specific risk-adjusted complication rates. We then adjusted these rates for reliability. To evaluate the extent to which surgeon-level variation was reduced, surgeons were placed into quartiles based on performance and complication rates were compared before and after reliability adjustment.
  
  
  RESULTS
  A total of 5033 patients (n = 345 surgeons) undergoing partial colectomy reported a risk-adjusted complication rate of 24.5%. Approximately 86% of the variability of complication rates across surgeons was explained by measurement noise, whereas the remaining 14% represented true signal. Risk-adjusted complication rates varied from 0% to 55.1% across quartiles before adjusting for reliability. Reliability adjustment greatly diminished this variation, generating a 1.2-fold difference (21.4%-25.6%). A caseload of 168 colectomies across 3 years was required to achieve a reliability of more than 0.7, which is considered a proficient level. Only 1 surgeon surpassed this volume threshold.
  
  
  CONCLUSIONS
  The vast majority of surgeons do not perform enough colectomies to generate a reliable surgeon-specific complication rate. Risk-adjusted complication rates should be viewed with caution when evaluating surgeons with low operative volume, as statistical noise is a large determinant in estimating their surgeon-specific complication rates.
authors.affiliation: *Department of Surgery, Academic Surgeon Development Program †Michigan Surgery Quality Collaborative, University of Michigan, Ann Arbor, MI.
authors.forename: Terry
authors.initials: T
authors.lastname: Shih
authors.forename: Adam I
authors.initials: AI
authors.lastname: Cole
authors.forename: Paul M
authors.initials: PM
authors.lastname: Al-Attar
authors.forename: Apurba
authors.initials: A
authors.lastname: Chakrabarti
authors.forename: Hussein A
authors.initials: HA
authors.lastname: Fardous
authors.forename: Peter F
authors.initials: PF
authors.lastname: Helvie
authors.forename: Michael T
authors.initials: MT
authors.lastname: Kemp
authors.forename: Chris
authors.initials: C
authors.lastname: Lee
authors.forename: Eytan
authors.initials: E
authors.lastname: Shtull-Leber
authors.forename: Darrell A
authors.initials: DA
authors.lastname: Campbell
authors.forename: Michael J
authors.initials: MJ
authors.lastname: Englesbe
mesh_terms: D000368:Aged; D000369:Aged, 80 and over; D002983:Clinical Competence; D003082:Colectomy / Q000706:statistics & numerical data*; D030881:Disclosure; D005260:Female; D006801:Humans; D008297:Male; D008824:Michigan / Q000453:epidemiology; D008875:Middle Aged; D017063:Outcome Assessment, Health Care; D011183:Postoperative Complications / Q000453:epidemiology*; D015203:Reproducibility of Results; D066231:Surgeons / Q000592:standards
references.citation: N Engl J Med. 2006 Feb 23;354(8):864-70
references.pmid: 16495401
references.citation: Health Serv Res. 2010 Dec;45(6 Pt 1):1614-29
references.pmid: 20722747
references.citation: Ann Surg. 2009 Sep;250(3):363-76
references.pmid: 19644350
references.citation: N Engl J Med. 2010 Mar 18;362(11):1014-21
references.pmid: 20237347
references.citation: Eval Health Prof. 2010 Sep;33(3):302-20
references.pmid: 20801974
references.citation: Ann Surg. 2010 Sep;252(3):514-9; discussion 519-20
references.pmid: 20739852
references.citation: J Vasc Surg. 2011 Jan;53(1):1-5
references.pmid: 21093202
references.citation: J Am Coll Surg. 2011 Aug;213(2):231-5
references.pmid: 21622011
references.citation: Ann Surg. 2012 Apr;255(4):703-7
references.pmid: 22388108
references.citation: J Am Coll Surg. 2013 Oct;217(4):636-647.e1
references.pmid: 23816383
references.citation: JAMA. 1991 Jun 12;265(22):3006-7
references.pmid: 2033775
references.citation: Am J Med Qual. 1994 Fall;9(3):104-15
references.pmid: 7950482
references.citation: N Engl J Med. 1995 May 4;332(18):1229-32
references.pmid: 7700321
references.citation: N Engl J Med. 1996 Jul 25;335(4):251-6
references.pmid: 8657242
references.citation: JAMA. 1999 Jun 9;281(22):2098-105
references.pmid: 10367820
references.citation: J Gen Intern Med. 2006 Jan;21(1):13-21
references.pmid: 16423118
references.citation: Am J Manag Care. 2008 Dec;14(12):833-8
references.pmid: 19067500