title: An Examination of American College of Surgeons NSQIP Surgical Risk Calculator Accuracy.
abstract: BACKGROUND
  The American College of Surgeons NSQIP offers a Surgical Risk Calculator (SRC) that provides detailed, patient-level, risk assessments for many adverse outcomes to surgeons, patients, and the general public. The SRC calculator was designed to help guide discussion and decisions by providing generally applicable (not hospital-specific) information about surgical risk using easily understood and broadly available preoperative variables. Although large, internal evaluations have shown that the SRC has good accuracy (model discrimination and calibration), external validations have been inconsistent and tend to favor a conclusion of inadequate performance.
  
  
  METHODS
  External studies, attempting to validate the SRC, were examined with respect to 3 design features: sample size (small samples reduce reliability), case-mix homogeneity (homogeneity reduces discrimination); and number of institutions providing data (few institutions reduces generalizability). The impact of each feature was then examined in several sets of simulation studies.
  
  
  RESULTS
  Each of the 3 design features has the potential to act as an artifactual cause for apparent SRC predictive failure. In addition, demonstrations that SRC estimates are inferior to those from models that use additional (sometimes operation-specific) predictor variables were seen as not relevant with respect to the SRC's intended scope.
  
  
  CONCLUSIONS
  The SRC predictive failures, reported by studies with the described design limitations, should not be misunderstood as disqualifying the SRC as an accurate and appropriate tool for its intended purpose of providing a general purpose risk calculator, applicable across many surgical domains, using easily understood and generally available predictive information.
authors.affiliation: Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL Department of Surgery, University of California Los Angeles, David Geffen School of Medicine, Los Angeles, CA VA Greater Los Angeles Healthcare System, Los Angeles, CA Department of Surgery, Washington University in St Louis, St Louis, MO Center for Health Policy and the Olin Business School, Washington University in St Louis, St Louis, MO John Cochran Veterans Affairs Medical Center, St Louis, MO BJC Healthcare, St Louis, MO.
authors.forename: Mark E
authors.initials: ME
authors.lastname: Cohen
authors.forename: Yaoming
authors.initials: Y
authors.lastname: Liu
authors.forename: Clifford Y
authors.initials: CY
authors.lastname: Ko
authors.forename: Bruce L
authors.initials: BL
authors.lastname: Hall
mesh_terms: D006801:Humans; D011183:Postoperative Complications / Q000453:epidemiology*; D058996:Quality Improvement; D015203:Reproducibility of Results; D018570:Risk Assessment; D014481:United States / Q000453:epidemiology