title: A comparative evaluation of risk-adjustment models for benchmarking amputation-free survival after lower extremity bypass.
abstract: BACKGROUND
  Providing patients and payers with publicly reported risk-adjusted quality metrics for the purpose of benchmarking physicians and institutions has become a national priority. Several prediction models have been developed to estimate outcomes after lower extremity revascularization for critical limb ischemia, but the optimal model to use in contemporary practice has not been defined. We sought to identify the highest-performing risk-adjustment model for amputation-free survival (AFS) at 1 year after lower extremity bypass (LEB).
  
  
  METHODS
  We used the national Society for Vascular Surgery Vascular Quality Initiative (VQI) database (2003-2012) to assess the performance of three previously validated risk-adjustment models for AFS. The Bypass versus Angioplasty in Severe Ischaemia of the Leg (BASIL), Finland National Vascular (FINNVASC) registry, and the modified Project of Ex-vivo vein graft Engineering via Transfection III (PREVENT III [mPIII]) risk scores were applied to the VQI cohort. A novel model for 1-year AFS was also derived using the VQI data set and externally validated using the PIII data set. The relative discrimination (Harrell c-index) and calibration (Hosmer-May goodness-of-fit test) of each model were compared.
  
  
  RESULTS
  Among 7754 patients in the VQI who underwent LEB for critical limb ischemia, the AFS was 74% at 1 year. Each of the previously published models for AFS demonstrated similar discriminative performance: c-indices for BASIL, FINNVASC, mPIII were 0.66, 0.60, and 0.64, respectively. The novel VQI-derived model had improved discriminative ability with a c-index of 0.71 and appropriate generalizability on external validation with a c-index of 0.68. The model was well calibrated in both the VQI and PIII data sets (goodness of fit P = not significant).
  
  
  CONCLUSIONS
  Currently available prediction models for AFS after LEB perform modestly when applied to national contemporary VQI data. Moreover, the performance of each model was inferior to that of the novel VQI-derived model. Because the importance of risk-adjusted outcome reporting continues to increase, national registries such as VQI should begin using this novel model for benchmarking quality of care.
authors.affiliation: Division of Vascular and Endovascular Surgery, University of Massachusetts Medical School, Worcester, Mass.
authors.forename: Jessica P
authors.initials: JP
authors.lastname: Simons
authors.affiliation: Section of Vascular Surgery, Dartmouth-Hitchcock Medical Center, Lebanon, NH.
authors.forename: Philip P
authors.initials: PP
authors.lastname: Goodney
authors.affiliation: Division of Vascular and Endovascular Surgery, University of Massachusetts Medical School, Worcester, Mass.
authors.forename: Julie
authors.initials: J
authors.lastname: Flahive
authors.affiliation: Division of Vascular Surgery, Feinberg School of Medicine, Northwestern University, Chicago, Ill.
authors.forename: Andrew W
authors.initials: AW
authors.lastname: Hoel
authors.affiliation: Division of Vascular Surgery, Roper St Francis, Charleston, SC.
authors.forename: John W
authors.initials: JW
authors.lastname: Hallett
authors.affiliation: Division of Vascular Surgery, University of Utah Health Sciences Center, Salt Lake City, Utah.
authors.forename: Larry W
authors.initials: LW
authors.lastname: Kraiss
authors.affiliation: Division of Vascular and Endovascular Surgery, University of Massachusetts Medical School, Worcester, Mass. Electronic address: andres.schanzer@umassmemorial.org.
authors.forename: Andres
authors.initials: A
authors.lastname: Schanzer
mesh_terms: D000368:Aged; D000369:Aged, 80 and over; D000671:Amputation, Surgical; D019985:Benchmarking / Q000592:standards; D003661:Decision Support Techniques; D018572:Disease-Free Survival; D005260:Female; D006801:Humans; D035002:Lower Extremity / Q000098:blood supply*; D008297:Male; D008875:Middle Aged; D018579:Patient Selection; D058729:Peripheral Arterial Disease / Q000175:diagnosis / Q000401:mortality / Q000503:physiopathology / Q000601:surgery*; D011183:Postoperative Complications / Q000175:diagnosis / Q000401:mortality / Q000601:surgery*; D011237:Predictive Value of Tests; D012086:Reoperation; D015203:Reproducibility of Results; D012189:Retrospective Studies; D018570:Risk Assessment; D012307:Risk Factors; D013997:Time Factors; D016896:Treatment Outcome; D058017:Vascular Grafting / Q000009:adverse effects / Q000401:mortality / Q000592:standards