title: Evaluation of Flagging Criteria of United States Kidney Transplant Center Performance: How to Best Define Outliers?
abstract: Scientific Registry of Transplant Recipients report cards of US organ transplant center performance are publicly available and used for quality oversight. Low center performance (LP) evaluations are associated with changes in practice including reduced transplant rates and increased waitlist removals. In 2014, Scientific Registry of Transplant Recipients implemented new Bayesian methodology to evaluate performance which was not adopted by Center for Medicare and Medicaid Services (CMS). In May 2016, CMS altered their performance criteria, reducing the likelihood of LP evaluations.
  
  
  
  Our aims were to evaluate incidence, survival rates, and volume of LP centers with Bayesian, historical (old-CMS) and new-CMS criteria using 6 consecutive program-specific reports (PSR), January 2013 to July 2015 among adult kidney transplant centers.
  
  
  
  Bayesian, old-CMS and new-CMS criteria identified 13.4%, 8.3%, and 6.1% LP PSRs, respectively. Over the 3-year period, 31.9% (Bayesian), 23.4% (old-CMS), and 19.8% (new-CMS) of centers had 1 or more LP evaluation. For small centers (<83 transplants/PSR), there were 4-fold additional LP evaluations (52 vs 13 PSRs) for 1-year mortality with Bayesian versus new-CMS criteria. For large centers (>183 transplants/PSR), there were 3-fold additional LP evaluations for 1-year mortality with Bayesian versus new-CMS criteria with median differences in observed and expected patient survival of -1.6% and -2.2%, respectively.
  
  
  
  A significant proportion of kidney transplant centers are identified as low performing with relatively small survival differences compared with expected. Bayesian criteria have significantly higher flagging rates and new-CMS criteria modestly reduce flagging. Critical appraisal of performance criteria is needed to assess whether quality oversight is meeting intended goals and whether further modifications could reduce risk aversion, more efficiently allocate resources, and increase transplant opportunities.
authors.affiliation: 1 Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH. 2 Cleveland Clinic Lerner College of Medicine, Case Western Reserve University, Cleveland, OH. 3 Digestive Disease Institute, Cleveland Clinic, Cleveland, OH. 4 Department of Surgery, Ohio State University, Columbus, OH. 5 Glickman Urological and Kidney Institute, Cleveland Clinic, Cleveland, OH. 6 Department of Surgery, University of Florida, Gainesville, FL.
authors.forename: Jesse D
authors.initials: JD
authors.lastname: Schold
authors.forename: Charles M
authors.initials: CM
authors.lastname: Miller
authors.forename: Mitchell L
authors.initials: ML
authors.lastname: Henry
authors.forename: Laura D
authors.initials: LD
authors.lastname: Buccini
authors.forename: Stuart M
authors.initials: SM
authors.lastname: Flechner
authors.forename: David A
authors.initials: DA
authors.lastname: Goldfarb
authors.forename: Emilio D
authors.initials: ED
authors.lastname: Poggio
authors.forename: Kenneth A
authors.initials: KA
authors.lastname: Andreoni
mesh_terms: D001499:Bayes Theorem; D014487:Centers for Medicare and Medicaid Services, U.S.; D061847:Hospitals, High-Volume / Q000592:standards* / Q000706:statistics & numerical data; D061846:Hospitals, Low-Volume / Q000592:standards* / Q000706:statistics & numerical data; D006801:Humans; D016030:Kidney Transplantation / Q000009:adverse effects / Q000401:mortality / Q000592:standards* / Q000706:statistics & numerical data; D015233:Models, Statistical; D017064:Process Assessment, Health Care / Q000592:standards* / Q000706:statistics & numerical data; D015397:Program Evaluation; D058996:Quality Improvement / Q000592:standards* / Q000706:statistics & numerical data; D019984:Quality Indicators, Health Care / Q000592:standards* / Q000706:statistics & numerical data; D013997:Time Factors; D016896:Treatment Outcome; D014481:United States; D014850:Waiting Lists