title: Evaluation of an automated safety surveillance system using risk adjusted sequential probability ratio testing.
abstract: BACKGROUND
  Automated adverse outcome surveillance tools and methods have potential utility in quality improvement and medical product surveillance activities. Their use for assessing hospital performance on the basis of patient outcomes has received little attention. We compared risk-adjusted sequential probability ratio testing (RA-SPRT) implemented in an automated tool to Massachusetts public reports of 30-day mortality after isolated coronary artery bypass graft surgery.
  
  
  METHODS
  A total of 23,020 isolated adult coronary artery bypass surgery admissions performed in Massachusetts hospitals between January 1, 2002 and September 30, 2007 were retrospectively re-evaluated. The RA-SPRT method was implemented within an automated surveillance tool to identify hospital outliers in yearly increments. We used an overall type I error rate of 0.05, an overall type II error rate of 0.10, and a threshold that signaled if the odds of dying 30-days after surgery was at least twice than expected. Annual hospital outlier status, based on the state-reported classification, was considered the gold standard. An event was defined as at least one occurrence of a higher-than-expected hospital mortality rate during a given year.
  
  
  RESULTS
  We examined a total of 83 hospital-year observations. The RA-SPRT method alerted 6 events among three hospitals for 30-day mortality compared with 5 events among two hospitals using the state public reports, yielding a sensitivity of 100% (5/5) and specificity of 98.8% (79/80).
  
  
  CONCLUSIONS
  The automated RA-SPRT method performed well, detecting all of the true institutional outliers with a small false positive alerting rate. Such a system could provide confidential automated notification to local institutions in advance of public reporting providing opportunities for earlier quality improvement interventions.
authors.affiliation: GRECC and Center for Health Services Research, Tennessee Valley Healthcare System, Veterans Administration, Nashville, TN, USA. michael.matheny@vanderbilt.edu
authors.forename: Michael E
authors.initials: ME
authors.lastname: Matheny
authors.forename: Sharon-Lise T
authors.initials: SL
authors.lastname: Normand
authors.forename: Thomas P
authors.initials: TP
authors.lastname: Gross
authors.forename: Danica
authors.initials: D
authors.lastname: Marinac-Dabic
authors.forename: Nilsa
authors.initials: N
authors.lastname: Loyo-Berrios
authors.forename: Venkatesan D
authors.initials: VD
authors.lastname: Vidi
authors.forename: Sharon
authors.initials: S
authors.lastname: Donnelly
authors.forename: Frederic S
authors.initials: FS
authors.lastname: Resnic
mesh_terms: D000328:Adult; D019985:Benchmarking; D001026:Coronary Artery Bypass / Q000592:standards / Q000706:statistics & numerical data; D003661:Decision Support Techniques; D006801:Humans; D008390:Markov Chains; D008404:Massachusetts; D019300:Medical Errors / Q000517:prevention & control; D008499:Medical Records; D015233:Models, Statistical; D009010:Monte Carlo Method; D016017:Odds Ratio; D017063:Outcome Assessment, Health Care; D010343:Patient Admission / Q000706:statistics & numerical data* / Q000639:trends; D019095:Point-of-Care Systems; D012189:Retrospective Studies; D020379:Risk Adjustment / Q000379:methods*; D017751:Safety Management / Q000458:organization & administration*; D018571:Sentinel Surveillance