title: Should policy-makers and managers trust PSI? An empirical validation study of five patient safety indicators in a national health service.
abstract: BACKGROUND
  Patient Safety Indicators (PSI) are being modestly used in Spain, somewhat due to concerns on their empirical properties. This paper provides evidence by answering three questions: a) Are PSI differences across hospitals systematic -rather than random?; b) Do PSI measure differences among hospital-providers -as opposed to differences among patients?; and, c) Are measurements able to detect hospitals with a higher than "expected" number of cases?
  
  
  METHODS
  An empirical validation study on administrative data was carried out. All 2005 and 2006 publicly-funded hospital discharges were used to retrieve eligible cases of five PSI: Death in low-mortality DRGs (MLM); decubitus ulcer (DU); postoperative pulmonary embolism or deep-vein thrombosis (PE-DVT); catheter-related infections (CRI), and postoperative sepsis (PS). Empirical Bayes statistic (EB) was used to estimate whether the variation was systematic; logistic-multilevel modelling determined what proportion of the variation was explained by the hospital; and, shrunken residuals, as provided by multilevel modelling, were plotted to flag hospitals performing worse than expected.
  
  
  RESULTS
  Variation across hospitals was observed to be systematic in all indicators, with EB values ranging from 0.19 (CI95%:0.12 to 0.28) in PE-DVT to 0.34 (CI95%:0.25 to 0.45) in DU. A significant proportion of the variance was explained by the hospital, once patient case-mix was adjusted: from a 6% in MLM (CI95%:3% to 11%) to a 24% (CI95%:20% to 30%) in CRI. All PSI were able to flag hospitals with rates over the expected, although this capacity decreased when the largest hospitals were analysed.
  
  
  CONCLUSIONS
  Five PSI showed reasonable empirical properties to screen healthcare performance in Spanish hospitals, particularly in the largest ones.
authors.affiliation: Instituto Aragonés de Ciencias de la Salud, Instituto de Investigación Sanitaria Aragón, Zaragoza, Spain. ebernal.iacs@aragon.es
authors.forename: Enrique
authors.initials: E
authors.lastname: Bernal-Delgado
authors.forename: Sandra
authors.initials: S
authors.lastname: García-Armesto
authors.forename: Natalia
authors.initials: N
authors.lastname: Martínez-Lizaga
authors.forename: Begoña
authors.initials: B
authors.lastname: Abadía-Taira
authors.forename: Joaquín
authors.initials: J
authors.lastname: Beltrán-Peribañez
authors.forename: Salvador
authors.initials: S
authors.lastname: Peiró
mesh_terms: D000288:Administrative Personnel / Q000523:psychology*; D036262:Empirical Research; D005260:Female; D006761:Hospitals / Q000592:standards; D006801:Humans; D016015:Logistic Models; D008297:Male; D009313:National Health Programs; D017063:Outcome Assessment, Health Care; D010351:Patient Discharge / Q000706:statistics & numerical data*; D061214:Patient Safety / Q000592:standards*; D011050:Policy Making; D019984:Quality Indicators, Health Care; D013030:Spain; D035502:Trust