title: Accuracy of commercial reporting systems to monitor quality of care in burns.
abstract: BACKGROUND
  Capse Healthcare Knowledge Systems (CHKS) is a global commercial organisation that operates health benchmarking programmes in the UK and internationally. In absence of a specialty-specific quality monitoring programme for burn services, CHKS has been producing comparative quality data for burn services for a number of years. The major quality indicator reported by CHKS is mortality as a Risk Adjusted Mortality Index (RAMI). The accuracy of RAMI is unknown in comparison to published burn-specific mortality prediction models.
  
  
  METHODS
  A retrospective study design was used to collect data for patients admitted to the Adult Burn Service at University Hospital South Manchester (UHSM) between January 2006 and December 2010. Data was collected from two sources, CHKS and Manchester Burn Injury Database (MBID). The demographic and injury characteristics of survivors and non-survivors were compared and Receiver Operator Curve (ROC), equivalence and non-inferiority analyses were used to assess accuracy of RAMI in comparison to Abbreviated Burn Severity Index (ABSI), Belgian Outcome of Burn Injury (BOBI) score, Baux score (Baux) and McGwin score (McGwin).
  
  
  RESULTS
  The accuracy of RAMI to discriminate between survivors and non-survivors (area under curve=0.79, 95% CI 0.50-0.81) was significantly inferior to that of ABSI, BOBI, Baux and McGwin scores. Equivalence and non-inferiority testing of ROC curves also showed RAMI score to be inferior to ABSI, BOBI, Baux and McGwin scores at 5% significance level.
  
  
  CONCLUSIONS
  CHKS RAMI provides an inaccurate and inferior monitoring of mortality as a quality indicator in burn patients compared to burn specific mortality prediction models. This study raises concerns about the ability of commercially reported systems to accurately monitor quality indicators of relevance to burn care.
authors.affiliation: University Hospital South Manchester, Southmoor Road, Manchester M23 9LT, United Kingdom. Electronic address: amh@doctors.net.uk.
authors.forename: Amer
authors.initials: A
authors.lastname: Hussain
authors.affiliation: University Hospital South Manchester, Southmoor Road, Manchester M23 9LT, United Kingdom. Electronic address: ken.dunn@uhsm.nhs.uk.
authors.forename: Ken W
authors.initials: KW
authors.lastname: Dunn
mesh_terms: D000328:Adult; D000367:Age Factors; D000368:Aged; D019540:Area Under Curve; D019985:Benchmarking / Q000592:standards*; D001830:Body Surface Area; D002053:Burn Units / Q000706:statistics & numerical data*; D002056:Burns / Q000150:complications / Q000401:mortality* / Q000628:therapy; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D019984:Quality Indicators, Health Care / Q000592:standards*; D011787:Quality of Health Care / Q000706:statistics & numerical data*; D012372:ROC Curve; D012189:Retrospective Studies; D012720:Severity of Illness Index; D015208:Smoke Inhalation Injury / Q000150:complications
keywords: Burn; Mortality; Outcome; Quality indicator(s); Risk-adjustment