title: Dead reckoning: can we trust estimates of mortality rates in clinical databases?
abstract: OBJECTIVE
  Databases almost invariably contain some errors and improvements to the quality of recorded data are costly. We sought to assess the extent to which given levels of error in a clinical database can lead to misleading mortality rates being derived.
  
  
  METHODS
  We deliberately seeded a large database concerning congenital heart surgery involving over 17,600 operations, which we assumed to be error free, with errors at known rates of 0-20%. The effects of three different types of random error were explored: data omission, outcome miscoding (alive or dead) and the miscoding of procedures. For each error type, we compared the mortality rates calculated from the 'seeded' database to those calculated from the pristine database.
  
  
  RESULTS
  Outcome miscoding typically results in overestimated mortality rates which for low-risk procedures may well give estimates over double the true value. Random data omission has relatively little effect. If procedure types are miscoded, procedure-specific mortality estimates for high-risk operations tend to be underestimates and those for low-risk operations overestimates. A mathematical model developed to examine these effects accurately forecasted the results of such error-seeding experiments. Software to implement this model is available free of charge on the Internet.
  
  
  CONCLUSIONS
  Even small levels of data error can substantially affect the accuracy of mortality rate estimates, especially for low-risk operations. Such inaccuracy could lead to misleading analysis of institutional and individual surgeons' results. Our results suggest that caution is warranted in interpreting the mortality estimates derived from clinical databases. Our analysis goes beyond the realms of surgical mortality and concerns all adverse events whose frequency is rare.
authors.affiliation: Clinical Operational Research Unit, University College, London, UK. s.gallivan@ucl.ac.uk
authors.forename: Steve
authors.initials: S
authors.lastname: Gallivan
authors.forename: Jaroslav
authors.initials: J
authors.lastname: Stark
authors.forename: Christina
authors.initials: C
authors.lastname: Pagel
authors.forename: Gail
authors.initials: G
authors.lastname: Williams
authors.forename: William G
authors.initials: WG
authors.lastname: Williams
mesh_terms: D006348:Cardiac Surgical Procedures / Q000401:mortality*; D016208:Databases, Factual / Q000592:standards*; D006330:Heart Defects, Congenital / Q000401:mortality / Q000601:surgery*; D017052:Hospital Mortality; D006801:Humans; D009026:Mortality