title: Automatic classification of diseases from free-text death certificates for real-time surveillance.
abstract: BACKGROUND
  Death certificates provide an invaluable source for mortality statistics which can be used for surveillance and early warnings of increases in disease activity and to support the development and monitoring of prevention or response strategies. However, their value can be realised only if accurate, quantitative data can be extracted from death certificates, an aim hampered by both the volume and variable nature of certificates written in natural language. This study aims to develop a set of machine learning and rule-based methods to automatically classify death certificates according to four high impact diseases of interest: diabetes, influenza, pneumonia and HIV.
  
  
  METHODS
  Two classification methods are presented: i) a machine learning approach, where detailed features (terms, term n-grams and SNOMED CT concepts) are extracted from death certificates and used to train a set of supervised machine learning models (Support Vector Machines); and ii) a set of keyword-matching rules. These methods were used to identify the presence of diabetes, influenza, pneumonia and HIV in a death certificate. An empirical evaluation was conducted using 340,142 death certificates, divided between training and test sets, covering deaths from 2000-2007 in New South Wales, Australia. Precision and recall (positive predictive value and sensitivity) were used as evaluation measures, with F-measure providing a single, overall measure of effectiveness. A detailed error analysis was performed on classification errors.
  
  
  RESULTS
  Classification of diabetes, influenza, pneumonia and HIV was highly accurate (F-measure 0.96). More fine-grained ICD-10 classification effectiveness was more variable but still high (F-measure 0.80). The error analysis revealed that word variations as well as certain word combinations adversely affected classification. In addition, anomalies in the ground truth likely led to an underestimation of the effectiveness.
  
  
  CONCLUSIONS
  The high accuracy and low cost of the classification methods allow for an effective means for automatic and real-time surveillance of diabetes, influenza, pneumonia and HIV deaths. In addition, the methods are generally applicable to other diseases of interest and to other sources of medical free-text besides death certificates.
authors.affiliation: Australian e-Health Research Centre, CSIRO, Royal Brisbane and Women's Hospital, Brisbane, Australia. bevan.koopman@csiro.au.
authors.forename: Bevan
authors.initials: B
authors.lastname: Koopman
authors.affiliation: Australian e-Health Research Centre, CSIRO, Royal Brisbane and Women's Hospital, Brisbane, Australia.
authors.forename: Sarvnaz
authors.initials: S
authors.lastname: Karimi
authors.affiliation: Australian e-Health Research Centre, CSIRO, Royal Brisbane and Women's Hospital, Brisbane, Australia.
authors.forename: Anthony
authors.initials: A
authors.lastname: Nguyen
authors.affiliation: NSW Ministry of Health, North Sydney, Sydney, Australia.
authors.forename: Rhydwyn
authors.initials: R
authors.lastname: McGuire
authors.affiliation: NSW Ministry of Health, North Sydney, Sydney, Australia.
authors.forename: David
authors.initials: D
authors.lastname: Muscatello
authors.affiliation: Australian e-Health Research Centre, CSIRO, Royal Brisbane and Women's Hospital, Brisbane, Australia.
authors.forename: Madonna
authors.initials: M
authors.lastname: Kemp
authors.affiliation: Australian e-Health Research Centre, CSIRO, Royal Brisbane and Women's Hospital, Brisbane, Australia.
authors.forename: Donna
authors.initials: D
authors.lastname: Truran
authors.affiliation: Australian e-Health Research Centre, CSIRO, Royal Brisbane and Women's Hospital, Brisbane, Australia.
authors.forename: Ming
authors.initials: M
authors.lastname: Zhang
authors.affiliation: NSW Ministry of Health, North Sydney, Sydney, Australia.
authors.forename: Sarah
authors.initials: S
authors.lastname: Thackway
mesh_terms: D002965:Classification; D003644:Death Certificates; D062665:Epidemiological Monitoring; D006801:Humans; D000069550:Machine Learning; D009517:New South Wales