title: A design of experiments approach to validation sampling for logistic regression modeling with error-prone medical records.
abstract: OBJECTIVE
  Electronic medical record (EMR) databases offer significant potential for developing clinical hypotheses and identifying disease risk associations by fitting statistical models that capture the relationship between a binary response variable and a set of predictor variables that represent clinical, phenotypical, and demographic data for the patient. However, EMR response data may be error prone for a variety of reasons. Performing a manual chart review to validate data accuracy is time consuming, which limits the number of chart reviews in a large database. The authors' objective is to develop a new design-of-experiments-based systematic chart validation and review (DSCVR) approach that is more powerful than the random validation sampling used in existing approaches.
  
  
  METHODS
  The DSCVR approach judiciously and efficiently selects the cases to validate (i.e., validate whether the response values are correct for those cases) for maximum information content, based only on their predictor variable values. The final predictive model will be fit using only the validation sample, ignoring the remainder of the unvalidated and unreliable error-prone data. A Fisher information based D-optimality criterion is used, and an algorithm for optimizing it is developed.
  
  
  RESULTS
  The authors' method is tested in a simulation comparison that is based on a sudden cardiac arrest case study with 23 041 patients' records. This DSCVR approach, using the Fisher information based D-optimality criterion, results in a fitted model with much better predictive performance, as measured by the receiver operating characteristic curve and the accuracy in predicting whether a patient will experience the event, than a model fitted using a random validation sample.
  
  
  CONCLUSIONS
  The simulation comparisons demonstrate that this DSCVR approach can produce predictive models that are significantly better than those produced from random validation sampling, especially when the event rate is low.
authors.affiliation: Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL 60208, USA.
authors.forename: Liwen
authors.initials: L
authors.lastname: Ouyang
authors.affiliation: Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL 60208, USA apley@northwestern.edu.
authors.forename: Daniel W
authors.initials: DW
authors.lastname: Apley
authors.affiliation: Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL 60208, USA.
authors.forename: Sanjay
authors.initials: S
authors.lastname: Mehrotra
mesh_terms: D000465:Algorithms; D019540:Area Under Curve; D003198:Computer Simulation; D016757:Death, Sudden, Cardiac; D057286:Electronic Health Records; D006801:Humans; D016015:Logistic Models; D054928:Validation Studies as Topic
keywords: design of experiments; electronic medical records; logistic regression; sudden cardiac arrest; validation sampling