title: How well do various health outcome definitions identify appropriate cases in observational studies?
abstract: BACKGROUND
  Observational data can be useful for drug safety research, but accurate measurement of adverse health outcomes is paramount. Best practices for identifying important health outcomes of interest (HOI) are needed.
  
  
  OBJECTIVE
  To evaluate the extent to which health outcome definitions commonly used in observational database studies identify cases that are consistent with expert panel assessment of the underlying data.
  
  
  METHODS
  Competing HOI definitions were used to identify potential cases of acute liver injury (ALI; n = 208), acute kidney injury (AKI; n = 200), and myocardial infarction (MI; n = 204) in the Truven MarketScan Lab Database (MSLR). Panelists reviewed patient-level data and answered questions about whether they believed the case actually reflected the HOI and their certainty of case classification on a 10-point scale (1 = unlikely to 10 = likely). Each patient was reviewed independently by two panelists. Case disagreements were resolved through consensus meetings. Positive predictive value (PPV) was calculated as the number of cases deemed to be true over the total number of sampled cases. Kappa statistics assessed inter-rater agreement.
  
  
  RESULTS
  PPV ranged from 0 to 52 % across ALI definitions, 12 to 82 % across AKI definitions, and 1 to 56 % across MI definitions. Certainty scores on the 10-point scale paralleled the PPV, with a range of mean values from 1.7 to 4.8 across ALI definitions, 3.1 to 6.0 across AKI definitions, and 2.8 to 5.7 across MI definitions. Inter-rater agreement was low to moderate (Kappa range 0.0-0.6).
  
  
  CONCLUSIONS
  Existing HOI definitions had relatively low PPV based on expert panel review. Experts commonly disagreed on case classification. Additional work is needed to refine HOI case definitions if observational data are to be reliably used for health outcome assessment.
authors.affiliation: Department of Pharmacy Care Systems, Harrison School of Pharmacy, Auburn University, 020 Foy Hall, Auburn, AL, 36849, USA, rah0019@auburn.edu.
authors.forename: Richard A
authors.initials: RA
authors.lastname: Hansen
authors.forename: Michael D
authors.initials: MD
authors.lastname: Gray
authors.forename: Brent I
authors.initials: BI
authors.lastname: Fox
authors.forename: Joshua C
authors.initials: JC
authors.lastname: Hollingsworth
authors.forename: Juan
authors.initials: J
authors.lastname: Gao
authors.forename: Peng
authors.initials: P
authors.lastname: Zeng
mesh_terms: D058186:Acute Kidney Injury / Q000139:chemically induced*; D056486:Chemical and Drug Induced Liver Injury / Q000175:diagnosis*; D064420:Drug-Related Side Effects and Adverse Reactions / Q000175:diagnosis*; D006801:Humans; D009203:Myocardial Infarction / Q000139:chemically induced*; D064887:Observational Studies as Topic; D011237:Predictive Value of Tests