title: Acute Respiratory Distress Syndrome Measurement Error. Potential Effect on Clinical Study Results.
abstract: Identifying patients with acute respiratory distress syndrome (ARDS) is a recognized challenge. Experts often have only moderate agreement when applying the clinical definition of ARDS to patients. However, no study has fully examined the implications of low reliability measurement of ARDS on clinical studies.
  
  
  
  To investigate how the degree of variability in ARDS measurement commonly reported in clinical studies affects study power, the accuracy of treatment effect estimates, and the measured strength of risk factor associations.
  
  
  
  We examined the effect of ARDS measurement error in randomized clinical trials (RCTs) of ARDS-specific treatments and cohort studies using simulations. We varied the reliability of ARDS diagnosis, quantified as the interobserver reliability (κ-statistic) between two reviewers. In RCT simulations, patients identified as having ARDS were enrolled, and when measurement error was present, patients without ARDS could be enrolled. In cohort studies, risk factors as potential predictors were analyzed using reviewer-identified ARDS as the outcome variable.
  
  
  
  Lower reliability measurement of ARDS during patient enrollment in RCTs seriously degraded study power. Holding effect size constant, the sample size necessary to attain adequate statistical power increased by more than 50% as reliability declined, although the result was sensitive to ARDS prevalence. In a 1,400-patient clinical trial, the sample size necessary to maintain similar statistical power increased to over 1,900 when reliability declined from perfect to substantial (κ = 0.72). Lower reliability measurement diminished the apparent effectiveness of an ARDS-specific treatment from a 15.2% (95% confidence interval, 9.4-20.9%) absolute risk reduction in mortality to 10.9% (95% confidence interval, 4.7-16.2%) when reliability declined to moderate (κ = 0.51). In cohort studies, the effect on risk factor associations was similar.
  
  
  
  ARDS measurement error can seriously degrade statistical power and effect size estimates of clinical studies. The reliability of ARDS measurement warrants careful attention in future ARDS clinical studies.
authors.affiliation: 1 Department of Internal Medicine and.|2 Institute for Healthcare Policy & Innovation, University of Michigan, Ann Arbor, Michigan.
authors.forename: Michael W
authors.initials: MW
authors.lastname: Sjoding
authors.affiliation: 1 Department of Internal Medicine and.|2 Institute for Healthcare Policy & Innovation, University of Michigan, Ann Arbor, Michigan.
authors.forename: Colin R
authors.initials: CR
authors.lastname: Cooke
authors.affiliation: 1 Department of Internal Medicine and.|3 VA Center for Clinical Management Research, Ann Arbor, Michigan.|4 Institute for Social Research, Ann Arbor, Michigan; and.|5 Australian and New Zealand Intensive Care Research Centre, Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
authors.forename: Theodore J
authors.initials: TJ
authors.lastname: Iwashyna
authors.affiliation: 1 Department of Internal Medicine and.|2 Institute for Healthcare Policy & Innovation, University of Michigan, Ann Arbor, Michigan.|3 VA Center for Clinical Management Research, Ann Arbor, Michigan.
authors.forename: Timothy P
authors.initials: TP
authors.lastname: Hofer
mesh_terms: D000328:Adult; D003951:Diagnostic Errors / Q000706:statistics & numerical data*; D006801:Humans; D016015:Logistic Models; D015999:Multivariate Analysis; D064887:Observational Studies as Topic; D016032:Randomized Controlled Trials as Topic; D015203:Reproducibility of Results; D012121:Respiration, Artificial / Q000379:methods; D012128:Respiratory Distress Syndrome / Q000175:diagnosis* / Q000401:mortality* / Q000628:therapy; D012307:Risk Factors
keywords: acute lung injury; bias; clinical trial; diagnosis; observational study