title: A critical review of methods to evaluate the impact of FDA regulatory actions.
abstract: OBJECTIVE
  To conduct a synthesis of the literature on methods to evaluate the impacts of FDA regulatory actions and identify best practices for future evaluations.
  
  
  METHODS
  We searched MEDLINE for manuscripts published between January 1948 and August 2011 that included terms related to FDA, regulatory actions, and empirical evaluation; the review additionally included FDA-identified literature. We used a modified Delphi method to identify preferred methodologies. We included studies with explicit methods to address threats to validity and identified designs and analytic methods with strong internal validity that have been applied to other policy evaluations.
  
  
  RESULTS
  We included 18 studies out of 243 abstracts and papers screened. Overall, analytic rigor in prior evaluations of FDA regulatory actions varied considerably; less than a quarter of studies (22%) included control groups. Only 56% assessed changes in the use of substitute products/services, and 11% examined patient health outcomes. Among studies meeting minimal criteria of rigor, 50% found no impact or weak/modest impacts of FDA actions and 33% detected unintended consequences. Among those studies finding significant intended effects of FDA actions, all cited the importance of intensive communication efforts. There are preferred methods with strong internal validity that have yet to be applied to evaluations of FDA regulatory actions.
  
  
  CONCLUSIONS
  Rigorous evaluations of the impact of FDA regulatory actions have been limited and infrequent. Several methods with strong internal validity are available to improve trustworthiness of future evaluations of FDA policies.
authors.affiliation: Division of Geriatric Medicine, University of Massachusetts Medical School, Worcester, MA 01605, USA. Becky.Briesacher@umassmed.edu
authors.forename: Becky A
authors.initials: BA
authors.lastname: Briesacher
authors.forename: Stephen B
authors.initials: SB
authors.lastname: Soumerai
authors.forename: Fang
authors.initials: F
authors.lastname: Zhang
authors.forename: Sengwee
authors.initials: S
authors.lastname: Toh
authors.forename: Susan E
authors.initials: SE
authors.lastname: Andrade
authors.forename: Joann L
authors.initials: JL
authors.lastname: Wagner
authors.forename: Azadeh
authors.initials: A
authors.lastname: Shoaibi
authors.forename: Jerry H
authors.initials: JH
authors.lastname: Gurwitz
mesh_terms: D003257:Consumer Product Safety / Q000331:legislation & jurisprudence*; D017277:Drug Approval / Q000331:legislation & jurisprudence*; D004348:Drug Labeling / Q000331:legislation & jurisprudence*; D023381:Endpoint Determination / Q000379:methods* / Q000706:statistics & numerical data; D033161:Government Regulation; D012107:Research Design / Q000706:statistics & numerical data; D014481:United States; D014486:United States Food and Drug Administration
keywords: FDA; evaluation methodology; pharmacoepidemiology; regulatory actions