title: A review of multiple hypothesis testing in otolaryngology literature.
abstract: OBJECTIVE
  Multiple hypothesis testing (or multiple testing) refers to testing more than one hypothesis within a single analysis, and can inflate the type I error rate (false positives) within a study. The aim of this review was to quantify multiple testing in recent large clinical studies in the otolaryngology literature and to discuss strategies to address this potential problem.
  
  
  METHODS
  Original clinical research articles with >100 subjects published in 2012 in the four general otolaryngology journals with the highest Journal Citation Reports 5-year impact factors.
  
  
  METHODS
  Articles were reviewed to determine whether the authors tested greater than five hypotheses in at least one family of inferences. For the articles meeting this criterion for multiple testing, type I error rates were calculated, and statistical correction was applied to the reported results.
  
  
  RESULTS
  Of the 195 original clinical research articles reviewed, 72% met the criterion for multiple testing. Within these studies, there was a mean 41% chance of a type I error and, on average, 18% of significant results were likely to be false positives. After the Bonferroni correction was applied, only 57% of significant results reported within the articles remained significant.
  
  
  CONCLUSIONS
  Multiple testing is common in recent large clinical studies in otolaryngology and deserves closer attention from researchers, reviewers, and editors. Strategies for adjusting for multiple testing are discussed.
authors.affiliation: Department of Otolaryngology/Head & Neck Surgery, University of Washington, Seattle, Washington.
authors.forename: Erin M
authors.initials: EM
authors.lastname: Kirkham
authors.forename: Edward M
authors.initials: EM
authors.lastname: Weaver
mesh_terms: D035843:Biomedical Research; D006801:Humans; D010036:Otolaryngology; D010506:Periodicals as Topic
keywords: Bonferroni; Bonferroni-Holm; Multiple testing; clinical research methodology; database; error rate per comparison; family-wise error rate; multiple comparisons; percent error rate; study design
references.citation: Invest Ophthalmol Vis Sci. 2012 Apr 06;53(4):1830-4
references.pmid: 22408004
references.citation: PM R. 2009 Dec;1(12):1098-103
references.pmid: 20006317
references.citation: Am J Public Health. 1996 May;86(5):726-8
references.pmid: 8629727
references.citation: PLoS One. 2011;6(12):e28422
references.pmid: 22164288
references.citation: Am J Epidemiol. 1998 Apr 1;147(7):615-9
references.pmid: 9554599
references.citation: J Clin Epidemiol. 2001 Apr;54(4):343-9
references.pmid: 11297884
references.citation: Biometrics. 1979 Sep;35(3):549-56
references.pmid: 497341
references.citation: Clin Ther. 2004 May;26(5):780-6
references.pmid: 15220022