title: An analysis of the use of multiple comparison corrections in ophthalmology research.
abstract: OBJECTIVE
  The probability of type I error, or a false-positive result, increases as the number of statistical comparisons in a study increases. Statisticians have developed numerous corrections to account for the multiple comparison problem. This study discusses recent guidelines involving multiple comparison corrections, calculates the prevalence of corrections in ophthalmic research, and estimates the corresponding number of false-positive results reported at a recent international research meeting.
  
  
  METHODS
  The 6415 abstracts presented at ARVO 2010 were searched for statistical comparisons (P values) and for use of multiple comparison corrections. Studies that reported five or more P values while reporting no correction factor were used in a simulation study. The simulation study was conducted to estimate the number of false-positive results reported in these studies.
  
  
  RESULTS
  Overall, 36% of abstracts reported P values and 1.2% of abstracts used some form of correction. Whereas 8% of abstracts reported at least five P values, only 5% of these used a multiple comparison correction. In these highly statistical studies, simulations resulted in 185 false-positive outcomes found in 30% of abstracts.
  
  
  CONCLUSIONS
  The paucity of multiple comparison corrections in ophthalmic research results in inflated type I error and may produce unwarranted shifts in clinical or surgical care. Researchers must make a conscious effort to decide if and when to use a correction factor to ensure the validity of the data.
authors.affiliation: Department of Medical Education, Riverside Methodist Hospital, Columbus, Ohio 43214, USA. astacey2@ohiohealth.com
authors.forename: Andrew W
authors.initials: AW
authors.lastname: Stacey
authors.forename: Severin
authors.initials: S
authors.lastname: Pouly
authors.forename: Craig N
authors.initials: CN
authors.lastname: Czyz
mesh_terms: D035843:Biomedical Research / Q000379:methods*; D006801:Humans; D009885:Ophthalmology; D017594:Publication Bias; D015203:Reproducibility of Results; D012189:Retrospective Studies