title: Review of null hypothesis significance testing in the ophthalmic literature: are most 'significant' P values false positives?
abstract: P values associated with null hypothesis significance testing (NHST) are almost universal in the ophthalmic literature. A P value < 0.05 is traditionally considered 'significant'. This concept may deflect further thought about the veracity of the results. P values influence the publishability of the data and have flow-on effects for funding success and the direction of future research. Despite their importance, the problems inherent in P values have been recognized since their inception, and in more recent years have been increasingly highlighted in some scientific fields. In this review, we aim to bring the problems associated with P values and NHST to the attention of the ophthalmic research community. We do not offer a universal solution to the problem of determining the veracity of a scientific claim; however, we demonstrate the need for caution in interpreting 'significant' P values by performing a Bayesian re-analysis of t-tests in the ophthalmic literature.
authors.affiliation: Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, University of Melbourne, Melbourne, Victoria, Australia.|Centre for Ophthalmology and Visual Science, Lions Eye Institute, University of Western Australia, Perth, Western Australia, Australia.
authors.forename: Paul G
authors.initials: PG
authors.lastname: Sanfilippo
authors.affiliation: Hanson Institute and University of Adelaide, Adelaide, South Australia, Australia.
authors.forename: Robert J
authors.initials: RJ
authors.lastname: Casson
authors.affiliation: Centre for Ophthalmology and Visual Science, Lions Eye Institute, University of Western Australia, Perth, Western Australia, Australia.
authors.forename: Seyhan
authors.initials: S
authors.lastname: Yazar
authors.affiliation: Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, University of Melbourne, Melbourne, Victoria, Australia.|Centre for Ophthalmology and Visual Science, Lions Eye Institute, University of Western Australia, Perth, Western Australia, Australia.|School of Medicine, Menzies Research Institute Tasmania, University of Tasmania, Hobart, Tasmania, Australia.
authors.forename: David A
authors.initials: DA
authors.lastname: Mackey
authors.affiliation: Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, University of Melbourne, Melbourne, Victoria, Australia.|Centre for Ophthalmology and Visual Science, Lions Eye Institute, University of Western Australia, Perth, Western Australia, Australia.|School of Medicine, Menzies Research Institute Tasmania, University of Tasmania, Hobart, Tasmania, Australia.
authors.forename: Alex W
authors.initials: AW
authors.lastname: Hewitt
mesh_terms: D003627:Data Interpretation, Statistical; D005189:False Positive Reactions; D006801:Humans; D009885:Ophthalmology; D011237:Predictive Value of Tests; D011336:Probability
keywords: biostatistics; epidemiology; journal