title: But what do the numbers really tell us? Arbitrary metrics and effect size reporting in sport psychology research.
abstract: Many of the measurements used in sport psychology research are arbitrary metrics, and researchers often cannot make the jump from scores on paper-and-pencil tests to what those scores actually mean in terms of real-world behaviors. Effect sizes for behavioral data are often interpretable, but the meaning of a small, medium, or large effect for an arbitrary metric is elusive. We reviewed all the issues in the 2005 volumes of the Journal of Sport & Exercise Psychology, The Sport Psychologist, and the Journal of Applied Sport Psychology to determine whether the arbitrary metrics used in sport psychology research were interpreted, or calibrated, against real-world variables. Of the 54 studies that used quantitative methods, 25 reported only paper-and-pencil arbitrary metrics with no connections to behavior or other real-world variables. Also, 44 of the 54 studies reported effect sizes, but only 7 studies, using both arbitrary and behavioral metrics, had calculated effect indicators and interpreted them in terms of real-world meaning.
authors: School of Human Movement, Recreation, and Performance, Centre for Ageing, Rehabilitation, Exercise, and Sport, Victoria University, Melbourne, Australia.
authors: Mark B
authors: MB
authors: Andersen
authors: Penny
authors: P
authors: McCullagh
authors: Gabriel J
authors: GJ
authors: Wilson
mesh_terms: D015706:Bibliometrics; D003627:Data Interpretation, Statistical; D006801:Humans; D011594:Psychometrics; D012106:Research; D013177:Sports / Q000523:psychology*