Stability of questionnaire items in sport and exercise psychology: bootstrap limits of agreement.
We describe an alternative method for assessing the stability of individual questionnaire items in sport and exercise psychology. To date, the Pearson product-moment correlation coefficient has been widely used in psychometrics. We propose an alternative non-parametric method based on proportion of agreement. Ninety-two male university students completed the revised 9-item Social Physique Anxiety Scale on two occasions, separated by a 2-week interval. Point estimates of the proportion of direct within-individual agreement between the two occasions were calculated separately for each item of the Social Physique Anxiety Scale. Estimates of uncertainty of the agreement were calculated using a bootstrapping resampling technique. For each item, 2000 bootstrap samples (each n = 92 pairs) were redrawn from the original sample. The sample statistic was calculated for each bootstrap sample to provide a bootstrap sampling distribution. The 95% confidence intervals (CI) were then calculated using the percentile method. The three most problematic items were items 7, 8 and 10 (as labelled in the original 12-item scale). These items demonstrated an agreement of 0.46 (95% CI= 0.36 0.56), 0.42 (95% CI = 0.33-0.52) and 0.41 (95% CI = 0.32-0.51) respectively. Our proposed method measures absolute agreement between test-retest responses, is free of normal assumptions, does not depend on high between-individuals variance, and can be applied successfully to individual items in the development of psychological tests.
Wilson
K
K
School of Social Sciences, University of Teesside, Middlesbrough, UK.
Batterham
A
A
D000328:Adult; D001828:Body Image; D016001:Confidence Intervals; D006801:Humans; D008297:Male; D010555:Personality Inventory / Q000592:standards* / Q000706:statistics & numerical data*; D011594:Psychometrics; D015203:Reproducibility of Results; D012494:Sampling Studies; D018709:Statistics, Nonparametric