Tests of measurement invariance failed to support the application of the "then-test".
OBJECTIVE
The use of then-test (retrospective pre-test) scores has frequently been proposed as a solution to potential confounding of change scores because of response shift, as it is assumed that then-test and post-test responses are provided from the same perspective. However, this assumption has not been formally tested using robust quantitative methods. The aim of this study was to compare the psychometric performance of then-test/post-test with traditional pre-test/post-test data and assessing whether the resulting data structures support the application of the then-test for evaluations of chronic disease self-management interventions.


METHODS
Pre-test, post-test, and then-test data were collected from 314 participants of self-management courses using the Health Education Impact Questionnaire (heiQ). The derived change scores (pre-test/post-test; then-test/post-test) were examined for their psychometric performance using tests of measurement invariance.


RESULTS
Few questionnaire items were noninvariant across pre-test/post-test, with four items identified and requiring removal to enable an unbiased comparison of factor means. In contrast, 12 items were identified and required removal in then-test/post-test data to avoid biased change score estimates.


CONCLUSIONS
Traditional pre-test/post-test data appear to be robust with little indication of response shift. In contrast, the weaker psychometric performance of then-test/post-test data suggests psychometric flaws that may be the result of implicit theory of change, social desirability, and recall bias.
Nolte
Sandra
S
Department of Medicine, AFV Centre for Rheumatic Diseases, The University of Melbourne, Royal Melbourne Hospital, Parkville, Victoria 3050, Australia. sandra.nolte@web.de
Elsworth
Gerald R
GR
Sinclair
Andrew J
AJ
Osborne
Richard H
RH
D000223:Adaptation, Psychological; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D001294:Attitude to Health; D002908:Chronic Disease / Q000523:psychology / Q000534:rehabilitation*; D004812:Epidemiologic Methods; D005260:Female; D006305:Health Status Indicators; D006801:Humans; D008297:Male; D008875:Middle Aged; D010353:Patient Education as Topic; D015397:Program Evaluation; D011594:Psychometrics; D012648:Self Care / Q000379:methods*; D016896:Treatment Outcome; D055815:Young Adult