Controlling for occasion-specific effects when assessing the test-retest reliability of self-report health questionnaires.
OBJECTIVE
This study proposes a method for self-report health questionnaires to adjust test-retest reliability for changes during the test-retest interval based on an external measure, and to distinguish such changes from random response errors.


METHODS
In our application, eighty participants completed the Symptoms of Illness Checklist (SIC) on two occasions, two weeks apart, immediately before interviews given on each occasion by one of two physicians in a crossover design. The physician interview scores served as external measures, and structural equation modeling was used to estimate the parameters of a model that corrected for the occasion-specific effect of participants' responses using information from the interviews.


RESULTS
Correcting for changes in symptoms during the test-retest interval increased SIC test-retest reliability from .744 to .804 and significantly improved model fit (chi2(diff)(1) = 30.78, p < .001).


CONCLUSIONS
The results suggest methods that can improve the evaluation of self-report health questionnaire test-retest reliability by identifying changes using an external measure, and distinguishing these from random response errors; these increased the estimated SIC test-retest reliability and indicated that the SIC was indeed able to measure changes over the studied time interval. This method can be applied across a broad range of questionnaires.
Olsen
Joseph A
JA
College of Family, Home and Social Sciences, Brigham Young University, Provo, UT, USA.
Bloch
Daniel A
DA
Bloch
George J
GJ
D006304:Health Status; D006305:Health Status Indicators; D006801:Humans; D008962:Models, Theoretical; D011581:Psychological Tests; D011594:Psychometrics; D015203:Reproducibility of Results; D018434:Sickness Impact Profile; D011795:Surveys and Questionnaires