title: The inclusion of 'then-test' questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation.
abstract: OBJECTIVE
  Program evaluations are frequently based on 'then-test' data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects' responses to post-test.
  
  
  METHODS
  To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314).
  
  
  RESULTS
  Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then-test questions reporting significantly higher post-test scores than respondents of the other groups.
  
  
  CONCLUSIONS
  This study provides evidence that the inclusion of then-test questions alters post-test responses, suggesting that change scores based on then-test data be interpreted with care.
authors.affiliation: Deakin University, Burwood, VIC, Australia. sandra.nolte@web.de
authors.forename: Sandra
authors.initials: S
authors.lastname: Nolte
authors.forename: Gerald R
authors.initials: GR
authors.lastname: Elsworth
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Sinclair
authors.forename: Richard H
authors.initials: RH
authors.lastname: Osborne
mesh_terms: D000223:Adaptation, Psychological; D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D001294:Attitude to Health; D001315:Australia; D002908:Chronic Disease / Q000523:psychology*; D005260:Female; D006305:Health Status Indicators; D006801:Humans; D008297:Male; D008875:Middle Aged; D010353:Patient Education as Topic; D015397:Program Evaluation; D011594:Psychometrics; D011788:Quality of Life; D012107:Research Design; D012648:Self Care; D011795:Surveys and Questionnaires