title: Application of generalizability theory confirmed lower reliability of the standard gamble than the feeling thermometer.
abstract: OBJECTIVE
  Recent studies suggest that rating clinical marker states (CMS) does not improve the measurement properties of the standard gamble (SG) and only slightly improves those of the feeling thermometer (FT). The poor intrarater (test-retest) reliability of CMS may explain their meager performance. Further, lack of interrater reliability may compromise the use of CMS in interpreting health state ratings. The aim of this study was to assess the reliability of CMS ratings for the SG and the FT.
  
  
  METHODS
  Two similar studies in patients with chronic obstructive pulmonary disease (COPD, n=91) and in patients with gastroesophageal reflux disease (GERD, n=112) provided data for this analysis. Patients rated three different CMS (mild, moderate, and severe disease) twice several weeks apart. We used generalizability theory to calculate reliability coefficients.
  
  
  RESULTS
  Test-retest reliability for CMS ratings was higher for the FT compared to the SG (COPD: 0.86 vs. 0.67; GERD: 0.86 vs. 0.67). Interrater reliability was much higher for the FT compared to the SG (COPD: 0.78 vs. 0.46; GERD: 0.71 vs. 0.26).
  
  
  CONCLUSIONS
  These results suggest that the markedly poorer reliability of CMS for the SG than the FT is driven largely by poor interrater reliability.
authors.affiliation: Department of Epidemiology, INFORMA Unit/CLARITY Research Group, Italian National Cancer Institute Regina Elena, Via Elio Chianesi 53, 00144 Rome, Italy. schuneh@mcmaster.ca
authors.forename: Holger J
authors.initials: HJ
authors.lastname: Schünemann
authors.forename: Geoff
authors.initials: G
authors.lastname: Norman
authors.forename: Milo A
authors.initials: MA
authors.lastname: Puhan
authors.forename: Elisabeth
authors.initials: E
authors.lastname: Ståhl
authors.forename: Lauren
authors.initials: L
authors.lastname: Griffith
authors.forename: Diane
authors.initials: D
authors.lastname: Heels-Ansdell
authors.forename: Victor M
authors.initials: VM
authors.lastname: Montori
authors.forename: Ingela
authors.initials: I
authors.lastname: Wiklund
authors.forename: Roger
authors.initials: R
authors.lastname: Goldstein
authors.forename: M Jeffery
authors.initials: MJ
authors.lastname: Mador
authors.forename: Gordon H
authors.initials: GH
authors.lastname: Guyatt
mesh_terms: D000328:Adult; D000368:Aged; D005260:Female; D005764:Gastroesophageal Reflux / Q000534:rehabilitation*; D006305:Health Status Indicators; D006801:Humans; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D017060:Patient Satisfaction; D029424:Pulmonary Disease, Chronic Obstructive / Q000534:rehabilitation*; D011788:Quality of Life; D015203:Reproducibility of Results; D011795:Surveys and Questionnaires