title: Inter-rater agreement on assessment of outcome within a trauma registry.
abstract: BACKGROUND
  To better evaluate the degree of ongoing disability in trauma patients, it has been recommended that trauma registries introduce routine long-term outcome measurement. One of the measures recommended for use is the Extended Glasgow Outcome Scale (GOS-E). However, few registries have adopted this measure and further research is required to determine its reliability with trauma populations. This study aimed to evaluate the inter-rater agreement of GOS-E scoring between an expert rater and trauma registry follow-up staff with a sample of detailed trauma case scenarios.
  
  
  METHODS
  Sixteen trauma registry telephone interviewers participated in the study. They were provided with a written summary of 15 theoretical adult trauma cases covering a spectrum of disability and asked to rate each case using the structured GOS-E interview. Their ratings were compared with those of an expert rater in order to calculate the inter-rater agreement for each individual rater-expert rater pair. Agreement was reported as the percentage of agreement, the kappa statistic, and weighted kappa. A multi-rater kappa value was also calculated for agreement between the 16 raters.
  
  
  RESULTS
  Across the 15 cases, the percentage of agreement between individual raters and the expert ranged from 63% to 100%. Across the 16 raters, the percentage of agreement with the expert rater ranged from 73-100% (mean=90%). Kappa values ranged from 0.65 to 1.00 across raters (mean=0.86) and weighted kappa values ranged from 0.73 to 1.00 (mean=0.89) The multi-rater kappa value was 0.78 (95% CI: 0.66, 0.89).
  
  
  CONCLUSIONS
  Sixteen follow-up staff achieved 'substantial' to 'almost perfect' agreement with an expert rater using the GOS-E outcome measure to score 15 sample trauma cases. The results of this study lend support to the use of the GOS-E within trauma populations and highlight the importance of ongoing training where multiple raters are involved to ensure reliable outcome reporting. It is also recommended that the structured GOS-E interview guide be used to achieve better agreement between raters. Ensuring the reliability of trauma outcome scores will enable more accurate evaluation of patient outcomes, and ultimately, more targeted trauma care.
authors: Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia. Electronic address: christina.ekegren@monash.edu.
authors: C L
authors: CL
authors: Ekegren
authors: Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia; Royal Melbourne Hospital, Melbourne, Australia.
authors: M J
authors: MJ
authors: Hart
authors: Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia.
authors: A
authors: A
authors: Brown
authors: Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia.
authors: B J
authors: BJ
authors: Gabbe
mesh_terms: D001315:Australia / Q000453:epidemiology; D016022:Case-Control Studies; D004185:Disability Evaluation; D006233:Persons with Disabilities; D005500:Follow-Up Studies; D023261:Glasgow Outcome Scale; D006801:Humans; D015588:Observer Variation; D011785:Quality Assurance, Health Care; D020127:Recovery of Function; D012042:Registries / Q000706:statistics & numerical data*; D015203:Reproducibility of Results; D015599:Trauma Severity Indices; D014947:Wounds and Injuries / Q000401:mortality* / Q000503:physiopathology / Q000534:rehabilitation
keywords: Extended Glasgow Outcome Scale; Inter-rater agreement; Outcome measures; Registry; Reliability; Trauma; Validity