title: Observer variation in the assessment of outcome in traumatic brain injury: experience from a multicenter, international randomized clinical trial.
abstract: OBJECTIVE
  Accurate and consistent outcome assessment is essential to randomized clinical trials. We aimed to explore observer variation in the assessment of outcome in a recently completed trial of dexanabinol in head injury and to consider steps to reduce such variation.
  
  
  METHODS
  Eight hundred sixty-one patients with severe traumatic brain injury who were admitted to 86 centers were included in a multicenter, placebo-controlled, Phase III trial. Outcome was assessed at 3 and 6 months postinjury using the extended Glasgow Outcome Scale; standardized assessment was facilitated by the use of a structured interview. Before initiation of trial centers, outcome ratings were obtained for sample cases to establish initial levels of agreement. Training sessions in outcome assessment were held, and problems in assigning outcome were investigated. During the trial, a process of central review was established to monitor performance. Interobserver variation was analyzed using the kappa statistic.
  
  
  RESULTS
  Substantial observer variation was found in the rating of sample cases (weighted kappa, 0.72; confidence interval, 0.68-0.75) and in assigning outcome based on completed structured interviews (weighted kappa, 0.61; confidence interval, 0.57-0.64). In the early stages of the trial, a relatively large number of discrepancies (29-37%) were identified on central review. This number declined as the trial progressed and coincided with investigator training and feedback from central review. Centers with higher enrollment rates showed better performance.
  
  
  CONCLUSIONS
  Observer variation in outcome assessment is a significant problem for head injury trials. Consistency can be improved by standardizing procedures, training assessors, and monitoring the quality of assessments and providing feedback to interviewers.
authors.affiliation: Department of Psychology, University of Stirling, Stirling, Scotland, United Kingdom. J.T.L.Wilson@stir.ac.uk
authors.forename: J T Lindsay
authors.initials: JT
authors.lastname: Wilson
authors.forename: Frans J A
authors.initials: FJ
authors.lastname: Slieker
authors.forename: Valerie
authors.initials: V
authors.lastname: Legrand
authors.forename: Gordon
authors.initials: G
authors.lastname: Murray
authors.forename: Nino
authors.initials: N
authors.lastname: Stocchetti
authors.forename: Andrew I R
authors.initials: AI
authors.lastname: Maas
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D001930:Brain Injuries / Q000175:diagnosis* / Q000453:epidemiology* / Q000628:therapy; D005260:Female; D006801:Humans; D038622:Internationality; D008297:Male; D008875:Middle Aged; D017063:Outcome Assessment, Health Care / Q000379:methods* / Q000706:statistics & numerical data*; D011379:Prognosis; D015203:Reproducibility of Results; D018570:Risk Assessment / Q000379:methods*; D012307:Risk Factors; D012680:Sensitivity and Specificity; D016896:Treatment Outcome