title: Outcomes of endovascular treatments of aneurysms: observer variability and implications for interpreting case series and planning randomized trials.
abstract: OBJECTIVE
  Angiographic results are commonly used as a surrogate marker of success of coiling of intracranial aneurysms. Inter- and intraobserver agreement in judging angiographic results remain poorly characterized. Our goal was to offer such an evaluation of a grading scale commonly used to evaluate results of endovascular treatment of aneurysms.
  
  
  METHODS
  A portfolio of 90 angiographic images from 45 patients selected from the core lab data base of a randomized trial was sent to 12 observers on 2 occasions more than 3 months apart. The variability of a 3-value grading scale used to score angiographic results and of a final judgment regarding the presence of a recurrence was studied using κ statistics.
  
  
  RESULTS
  Ten participants responded once and 6 responded twice. Agreement was poor to moderate (κ = 0.28-0.5) for senior and junior observers judging angiographic results immediately or 12-18 months after treatment. Agreement reached a reassuring "substantial" (κ = 0.62) level, with a dichotomous presence-absence of a major recurrence, and intraobserver agreement was better in experienced core lab assessors.
  
  
  CONCLUSIONS
  There is an important variability in the assessment of angiographic outcomes of endovascular treatments, rendering comparisons between publications risky, if not invalid. A simple dichotomous judgment can be used as a surrogate outcome in randomized trials designed to assess the value of new endovascular devices.
authors.affiliation: Department of Neuroradiology, Centre Hospitalier de l'Université de Montréal, Montreal, Canada.
authors.forename: É
authors.initials: É
authors.lastname: Tollard
authors.forename: T E
authors.initials: TE
authors.lastname: Darsaut
authors.forename: F
authors.initials: F
authors.lastname: Bing
authors.forename: F
authors.initials: F
authors.lastname: Guilbert
authors.forename: G
authors.initials: G
authors.lastname: Gevry
authors.forename: J
authors.initials: J
authors.lastname: Raymond
mesh_terms: D000328:Adult; D000368:Aged; D002533:Cerebral Angiography; D057510:Endovascular Procedures / Q000295:instrumentation* / Q000379:methods*; D006801:Humans; D002532:Intracranial Aneurysm / Q000000981:diagnostic imaging* / Q000601:surgery*; D008297:Male; D061185:Mechanical Thrombolysis / Q000295:instrumentation*; D008875:Middle Aged; D015588:Observer Variation; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D016896:Treatment Outcome