title: Observer agreement in the assessment of endovascular aneurysm therapy and aneurysm recurrence.
abstract: OBJECTIVE
  Assessments of completeness of endovascular cerebral aneurysm therapy are commonly reported in the literature. We studied several aneurysm assessment scales with regard to observer variability, which directly affects validity of these scales.
  
  
  METHODS
  Initial aneurysm occlusion and occlusion at a follow-up angiogram at 3-6 months were assessed independently by 2 experienced observers. Assessments of each aneurysm were made using 3 different scales: 4-response (complete, dog ear, neck remnant, incomplete), 3-response (complete, near-complete, incomplete), and 2-response (complete or near-complete, incomplete). Assessments were also made of comparisons of initial treatment angiogram with follow-up angiogram using 2 different scales: 3-response (better, same, worse) and 2-point response (not worse, worse).
  
  
  RESULTS
  With assessments of both initial and follow-up angiograms, interobserver and intraobserver agreement was progressively worse with increasing response choices in the scales. Observer agreement on assessments of initial angiograms (kappa values 0.48-0.67) was worse than that for follow-up angiograms (kappa values 0.66-0.97). For the comparisons of the initial angiogram with the follow-up angiogram, there was worse observer agreement with the 3-response scale (kappa values 0.64-0.71) than with the 2-response scale (kappa values 0.78-0.89).
  
  
  CONCLUSIONS
  Interobserver and intraobserver variability are inherent to assessment scales of completeness of cerebral aneurysm therapy. Observer variability is substantially better in scales that offer fewer observer responses. However, scales with fewer observer responses may not identify aneurysm subgroups that have differing risks of recurrence and/or rehemorrhage.
authors.affiliation: Department of Radiology, Mayo Clinic, Rochester, MN 55905, USA. cloft.harry@mayo.edu
authors.forename: H J
authors.initials: HJ
authors.lastname: Cloft
authors.forename: T
authors.initials: T
authors.lastname: Kaufmann
authors.forename: D F
authors.initials: DF
authors.lastname: Kallmes
mesh_terms: D002533:Cerebral Angiography / Q000592:standards* / Q000706:statistics & numerical data; D004621:Embolization, Therapeutic; D005500:Follow-Up Studies; D006801:Humans; D002532:Intracranial Aneurysm / Q000000981:diagnostic imaging* / Q000628:therapy*; D015588:Observer Variation; D011183:Postoperative Complications / Q000000981:diagnostic imaging*; D012008:Recurrence; D015203:Reproducibility of Results