title: Interrater reliability among epilepsy centers: multicenter study of epilepsy surgery.
abstract: OBJECTIVE
  To measure the interrater reliability of presurgical testing and surgical decisions among epilepsy centers.
  
  
  METHODS
  Seven centers participating in an ongoing, prospective multicenter study of resective epilepsy surgery agreed to conform to a detailed protocol regarding presurgical evaluation and surgery. To assess quality assurance, each center independently reviewed 21 randomly selected surgical cases for preoperative study lateralization and localization, and surgical decisions. Interrater reliability was assessed by using intraclass correlation coefficients (ICCs), validated for use with multiple raters, and calculated in a two-way random model based on absolute agreement.
  
  
  RESULTS
  Agreement for ICC values: > or = 0.75, excellent; 0.60-0.74, good; 0.40-0.59, fair; < or = 0.39, poor. One center was excluded for missing data. Agreement was excellent for extracranial EEG lateralization (0.8039), magnetic resonance imaging (MRI) lateralization (0.9521) and localization (0.9130), Wada lateralization (0.9453), and intracranial EEG localization (0.7905). Agreement was good for extracranial EEG localization (0.7384) and neuropsychological testing lateralization (0.7178) and localization (0.6891). Consensus about the decision to perform intracranial monitoring was fair (0.5397), in part reflecting one center's tendency toward intracranial monitoring. Overall agreements on whether to perform surgery (0.8311) and specific surgery recommended (0.8164) were excellent.
  
  
  CONCLUSIONS
  High interrater reliability among six epilepsy centers was present for interpretation of most components of presurgical testing. Although consensus for the decision to perform intracranial monitoring was only fair, agreements for the ultimate decision about resective surgery and specific choice of resection were excellent. We believe that this study demonstrates the feasibility of implementing multicenter protocols for neurologic management, especially those involving localization, as well as protocols combining study results with clinical decision making.
authors.affiliation: Comprehensive Epilepsy Management Center and Departments of Neurology and Pediatrics of the Montefiore Medical Center and the Albert Einstein College of Medicine, Bronx, New York, USA.
authors.forename: Sheryl R
authors.initials: SR
authors.lastname: Haut
authors.forename: Anne T
authors.initials: AT
authors.lastname: Berg
authors.forename: Shlomo
authors.initials: S
authors.lastname: Shinnar
authors.forename: Hillel W
authors.initials: HW
authors.lastname: Cohen
authors.forename: Carl W
authors.initials: CW
authors.lastname: Bazil
authors.forename: Michael R
authors.initials: MR
authors.lastname: Sperling
authors.forename: John T
authors.initials: JT
authors.lastname: Langfitt
authors.forename: Steven V
authors.initials: SV
authors.lastname: Pacia
authors.forename: Thaddeus S
authors.initials: TS
authors.lastname: Walczak
authors.forename: Susan S
authors.initials: SS
authors.lastname: Spencer
mesh_terms: D004569:Electroencephalography / Q000706:statistics & numerical data; D004827:Epilepsy / Q000175:diagnosis* / Q000523:psychology / Q000601:surgery*; D005240:Feasibility Studies; D006801:Humans; D009483:Neuropsychological Tests / Q000706:statistics & numerical data; D019635:Neurosurgical Procedures / Q000706:statistics & numerical data*; D015588:Observer Variation; D011446:Prospective Studies; D014055:Tomography, Emission-Computed / Q000706:statistics & numerical data; D015899:Tomography, Emission-Computed, Single-Photon / Q000706:statistics & numerical data