title: Visual and region of interest-based inter-rater agreement in the assessment of the diffusion-weighted imaging- fluid-attenuated inversion recovery mismatch.
abstract: OBJECTIVE
  WAKE-UP is a randomized, placebo-controlled MRI-based trial of thrombolysis in wake-up stroke using the mismatch between a lesion's visibility in diffusion-weighted imaging and fluid-attenuated inversion recovery (FLAIR) sequences as its main imaging inclusion criterion. Visual judgment of lesion conspicuity on FLAIR is however methodically limited by moderate inter-rater agreement. We therefore sought to improve rating homogeneity by incorporating quantitative signal intensity measurements.
  
  
  METHODS
  One hundred forty-three data sets of patients with acute ischemic stroke were visually rated by 8 raters with respect to WAKE-UP study inclusion and exclusion criteria, and inter-rater agreement was calculated. A subanalysis was performed on 45 cases to determine a threshold value of relative signal intensity (rSI) between the ischemic lesion and contralateral healthy tissue which best corresponded to a visually established verdict of FLAIR positivity. The usefulness of this threshold in improving inter-rater agreement was evaluated in an additional sample of 50 patients.
  
  
  RESULTS
  Inter-rater agreement for inclusion into the WAKE-UP trial was 73% with a free-marginal κ of 0.46. A threshold of rSI which best correlated with the visual rating of lesions as FLAIR positive was 1.20. The addition of rSI measurements to visual evaluation did not change the inter-rater agreement.
  
  
  CONCLUSIONS
  Introducing a semiquantitative measure for FLAIR rSI did not improve the agreement between individual raters. However, enhancing visual assessment with rSI measurements can provide reassurance to local investigators in cases of uncertainty.
authors.affiliation: From the Center for Stroke Research Berlin (CSB), Charité-Universitätsmedizin Berlin, Berlin, Germany (I.G., L.N., J.B.F.); Department of Radiology (IDI), Girona Biomedical Research Institute (IDIBGI), Hospital Universitari de Girona Dr Josep Trueta, Girona, Spain (J.P., J.G., S.P.); and Departments of Neurology (A.K., S.S., B.C., G.T.) and Diagnostic and Interventional Neuroradiology (J.F.), University-Medical Center Hamburg-Eppendorf, Hamburg, Germany.
authors.forename: Ivana
authors.initials: I
authors.lastname: Galinovic
authors.forename: Josep
authors.initials: J
authors.lastname: Puig
authors.forename: Lars
authors.initials: L
authors.lastname: Neeb
authors.forename: Jorge
authors.initials: J
authors.lastname: Guibernau
authors.forename: Andre
authors.initials: A
authors.lastname: Kemmling
authors.forename: Susanne
authors.initials: S
authors.lastname: Siemonsen
authors.forename: Salvador
authors.initials: S
authors.lastname: Pedraza
authors.forename: Bastian
authors.initials: B
authors.lastname: Cheng
authors.forename: Götz
authors.initials: G
authors.lastname: Thomalla
authors.forename: Jens
authors.initials: J
authors.lastname: Fiehler
authors.forename: Jochen B
authors.initials: JB
authors.lastname: Fiebach
mesh_terms: D000368:Aged; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D000208:Acute Disease; D002545:Brain Ischemia / Q000473:pathology; D038524:Diffusion Magnetic Resonance Imaging / Q000379:methods / Q000706:statistics & numerical data; D015588:Observer Variation; D020127:Recovery of Function; D020521:Stroke / Q000473:pathology; D015912:Thrombolytic Therapy / Q000379:methods; D014795:Visual Pathways / Q000473:pathology
keywords: magnetic resonance imaging; stroke