title: Evaluation of current algorithms for segmentation of scar tissue from late gadolinium enhancement cardiovascular magnetic resonance of the left atrium: an open-access grand challenge.
abstract: BACKGROUND
  Late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging can be used to visualise regions of fibrosis and scarring in the left atrium (LA) myocardium. This can be important for treatment stratification of patients with atrial fibrillation (AF) and for assessment of treatment after radio frequency catheter ablation (RFCA). In this paper we present a standardised evaluation benchmarking framework for algorithms segmenting fibrosis and scar from LGE CMR images. The algorithms reported are the response to an open challenge that was put to the medical imaging community through an ISBI (IEEE International Symposium on Biomedical Imaging) workshop.
  
  
  METHODS
  The image database consisted of 60 multicenter, multivendor LGE CMR image datasets from patients with AF, with 30 images taken before and 30 after RFCA for the treatment of AF. A reference standard for scar and fibrosis was established by merging manual segmentations from three observers. Furthermore, scar was also quantified using 2, 3 and 4 standard deviations (SD) and full-width-at-half-maximum (FWHM) methods. Seven institutions responded to the challenge: Imperial College (IC), Mevis Fraunhofer (MV), Sunnybrook Health Sciences (SY), Harvard/Boston University (HB), Yale School of Medicine (YL), King's College London (KCL) and Utah CARMA (UTA, UTB). There were 8 different algorithms evaluated in this study.
  
  
  RESULTS
  Some algorithms were able to perform significantly better than SD and FWHM methods in both pre- and post-ablation imaging. Segmentation in pre-ablation images was challenging and good correlation with the reference standard was found in post-ablation images. Overlap scores (out of 100) with the reference standard were as follows: Pre: IC = 37, MV = 22, SY = 17, YL = 48, KCL = 30, UTA = 42, UTB = 45; Post: IC = 76, MV = 85, SY = 73, HB = 76, YL = 84, KCL = 78, UTA = 78, UTB = 72.
  
  
  CONCLUSIONS
  The study concludes that currently no algorithm is deemed clearly better than others. There is scope for further algorithmic developments in LA fibrosis and scar quantification from LGE CMR images. Benchmarking of future scar segmentation algorithms is thus important. The proposed benchmarking framework is made available as open-source and new participants can evaluate their algorithms via a web-based interface.
authors.affiliation: Department of Imaging Sciences & Biomedical Engineering, King's College London, London, UK. rashed.karim@kcl.ac.uk.
authors.forename: Rashed
authors.initials: R
authors.lastname: Karim
authors.forename: R James
authors.initials: RJ
authors.lastname: Housden
authors.forename: Mayuragoban
authors.initials: M
authors.lastname: Balasubramaniam
authors.forename: Zhong
authors.initials: Z
authors.lastname: Chen
authors.forename: Daniel
authors.initials: D
authors.lastname: Perry
authors.forename: Ayesha
authors.initials: A
authors.lastname: Uddin
authors.forename: Yosra
authors.initials: Y
authors.lastname: Al-Beyatti
authors.forename: Ebrahim
authors.initials: E
authors.lastname: Palkhi
authors.forename: Prince
authors.initials: P
authors.lastname: Acheampong
authors.forename: Samantha
authors.initials: S
authors.lastname: Obom
authors.forename: Anja
authors.initials: A
authors.lastname: Hennemuth
authors.forename: Yingli
authors.initials: Y
authors.lastname: Lu
authors.forename: Wenjia
authors.initials: W
authors.lastname: Bai
authors.forename: Wenzhe
authors.initials: W
authors.lastname: Shi
authors.forename: Yi
authors.initials: Y
authors.lastname: Gao
authors.forename: Heinz-Otto
authors.initials: HO
authors.lastname: Peitgen
authors.forename: Perry
authors.initials: P
authors.lastname: Radau
authors.forename: Reza
authors.initials: R
authors.lastname: Razavi
authors.forename: Allen
authors.initials: A
authors.lastname: Tannenbaum
authors.forename: Daniel
authors.initials: D
authors.lastname: Rueckert
authors.forename: Josh
authors.initials: J
authors.lastname: Cates
authors.forename: Tobias
authors.initials: T
authors.lastname: Schaeffter
authors.forename: Dana
authors.initials: D
authors.lastname: Peters
authors.forename: Rob
authors.initials: R
authors.lastname: MacLeod
authors.forename: Kawal
authors.initials: K
authors.lastname: Rhode
mesh_terms: D000465:Algorithms; D001281:Atrial Fibrillation / Q000175:diagnosis* / Q000473:pathology; D019985:Benchmarking; D002921:Cicatrix / Q000175:diagnosis* / Q000473:pathology; D003287:Contrast Media; D016208:Databases, Factual; D005060:Europe; D005355:Fibrosis; D006325:Heart Atria / Q000473:pathology*; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000592:standards; D008279:Magnetic Resonance Imaging / Q000592:standards; D015588:Observer Variation; D011237:Predictive Value of Tests; D015203:Reproducibility of Results; D014481:United States
chemical_list: D003287:Contrast Media