title: A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images.
abstract: A collaborative framework was initiated to establish a community resource of ground truth segmentations from cardiac MRI. Multi-site, multi-vendor cardiac MRI datasets comprising 95 patients (73 men, 22 women; mean age 62.73±11.24years) with coronary artery disease and prior myocardial infarction, were randomly selected from data made available by the Cardiac Atlas Project (Fonseca et al., 2011). Three semi- and two fully-automated raters segmented the left ventricular myocardium from short-axis cardiac MR images as part of a challenge introduced at the STACOM 2011 MICCAI workshop (Suinesiaputra et al., 2012). Consensus myocardium images were generated based on the Expectation-Maximization principle implemented by the STAPLE algorithm (Warfield et al., 2004). The mean sensitivity, specificity, positive predictive and negative predictive values ranged between 0.63 and 0.85, 0.60 and 0.98, 0.56 and 0.94, and 0.83 and 0.92, respectively, against the STAPLE consensus. Spatial and temporal agreement varied in different amounts for each rater. STAPLE produced high quality consensus images if the region of interest was limited to the area of discrepancy between raters. To maintain the quality of the consensus, an objective measure based on the candidate automated rater performance distribution is proposed. The consensus segmentation based on a combination of manual and automated raters were more consistent than any particular rater, even those with manual input. The consensus is expected to improve with the addition of new automated contributions. This resource is open for future contributions, and is available as a test bed for the evaluation of new segmentation algorithms, through the Cardiac Atlas Project (www.cardiacatlas.org).
authors.affiliation: Department of Anatomy with Radiology, University of Auckland, New Zealand. Electronic address: a.suinesiaputra@auckland.ac.nz.
authors.forename: Avan
authors.initials: A
authors.lastname: Suinesiaputra
authors.forename: Brett R
authors.initials: BR
authors.lastname: Cowan
authors.forename: Ahmed O
authors.initials: AO
authors.lastname: Al-Agamy
authors.forename: Mustafa A
authors.initials: MA
authors.lastname: Elattar
authors.forename: Nicholas
authors.initials: N
authors.lastname: Ayache
authors.forename: Ahmed S
authors.initials: AS
authors.lastname: Fahmy
authors.forename: Ayman M
authors.initials: AM
authors.lastname: Khalifa
authors.forename: Pau
authors.initials: P
authors.lastname: Medrano-Gracia
authors.forename: Marie-Pierre
authors.initials: MP
authors.lastname: Jolly
authors.forename: Alan H
authors.initials: AH
authors.lastname: Kadish
authors.forename: Daniel C
authors.initials: DC
authors.lastname: Lee
authors.forename: Ján
authors.initials: J
authors.lastname: Margeta
authors.forename: Simon K
authors.initials: SK
authors.lastname: Warfield
authors.forename: Alistair A
authors.initials: AA
authors.lastname: Young
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003324:Coronary Artery Disease / Q000150:complications / Q000473:pathology*; D005260:Female; D006352:Heart Ventricles / Q000473:pathology*; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D016013:Likelihood Functions; D019028:Magnetic Resonance Imaging, Cine / Q000379:methods*; D008297:Male; D008875:Middle Aged; D010363:Pattern Recognition, Automated / Q000379:methods*; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D013382:Subtraction Technique; D018487:Ventricular Dysfunction, Left / Q000209:etiology / Q000473:pathology*
keywords: Consensus images; LV myocardium; Segmentation challenge