title: The 2014 liver ultrasound tracking benchmark.
abstract: The Challenge on Liver Ultrasound Tracking (CLUST) was held in conjunction with the MICCAI 2014 conference to enable direct comparison of tracking methods for this application. This paper reports the outcome of this challenge, including setup, methods, results and experiences. The database included 54 2D and 3D sequences of the liver of healthy volunteers and tumor patients under free breathing. Participants had to provide the tracking results of 90% of the data (test set) for pre-defined point-landmarks (healthy volunteers) or for tumor segmentations (patient data). In this paper we compare the best six methods which participated in the challenge. Quantitative evaluation was performed by the organizers with respect to manual annotations. Results of all methods showed a mean tracking error ranging between 1.4 mm and 2.1 mm for 2D points, and between 2.6 mm and 4.6 mm for 3D points. Fusing all automatic results by considering the median tracking results, improved the mean error to 1.2 mm (2D) and 2.5 mm (3D). For all methods, the performance is still not comparable to human inter-rater variability, with a mean tracking error of 0.5-0.6 mm (2D) and 1.2-1.8 mm (3D). The segmentation task was fulfilled only by one participant, resulting in a Dice coefficient ranging from 76.7% to 92.3%. The CLUST database continues to be available and the online leader-board will be updated as an ongoing challenge.
authors.affiliation: Computer Vision Lab, ETH Zurich, 8092 Zurich, Switzerland.
authors.forename: V
authors.initials: V
authors.lastname: De Luca
authors.forename: T
authors.initials: T
authors.lastname: Benz
authors.forename: S
authors.initials: S
authors.lastname: Kondo
authors.forename: L
authors.initials: L
authors.lastname: König
authors.forename: D
authors.initials: D
authors.lastname: Lübke
authors.forename: S
authors.initials: S
authors.lastname: Rothlübbers
authors.forename: O
authors.initials: O
authors.lastname: Somphone
authors.forename: S
authors.initials: S
authors.lastname: Allaire
authors.forename: M A
authors.initials: MA
authors.lastname: Lediju Bell
authors.forename: D Y F
authors.initials: DY
authors.lastname: Chung
authors.forename: A
authors.initials: A
authors.lastname: Cifor
authors.forename: C
authors.initials: C
authors.lastname: Grozea
authors.forename: M
authors.initials: M
authors.lastname: Günther
authors.forename: J
authors.initials: J
authors.lastname: Jenne
authors.forename: T
authors.initials: T
authors.lastname: Kipshagen
authors.forename: M
authors.initials: M
authors.lastname: Kowarschik
authors.forename: N
authors.initials: N
authors.lastname: Navab
authors.forename: J
authors.initials: J
authors.lastname: Rühaak
authors.forename: J
authors.initials: J
authors.lastname: Schwaab
authors.forename: C
authors.initials: C
authors.lastname: Tanner
mesh_terms: D000465:Algorithms; D019985:Benchmarking; D016022:Case-Control Studies; D003226:Congresses as Topic; D016208:Databases, Factual / Q000592:standards*; D006801:Humans; D007091:Image Processing, Computer-Assisted; D021621:Imaging, Three-Dimensional / Q000592:standards*; D008099:Liver / Q000000981:diagnostic imaging*; D008113:Liver Neoplasms / Q000000981:diagnostic imaging* / Q000473:pathology; D008279:Magnetic Resonance Imaging / Q000592:standards*; D012119:Respiration; D014463:Ultrasonography / Q000592:standards*