title: Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge.
abstract: EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.
authors.affiliation: Image Sciences Institute, University Medical Center, Utrecht, The Netherlands.
authors.forename: Keelin
authors.initials: K
authors.lastname: Murphy
authors.forename: Bram
authors.initials: B
authors.lastname: van Ginneken
authors.forename: Joseph M
authors.initials: JM
authors.lastname: Reinhardt
authors.forename: Sven
authors.initials: S
authors.lastname: Kabus
authors.forename: Kai
authors.initials: K
authors.lastname: Ding
authors.forename: Xiang
authors.initials: X
authors.lastname: Deng
authors.forename: Kunlin
authors.initials: K
authors.lastname: Cao
authors.forename: Kaifang
authors.initials: K
authors.lastname: Du
authors.forename: Gary E
authors.initials: GE
authors.lastname: Christensen
authors.forename: Vincent
authors.initials: V
authors.lastname: Garcia
authors.forename: Tom
authors.initials: T
authors.lastname: Vercauteren
authors.forename: Nicholas
authors.initials: N
authors.lastname: Ayache
authors.forename: Olivier
authors.initials: O
authors.lastname: Commowick
authors.forename: Grégoire
authors.initials: G
authors.lastname: Malandain
authors.forename: Ben
authors.initials: B
authors.lastname: Glocker
authors.forename: Nikos
authors.initials: N
authors.lastname: Paragios
authors.forename: Nassir
authors.initials: N
authors.lastname: Navab
authors.forename: Vladlena
authors.initials: V
authors.lastname: Gorbunova
authors.forename: Jon
authors.initials: J
authors.lastname: Sporring
authors.forename: Marleen
authors.initials: M
authors.lastname: de Bruijne
authors.forename: Xiao
authors.initials: X
authors.lastname: Han
authors.forename: Mattias P
authors.initials: MP
authors.lastname: Heinrich
authors.forename: Julia A
authors.initials: JA
authors.lastname: Schnabel
authors.forename: Mark
authors.initials: M
authors.lastname: Jenkinson
authors.forename: Cristian
authors.initials: C
authors.lastname: Lorenz
authors.forename: Marc
authors.initials: M
authors.lastname: Modat
authors.forename: Jamie R
authors.initials: JR
authors.lastname: McClelland
authors.forename: Sébastien
authors.initials: S
authors.lastname: Ourselin
authors.forename: Sascha E A
authors.initials: SE
authors.lastname: Muenzing
authors.forename: Max A
authors.initials: MA
authors.lastname: Viergever
authors.forename: Dante
authors.initials: D
authors.lastname: De Nigris
authors.forename: D Louis
authors.initials: DL
authors.lastname: Collins
authors.forename: Tal
authors.initials: T
authors.lastname: Arbel
authors.forename: Marta
authors.initials: M
authors.lastname: Peroni
authors.forename: Rui
authors.initials: R
authors.lastname: Li
authors.forename: Gregory C
authors.initials: GC
authors.lastname: Sharp
authors.forename: Alexander
authors.initials: A
authors.lastname: Schmidt-Richberg
authors.forename: Jan
authors.initials: J
authors.lastname: Ehrhardt
authors.forename: René
authors.initials: R
authors.lastname: Werner
authors.forename: Dirk
authors.initials: D
authors.lastname: Smeets
authors.forename: Dirk
authors.initials: D
authors.lastname: Loeckx
authors.forename: Gang
authors.initials: G
authors.lastname: Song
authors.forename: Nicholas
authors.initials: N
authors.lastname: Tustison
authors.forename: Brian
authors.initials: B
authors.lastname: Avants
authors.forename: James C
authors.initials: JC
authors.lastname: Gee
authors.forename: Marius
authors.initials: M
authors.lastname: Staring
authors.forename: Stefan
authors.initials: S
authors.lastname: Klein
authors.forename: Berend C
authors.initials: BC
authors.lastname: Stoel
authors.forename: Martin
authors.initials: M
authors.lastname: Urschler
authors.forename: Manuel
authors.initials: M
authors.lastname: Werlberger
authors.forename: Jef
authors.initials: J
authors.lastname: Vandemeulebroucke
authors.forename: Simon
authors.initials: S
authors.lastname: Rit
authors.forename: David
authors.initials: D
authors.lastname: Sarrut
authors.forename: Josien P W
authors.initials: JP
authors.lastname: Pluim
mesh_terms: D000465:Algorithms; D000818:Animals; D016208:Databases, Factual; D008168:Lung / Q000000981:diagnostic imaging*; D015588:Observer Variation; D011856:Radiographic Image Enhancement; D011857:Radiographic Image Interpretation, Computer-Assisted / Q000379:methods*; D013902:Radiography, Thoracic / Q000379:methods*; D012015:Reference Standards; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D012756:Sheep; D012986:Software Validation; D013909:Thorax; D014057:Tomography, X-Ray Computed / Q000379:methods*