title: Classification and evaluation strategies of auto-segmentation approaches for PET: Report of AAPM task group No. 211.
abstract: OBJECTIVE
  The purpose of this educational report is to provide an overview of the present state-of-the-art PET auto-segmentation (PET-AS) algorithms and their respective validation, with an emphasis on providing the user with help in understanding the challenges and pitfalls associated with selecting and implementing a PET-AS algorithm for a particular application.
  
  
  METHODS
  A brief description of the different types of PET-AS algorithms is provided using a classification based on method complexity and type. The advantages and the limitations of the current PET-AS algorithms are highlighted based on current publications and existing comparison studies. A review of the available image datasets and contour evaluation metrics in terms of their applicability for establishing a standardized evaluation of PET-AS algorithms is provided. The performance requirements for the algorithms and their dependence on the application, the radiotracer used and the evaluation criteria are described and discussed. Finally, a procedure for algorithm acceptance and implementation, as well as the complementary role of manual and auto-segmentation are addressed.
  
  
  RESULTS
  A large number of PET-AS algorithms have been developed within the last 20 years. Many of the proposed algorithms are based on either fixed or adaptively selected thresholds. More recently, numerous papers have proposed the use of more advanced image analysis paradigms to perform semi-automated delineation of the PET images. However, the level of algorithm validation is variable and for most published algorithms is either insufficient or inconsistent which prevents recommending a single algorithm. This is compounded by the fact that realistic image configurations with low signal-to-noise ratios (SNR) and heterogeneous tracer distributions have rarely been used. Large variations in the evaluation methods used in the literature point to the need for a standardized evaluation protocol.
  
  
  CONCLUSIONS
  Available comparison studies suggest that PET-AS algorithms relying on advanced image analysis paradigms provide generally more accurate segmentation than approaches based on PET activity thresholds, particularly for realistic configurations. However, this may not be the case for simple shape lesions in situations with a narrower range of parameters, where simpler methods may also perform well. Recent algorithms which employ some type of consensus or automatic selection between several PET-AS methods have potential to overcome the limitations of the individual methods when appropriately trained. In either case, accuracy evaluation is required for each different PET scanner and scanning and image reconstruction protocol. For the simpler, less robust approaches, adaptation to scanning conditions, tumor type, and tumor location by optimization of parameters is necessary. The results from the method evaluation stage can be used to estimate the contouring uncertainty. All PET-AS contours should be critically verified by a physician. A standard test, i.e., a benchmark dedicated to evaluating both existing and future PET-AS algorithms needs to be designed, to aid clinicians in evaluating and selecting PET-AS algorithms and to establish performance limits for their acceptance for clinical use. The initial steps toward designing and building such a standard are undertaken by the task group members.
authors.affiliation: INSERM, UMR 1101, LaTIM, University of Brest, IBSAM, Brest, France.
authors.forename: Mathieu
authors.initials: M
authors.lastname: Hatt
authors.affiliation: Université catholique de Louvain (IREC/MIRO) & FNRS, Brussels, 1200, Belgium.
authors.forename: John A
authors.initials: JA
authors.lastname: Lee
authors.affiliation: Memorial Sloan Kettering Cancer Center, New York, NY, 10065, USA.
authors.forename: Charles R
authors.initials: CR
authors.lastname: Schmidtlein
authors.affiliation: University of Michigan, Ann Arbor, MI, 48103, USA.
authors.forename: Issam El
authors.initials: IE
authors.lastname: Naqa
authors.affiliation: Sunnybrook Health Sciences Center, Toronto, ON, M4N 3M5, Canada.
authors.forename: Curtis
authors.initials: C
authors.lastname: Caldwell
authors.affiliation: University of Milano-Bicocca, Monza, Italy.
authors.forename: Elisabetta
authors.initials: E
authors.lastname: De Bernardi
authors.affiliation: Memorial Sloan Kettering Cancer Center, New York, NY, 10065, USA.
authors.forename: Wei
authors.initials: W
authors.lastname: Lu
authors.affiliation: University of North Carolina, Chapel Hill, NC, 27599, USA.
authors.forename: Shiva
authors.initials: S
authors.lastname: Das
authors.affiliation: Université catholique de Louvain (IREC/MIRO) & FNRS, Brussels, 1200, Belgium.
authors.forename: Xavier
authors.initials: X
authors.lastname: Geets
authors.affiliation: Université catholique de Louvain (IREC/MIRO) & FNRS, Brussels, 1200, Belgium.
authors.forename: Vincent
authors.initials: V
authors.lastname: Gregoire
authors.affiliation: University of Wisconsin, Madison, WI, 53705, USA.
authors.forename: Robert
authors.initials: R
authors.lastname: Jeraj
authors.affiliation: Peter MacCallum Cancer Centre, Melbourne, Australia.
authors.forename: Michael P
authors.initials: MP
authors.lastname: MacManus
authors.affiliation: MD Anderson Cancer Center, Houston, TX, 77030, USA.
authors.forename: Osama R
authors.initials: OR
authors.lastname: Mawlawi
authors.affiliation: Universitätsklinikum Freiburg, Freiburg, 79106, Germany.
authors.forename: Ursula
authors.initials: U
authors.lastname: Nestle
authors.affiliation: University of Texas Southwestern Medical Center, Dallas, TX, 75390, USA.
authors.forename: Andrei B
authors.initials: AB
authors.lastname: Pugachev
authors.affiliation: Memorial Sloan Kettering Cancer Center, New York, NY, 10065, USA.
authors.forename: Heiko
authors.initials: H
authors.lastname: Schöder
authors.affiliation: Turku University Hospital, Turku, 20521, Finland.
authors.forename: Tony
authors.initials: T
authors.lastname: Shepherd
authors.affiliation: School of Engineering, Cardiff University, Cardiff, Wales, United Kingdom.
authors.forename: Emiliano
authors.initials: E
authors.lastname: Spezi
authors.affiliation: INSERM, UMR 1101, LaTIM, University of Brest, IBSAM, Brest, France.
authors.forename: Dimitris
authors.initials: D
authors.lastname: Visvikis
authors.affiliation: Geneva University Hospital, Geneva, CH-1211, Switzerland.
authors.forename: Habib
authors.initials: H
authors.lastname: Zaidi
authors.affiliation: Memorial Sloan Kettering Cancer Center, New York, NY, 10065, USA.
authors.forename: Assen S
authors.initials: AS
authors.lastname: Kirov
mesh_terms: D000465:Algorithms; D006801:Humans; D007091:Image Processing, Computer-Assisted; D049268:Positron-Emission Tomography; D059629:Signal-To-Noise Ratio; D014057:Tomography, X-Ray Computed
keywords: PET segmentation; PET/CT; treatment assessment; treatment planning