title: Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset.
abstract: OBJECTIVE
  This paper presents benchmarking results of human epithelial type 2 (HEp-2) interphase cell image classification methods on a very large dataset. The indirect immunofluorescence method applied on HEp-2 cells has been the gold standard to identify connective tissue diseases such as systemic lupus erythematosus and Sjögren's syndrome. However, the method suffers from numerous issues such as being subjective, time consuming and labor intensive. This has been the main motivation for the development of various computer-aided diagnosis systems whose main task is to automatically classify a given cell image into one of the predefined classes.
  
  
  METHODS
  The benchmarking was performed in the form of an international competition held in conjunction with the International Conference of Image Processing in 2013: fourteen teams, composed of practitioners and researchers in this area, took part in the initiative. The system developed by each team was trained and tested on a very large HEp-2 cell dataset comprising over 68,000 images of HEp-2 cell. The dataset contains cells with six different staining patterns and two levels of fluorescence intensity. For each method we provide a brief description highlighting the design choices and an in-depth analysis on the benchmarking results.
  
  
  RESULTS
  The staining pattern recognition accuracy attained by the methods varies between 47.91% and slightly above 83.65%. However, the difference between the top performing method and the seventh ranked method is only 5%. In the paper, we also study the performance achieved by fusing the best methods, finding that a recognition rate of 85.60% is reached when the top seven methods are employed.
  
  
  CONCLUSIONS
  We found that highest performance is obtained when using a strong classifier (typically a kernelised support vector machine) in conjunction with features extracted from local statistics. Furthermore, the misclassification profiles of the different methods highlight that some staining patterns are intrinsically more difficult to recognize. We also noted that performance is strongly affected by the fluorescence intensity level. Thus, low accuracy is to be expected when analyzing low contrasted images.
authors: Sullivan Nicolaides Pathology, 134 Whitmore street, Taringa, Queensland 4068, Australia. Electronic address: peter_hobson@snp.com.au.
authors: Peter
authors: P
authors: Hobson
authors: School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Queensland 4072, Australia. Electronic address: lovell@itee.uq.edu.au.
authors: Brian C
authors: BC
authors: Lovell
authors: Department of Information Engineering, Electrical Engineering and Applied Mathematics, University of Salerno, Via Giovanni Paolo II, 132, Fisciano, SA I-84084, Italy. Electronic address: pergen@unisa.it.
authors: Gennaro
authors: G
authors: Percannella
authors: Department of Information Engineering, Electrical Engineering and Applied Mathematics, University of Salerno, Via Giovanni Paolo II, 132, Fisciano, SA I-84084, Italy. Electronic address: mvento@unisa.it.
authors: Mario
authors: M
authors: Vento
authors: School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Queensland 4072, Australia. Electronic address: a.wiliem@uq.edu.au.
authors: Arnold
authors: A
authors: Wiliem
mesh_terms: D000465:Algorithms; D003240:Connective Tissue Diseases / Q000175:diagnosis*; D003936:Diagnosis, Computer-Assisted / Q000379:methods*; D004847:Epithelial Cells / Q000145:classification*; D019084:Fluorescent Antibody Technique, Indirect; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D007399:Interphase; D010363:Pattern Recognition, Automated / Q000379:methods*; D012680:Sensitivity and Specificity; D060388:Support Vector Machine
keywords: Computer-aided diagnosis systems; Hep-2 cell classification; Indirect immunofluorescence; Large-scale benchmarking