title: Visual versus automated evaluation of chest computed tomography for the presence of chronic obstructive pulmonary disease.
abstract: BACKGROUND
  Incidental CT findings may provide an opportunity for early detection of chronic obstructive pulmonary disease (COPD), which may prove important in CT-based lung cancer screening setting. We aimed to determine the diagnostic performance of human observers to visually evaluate COPD presence on CT images, in comparison to automated evaluation using quantitative CT measures.
  
  
  METHODS
  This study was approved by the Dutch Ministry of Health and the institutional review board. All participants provided written informed consent. We studied 266 heavy smokers enrolled in a lung cancer screening trial. All subjects underwent volumetric inspiratory and expiratory chest computed tomography (CT). Pulmonary function testing was used as the reference standard for COPD. We evaluated the diagnostic performance of eight observers and one automated model based on quantitative CT measures.
  
  
  RESULTS
  The prevalence of COPD in the study population was 44% (118/266), of whom 62% (73/118) had mild disease. The diagnostic accuracy was 74.1% in the automated evaluation, and ranged between 58.3% and 74.3% for the visual evaluation of CT images. The positive predictive value was 74.3% in the automated evaluation, and ranged between 52.9% and 74.7% for the visual evaluation. Interobserver variation was substantial, even within the subgroup of experienced observers. Agreement within observers yielded kappa values between 0.28 and 0.68, regardless of the level of expertise. The agreement between the observers and the automated CT model showed kappa values of 0.12-0.35.
  
  
  CONCLUSIONS
  Visual evaluation of COPD presence on chest CT images provides at best modest accuracy and is associated with substantial interobserver variation. Automated evaluation of COPD subjects using quantitative CT measures appears superior to visual evaluation by human observers.
authors: Department of Radiology, University Medical Center Utrecht, Utrecht, The Netherlands. o.m.mets@umcutrecht.nl
authors: Onno M
authors: OM
authors: Mets
authors: Ewoud J
authors: EJ
authors: Smit
authors: Firdaus A A
authors: FA
authors: Mohamed Hoesein
authors: Hester A
authors: HA
authors: Gietema
authors: Reinoud P H
authors: RP
authors: Bokkers
authors: Mohamed
authors: M
authors: Attrach
authors: Saskia
authors: S
authors: van Amelsvoort-van de Vorst
authors: Ernst Th
authors: ET
authors: Scholten
authors: Constantinus F M
authors: CF
authors: Buckens
authors: Matthijs
authors: M
authors: Oudkerk
authors: Jan-Willem J
authors: JW
authors: Lammers
authors: Mathias
authors: M
authors: Prokop
authors: Pim A
authors: PA
authors: de Jong
mesh_terms: D000368:Aged; D001331:Automation; D006801:Humans; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D008297:Male; D008875:Middle Aged; D015588:Observer Variation; D029424:Pulmonary Disease, Chronic Obstructive / Q000000981:diagnostic imaging*; D013902:Radiography, Thoracic / Q000379:methods*; D014057:Tomography, X-Ray Computed / Q000379:methods*