title: Inter-rater agreement of sputum cytology for lung cancer screening in Japan.
abstract: BACKGROUND
  To compare lung cancer detection rate by sputum cytology, we need some assurance that the estimates do not vary widely if different observers evaluate the same specimens. The aim of this study was to determine inter-rater agreement of sputum cytology diagnoses.
  
  
  METHODS
  Slides of sputum cytology from 150 subjects were selected from a pool of slides held by six of the laboratories that had participated in a population-based lung cancer screening program over the last ten years in Japan. The cytotechnologists in these laboratories had considerable experience with sputum cytology. Each case was re-evaluated six times. Cases that were diagnosed as the same category by all six laboratories were selected as consensus cases to serve as standardized sputum cytology cases. Thirty-seven cytotechnologists with various levels of experience in sputum cytology then re-evaluated these consensus cases. Inter-rater agreement was calculated by kappa statistics including Fleiss' kappa.
  
  
  RESULTS
  All pairs of interlaboratory agreement for the 150 cases showed statistically significant kappa values, most pairs showing substantial agreement. Fleiss' kappa value across the six laboratories was 0.5. Fourteen cases were identified as the consensus cases, and the agreement among observers with less experience of sputum cytology showed significantly lower than the agreement among those with considerable experience (Fleiss' kappa value 0.27 vs. 0.45, P < 0.05). Moreover, cytotechnologists with less experience under-diagnosed the slides significantly more often than those with considerable experience.
  
  
  CONCLUSIONS
  When the observers have considerable experience with sputum cytology, inter-observer agreement is good.
authors: Department of Thoracic Surgery, Tohoku University Hospital, Sendai, Japan.
authors: Chiaki
authors: C
authors: Endo
authors: Department of Cytology, Miyagi Cancer Society, Sendai, Japan.
authors: Ryutaro
authors: R
authors: Nakashima
authors: Department of Pathology and Cytology, Chiba Foundation for Health Promotion and Disease Prevention, Chiba, Japan.
authors: Akemi
authors: A
authors: Taguchi
authors: Department of Cytology, Osaka Medical Association, Osaka, Japan.
authors: Kazunobu
authors: K
authors: Yahata
authors: Department of Clinical Laboratory Science, Kanazawa University, Kanazawa, Japan.
authors: Ei
authors: E
authors: Kawahara
authors: Department of Cytology, Niigata Health Service Center, Niigata, Japan.
authors: Nikako
authors: N
authors: Shimagaki
authors: Department of Cytology, Fukushima Preservative Service Association of Health, Fukushima, Japan.
authors: Junko
authors: J
authors: Kamio
authors: Department of Thoracic Surgery, Sendai Medical Center, Sendai, Japan.
authors: Yasuki
authors: Y
authors: Saito
authors: Department of Surgery, Tokyo Medical University, Tokyo, Japan.
authors: Norihiko
authors: N
authors: Ikeda
authors: Department of General Thoracic Surgery, Graduate School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan.
authors: Masami
authors: M
authors: Sato
mesh_terms: D003581:Cytodiagnosis; D055088:Early Detection of Cancer; D006801:Humans; D007150:Immunohistochemistry; D007564:Japan; D059021:Laboratory Proficiency Testing / Q000706:statistics & numerical data*; D008168:Lung / Q000473:pathology*; D008175:Lung Neoplasms / Q000145:classification / Q000175:diagnosis* / Q000473:pathology; D015588:Observer Variation; D011786:Quality Control; D013183:Sputum / Q000166:cytology*
keywords: Fleiss’; interrater agreement; kappa; kappa statistics; lung cancer screening; quality assurance; sputum cytology