title: Does training improve diagnostic accuracy and inter-rater agreement in applying the Berlin radiographic definition of acute respiratory distress syndrome? A multicenter prospective study.
abstract: Poor inter-rater reliability in chest radiograph interpretation has been reported in the context of acute respiratory distress syndrome (ARDS), although not for the Berlin definition of ARDS. We sought to examine the effect of training material on the accuracy and consistency of intensivists' chest radiograph interpretations for ARDS diagnosis.
  
  
  
  We conducted a rater agreement study in which 286 intensivists (residents 41.3%, junior attending physicians 35.3%, and senior attending physician 23.4%) independently reviewed the same 12 chest radiographs developed by the ARDS Definition Task Force ("the panel") before and after training. Radiographic diagnoses by the panel were classified into the consistent (n = 4), equivocal (n = 4), and inconsistent (n = 4) categories and were used as a reference. The 1.5-hour training course attended by all 286 intensivists included introduction of the diagnostic rationale, and a subsequent in-depth discussion to reach consensus for all 12 radiographs.
  
  
  
  Overall diagnostic accuracy, which was defined as the percentage of chest radiographs that were interpreted correctly, improved but remained poor after training (42.0 ± 14.8% before training vs. 55.3 ± 23.4% after training, p < 0.001). Diagnostic sensitivity and specificity improved after training for all diagnostic categories (p < 0.001), with the exception of specificity for the equivocal category (p = 0.883). Diagnostic accuracy was higher for the consistent category than for the inconsistent and equivocal categories (p < 0.001). Comparisons of pre-training and post-training results revealed that inter-rater agreement was poor and did not improve after training, as assessed by overall agreement (0.450 ± 0.406 vs. 0.461 ± 0.575, p = 0.792), Fleiss's kappa (0.133 ± 0.575 vs. 0.178 ± 0.710, p = 0.405), and intraclass correlation coefficient (ICC; 0.219 vs. 0.276, p = 0.470).
  
  
  
  The radiographic diagnostic accuracy and inter-rater agreement were poor when the Berlin radiographic definition was used, and were not significantly improved by the training set of chest radiographs developed by the ARDS Definition Task Force.
  
  
  
  The study was registered at ClinicalTrials.gov (registration number NCT01704066 ) on 6 October 2012.
authors.affiliation: Medical ICU, Peking Union Medical College Hospital, 1 Shuai Fu Yuan, Beijing, 100730, People's Republic of China.
authors.forename: Jin-Min
authors.initials: JM
authors.lastname: Peng
authors.affiliation: Department of Emergency Medicine, The First Affiliated Hospital of Kunming Medical University, 295 Xichang Street, Kunming, 650032, People's Republic of China.
authors.forename: Chuan-Yun
authors.initials: CY
authors.lastname: Qian
authors.affiliation: Department of Critical Care Medicine, First Affiliated Hospital, Xinjiang Medical University, 1 Liyushan Road, Urumqi, 830054, People's Republic of China.
authors.forename: Xiang-You
authors.initials: XY
authors.lastname: Yu
authors.affiliation: Department of Critical Care Medicine, The First Affiliated Hospital, Harbin Medical University, 23 Youzheng Street, Harbin, 150001, People's Republic of China.
authors.forename: Ming-Yan
authors.initials: MY
authors.lastname: Zhao
authors.affiliation: Department of Critical Care Medicine, Tongji Hospital, Tongji Medical College, Huazhong University of Science & Technology, 1095 Jiefang Road, Wuhan, 430030, People's Republic of China.
authors.forename: Shu-Sheng
authors.initials: SS
authors.lastname: Li
authors.affiliation: Department of Critical Care Medicine, The First Affiliated Hospital of China Medical University, 155 North Nanjing Street, Shenyang, 110001, People's Republic of China.
authors.forename: Xiao-Chun
authors.initials: XC
authors.lastname: Ma
authors.affiliation: Department of Critical Care Medicine, West China Hospital, Sichuan University, 37 Guoxue Alley, Chengdu, 610041, People's Republic of China.
authors.forename: Yan
authors.initials: Y
authors.lastname: Kang
authors.affiliation: Department of Critical Care Medicine, The First Affiliated Hospital, Chongqing Medical University, 1 Youyi Road, Yuanjiagang, Chongqing, 400016, People's Republic of China.
authors.forename: Fa-Chun
authors.initials: FC
authors.lastname: Zhou
authors.affiliation: Department of Critical Care Medicine, Hainan Provincial People's Hospital, No. 19 Xiuhua Road, Haikou, 570311, People's Republic of China.
authors.forename: Zhen-Yang
authors.initials: ZY
authors.lastname: He
authors.affiliation: Department of Critical Care Medicine, Guangdong General Hospital, 106 Zhongshan Er Road, Guangzhou, 510080, People's Republic of China.
authors.forename: Tie-He
authors.initials: TH
authors.lastname: Qin
authors.affiliation: Department of Emergency and Critical Care Medicine, The Second Hospital of Jilin University, 18 Ziqiang Street, Changchun, 130041, People's Republic of China.
authors.forename: Yong-Jie
authors.initials: YJ
authors.lastname: Yin
authors.affiliation: Department of Critical Care Medicine, Fuxing Hospital, Capital Medical University, A20 Fuxingmenwai Street, Beijing, 100038, People's Republic of China.
authors.forename: Li
authors.initials: L
authors.lastname: Jiang
authors.affiliation: Department of Critical Care Medicine, Hebei Medical University Fourth Hospital, 12 Jiankang Road, Shijiazhuang, 050011, People's Republic of China.
authors.forename: Zhen-Jie
authors.initials: ZJ
authors.lastname: Hu
authors.affiliation: Department of Critical Care Medicine, Zhejiang Provincial People's Hospital, 158 Shangtang Road, Hangzhou, 310014, People's Republic of China.
authors.forename: Ren-Hua
authors.initials: RH
authors.lastname: Sun
authors.affiliation: Department of Critical Care Medicine, The First Affiliated Hospital of Fujian Medical University, 20 Chazhong Road, Fuzhou, 350005, People's Republic of China.
authors.forename: Jian-Dong
authors.initials: JD
authors.lastname: Lin
authors.affiliation: Department of Critical Care Medicine, Beijing Tongren Hospital, Capital Medical University, 2 Chongwenmennei Street, Beijing, 100730, People's Republic of China.
authors.forename: Tong
authors.initials: T
authors.lastname: Li
authors.affiliation: Department of Critical Care Medicine, Qilu Hospital, Shandong University, 107 Wenhua Xi Road, Jinan, 250012, People's Republic of China.
authors.forename: Da-Wei
authors.initials: DW
authors.lastname: Wu
authors.affiliation: Department of Critical Care Medicine, Peking University People's Hospital, 11 Xizhimen South Street, Beijing, 100044, People's Republic of China.
authors.forename: You-Zhong
authors.initials: YZ
authors.lastname: An
authors.affiliation: Department of Critical Care Medicine, Xiangya Hospital, Central South University, 87 Xiangya Road, Changsha, 410008, People's Republic of China.
authors.forename: Yu-Hang
authors.initials: YH
authors.lastname: Ai
authors.affiliation: Department of Critical Care Medicine, The Affiliated Hospital of Inner Mongolia Medical College, 1 Tongdao North Street, Huhhot, 010050, People's Republic of China.
authors.forename: Li-Hua
authors.initials: LH
authors.lastname: Zhou
authors.affiliation: Department of Critical Care Medicine, Affiliated Hospital of Ningxia Medical University, 804 Shengli South Street, Yinchuan, 750004, People's Republic of China.
authors.forename: Xiang-Yuan
authors.initials: XY
authors.lastname: Cao
authors.affiliation: Surgical ICU, Department of Anesthesia, Xijing Hospital, 127 Chang Le Xi Road, Xi'an, 710032, People's Republic of China.
authors.forename: Xi-Jing
authors.initials: XJ
authors.lastname: Zhang
authors.affiliation: Surgical ICU, The First Affiliated Hospital of Zhengzhou University, 1 Jianshe Road, Zhengzhou, 450052, Henan, People's Republic of China.
authors.forename: Rong-Qing
authors.initials: RQ
authors.lastname: Sun
authors.affiliation: Ruijin Hospital, Shanghai Jiao Tong University, No. 197 Ruijin Er Road, Shanghai, 200025, People's Republic of China.
authors.forename: Er-Zhen
authors.initials: EZ
authors.lastname: Chen
authors.affiliation: Medical ICU, Peking Union Medical College Hospital, 1 Shuai Fu Yuan, Beijing, 100730, People's Republic of China. dubin98@gmail.com.
authors.forename: Bin
authors.initials: B
authors.lastname: Du
mesh_terms: D002983:Clinical Competence / Q000592:standards* / Q000706:statistics & numerical data; D005260:Female; D006801:Humans; D008297:Male; D015588:Observer Variation; D011446:Prospective Studies; D013902:Radiography, Thoracic / Q000379:methods* / Q000706:statistics & numerical data; D015203:Reproducibility of Results; D012128:Respiratory Distress Syndrome / Q000175:diagnosis* / Q000000981:diagnostic imaging; D013663:Teaching / Q000592:standards* / Q000706:statistics & numerical data
keywords: Acute respiratory distress syndrome; Chest radiograph; Diagnostic accuracy; Inter-rater variability