title: Process and outcome for international reliability in sleep scoring.
abstract: OBJECTIVE
  The aim was to evaluate the inter-rater reliability in scoring sleep stages in two sleep labs in Berlin Germany and Beijing China.
  
  
  METHODS
  The subjects consist of polysomnography (PSGs) from 15 subjects in a German sleep laboratory, with 7 mild to moderate sleep apnea hypopnea syndrome (SAHS) patients and 8 healthy controls, and PSGs from 15 narcolepsy patients in a Chinese sleep laboratory. Five experienced technologists including two Chinese and three Germans without common training scored the PSGs following the 2007 AASM manual except the EEG signals included only two EEG leads (C3/A2 and C4/A1). Differences in inter-scorer agreement were analyzed based on epoch-by-epoch comparison by means of Cohen's κ, and quantitative sleep parameters by means of intra-class correlation coefficients.
  
  
  RESULTS
  Inter-laboratory epoch-by-epoch agreement comparison between scorers from the two countries yielded a moderate agreement with a mean κ value of 0.57 for controls, 0.58 for SAHS, and 0.54 for narcolepsy. When compared with controls, the inter-scoring agreement is higher for wake and N3 stage scoring in SAHS and N1 and N3 scoring in narcolepsy (p < 0.05). The only sleep stage with lower scoring agreement in both SAHS (κ 0.69 vs. 0.79, p = 0.034) and narcolepsy (0.66 vs 0.79, p = 0.022) was stage REM. Inter-laboratory comparisons showed that the most common combinations of deviating scorings were N1 and N2, N2 and N3, and N1 and wake. A 6.5 % deviating scoring rate of wake and REM and a 13.4 % deviating scoring rate of N1 and REM indicated that inter-laboratory scoring in narcolepsy was about twice as in SAHS and controls confused. This was further confirmed by agreement analysis of quantitative parameters using intra-class correlation coefficients ICC(2,1) indicating REM sleep scoring agreement was lower in narcolepsy than in controls (p < 0.05).
  
  
  CONCLUSIONS
  Low REM stage scoring agreement exists for narcoleptics and SAHS, indicating the necessity to study sleep stage scoring agreement for a specific sleep disorder. Intensive training is needed for the scoring of sleep in international multiple center studies to improve the scoring agreement.
authors.affiliation: Department of Pulmonary Medicine, Peking University People's Hospital, No.11 Xizhimen South Street, Beijing, 100044, China.
authors.forename: Xiaozhe
authors.initials: X
authors.lastname: Zhang
authors.forename: Xiaosong
authors.initials: X
authors.lastname: Dong
authors.forename: Jan W
authors.initials: JW
authors.lastname: Kantelhardt
authors.forename: Jing
authors.initials: J
authors.lastname: Li
authors.forename: Long
authors.initials: L
authors.lastname: Zhao
authors.forename: Carmen
authors.initials: C
authors.lastname: Garcia
authors.forename: Martin
authors.initials: M
authors.lastname: Glos
authors.forename: Thomas
authors.initials: T
authors.lastname: Penzel
authors.forename: Fang
authors.initials: F
authors.lastname: Han
mesh_terms: D000328:Adult; D000368:Aged; D001604:Berlin; D002681:China; D003431:Cross-Cultural Comparison; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D009290:Narcolepsy / Q000145:classification* / Q000175:diagnosis*; D015588:Observer Variation; D010043:Outcome and Process Assessment, Health Care; D017286:Polysomnography / Q000145:classification*; D015203:Reproducibility of Results; D020181:Sleep Apnea, Obstructive / Q000145:classification* / Q000175:diagnosis*; D012894:Sleep Stages