title: Agreement in computer-assisted manual scoring of polysomnograms across sleep centers.
abstract: OBJECTIVE
  To determine intersite agreement in respiratory event scoring of polysomnograms (PSGs) using different hypopnea definitions.
  
  
  METHODS
  Technical assessment.
  
  
  METHODS
  Five academic medical centers.
  
  
  METHODS
  N/A.
  
  
  METHODS
  N/A.
  
  
  RESULTS
  Seventy good-quality PSGs performed in middle-aged women were manually scored by two experienced technologists at each of the five sleep centers using the particular laboratory's own software system. Studies were scored once by each scorer using American Academy of Sleep Medicine (AASM) standards for scoring sleep stages, arousals, and apneas. Hypopneas were then scored using three different AASM criteria: recommended, alternate, and research (Chicago). Means of each PSG variable for the scorers at each site were used to calculate an across-site intraclass correlation coefficient (ICC). Average AHI across the 10 scorers was 7.4 ± 12.3 (standard deviation) events/h using recommended criteria (ICC 0.984; 95% confidence interval [CI] 0.977-0.990), 12.1 ± 13.3 events/h using alternate criteria (ICC 0.947; 95% CI 0.889-0.972), and 15.1 ± 13.9 events/h with Chicago criteria (ICC 0.800; 95% CI 0.768-0.828). ICC across sites was 0.870 (95% CI = 0.847-0.889) for total sleep time, 0.861 (95% CI 0.837-0.881) for number of obstructive apneas and 0.683 (95% CI 0.640-0.722) for number of central apneas. ICCs across sites for hypopneas were very good using recommended criteria (ICC 0.843; 95% CI 0.820-0.870) but decreased when alternate criteria (ICC 0.728; 95% CI 0.689-0.763) and Chicago criteria (ICC 0.535; 95% CI 0.485-0.583) were used.
  
  
  CONCLUSIONS
  Experienced scorers at different laboratories have very good agreement in hypopnea and AHI results when good-quality PSGs are scored using AASM-recommended criteria. Substantial degradation of reliability was observed for alternative definitions of hypopneas, particularly that proposed for research.
authors.affiliation: Department of Medicine and Center for Sleep and Circadian Neurobiology, University of Pennsylvania, Philadelphia, PA, USA. skuna@mail.med.upenn.edu
authors.forename: Samuel T
authors.initials: ST
authors.lastname: Kuna
authors.forename: Ruth
authors.initials: R
authors.lastname: Benca
authors.forename: Clete A
authors.initials: CA
authors.lastname: Kushida
authors.forename: James
authors.initials: J
authors.lastname: Walsh
authors.forename: Magdy
authors.initials: M
authors.lastname: Younes
authors.forename: Bethany
authors.initials: B
authors.lastname: Staley
authors.forename: Alexandra
authors.initials: A
authors.lastname: Hanlon
authors.forename: Allan I
authors.initials: AI
authors.lastname: Pack
authors.forename: Grace W
authors.initials: GW
authors.lastname: Pien
authors.forename: Atul
authors.initials: A
authors.lastname: Malhotra
mesh_terms: D000328:Adult; D003936:Diagnosis, Computer-Assisted / Q000379:methods / Q000592:standards*; D005260:Female; D006801:Humans; D008875:Middle Aged; D015588:Observer Variation; D017286:Polysomnography / Q000379:methods* / Q000592:standards*; D015203:Reproducibility of Results; D012891:Sleep Apnea Syndromes / Q000175:diagnosis*; D012984:Software
keywords: Apnea-hypopnea index; polysomnography; reliability; scoring
references.citation: Am J Respir Crit Care Med. 1998 Aug;158(2):358-62
references.pmid: 9700107
references.citation: J Clin Sleep Med. 2007 Mar 15;3(2):121-31
references.pmid: 17557422
references.citation: J Clin Anesth. 1997 May;9(3):184-8
references.pmid: 9172023
references.citation: Am J Respir Crit Care Med. 2000 Feb;161(2 Pt 1):369-74
references.pmid: 10673173
references.citation: Sleep. 2009 Feb;32(2):150-7
references.pmid: 19238801
references.citation: Am J Respir Crit Care Med. 2011 May 1;183(9):1238-44
references.pmid: 21471093
references.citation: Sleep. 2011 Jan 01;34(1):73-81
references.pmid: 21203376
references.citation: Sleep. 1999 Aug 1;22(5):667-89
references.pmid: 10450601
references.citation: Biometrics. 1977 Mar;33(1):159-74
references.pmid: 843571
references.citation: Sleep Med. 2002 Jan;3(1):43-7
references.pmid: 14592253
references.citation: Sleep. 1998 Nov 1;21(7):749-57
references.pmid: 11286351
references.citation: Chest. 2005 Jan;127(1):80-8
references.pmid: 15653966
references.citation: Sleep. 1999 Nov 1;22(7):916-20
references.pmid: 10566909
references.citation: Sleep. 1989 Aug;12(4):354-62
references.pmid: 2762689
references.citation: Sleep. 1998 Nov 1;21(7):759-67
references.pmid: 11300121
references.citation: Sleep. 2013 Apr 01;36(4):591-6
references.pmid: 23565005
references.citation: Sleep. 2010 Feb;33(2):260-6
references.pmid: 20175410
references.citation: Sleep. 2013 Apr 01;36(4):573-82
references.pmid: 23565003
references.citation: Sleep. 2000 Nov 1;23(7):901-8
references.pmid: 11083599
references.citation: Diabetes Care. 2009 Jun;32(6):1017-9
references.pmid: 19279303
references.citation: Electroencephalogr Clin Neurophysiol. 1992 May;82(5):391-3
references.pmid: 1374708
references.citation: J Sleep Res. 2009 Mar;18(1):74-84
references.pmid: 19250176