title: Performance of an automated polysomnography scoring system versus computer-assisted manual scoring.
abstract: OBJECTIVE
  Manual scoring of polysomnograms (PSG) is labor intensive and has considerable variance between scorers. Automation of scoring could reduce cost and improve reproducibility. The purpose of this study was to compare a new automated scoring system (YST-Limited, Winnipeg, Canada) with computer-assisted manual scoring.
  
  
  METHODS
  Technical assessment.
  
  
  METHODS
  Five academic medical centers.
  
  
  METHODS
  N/A.
  
  
  METHODS
  N/A.
  
  
  RESULTS
  Seventy PSG files were selected at University of Pennsylvania (Penn) and distributed to five US academic sleep centers. Two blinded technologists from each center scored each file. Automatic scoring was performed at Penn by a YST Limited technician using a laptop containing the software. Variables examined were sleep stages, arousals, and apnea-hypopnea index (AHI) using three methods of identifying hypopneas. Automatic scores were not edited and were compared to the average scores of the 10 technologists. Intraclass correlation coefficient (ICC) was obtained for the 70 pairs and compared to across-sites ICCs for manually scored results. ICCs for automatic versus manual scoring were > 0.8 for total sleep time, stage N2, and nonrapid eye movement arousals and > 0.9 for AHI scored by primary and secondary American Academy of Sleep Medicine criteria. ICCs for other variables were not as high but were comparable to the across-site ICCs for manually scored results.
  
  
  CONCLUSIONS
  The automatic system yielded results that were similar to those obtained by experienced technologists. Very good ICCs were obtained for many primary PSG outcome measures. This automated scoring software, particularly if supplemented with manual editing, may increase laboratory efficiency and standardize PSG scoring results within and across sleep centers.
authors.affiliation: Department of Medicine, Harvard University, Boston, MA, USA. amalhotra1@partners.org
authors.forename: Atul
authors.initials: A
authors.lastname: Malhotra
authors.forename: Magdy
authors.initials: M
authors.lastname: Younes
authors.forename: Samuel T
authors.initials: ST
authors.lastname: Kuna
authors.forename: Ruth
authors.initials: R
authors.lastname: Benca
authors.forename: Clete A
authors.initials: CA
authors.lastname: Kushida
authors.forename: James
authors.initials: J
authors.lastname: Walsh
authors.forename: Alexandra
authors.initials: A
authors.lastname: Hanlon
authors.forename: Bethany
authors.initials: B
authors.lastname: Staley
authors.forename: Allan I
authors.initials: AI
authors.lastname: Pack
authors.forename: Grace W
authors.initials: GW
authors.lastname: Pien
mesh_terms: D001143:Arousal / Q000502:physiology; D003936:Diagnosis, Computer-Assisted / Q000379:methods* / Q000592:standards*; D005260:Female; D006801:Humans; D008875:Middle Aged; D015588:Observer Variation; D017286:Polysomnography / Q000379:methods* / Q000592:standards*; D015203:Reproducibility of Results; D012891:Sleep Apnea Syndromes / Q000175:diagnosis*; D012894:Sleep Stages / Q000502:physiology; D012984:Software
keywords: apnea-hypopnea index; lung; polysomnography; reliability; scoring; sleep
references.citation: Lancet. 1986 Feb 8;1(8476):307-10
references.pmid: 2868172
references.citation: Am J Respir Crit Care Med. 1998 Aug;158(2):358-62
references.pmid: 9700107
references.citation: Sleep. 1989 Dec;12(6):550-8
references.pmid: 2595177
references.citation: Am J Respir Crit Care Med. 1998 May;157(5 Pt 1):1461-7
references.pmid: 9603124
references.citation: J Clin Sleep Med. 2007 Mar 15;3(2):109-20
references.pmid: 17557421
references.citation: J Neurosci Methods. 1984 Nov;12(1):49-56
references.pmid: 6513591
references.citation: Sleep. 2000 Nov 1;23(7):901-8
references.pmid: 11083599
references.citation: Sleep. 2004 Nov 1;27(7):1394-403
references.pmid: 15586793
references.citation: Sleep. 1999 Aug 1;22(5):667-89
references.pmid: 10450601
references.citation: Sleep. 2013 Apr 01;36(4):583-9
references.pmid: 23565004
references.citation: Sleep. 1998 Nov 1;21(7):749-57
references.pmid: 11286351
references.citation: JAMA. 2010 Jan 20;303(3):235-41
references.pmid: 20071471
references.citation: Sleep. 1999 Nov 1;22(7):916-20
references.pmid: 10566909
references.citation: Sleep. 1989 Aug;12(4):354-62
references.pmid: 2762689
references.citation: Ann Intern Med. 2007 Feb 6;146(3):157-66
references.pmid: 17283346
references.citation: Neuropsychobiology. 2005;51(3):115-33
references.pmid: 15838184
references.citation: Sleep. 2007 Nov;30(11):1562-74
references.pmid: 18041489
references.citation: N Engl J Med. 1993 Apr 29;328(17):1230-5
references.pmid: 8464434
references.citation: Neuropsychobiology. 2005;51(3):134-47
references.pmid: 15838185
references.citation: Neuropsychobiology. 2005;51(3):148-63
references.pmid: 15838186
references.citation: Neuropsychobiology. 2010;62(4):250-64
references.pmid: 20829636
references.citation: Sleep Med. 2002 Jan;3(1):43-7
references.pmid: 14592253
references.citation: Health Technol Assess. 2007 Dec;11(50):iii, ix-51
references.pmid: 18021577