title: Comparison of manual sleep staging with automated neural network-based analysis in clinical practice.
abstract: We have compared sleep staging by an automated neural network (ANN) system, BioSleep (Oxford BioSignals) and a human scorer using the Rechtschaffen and Kales scoring system. Sleep study recordings from 114 patients with suspected obstructed sleep apnoea syndrome (OSA) were analysed by ANN and by a blinded human scorer. We also examined human scorer reliability by calculating the agreement between the index scorer and a second independent blinded scorer for 28 of the 114 studies. For each study, we built contingency tables on an epoch-by-epoch (30 s epochs) comparison basis. From these, we derived kappa (kappa) coefficients for different combinations of sleep stages. The overall agreement of automatic and manual scoring for the 114 studies for the classification {wake / light-sleep / deep-sleep / REM} was poor (median kappa = 0.305) and only a little better (kappa = 0.449) for the crude {wake / sleep} distinction. For the subgroup of 28 randomly selected studies, the overall agreement of automatic and manual scoring was again relatively low (kappa = 0.331 for {wake light-sleep / deep-sleep REM} and kappa = 0.505 for {wake / sleep}), whereas inter-scorer reliability was higher (kappa = -0.641 for {wake / light-sleep / deep-sleep / REM} and kappa = 0.737 for {wake / sleep}). We conclude that such an ANN-based analysis system is not sufficiently accurate for sleep study analyses using the R&K classification system.
authors.affiliation: Medical Physics, Freeman Hospital, NE7 7DN Newcastle upon Tyne, England.
authors.forename: Jennifer
authors.initials: J
authors.lastname: Caffarel
authors.forename: G John
authors.initials: GJ
authors.lastname: Gibson
authors.forename: J Phil
authors.initials: JP
authors.lastname: Harrison
authors.forename: Clive J
authors.initials: CJ
authors.lastname: Griffiths
authors.forename: Michael J
authors.initials: MJ
authors.lastname: Drinnan
mesh_terms: D000328:Adult; D003936:Diagnosis, Computer-Assisted; D001330:Electronic Data Processing; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D016571:Neural Networks, Computer; D017286:Polysomnography; D012680:Sensitivity and Specificity; D016037:Single-Blind Method; D012891:Sleep Apnea Syndromes / Q000175:diagnosis*; D012894:Sleep Stages