Seizure detection: correlation of human experts.
OBJECTIVE
The description and application of a new, overlap-integral comparison method and the quantification of human vs. human accuracies that can be used as goals for algorithms.


METHODS
Four human experts marked ten 8 h electroencephalography (EEG) records from seizure patients. The seizures varied in origin and type, including complex partial, generalized absence, secondarily generalized and primary generalized tonic-clonic. The traditional any-overlap comparison method is used in addition to the overlap-integral method, which is sensitive to the correct placement of the seizure endpoints.


RESULTS
The number of events marked by each reader ranged from 57 to 77. The average any-overlap sensitivity and false positives per hour rate are 0.92 and 0.117. The average overlap-integral correlation, sensitivity and specificity are 0.80, 0.82 and 0.9926. As expected, the correspondence between readers is high, but confounding issues resulted in overlap-integral sensitivities less than 0.5 for 10% of the records. Seven percent of the any-overlap sensitivities are less than 0.5. A comparison of the methods by record shows that the overlap-integral specificity and the any-overlap false positive rate measure different features.


CONCLUSIONS
There was little variation between readers and they were essentially interchangeable. High seizure rate (many per hour), short seizure durations (<10 s) and long seizure durations (approximately 10 min) with ambiguous offsets can complicate the analysis and result in poor correlation. There may be any number of unmarked events in rigorously marked records and it may be preferable to use records from non-epilepsy patients to compute the false positive rate. The any-overlap and overlap-integral comparison methods are complementary.


CONCLUSIONS
Correlation between expert human readers can be low on some records, which will complicate testing of seizure detection algorithms.
Persyst Development Corporation, 1060 Sandretto Drive, Suite E2, Prescott, AZ 86305, USA. scottw@eeg-persyst.com
Scott B
SB
Wilson
Mark L
ML
Scheuer
Cheryl
C
Plummer
Bryan
B
Young
Steve
S
Pacia
D000465:Algorithms; D004569:Electroencephalography / Q000706:statistics & numerical data*; D017029:Epilepsy, Complex Partial / Q000175:diagnosis*; D004829:Epilepsy, Generalized / Q000175:diagnosis*; D006801:Humans; D015233:Models, Statistical; D009462:Neurology / Q000706:statistics & numerical data; D015588:Observer Variation; D012680:Sensitivity and Specificity