title: Multicenter accuracy and interobserver agreement of spot sign identification in acute intracerebral hemorrhage.
abstract: OBJECTIVE
  Rapid, accurate, and reliable identification of the computed tomography angiography spot sign is required to identify patients with intracerebral hemorrhage for trials of acute hemostatic therapy. We sought to assess the accuracy and interobserver agreement for spot sign identification.
  
  
  METHODS
  A total of 131 neurology, emergency medicine, and neuroradiology staff and fellows underwent imaging certification for spot sign identification before enrolling patients in 3 trials targeting spot-positive intracerebral hemorrhage for hemostatic intervention (STOP-IT, SPOTLIGHT, STOP-AUST). Ten intracerebral hemorrhage cases (spot-positive/negative ratio, 1:1) were presented for evaluation of spot sign presence, number, and mimics. True spot positivity was determined by consensus of 2 experienced neuroradiologists. Diagnostic performance, agreement, and differences by training level were analyzed.
  
  
  RESULTS
  Mean accuracy, sensitivity, and specificity for spot sign identification were 87%, 78%, and 96%, respectively. Overall sensitivity was lower than specificity (P<0.001) because of true spot signs incorrectly perceived as spot mimics. Interobserver agreement for spot sign presence was moderate (k=0.60). When true spots were correctly identified, 81% correctly identified the presence of single or multiple spots. Median time needed to evaluate the presence of a spot sign was 1.9 minutes (interquartile range, 1.2-3.1 minutes). Diagnostic performance, interobserver agreement, and time needed for spot sign evaluation were similar among staff physicians and fellows.
  
  
  CONCLUSIONS
  Accuracy for spot identification is high with opportunity for improvement in spot interpretation sensitivity and interobserver agreement particularly through greater reliance on computed tomography angiography source data and awareness of limitations of multiplanar images. Further prospective study is needed.
authors.affiliation: From the Divisions of Neuroradiology (T.J.H., T.L.C., S.P.S., R.I.A.) and Division of Neurology, Department of Medicine, and Brain Sciences Program (D.J.G.), and Department of Emergency Medicine (J.C.), Sunnybrook Health Sciences and University of Toronto, Toronto, Canada; Department of Neurology, University of Cincinnati, OH (M.L.F., J.P.B.); Departments of Clinical Neurosciences and Radiology, Hotchkiss Brain Institute, University of Calgary, Calagary, Canada (A.M.D.); Department of Neurology, University of Ottawa and Ottawa Hospital Research Institute, Ottawa, Canada (D.D.); Departments of Medicine and Neurology (A.M., S.M.D.) and Neurointerventional Radiology (P.J.M.), Royal Melbourne Hospital, University of Melbourne, Melbourne, Australia; and Department of Public Health Sciences, University of Toronto, Toronto, Canada (G.A.T.).
authors.forename: Thien J
authors.initials: TJ
authors.lastname: Huynh
authors.forename: Matthew L
authors.initials: ML
authors.lastname: Flaherty
authors.forename: David J
authors.initials: DJ
authors.lastname: Gladstone
authors.forename: Joseph P
authors.initials: JP
authors.lastname: Broderick
authors.forename: Andrew M
authors.initials: AM
authors.lastname: Demchuk
authors.forename: Dar
authors.initials: D
authors.lastname: Dowlatshahi
authors.forename: Atte
authors.initials: A
authors.lastname: Meretoja
authors.forename: Stephen M
authors.initials: SM
authors.lastname: Davis
authors.forename: Peter J
authors.initials: PJ
authors.lastname: Mitchell
authors.forename: George A
authors.initials: GA
authors.lastname: Tomlinson
authors.forename: Jordan
authors.initials: J
authors.lastname: Chenkin
authors.forename: Tze L
authors.initials: TL
authors.lastname: Chia
authors.forename: Sean P
authors.initials: SP
authors.lastname: Symons
authors.forename: Richard I
authors.initials: RI
authors.lastname: Aviv
mesh_terms: D002543:Cerebral Hemorrhage / Q000175:diagnosis*; D002568:Certification; D002983:Clinical Competence; D016001:Confidence Intervals; D003627:Data Interpretation, Statistical; D004502:Education, Medical, Continuing / Q000379:methods*; D006801:Humans; D007091:Image Processing, Computer-Assisted; D020407:Internet; D059906:Neuroimaging; D015588:Observer Variation; D010820:Physicians; D015203:Reproducibility of Results; D014057:Tomography, X-Ray Computed
keywords: angiography; cerebral hemorrhage; diagnosis; multidetector computed tomography; stroke