Early CT changes in patients admitted for thrombectomy: Intrarater and interrater agreement.
OBJECTIVE
To systematically review the literature and assess agreement on the Alberta Stroke Program Early CT Score (ASPECTS) among clinicians involved in the management of thrombectomy candidates.


METHODS
Studies assessing agreement using ASPECTS published from 2000 to 2015 were reviewed. Fifteen raters reviewed and scored the anonymized CT scans of 30 patients recruited in a local thrombectomy trial during 2 independent sessions, in order to study intrarater and interrater agreement. Agreement was measured using intraclass correlation coefficients (ICCs) and Fleiss kappa statistics for ASPECTS and dichotomized ASPECTS at various cutoff values.


RESULTS
The review yielded 30 articles reporting 40 measures of agreement. Populations, methods, analyses, and results were heterogeneous (slight to excellent agreement), precluding a meta-analysis. When analyzed as a categorical variable, intrarater agreement was slight to moderate (κ = 0.042-0.469); it reached a substantial level (κ > 0.6) in 11/15 raters when the score was dichotomized (0-5 vs 6-10). The interrater ICCs varied between 0.672 and 0.811, but agreement was slight to moderate (κ = 0.129-0.315). Even in the best of cases, when ASPECTS was dichotomized as 0-5 vs 6-10, interrater agreement did not reach a substantial level (κ = 0.561), which translates into at least 5 of 15 raters not giving the same dichotomized verdict in 15% of patients.


CONCLUSIONS
In patients considered for thrombectomy, there may be insufficient agreement between clinicians for ASPECTS to be reliably used as a criterion for treatment decisions.
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Behzad
B
Farzin
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Robert
R
Fahed
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Francois
F
Guilbert
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Alexandre Y
AY
Poppe
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Nicole
N
Daneault
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
André P
AP
Durocher
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Sylvain
S
Lanthier
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Hayet
H
Boudjani
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Naim N
NN
Khoury
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Daniel
D
Roy
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Alain
A
Weill
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Jean-Christophe
JC
Gentric
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
André L
AL
Batista
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Laurent
L
Létourneau-Guillon
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
François
F
Bergeron
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Marc-Antoine
MA
Henry
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada.
Tim E
TE
Darsaut
From the Departments of Radiology (B.F., R.F., F.G., N.N.K., D.R., A.W., J.-C.G., A.L.B., L.L.-G., F.B., M.-A.H., J.R.) and Neurosciences (A.Y.P., N.D., A.P.D., S.L., H.B.), Centre Hospitalier de l'Université de Montréal (CHUM), Notre-Dame Hospital, Montreal; and the Department of Surgery, Division of Neurosurgery (T.E.D.), University of Alberta Hospital, Mackenzie Health Sciences Centre, Edmonton, Canada. jean.raymond@umontreal.ca.
Jean
J
Raymond
D006801:Humans; D015588:Observer Variation; D017131:Thrombectomy / Q000379:methods*; D014057:Tomography, X-Ray Computed