title: The objectivity of the Autism Diagnostic Observation Schedule (ADOS) in naturalistic clinical settings.
abstract: The Autism Diagnostic Observation Schedule (ADOS) is a first-choice diagnostic tool in autism spectrum disorder (ASD). Excellent interpersonal objectivity (interrater reliability) has been demonstrated for the ADOS under optimal conditions, i.e., within groups of highly trained "research reliable" examiners in research setting. We investigated the spontaneous interrater reliability among clinically trained ADOS users across multiple sites in clinical routine. Forty videotaped administrations of the ADOS modules 1-4 were rated by five different raters each from a pool of in total 15 raters affiliated to 13 different clinical sites. G(q,k) coefficients (analogous to intraclass correlations), kappas (ĸ) and percent agreement (PA) were calculated. The median interrater reliability for items across the four modules was G(q,k) = .74-.83, with the single ADOS items ranging from .23 to .94. G(q,k) for total scores was .85-.92. For diagnostic classification (ASD/non-spectrum), PA was 64-82 % and Fleiss' ĸ .19-.55. Objectivity was lower for pervasive developmental disorder not otherwise specified and non-spectrum diagnoses as compared to autism. Interrater reliabilities of the ADOS items and domain totals among clinical users across multiple sites were in the same range as previously reported for research reliable users, while the one for diagnostic classification was lower. Differences in sample characteristics, rater skills and statistics compared with previous studies are discussed. Findings endorse the objectivity of the ADOS in naturalistic clinical settings, but also pinpoint its limitations and the need and value of adequate and continuous rater training.
authors.affiliation: Pediatric Neuropsychiatry Unit, Department of Women's and Children's Health, Center of Neurodevelopmental Disorders (KIND), Karolinska Institutet, Gävlegatan 22B, 113 30, Stockholm, Sweden. eric.zander@ki.se.|Child and Adolescent Psychiatry, Center for Psychiatry Research, Stockholm County Council, Stockholm, Sweden. eric.zander@ki.se.|Neurodevelopmental Psychiatry Unit South East, Child and Adolescent Psychiatry, Stockholm County Council, Stockholm, Sweden. eric.zander@ki.se.
authors.forename: Eric
authors.identifier: http://orcid.org/0000-0002-2267-5813
authors.initials: E
authors.lastname: Zander
authors.affiliation: Pediatric Neuropsychiatry Unit, Department of Women's and Children's Health, Center of Neurodevelopmental Disorders (KIND), Karolinska Institutet, Gävlegatan 22B, 113 30, Stockholm, Sweden.|Child and Adolescent Psychiatry, Center for Psychiatry Research, Stockholm County Council, Stockholm, Sweden.
authors.forename: Charlotte
authors.initials: C
authors.lastname: Willfors
authors.affiliation: Pediatric Neuropsychiatry Unit, Department of Women's and Children's Health, Center of Neurodevelopmental Disorders (KIND), Karolinska Institutet, Gävlegatan 22B, 113 30, Stockholm, Sweden.|Child and Adolescent Psychiatry, Center for Psychiatry Research, Stockholm County Council, Stockholm, Sweden.
authors.forename: Steve
authors.initials: S
authors.lastname: Berggren
authors.affiliation: Pediatric Neuropsychiatry Unit, Department of Women's and Children's Health, Center of Neurodevelopmental Disorders (KIND), Karolinska Institutet, Gävlegatan 22B, 113 30, Stockholm, Sweden.|Child and Adolescent Psychiatry, Center for Psychiatry Research, Stockholm County Council, Stockholm, Sweden.|BUP Södertälje, Child and Adolescent Psychiatry, Stockholm County Council, Södertälje, Sweden.
authors.forename: Nora
authors.initials: N
authors.lastname: Choque-Olsson
authors.affiliation: Pediatric Neuropsychiatry Unit, Department of Women's and Children's Health, Center of Neurodevelopmental Disorders (KIND), Karolinska Institutet, Gävlegatan 22B, 113 30, Stockholm, Sweden.|Child and Adolescent Psychiatry, Center for Psychiatry Research, Stockholm County Council, Stockholm, Sweden.|Neuropediatric Unit, Sachs' Children and Youth Hospital, Stockholm County Council, Stockholm, Sweden.
authors.forename: Christina
authors.initials: C
authors.lastname: Coco
authors.affiliation: PRIMA Child and Adolescent Psychiatry, Stockholm, Sweden.|Citypsykologhus, Stockholm, Sweden.
authors.forename: Anna
authors.initials: A
authors.lastname: Elmund
authors.affiliation: BUP Sollentuna, Child and Adolescent Psychiatry, Stockholm County Council, Sollentuna, Sweden.
authors.forename: Åsa Hedfors
authors.initials: ÅH
authors.lastname: Moretti
authors.affiliation: Astrid Lindgrens Children's Hospital, Karolinska University Hospital Solna, Stockholm County Council, Solna, Sweden.
authors.forename: Anette
authors.initials: A
authors.lastname: Holm
authors.affiliation: Astrid Lindgrens Children's Hospital, Karolinska University Hospital Huddinge, Stockholm County Council, Huddinge, Sweden.|Pupil Health Unit, Tiohundra, Norrtälje, Sweden.
authors.forename: Ida
authors.initials: I
authors.lastname: Jifält
authors.affiliation: BUP Malmö, Child and Adolescent Psychiatry, Region Skåne, Malmö, Sweden.|Pupil Health Unit, Resource Team for Learning Disabled, City of Malmö, Sweden.
authors.forename: Renata
authors.initials: R
authors.lastname: Kosieradzki
authors.affiliation: Södra Älvsborg Hospital (SÄS), Child and Adolescent Psychiatry, Region Västra Götaland, Borås, Sweden.
authors.forename: Jenny
authors.initials: J
authors.lastname: Linder
authors.affiliation: Neuropediatric Unit, Sachs' Children and Youth Hospital, Stockholm County Council, Stockholm, Sweden.
authors.forename: Viviann
authors.initials: V
authors.lastname: Nordin
authors.affiliation: BUP Lund, Child and Adolescent Psychiatry, Region Skåne, Lund, Sweden.
authors.forename: Karin
authors.initials: K
authors.lastname: Olafsdottir
authors.affiliation: PRIMA Child and Adolescent Psychiatry, Stockholm, Sweden.
authors.forename: Lina
authors.initials: L
authors.lastname: Poltrago
authors.affiliation: Pediatric Neuropsychiatry Unit, Department of Women's and Children's Health, Center of Neurodevelopmental Disorders (KIND), Karolinska Institutet, Gävlegatan 22B, 113 30, Stockholm, Sweden.|Child and Adolescent Psychiatry, Center for Psychiatry Research, Stockholm County Council, Stockholm, Sweden.
authors.forename: Sven
authors.initials: S
authors.lastname: Bölte
mesh_terms: D000067877:Autism Spectrum Disorder / Q000175:diagnosis*; D002648:Child; D002675:Child, Preschool; D005260:Female; D006801:Humans; D008297:Male; D011569:Psychiatric Status Rating Scales / Q000592:standards*; D011594:Psychometrics / Q000295:instrumentation*; D015203:Reproducibility of Results
keywords: Autism spectrum disorder; Diagnostic instrument; Interrater reliability