title: Comparison of youth, caregiver, therapist, trained, and treatment expert raters of therapist adherence to a substance abuse treatment protocol.
abstract: OBJECTIVE
  This study evaluated the accuracy of youth, caregiver, therapist, and trained raters relative to treatment experts on ratings of therapist adherence to a substance abuse treatment protocol for adolescents.
  
  
  METHODS
  Adherence ratings were provided by youth and caregivers in an ongoing trial evaluating a Contingency Management (CM) intervention for youth in juvenile drug court. These ratings were compared to those provided by therapists and trained raters, and each rater type was compared to ratings provided by CM treatment experts. Data were analyzed using item-response-theory-based Many-Facet Rasch Models.
  
  
  RESULTS
  Relative to treatment experts, youth and caregivers were significantly more likely to endorse the occurrence of CM components. In contrast, therapists and trained raters were much more consistent with treatment experts. In terms of practical significance, youth and caregivers each had a 97% estimated probability of indicating that a typical treatment component had occurred. By comparison, the probability was 31%, 19%, and 26% for therapists, trained raters, and treatment experts, respectively.
  
  
  CONCLUSIONS
  Youth and caregivers were highly inaccurate relative to treatment experts, whereas, therapists and trained raters were generally consistent with treatment experts. The implications of these findings for therapist adherence measurement are considered.
authors.affiliation: Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Family Services Research Center, 67 President Street, STE MC406, PO Box 2508861, Charleston, SC 29425, USA. chapmaja@musc.edu
authors.forename: Jason E
authors.initials: JE
authors.lastname: Chapman
authors.forename: Michael R
authors.initials: MR
authors.lastname: McCart
authors.forename: Elizabeth J
authors.initials: EJ
authors.lastname: Letourneau
authors.forename: Ashli J
authors.initials: AJ
authors.lastname: Sheidow
mesh_terms: D000293:Adolescent; D000328:Adult; D017028:Caregivers / Q000523:psychology; D002985:Clinical Protocols; D019468:Disease Management; D005196:Family Therapy / Q000592:standards*; D005260:Female; D019983:Guideline Adherence / Q000592:standards*; D006282:Health Personnel / Q000592:standards*; D006801:Humans; D007604:Juvenile Delinquency / Q000534:rehabilitation; D008297:Male; D019966:Substance-Related Disorders / Q000628:therapy*; D000078329:Workforce