Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: correspondence between therapists and observers.
Developing therapist-report fidelity tools to support quality delivery of evidence-based practices in usual care is a top priority for implementation science. This study tested the reliability and accuracy of two groups of community therapists who reported on their use of family therapy (FT) and motivational interviewing/cognitive-behavioral therapy (MI/CBT) interventions during routine treatment of inner-city adolescents with conduct and substance use problems. Study cases (n = 45) were randomized into two conditions: (a) Routine Family Therapy (RFT), consisting of a single site that featured family therapy as its standard of care for behavioral treatment; or (b) Treatment As Usual (TAU), consisting of five sites that featured non-family approaches. Therapists and trained observational raters provided FT and MI/CBT adherence ratings on 157 sessions (104 RFT, 53 TAU). Overall therapist reliability was adequate for averaged FT ratings (ICC = .66) but almost non-existent for MI/CBT (ICC = .06); moreover, both RFT and TAU therapists were more reliable in reporting on FT than on MI/CBT. Both groups of therapists overestimated the extent to which they implemented FT and MI/CBT interventions. Results offer support for the feasibility of using existing therapist-report methods to anchor quality assurance procedures for FT interventions in real-world settings, though not for MI/CBT.
Treatment Research Division, The National Center on Addiction and Substance Abuse at Columbia University, 633 Third Avenue, 19th floor, New York, NY, 10017, USA, athogue@aol.com.
Aaron
A
Hogue
Sarah
S
Dauber
Emily
E
Lichvar
Molly
M
Bobek
Craig E
CE
Henderson
D000293:Adolescent; D001008:Anxiety Disorders / Q000628:therapy; D001289:Attention Deficit Disorder with Hyperactivity / Q000628:therapy*; D019958:Attention Deficit and Disruptive Behavior Disorders / Q000628:therapy; D002985:Clinical Protocols; D015928:Cognitive Behavioral Therapy / Q000592:standards*; D019955:Conduct Disorder / Q000628:therapy*; D055317:Evidence-Based Practice / Q000592:standards*; D005196:Family Therapy / Q000592:standards*; D005260:Female; D019983:Guideline Adherence; D006801:Humans; D008297:Male; D062405:Motivational Interviewing / Q000592:standards*; D011613:Psychotherapy; D015203:Reproducibility of Results; D057566:Self Report; D019966:Substance-Related Disorders / Q000628:therapy