Tracking evidence based practice with youth: validity of the MATCH and standard manual consultation records.
This study sought to evaluate the agreement between therapist report and coder observation of therapy practices. The study sampled session data from a community-based, randomized trial of treatment for youth ages 7 to 13. We used therapist report of session content and coverage gathered using formal Consultation Records and developed complimentary records for coders to use when watching or listening to therapy tape. We established initial reliability between coders and then conducted a random, stratified, and comprehensive sample of sessions across youth (N = 121), therapists (N = 57), conditions (MATCH and Standard Manuals), and study sites (Honolulu and Boston) to code and compare with therapist record reports. Intraclass correlation coefficients (ICCs) representing coder versus therapist agreement on manual content delivered ranged from .42 to 1.0 across conditions and problem areas. Analyses revealed marked variability in agreement regarding whether behavioral rehearsals took place (ICCs from -.01 to 1.0) but strong agreement on client comprehension of therapy content and homework assignments. Overall, the findings indicate that therapists can be accurate reporters of the therapeutic practices they deliver, although they may need more support in reporting subtle but valuable aspects of implementation such as types of behavioral rehearsals. Developing means to support accurate reporting is important to developing future clinical feedback methodology applicable to the implementation of evidence-based treatments in the real world.
Department of Psychology, University of California, Los Angeles, CA 90095, USA. alymargaret@gmail.com
Alyssa M
AM
Ward
Jennifer
J
Regan
Bruce F
BF
Chorpita
Nicole
N
Starace
Adriana
A
Rodriguez
Kelsie
K
Okamura
Eric L
EL
Daleiden
Sarah Kate
SK
Bearman
John R
JR
Weisz
D000293:Adolescent; D000553:Ambulatory Care / Q000379:methods; D001521:Behavior Therapy / Q000379:methods; D002648:Child; D059019:Clinical Coding / Q000379:methods*; D003156:Community Mental Health Services / Q000379:methods*; D055317:Evidence-Based Practice; D005196:Family Therapy / Q000379:methods*; D005260:Female; D005568:Forms and Records Control / Q000379:methods*; D006801:Humans; D008297:Male; D008499:Medical Records; D011361:Professional Competence / Q000706:statistics & numerical data*; D011787:Quality of Health Care