Toward Evidence-Based Measures of Implementation: Examining the Relationship Between Implementation Outcomes and Client Outcomes.
Developing consistent, valid, and efficient implementation outcome measures is necessary to advance implementation science. However, development of such measures has been limited to date, especially for validating the extent to which such measures are associated with important improvements in client outcomes. This study seeks to address this gap by developing one or more evidence-based measures of implementation (EBMIs; i.e., implementation outcome measure that is predictive of improvements in key client outcomes) for the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based practice (EBP) for adolescent substance use.



Data for the current study were collected as part of a large-scale federally funded EBP dissemination and implementation initiative. The multilevel dataset included 65 substance use treatment organizations, 308 clinicians, and 5873 adolescent clients. Adjusted multilevel regression analyses were used to examine the extent to which client-level outcome measures assessed at 6-month follow-up (i.e., substance use, emotional problems) could be predicted by four implementation outcomes: two measures of fidelity (i.e., session exposure, procedure exposure) and two measures of penetration (i.e., absolute client penetration, absolute staff penetration).



Adjusting for client substance use at intake, as well as several client characteristics (e.g., age, race, criminal justice involvement), client substance use at follow-up was significantly lower for treatment organizations that had higher procedure exposure (B=-1.227, standard error [SE]=0.583, 95% confidence interval=-2.370, 0.252; p<.05). None of the other three implementation outcome measures were found to predict improvements in client outcomes.



The current study provides support for procedure exposure as an organizational-level EBMI for A-CRA. Thus, future efforts focused on implementing A-CRA could be improved by measuring and monitoring the extent to which A-CRA procedures are being delivered to clients. Additionally, given the dearth of studies that have examined the relationship between organizational-level measures of implementation and client outcomes, this article provides a prototype for future research to identify EBMIs for other behavioral treatments.
RTI International, P. O. Box 12194, Research Triangle Park, NC, 27709-2194, USA. Electronic address: bgarner@rti.org.
Bryan R
BR
Garner
RAND, P.O. Box 2138, Santa Monica, CA, 90407-2138, USA. Electronic address: shunter@rand.org.
Sarah B
SB
Hunter
Chestnut Health Systems, 448 Wylie Dr., Normal, IL, 61761, USA. Electronic address: rfunk@chestnut.org.
Rodney R
RR
Funk
RAND, P.O. Box 2138, Santa Monica, CA, 90407-2138, USA. Electronic address: bethg@rand.org.
Beth Ann
BA
Griffin
Chestnut Health Systems, 448 Wylie Dr., Normal, IL, 61761, USA. Electronic address: sgodley@chestnut.org.
Susan H
SH
Godley
D000293:Adolescent; D000328:Adult; D000368:Aged; D055317:Evidence-Based Practice / Q000379:methods*; D005260:Female; D005500:Follow-Up Studies; D006801:Humans; D008297:Male; D008875:Middle Aged; D017063:Outcome Assessment, Health Care / Q000379:methods*; D012044:Regression Analysis; D019966:Substance-Related Disorders / Q000534:rehabilitation*; D016896:Treatment Outcome; D055815:Young Adult
Adolescent; Implementation outcomes; Substance use