Efficacy vs effectiveness trial results of an indicated "model" substance abuse program: implications for public health.
OBJECTIVE
The US Department of Education requires schools to choose substance abuse and violence prevention programs that meet standards of effectiveness. The Substance Abuse and Mental Health Services Agency certifies "model" programs that meet this standard. We compared findings from a large, multisite effectiveness trial of 1 model program to its efficacy trial findings, upon which the certification was based.


METHODS
1370 high-risk youths were randomized to experimental or control groups across 9 high schools in 2 large urban school districts. We used intent-to-treat and on-treatment approaches to examine baseline equivalence, attrition, and group differences in outcomes at the end of the program and at a 6-month follow-up.


RESULTS
Positive efficacy trial findings were not replicated in the effectiveness trial. All main effects were either null or worse for the experimental than for the control group.


CONCLUSIONS
These findings suggest that small efficacy trials conducted by developers provide insufficient evidence of effectiveness. Federal agencies and public health scientists must work together to raise the standards of evidence and ensure that data from new trials are incorporated into ongoing assessments of program effects.
Hallfors
Denise
D
Pacific Institute for Research and Evaluation, Chapel Hill, NC, USA. hallfors@pire.org
Cho
Hyunsan
H
Sanchez
Victoria
V
Khatapoush
Shereen
S
Kim
Hyung Min
HM
Bauer
Daniel
D
D000293:Adolescent; D002568:Certification; D005260:Female; D006801:Humans; D008297:Male; D017146:Models, Organizational; D017063:Outcome Assessment, Health Care; D010379:Peer Group; D015397:Program Evaluation; D011635:Public Health Administration; D012572:School Health Services / Q000458:organization & administration* / Q000592:standards; D013332:Student Dropouts / Q000523:psychology*; D019966:Substance-Related Disorders / Q000208:ethnology / Q000517:prevention & control*; D014481:United States; D014482:United States Substance Abuse and Mental Health Services Administration; D019037:Urban Health Services / Q000458:organization & administration* / Q000592:standards