A longitudinal perspective on monitoring outcomes of an innovative program.
OBJECTIVE
The use of outcome assessment to evaluate the performance of programs over many years of operation is becoming an increasingly important aspect of health care management. Over a five-year period of program monitoring, this study examined changes in individual client outcomes three months after discharge from a residential work therapy program for veterans with severe substance use disorders. The study also examined the relationship between these outcomes and changing program features, such as staffing, treatment variables, and follow-up rates.


METHODS
Data on admissions characteristics, services delivered during treatment, and status at discharges were collected for 3,390 veterans who were treated in 25 sites in the Department of Veterans Affairs' Compensated Work Therapy/Transitional Residence program. Follow-up data were gathered three months after discharge for 1,771 veterans (52.2 percent). Hierarchical linear modeling was used to examine the association between year of discharge, site-level measures of program staffing and follow-up rate, and individual patient-level treatment variables and outcomes.


RESULTS
Over the five-year monitoring period, site staff-to-bed ratios and follow-up rates dropped substantially, and veterans attended more Alcoholics Anonymous and Narcotics Anonymous meetings and had more toxicology screens. Higher staff-to-bed ratios were associated with more positive employment outcomes, and higher follow-up rates were associated with poorer outcomes in substance abuse and lower total income. However, no significant outcome trends were observed in clinical follow-up measures after the analyses adjusted for these factors, suggesting that program effectiveness did not deteriorate during a period of program change.


CONCLUSIONS
Long-term evaluations of program process and follow-up status can usefully document sustained program effectiveness over many years. Such efforts should enhance their value by examining and adjusting for the impact of factors such as changing staffing levels and follow-up rates.
Rosenheck
Robert A
RA
Department of Veterans Affairs Connecticut Healthcare System, West Haven, Connecticut 06516, USA. robert.rosenheck@yale.edu
Seibyl
Catherine L
CL
D000328:Adult; D000436:Alcoholics Anonymous; D000437:Alcoholism / Q000175:diagnosis / Q000453:epidemiology / Q000517:prevention & control; D015897:Comorbidity; D005260:Female; D005500:Follow-Up Studies; D006801:Humans; D008297:Male; D001523:Mental Disorders / Q000175:diagnosis / Q000453:epidemiology / Q000628:therapy; D008605:Mental Health Services / Q000458:organization & administration*; D008875:Middle Aged; D009936:Organizational Innovation; D017063:Outcome Assessment, Health Care; D010343:Patient Admission / Q000706:statistics & numerical data; D010349:Patient Compliance / Q000706:statistics & numerical data; D012114:Residential Treatment; D012559:Schizophrenia / Q000534:rehabilitation / Q000628:therapy*; D012720:Severity of Illness Index; D019966:Substance-Related Disorders / Q000175:diagnosis / Q000453:epidemiology* / Q000534:rehabilitation*; D014481:United States; D014493:United States Department of Veterans Affairs; D014728:Veterans / Q000523:psychology*