Subject eligibility criteria can substantially influence the results of alcohol-treatment outcome research.
OBJECTIVE
Most alcohol-treatment studies exclude some patients from participation based on particular criteria (e.g., comorbid illegal drug abuse, homelessness). The current study evaluated whether such eligibility criteria can change the outcome results a study obtains.


METHODS
Five widely used treatment research eligibility criteria--(1) psychiatric problems, (2) medical problems, (3) social-residential instability, (4) low motivation/noncompliance, and (5) drug problems--were applied to two samples of real-world alcohol patients whose outcomes were known. Comparing outcomes of the samples with and without the application of eligibility criteria produced estimates of bias in outcome results, as well as an assessment of change in statistical power.


RESULTS
Medical and psychiatric eligibility criteria produced a moderate bias in outcome estimates (e.g., a 10% or less change in outcome results). In contrast, social-residential instability, low motivation/noncompliance, and drug use produced a large (e.g., up to an 18% change) to a very large (e.g., up to a 51% change) bias in outcome estimates. Sensitivity analyses showed that these biases are even larger if eligibility criteria are operationalized in a broad rather than a narrow fashion. Contrary to expectation, eligibility criteria did not produce their theoretically expected benefit of increased statistical power.


CONCLUSIONS
Researchers who use eligibility criteria should do so judiciously and interpret outcome results in light of potential bias introduced by the ineligibility of some patients for study enrollment. Efforts to integrate findings across treatment outcome studies should also consider how conclusions might be affected by the eligibility criteria used in different research areas.
Humphreys
Keith
K
Center for Health Care Evaluation, Veterans Affairs Palo Alto Health Care System, 795 Willow Road, (152), Menlo Park, California 94025, USA. KNH@stanford.edu
Harris
Alex H S
AH
Weingardt
Kenneth R
KR
D000437:Alcoholism / Q000534:rehabilitation*; D015982:Bias; D002985:Clinical Protocols; D002986:Clinical Trials as Topic / Q000379:methods; D006801:Humans; D017063:Outcome Assessment, Health Care / Q000379:methods*; D018579:Patient Selection; D012107:Research Design / Q000592:standards; D016896:Treatment Outcome
Am J Psychiatry. 2000 Apr;157(4):588-94
10739418
Am J Clin Oncol. 1984 Apr;7(2):173-82
6702720
J Am Geriatr Soc. 1997 Aug;45(8):923-8
9256842
Compr Psychiatry. 2002 Mar-Apr;43(2):88-94
11893985
Drug Alcohol Depend. 2008 Nov 1;98(1-2):123-8
18579319
J Nerv Ment Dis. 1985 Jul;173(7):412-23
4009158
Stat Med. 1984 Oct-Dec;3(4):375-87
6396793
Alcohol Clin Exp Res. 2007 Jun;31(6):988-95
17428295
Ann N Y Acad Sci. 1993 Dec 31;703:314-40
8192313
J Nerv Ment Dis. 1980 Jan;168(1):26-33
7351540
Psychiatr Serv. 2003 Dec;54(12):1576
14645790
Addiction. 2005 Sep;100(9):1249-57
16128714
J Psychoactive Drugs. 2002 Jan-Mar;34(1):1-6
12003107
JAMA. 2007 Mar 21;297(11):1233-40
17374817
Addiction. 1998 Jun;93(6):811-23
9744117
Psychol Addict Behav. 2001 Sep;15(3):171-6
11563793