Effects of psychological therapies in randomized trials and practice-based studies.
BACKGROUND
Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies.


METHODS
Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data.


RESULTS
Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12%. There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials.


CONCLUSIONS
Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.
Barkham
Michael
M
Centre for Psychological Services Research, University of Sheffield, Sheffield, UK. m.barkham@sheffield.ac.uk
Stiles
William B
WB
Connell
Janice
J
Twigg
Elspeth
E
Leach
Chris
C
Lucock
Mike
M
Mellor-Clark
John
J
Bower
Peter
P
King
Michael
M
Shapiro
David A
DA
Hardy
Gillian E
GE
Greenberg
Leslie
L
Angus
Lynne
L
D002985:Clinical Protocols / Q000592:standards*; D003866:Depressive Disorder / Q000175:diagnosis / Q000523:psychology / Q000628:therapy; D006801:Humans; D017063:Outcome Assessment, Health Care; D010555:Personality Inventory; D011364:Professional Practice / Q000706:statistics & numerical data*; D011613:Psychotherapy / Q000379:methods* / Q000592:standards; D016032:Randomized Controlled Trials as Topic / Q000592:standards / Q000706:statistics & numerical data*; D012106:Research / Q000592:standards; D012107:Research Design; D016896:Treatment Outcome