title: Effects of psychological therapies in randomized trials and practice-based studies.
abstract: BACKGROUND
  Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies.
  
  
  METHODS
  Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data.
  
  
  RESULTS
  Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12%. There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials.
  
  
  CONCLUSIONS
  Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.
authors.affiliation: Centre for Psychological Services Research, University of Sheffield, Sheffield, UK. m.barkham@sheffield.ac.uk
authors.forename: Michael
authors.initials: M
authors.lastname: Barkham
authors.forename: William B
authors.initials: WB
authors.lastname: Stiles
authors.forename: Janice
authors.initials: J
authors.lastname: Connell
authors.forename: Elspeth
authors.initials: E
authors.lastname: Twigg
authors.forename: Chris
authors.initials: C
authors.lastname: Leach
authors.forename: Mike
authors.initials: M
authors.lastname: Lucock
authors.forename: John
authors.initials: J
authors.lastname: Mellor-Clark
authors.forename: Peter
authors.initials: P
authors.lastname: Bower
authors.forename: Michael
authors.initials: M
authors.lastname: King
authors.forename: David A
authors.initials: DA
authors.lastname: Shapiro
authors.forename: Gillian E
authors.initials: GE
authors.lastname: Hardy
authors.forename: Leslie
authors.initials: L
authors.lastname: Greenberg
authors.forename: Lynne
authors.initials: L
authors.lastname: Angus
mesh_terms: D002985:Clinical Protocols / Q000592:standards*; D003866:Depressive Disorder / Q000175:diagnosis / Q000523:psychology / Q000628:therapy; D006801:Humans; D017063:Outcome Assessment, Health Care; D010555:Personality Inventory; D011364:Professional Practice / Q000706:statistics & numerical data*; D011613:Psychotherapy / Q000379:methods* / Q000592:standards; D016032:Randomized Controlled Trials as Topic / Q000592:standards / Q000706:statistics & numerical data*; D012106:Research / Q000592:standards; D012107:Research Design; D016896:Treatment Outcome