title: A scale for distinguishing efficacy from effectiveness was adapted and applied to stroke rehabilitation studies.
abstract: OBJECTIVE
  To apply a tool that purports to differentiate between efficacy and effectiveness studies to stroke rehabilitation trials and to evaluate its applicability and reliability.
  
  
  METHODS
  Three raters developed item operational definitions before independently applying the seven-item scale to 151 randomized controlled trials (RCT), published during or after 1997, that evaluated either a pharmacologic (P, n=78) or a nonpharmacologic (NP, n=73) intervention. Inter-rater reliability was assessed for both individual items and total scores, separately for P and NP trials.
  
  
  RESULTS
  Item inter-rater reliability (multiple-rater kappa) ranged from 0.00 (95% CI [confidence interval]: -0.13, 0.13) to 0.85 (95% CI: 0.73, 0.98) and from 0.21 (95% CI: 0.08, 0.34) to 0.79 (95% CI: 0.66, 0.92) for P and NP RCTs, respectively. For the total score (dichotomized), kappa values were 0.43 (95% CI: 0.31, 0.56) and 0.51 (95% CI: 0.37, 0.64) for P and NP trials, respectively.
  
  
  CONCLUSIONS
  The tool provides a solid foundation on which to base further discussion of the differential criteria of efficacy-effectiveness trial design. Scale items should be properly operationalized depending on the research question of interest and evaluated for reliability before the scale is used for definitively judging a given study's design or the external validity of its results.
authors.affiliation: Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, The University of Western Ontario, London, Ontario, Canada. lauraz@chatham-kent.ca
authors.forename: Laura L
authors.initials: LL
authors.lastname: Zettler
authors.forename: Mark R
authors.initials: MR
authors.lastname: Speechley
authors.forename: Norine C
authors.initials: NC
authors.lastname: Foley
authors.forename: Katherine L
authors.initials: KL
authors.lastname: Salter
authors.forename: Robert W
authors.initials: RW
authors.lastname: Teasell
mesh_terms: D019317:Evidence-Based Medicine / Q000379:methods; D006801:Humans; D015588:Observer Variation; D016032:Randomized Controlled Trials as Topic / Q000592:standards*; D015203:Reproducibility of Results; D012107:Research Design; D000071939:Stroke Rehabilitation; D016896:Treatment Outcome