title: Reproducibility and responsiveness of evaluative outcome measures. Theoretical considerations illustrated by an empirical example.
abstract: OBJECTIVE
  This article outlines basic methods for quantifying reproducibility and responsiveness of evaluative outcome measures.
  
  
  METHODS
  The background noise in stable patients provides the desired information to quantify the reproducibility. From this, the smallest real difference (SRD) for longitudinal differences can be derived. We propose to use the SRD to define responsiveness: from all patients who change according to an external criterion, we take the percentage that changes at least SRD on the outcome measure. A more complete picture of the responsiveness of the outcome measure arises when the receiver operating characteristic (ROC) is constructed, corresponding to all possible cut-off change scores. The proposed methods are illustrated with an empirical example.
  
  
  RESULTS
  In the illustration the methods appeared to be very useful and complemented each other. We could evaluate whether the observed change score was larger than that expected due to chance. With the methods it was possible to evaluate both the ability of an instrument to detect change if there is a real change in health status (sensitivity to change) and the ability to detect absence of change if there is no real change (specificity to change).
  
  
  CONCLUSIONS
  We presented the use of SRDs and ROC curves for quantifying reproducibility and responsiveness. We started with the basic notions and arrived at methods that are both understandable and useful.
authors.affiliation: Vrije Universiteit Medical Centre.
authors.forename: H C
authors.initials: HC
authors.lastname: de Vet
authors.forename: L M
authors.initials: LM
authors.lastname: Bouter
authors.forename: P D
authors.initials: PD
authors.lastname: Bezemer
authors.forename: A J
authors.initials: AJ
authors.lastname: Beurskens
mesh_terms: D006305:Health Status Indicators; D006801:Humans; D008137:Longitudinal Studies; D017116:Low Back Pain / Q000628:therapy; D008962:Models, Theoretical; D009426:Netherlands / Q000453:epidemiology; D017063:Outcome Assessment, Health Care / Q000379:methods* / Q000592:standards; D012372:ROC Curve; D016032:Randomized Controlled Trials as Topic / Q000706:statistics & numerical data; D015203:Reproducibility of Results; D013673:Technology Assessment, Biomedical / Q000379:methods; D014143:Traction; D016896:Treatment Outcome