title: Secular trends and evaluation of complex interventions: the rising tide phenomenon.
abstract: Evaluations of service delivery interventions with contemporaneous controls often yield null results, even when the intervention appeared promising in advance. There can be many reasons for null results. In this paper we introduce the concept of a 'rising tide' phenomenon being a possible explanation of null results. We note that evaluations of service delivery interventions often occur when awareness of the problems they intend to address is already heightened, and pressure to tackle them is mounting throughout a health system. An evaluation may therefore take place in a setting where the system as a whole is improving - where there is a pronounced temporal trend or a 'rising tide causing all vessels to rise'. As a consequence, control sites in an intervention study will improve. This reduces the difference between intervention and control sites and predisposes the study to a null result, leading to the conclusion that the intervention has no effect. We discuss how a rising tide may be distinguished from other causes of improvement in both control and intervention groups, and give examples where the rising tide provides a convincing explanation of such a finding. We offer recommendations for interpretation of research findings where improvements in the intervention group are matched by improvements in the control group. Understanding the rising tide phenomenon is important for a more nuanced interpretation of null results arising in the context of system-wide improvement. Recognition that a rising tide may have predisposed to a null result in one health system cautions against generalising the result to another health system where strong secular trends are absent.
authors.affiliation: Warwick Centre for Applied Health Research & Delivery, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK.
authors.forename: Yen-Fu
authors.initials: YF
authors.lastname: Chen
authors.affiliation: School of Health and Population Sciences, University of Birmingham, Birmingham, UK.
authors.forename: Karla
authors.initials: K
authors.lastname: Hemming
authors.affiliation: School of Health and Population Sciences, University of Birmingham, Birmingham, UK.
authors.forename: Andrew J
authors.initials: AJ
authors.lastname: Stevens
authors.affiliation: Warwick Centre for Applied Health Research & Delivery, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK.
authors.forename: Richard J
authors.initials: RJ
authors.lastname: Lilford
mesh_terms: D003695:Delivery of Health Care / Q000639:trends*; D006801:Humans; D058996:Quality Improvement / Q000592:standards*; D016032:Randomized Controlled Trials as Topic; D012016:Reference Values; D014481:United States
keywords: Cluster trials; Evaluation methodology; Health services research; Quality improvement; Randomised controlled trial