title: A process evaluation accompanying an attempted randomized controlled trial of an evidence service for health system policymakers.
abstract: BACKGROUND
  We developed an evidence service that draws inputs from Health Systems Evidence (HSE), which is a comprehensive database of research evidence about governance, financial and delivery arrangements within health systems and about implementation strategies relevant to health systems. Our goal was to evaluate whether, how and why a 'full-serve' evidence service increases the use of synthesized research evidence by policy analysts and advisors in the Ontario Ministry of Health and Long-Term Care as compared to a 'self-serve' evidence service.
  
  
  METHODS
  We attempted to conduct a two-arm, 10-month randomized controlled trial (RCT), along with a follow-up qualitative process evaluation, but we terminated the RCT when we failed to reach our recruitment target. For the qualitative process evaluation we modified the original interview guide to allow us to explore the (1) factors influencing participation in the trial; (2) usage of HSE, factors explaining usage patterns, and strategies to increase usage; (3) participation in training workshops and use of other supports; and (4) views about and experiences with key HSE features.
  
  
  RESULTS
  We terminated the RCT given our 15% recruitment rate. Six factors were identified by those who had agreed to participate in the trial as encouraging their participation: relevance of the study to participants' own work; familiarity with the researchers; personal view of the importance of using research evidence in policymaking; academic background; support from supervisors; and participation of colleagues. Most reported that they never, infrequently or inconsistently used HSE and suggested strategies to increase its use, including regular email reminders and employee training. However, only two participants indicated that employee training, in the form of a workshop about finding and using research evidence, had influenced their use of HSE. Most participants found HSE features to be intuitive and helpful, although registration/sign-in and some page formats (particularly the advanced search page and detailed search results page) discouraged their use or did not optimize the user experience.
  
  
  CONCLUSIONS
  The qualitative findings informed a re-design of HSE, which allows users to more efficiently find and use research evidence about how to strengthen or reform health systems or in how to get cost-effective programs, services and drugs to those who need them. Our experience with RCT recruitment suggests the need to consider changing the unit of allocation to divisions instead of individuals within divisions, among other lessons.
  
  
  BACKGROUND
  This protocol for this study is published in Implementation Science and registered with ClinicalTrials.gov ( HHS/FHS REB 10-267 ).
authors.affiliation: McMaster Health Forum, McMaster University, Hamilton, Canada. wilsom2@mcmaster.ca.|Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, Canada. wilsom2@mcmaster.ca.|Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada. wilsom2@mcmaster.ca.
authors.forename: Michael G
authors.initials: MG
authors.lastname: Wilson
authors.affiliation: Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. jgrimshaw@ohri.ca.|Department of Medicine, University of Ottawa, Ottawa, Canada. jgrimshaw@ohri.ca.
authors.forename: Jeremy M
authors.initials: JM
authors.lastname: Grimshaw
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada. bhaynes@mcmaster.ca.|Health Information Research Unit, McMaster University, Hamilton, Canada. bhaynes@mcmaster.ca.
authors.forename: R Brian
authors.initials: RB
authors.lastname: Haynes
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada. hannas@mcmaster.ca.|Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore. hannas@mcmaster.ca.|Department of Political Science, Université Laval, Québec, Canada. hannas@mcmaster.ca.|Centre de Recherche du Centre Hospitalier Universitaire de Québec, Québec, Canada. hannas@mcmaster.ca.
authors.forename: Steven E
authors.initials: SE
authors.lastname: Hanna
authors.affiliation: Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada. praina@mcmaster.ca.|McMaster Evidence Review and Synthesis Centre, McMaster University, Hamilton, Canada. praina@mcmaster.ca.
authors.forename: Parminder
authors.initials: P
authors.lastname: Raina
authors.affiliation: Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore. r.gruen@alfred.org.au.
authors.forename: Russell
authors.initials: R
authors.lastname: Gruen
authors.affiliation: Department of Political Science, Université Laval, Québec, Canada. mathieu.ouimet@pol.ulaval.ca.|Centre de Recherche du Centre Hospitalier Universitaire de Québec, Québec, Canada. mathieu.ouimet@pol.ulaval.ca.
authors.forename: Mathieu
authors.initials: M
authors.lastname: Ouimet
authors.affiliation: McMaster Health Forum, McMaster University, Hamilton, Canada. lavisj@mcmaster.ca.|Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, Canada. lavisj@mcmaster.ca.|Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada. lavisj@mcmaster.ca.|Department of Political Science, McMaster University, Hamilton, Canada. lavisj@mcmaster.ca.|Department of Global Health and Population, Harvard School of Public Health, Cambridge, USA. lavisj@mcmaster.ca.
authors.forename: John N
authors.initials: JN
authors.lastname: Lavis
mesh_terms: D003362:Cost-Benefit Analysis; D003627:Data Interpretation, Statistical; D016208:Databases, Factual; D055317:Evidence-Based Practice / Q000191:economics / Q000379:methods / Q000592:standards; D006302:Health Services Research / Q000191:economics / Q000379:methods / Q000592:standards; D006801:Humans; D007407:Interviews as Topic; D008134:Long-Term Care / Q000191:economics / Q000379:methods / Q000592:standards; D009864:Ontario; D010043:Outcome and Process Assessment, Health Care; D018579:Patient Selection; D011050:Policy Making; D036301:Qualitative Research; D016032:Randomized Controlled Trials as Topic; D018401:Sample Size