title: Using the Nine Common Themes of Good Practice checklist as a tool for evaluating the research priority setting process of a provincial research and program evaluation program.
abstract: BACKGROUND
  Given the context-specific nature of health research prioritization and the obligation to effectively allocate resources to initiatives that will achieve the greatest impact, evaluation of priority setting processes can refine and strengthen such exercises and their outcomes. However, guidance is needed on evaluation tools that can be applied to research priority setting. This paper describes the adaption and application of a conceptual framework to evaluate a research priority setting exercise operating within the public health sector in Ontario, Canada.
  
  
  METHODS
  The Nine Common Themes of Good Practice checklist, described by Viergever et al. (Health Res Policy Syst 8:36, 2010) was used as the conceptual framework to evaluate the research priority setting process developed for the Locally Driven Collaborative Projects (LDCP) program in Ontario, Canada. Multiple data sources were used to inform the evaluation, including a review of selected priority setting approaches, surveys with priority setting participants, document review, and consultation with the program advisory committee.
  
  
  RESULTS
  The evaluation assisted in identifying improvements to six elements of the LDCP priority setting process. The modifications were aimed at improving inclusiveness, information gathering practices, planning for project implementation, and evaluation. In addition, the findings identified that the timing of priority setting activities and level of control over the process were key factors that influenced the ability to effectively implement changes.
  
  
  CONCLUSIONS
  The findings demonstrate the novel adaptation and application of the 'Nine Common Themes of Good Practice checklist' as a tool for evaluating a research priority setting exercise. The tool can guide the development of evaluation questions and enables the assessment of key constructs related to the design and delivery of a research priority setting process.
authors.affiliation: Public Health Ontario, Santé publique Ontario, 480 University Avenue, Suite 300, Toronto, ON, M5G 1V2, Canada. rebecca.mador@oahpp.ca.
authors.forename: Rebecca L
authors.initials: RL
authors.lastname: Mador
authors.affiliation: Public Health Ontario, Santé publique Ontario, 480 University Avenue, Suite 300, Toronto, ON, M5G 1V2, Canada.
authors.forename: Kathy
authors.initials: K
authors.lastname: Kornas
authors.affiliation: Public Health Ontario, Santé publique Ontario, 480 University Avenue, Suite 300, Toronto, ON, M5G 1V2, Canada.
authors.forename: Anne
authors.initials: A
authors.lastname: Simard
authors.affiliation: Ontario Long-Term Care Association, 425 University Avenue, Suite 500, Toronto, ON, M5G 1T6, Canada.
authors.forename: Vinita
authors.initials: V
authors.lastname: Haroun
mesh_terms: D035843:Biomedical Research / Q000379:methods*; D057189:Checklist / Q000379:methods*; D003299:Cooperative Behavior; D019317:Evidence-Based Medicine; D006801:Humans; D009864:Ontario; D015397:Program Evaluation / Q000379:methods*; D017150:Public Sector; D012107:Research Design
keywords: Evaluation; Priority setting; Public health; Research priorities