title: Advancing implementation science through measure development and evaluation: a study protocol.
abstract: BACKGROUND
  Significant gaps related to measurement issues are among the most critical barriers to advancing implementation science. Three issues motivated the study aims: (a) the lack of stakeholder involvement in defining pragmatic measure qualities; (b) the dearth of measures, particularly for implementation outcomes; and (c) unknown psychometric and pragmatic strength of existing measures. Aim 1: Establish a stakeholder-driven operationalization of pragmatic measures and develop reliable, valid rating criteria for assessing the construct. Aim 2: Develop reliable, valid, and pragmatic measures of three critical implementation outcomes, acceptability, appropriateness, and feasibility. Aim 3: Identify Consolidated Framework for Implementation Research and Implementation Outcome Framework-linked measures that demonstrate both psychometric and pragmatic strength.
  
  
  METHODS
  For Aim 1, we will conduct (a) interviews with stakeholder panelists (N = 7) and complete a literature review to populate pragmatic measure construct criteria, (b) Q-sort activities (N = 20) to clarify the internal structure of the definition, (c) Delphi activities (N = 20) to achieve consensus on the dimension priorities, (d) test-retest and inter-rater reliability assessments of the emergent rating system, and (e) known-groups validity testing of the top three prioritized pragmatic criteria. For Aim 2, our systematic development process involves domain delineation, item generation, substantive validity assessment, structural validity assessment, reliability assessment, and predictive validity assessment. We will also assess discriminant validity, known-groups validity, structural invariance, sensitivity to change, and other pragmatic features. For Aim 3, we will refine our established evidence-based assessment (EBA) criteria, extract the relevant data from the literature, rate each measure using the EBA criteria, and summarize the data.
  
  
  CONCLUSIONS
  The study outputs of each aim are expected to have a positive impact as they will establish and guide a comprehensive measurement-focused research agenda for implementation science and provide empirically supported measures, tools, and methods for accomplishing this work.
authors.affiliation: Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN, 47405, USA. lewiscc@indiana.edu.|Department of Psychiatry and Behavioral Sciences, School of Medicine, Harborview Medical Center, University of Washington, 325 9th Ave, Box 359911, Seattle, WA, 98104, USA. lewiscc@indiana.edu.
authors.forename: Cara C
authors.initials: CC
authors.lastname: Lewis
authors.affiliation: 1102-C McGavran-Greenberg Hall, University of North Carolina at Chapel Hill, 135 Dauer Drive, Campus Box 7411, Chapel Hill, NC, 27599-7411, USA. weiner@email.unc.edu.
authors.forename: Bryan J
authors.initials: BJ
authors.lastname: Weiner
authors.affiliation: Department of Psychology, University of Montana, 32 Campus Dr., Skaggs Bldg. 202, Missoula, MT, 59812, USA. cameo.stanick@umontana.edu.
authors.forename: Cameo
authors.initials: C
authors.lastname: Stanick
authors.affiliation: Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN, 47405, USA. sarafisc@umail.iu.edu.
authors.forename: Sarah M
authors.initials: SM
authors.lastname: Fischer
mesh_terms: D006801:Humans; D016730:Program Development / Q000379:methods* / Q000592:standards; D015397:Program Evaluation / Q000379:methods* / Q000592:standards; D012107:Research Design