title: Measuring the payback of research activities: a feasible ex-post evaluation methodology in epidemiology and public health.
abstract: Most ex-post evaluations of research funding programs are based on bibliometric methods and, although this approach has been widely used, it only examines one facet of the project's impact, that is, scientific productivity. More comprehensive models of payback assessment of research activities are designed for large-scale projects with extensive funding. The purpose of this study was to design and implement a methodology for the ex-post evaluation of small-scale projects that would take into account both the fulfillment of projects' stated objectives as well as other wider benefits to society as payback measures. We used a two-phase ex-post approach to appraise impact for 173 small-scale projects funded in 2007 and 2008 by a Spanish network center for research in epidemiology and public health. In the internal phase we used a questionnaire to query the principal investigator (PI) on the outcomes as well as actual and potential impact of each project; in the external phase we sent a second questionnaire to external reviewers with the aim of assessing (by peer-review) the performance of each individual project. Overall, 43% of the projects were rated as having completed their objectives "totally", and 40% "considerably". The research activities funded were reported by PIs as socially beneficial their greatest impact being on research capacity (50% of payback to society) and on knowledge translation (above 11%). The method proposed showed a good discriminating ability that makes it possible to measure, reliably, the extent to which a project's objectives were met as well as the degree to which the project contributed to enhance the group's scientific performance and of its social payback.
authors.affiliation: Laboratory of Translational Medicine and Decision Science, TransLab Research Group, Department of Medical Sciences, Faculty of Medicine, University of Girona, 17071 Girona, Catalonia, Spain. marta.aymerich@udg.edu
authors.forename: Marta
authors.initials: M
authors.lastname: Aymerich
authors.forename: Carme
authors.initials: C
authors.lastname: Carrion
authors.forename: Pedro
authors.initials: P
authors.lastname: Gallo
authors.forename: Maria
authors.initials: M
authors.lastname: Garcia
authors.forename: Abel
authors.initials: A
authors.lastname: López-Bermejo
authors.forename: Miquel
authors.initials: M
authors.lastname: Quesada
authors.forename: Rafel
authors.initials: R
authors.lastname: Ramos
mesh_terms: D004812:Epidemiologic Methods; D006801:Humans; D011634:Public Health; D015203:Reproducibility of Results; D012106:Research / Q000592:standards; D012107:Research Design; D011795:Surveys and Questionnaires