title: Automated Research Impact Assessment: A New Bibliometrics Approach.
abstract: As federal programs are held more accountable for their research investments, The National Institute of Environmental Health Sciences (NIEHS) has developed a new method to quantify the impact of our funded research on the scientific and broader communities. In this article we review traditional bibliometric analyses, address challenges associated with them, and describe a new bibliometric analysis method, the Automated Research Impact Assessment (ARIA). ARIA taps into a resource that has only rarely been used for bibliometric analyses: references cited in "important" research artifacts, such as policies, regulations, clinical guidelines, and expert panel reports. The approach includes new statistics that science managers can use to benchmark contributions to research by funding source. This new method provides the ability to conduct automated impact analyses of federal research that can be incorporated in program evaluations. We apply this method to several case studies to examine the impact of NIEHS funded research.
authors.affiliation: National Institute of Environmental Health Sciences, Program Analysis Branch.
authors.forename: Christina H
authors.initials: CH
authors.lastname: Drew
authors.affiliation: National Institute of Environmental Health Sciences, Program Analysis Branch.
authors.forename: Kristianna G
authors.initials: KG
authors.lastname: Pettibone
authors.affiliation: Open Intelligence, Inc.
authors.forename: Fallis Owen
authors.initials: FO
authors.lastname: Finch
authors.affiliation: Open Intelligence, Inc.
authors.forename: Douglas
authors.initials: D
authors.lastname: Giles
authors.affiliation: National Institutes of Health, Office of Extramural Research, Office of Data Analysis Tools and Systems.
authors.forename: Paul
authors.initials: P
authors.lastname: Jordan
keywords: automated impact analysis; bibliometrics; research evaluation; science of science management