Improving credibility and transparency of conservation impact evaluations through the partial identification approach.
The fundamental challenge of evaluating the impact of conservation interventions is that researchers must estimate the difference between the outcome after an intervention occurred and what the outcome would have been without it (counterfactual). Because the counterfactual is unobservable, researchers must make an untestable assumption that some units (e.g., organisms or sites) that were not exposed to the intervention can be used as a surrogate for the counterfactual (control). The conventional approach is to make a point estimate (i.e., single number along with a confidence interval) of impact, using, for example, regression. Point estimates provide powerful conclusions, but in nonexperimental contexts they depend on strong assumptions about the counterfactual that often lack transparency and credibility. An alternative approach, called partial identification (PI), is to first estimate what the counterfactual bounds would be if the weakest possible assumptions were made. Then, one narrows the bounds by using stronger but credible assumptions based on an understanding of why units were selected for the intervention and how they might respond to it. We applied this approach and compared it with conventional approaches by estimating the impact of a conservation program that removed invasive trees in part of the Cape Floristic Region. Even when we used our largest PI impact estimate, the program's control costs were 1.4 times higher than previously estimated. PI holds promise for applications in conservation science because it encourages researchers to better understand and account for treatment selection biases; can offer insights into the plausibility of conventional point-estimate approaches; could reduce the problem of advocacy in science; might be easier for stakeholders to agree on a bounded estimate than a point estimate where impacts are contentious; and requires only basic arithmetic skills.
Department of Botany & Zoology, Centre for Invasion Biology, Stellenbosch University, Private Bag X1, Matieland, 7602, South Africa.
Matthew M
MM
McConnachie
Department of Biology, USA & Centre for International Forestry Research, University of Florida, Gainesville, FL, U.S.A.
Claudia
C
Romero
Department of Economics, Andrew Young School of Policy Studies, Georgia State University, Atlanta, GA, U.S.A.
Paul J
PJ
Ferraro
Department of Botany & Zoology, Centre for Invasion Biology, Stellenbosch University, Private Bag X1, Matieland, 7602, South Africa.
Brian W
BW
van Wilgen
D044822:Biodiversity; D003247:Conservation of Natural Resources / Q000191:economics / Q000379:methods*; D003362:Cost-Benefit Analysis; D058865:Introduced Species; D013019:South Africa; D014197:Trees / Q000502:physiology*; D058728:Weed Control
adaptive management; basado en evidencia; ciencia de la sustentabilidad; especies invasoras; evidence based; impact evaluation methods; invasive species; manejo adaptativo; métodos de evaluación de impacto; sustainability science