title: Why so many "rigorous" evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute.
abstract: Many widely-used impact evaluation designs, including randomized control trials (RCTs) and quasi-experimental designs (QEDs), frequently fail to detect what are often quite serious unintended consequences of development programs. This seems surprising as experienced planners and evaluators are well aware that unintended consequences frequently occur. Most evaluation designs are intended to determine whether there is credible evidence (statistical, theory-based or narrative) that programs have achieved their intended objectives and the logic of many evaluation designs, even those that are considered the most "rigorous," does not permit the identification of outcomes that were not specified in the program design. We take the example of RCTs as they are considered by many to be the most rigorous evaluation designs. We present a numbers of cases to illustrate how infusing RCTs with a mixed-methods approach (sometimes called an "RCT+" design) can strengthen the credibility of these designs and can also capture important unintended consequences. We provide a Mixed Methods Evaluation Framework that identifies 9 ways in which UCs can occur, and we apply this framework to two of the case studies.
authors.affiliation: 6295 S.W. Elm Avenue, Beaverton, OR 97005, USA. Electronic address: jmichaelbamberger@gmail.com.
authors.forename: Michael
authors.initials: M
authors.lastname: Bamberger
authors.affiliation: UNESCO Evaluation Office (Suite 6.063), 7 Place de Fontenoy, Paris 75007, France. Electronic address: mitarsi@hotmail.com.
authors.forename: Michele
authors.initials: M
authors.lastname: Tarsilla
authors.affiliation: Department of Sociology, Boston College and Director of the Women's and Gender Studies Program, McGuinn Hall 419, 140 Commonwealth Avenue, Chestnut Hill, MA 02467, USA. Electronic address: sharlene.hesse-biber@bc.edu.
authors.forename: Sharlene
authors.initials: S
authors.lastname: Hesse-Biber
mesh_terms: D001210:Asia, Southeastern; D002489:Central America; D005260:Female; D051639:Gardening; D006801:Humans; D008297:Male; D011204:Poverty Areas; D016730:Program Development; D015397:Program Evaluation / Q000379:methods*; D016032:Randomized Controlled Trials as Topic; D012107:Research Design; D018570:Risk Assessment
keywords: Evaluation design; Mixed-methods; Randomized control trials; Uninintended consequences