Behind the scenes: Managing and conducting large scale impact evaluations in Colombia
by Bertha Briceño (Water and Sanitation Program, World Bank), Laura Cuesta (University of Wisconsin-Madison) and Orazio Attanasio (University College London)
3ie Working Paper 14
As more resources are being allocated to impact evaluation of development programs,the need to map out the utilization and influence of evaluations has been increasingly highlighted. This paper aims at filling this gap by describing and discussing experiences from four large impact evaluations in Colombia on a case study-basis. On the basis of (1) learning from our prior experience in both managing and conducting impact evaluations, (2) desk review of available documentation from the Monitoring & Evaluation system, and (3) structured interviews with government actors, evaluators and program managers, we benchmark each evaluation against eleven standards of quality. From this benchmarking exercise, we derive five key lessons for conducting high quality and influential impact evaluations: (1) investing in the preparation of good terms of reference and identification of evaluation questions; (2) choosing the best methodological approach to address the evaluation questions; (3) adopting mechanisms to ensure evaluation quality; (4) laying out the incentives for involved parties in order to foster evaluation buy-in; and (5) carrying out a plan for quality dissemination.
An ideal impact evaluation process
Before starting the IE
During implementation of the IE
After implementing the IE
Facing the real world: learning from the experience of selected impact evaluations in Colombia
A. The impact evaluation of the Familias en Acción (FeA) conditional cash transfer program
B. The impact evaluation of the Jóvenes en Acción (JeA) youth labor training program
C. The impact evaluation of the Hogares Comunitarios de Bienestar (HCB) nursery program
D. The evaluation of the Sistema General de Participaciones (SGP) fiscal transfers’ policy.
Conclusions and key lessons