Evaluating the Impact of Science, Technology and Innovation Programs: a Methodological Toolkit
Gustavo Crespi, Alessandro Maffioli, Pierre Mohnen and Gonzalo Vázquez
The purpose of this guideline is to provide ideas and technical advice on how to measure the effectiveness of Science, Technology and Innovation Programs (STIP). The paper addresses the specific challenges of evaluating STIP, from the assessment of the intervention logic to the choice of the most appropriate method to solve the attribution problem. Much attention is devoted to the topic of data, discussing pros and cons of different data sources, data quality issues, and strategies for data collection. The paper analyzes in detail the potential application of experimental and quasi-experimental methods to STIP. For each method, the paper highlights characteristics and assumptions, practical issues related to the implementation, and strengths and weakness specifically related to the application to STIP. Other specific issues related to the evaluation of STIP are also considered: the timing of effects, intensity of treatment, multiple treatments, impact heterogeneity, externalities, and general equilibrium effects. Concrete examples of rigorous evaluations of STIP support the discussion of the various topics throughout the guideline.
2. Economics, of Science, Technology and Innovation Programs
3. Conceptual Framework and Main Challenges in an STIP Impact Evaluation
5. A Review of Evaluation Methods
6. Dealing with Key Issues of an STIP Impact Evaluation
Inter-American Development Bank, 2011