Elusive Keys to Economic Development

ommon definitions of ‘impact’ used in evaluation generally refer to the totality of longer-term consequences associated with an intervention on quality-of-life outcomes. For example, the Organization for Economic Cooperation and Development’s Development Assistance Committee (OECD-DAC) defines impact as the “positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended”.[20] A number of international agencies have also adopted this definition of impact. For example, UNICEF defines impact as “The longer term results of a program – technical, economic, socio-cultural, institutional, environmental or other – whether intended or unintended. The intended impact should correspond to the program goal.”[21] Similarly, defines impact evaluation as an evaluation that looks beyond the immediate results of policies, instruction, or services to identify longer-term as well as unintended program effects.[22]

Technically, an evaluation could be conducted to assess ‘impact’ as defined here without reference to a counterfactual. However, much of the existing literature (e.g. NONIE Guidelines on Impact Evaluation[23] adopts the OECD-DAC definition of impact while referring to the techniques used to attribute impact to an intervention as necessarily based on counterfactual analysis.

What is missing from the term ‘impact’ evaluation is the way ‘impact’ shows up long-term. For instance, most Monitoring and Evaluation ‘logical framework’ plans have inputs-outputs-outcomes and… impacts. While the first three appear during the project duration itself, impact takes far longer to take place. For instance, in a 5-year agricultural project, seeds are inputs, farmers trained in using them our outputs, changes in crop yields as a result of the seeds being planted properly in an outcome and families being more sustainably food secure over time is an impact. Such post-project impact evaluations are very rare. They are also called ex-post evaluations or we are coining the term sustained impact evaluations. While hundreds of thousands of documents call for them, rarely do donors have the funding flexibility – or interest – to return to see how sustained, and durable our interventions remained after project close out, after resources were withdrawn. There are many lessons to be learned for design, implementation, M&E and how to foster country-ownership.

0 Comment

Leave a Reply