Careful judgment must be used in selecting programs and projects for rigorous impact evaluations. As described in the evaluation report’s Annex II, the ADA program was originally designed to foster a rigorous impact evaluation which was later impossible due to data limitations and implementation changes. MCC has two key takeaways from this evaluation’s failure to produce a rigorous measurement of impacts: MCC should only select those programs for rigorous impact evaluations which have clearly measurable outcomes and a clear theory of change which maps out the expected realization of those outcomes. Intensive mid-process technical reviews of impact evaluation design and implementation could detect and remedy these problems, a process MCC has already begun implementing across its evaluation portfolio in a format known as “Evaluation Management Committees.”
Lesson Learned