Lesson Learned

Careful judgment must be used in selecting programs and projects for rigorous impact evaluations.

Careful judgment must be used in selecting programs and projects for rigorous impact evaluations. As described in the evaluation report’s Annex II, the ADA program was originally designed to foster a rigorous impact evaluation which was later impossible due to data limitations and implementation changes. MCC has two key takeaways from this evaluation’s failure to produce a rigorous measurement of impacts: MCC should only select those programs for rigorous impact evaluations which have clearly measurable outcomes and a clear theory of change which maps out the expected realization of those outcomes. Intensive mid-process technical reviews of impact evaluation design and implementation could detect and remedy these problems, a process MCC has already begun implementing across its evaluation portfolio in a format known as “Evaluation Management Committees.”