January 31, 2014
MCC Independent Evaluations
Evaluation is the objective, systematic assessment of a program’s design, implementation and results. Through its investments in evaluations, the Millennium Challenge Corporation seeks to achieve its goals of transparency, accountability and improvement through learning by addressing three key questions:
- Was MCC’s investment implemented according to plan?
- What changes in outcomes, particularly in income, for program participants are attributable to MCC’s investment?
- What lessons can be generalized from past experience to improve the cost-effectiveness of future investments?
Independent evaluations are expected to build on and validate self-evaluations by MCC and Millennium Challenge Accounts (local organizations that implement compacts) that document the original program logic, design and implementation by answering questions such as:
- Did the program reach intended beneficiaries?
- Were there any unanticipated externalities (positive or negative)?
MCC’s independent evaluations are conducted by professional researchers selected through a competitive process. MCC’s use of independent, reputable professionals is intended to produce unbiased assessments of the activities being studied.
MCC publicly publishes findings from every independent evaluation, as well as each evaluation’s methodology and primary data collected whenever possible to allow the broader development community to learn from its experience.
Impact and Performance Evaluations
MCC invests in two different types of independent evaluations: impact and performance. Impact evaluations are more rigorous; they are designed to distinguish impacts caused specifically by an MCC investment from those resulting from common external factors that affected both program participants and non-participants, such as market prices for agricultural goods, national policy changes or favorable weather conditions. Impact evaluations compare what happened with the MCC investment to what would have happened without it, through use of a counterfactual.
Performance evaluations are another valuable way of gauging the effectiveness of MCC investments. Performance evaluations are less rigorous and cannot attribute causal impact to MCC investments because they do not use a statistically valid counterfactual—what would have happened to the same group of program participants if they had not received MCC’s assistance. However, they are useful to compare changes in the situation before and after MCC’s investment and provide details on how an investment might have contributed to changes in outcomes and—importantly—why or why not.
There are several critical factors that MCC considers when deciding to invest in an impact or a performance evaluation, including feasibility, learning potential, strong stakeholder commitment, and proper coordination.
Incorporating evaluations into program operations is not easy—particularly for impact evaluations—and establishing a statistically valid counterfactual for an impact evaluation can be a painstaking process. But these are challenges that MCC embraces to ensure accountability for results and to improve learning about what works and what doesn’t.
Contribution of MCC’s evaluations to development
With investments in more than 150 independent evaluations, MCC is a leader in the international development community for its commitment to accountability, transparency and learning. The benefits of these investments include:
- Testing traditional assumptions about what works. All MCC programs are selected, designed and implemented using certain assumptions about how the inputs and expected outputs will lead to poverty reduction through economic growth and who is expected to benefit. Evaluations can also be structured to evaluate the effectiveness of different program design elements, or comparing one approach to another.
- Understanding who benefits from investments and why. Evaluations can also be structured to understand how different social groups are able to benefit from a project. For this reason, MCC has independent evaluators disaggregate results by key characteristics according to the program logic, such as gender, age and poverty level. For example, understanding the relationship between intra-household assets and poverty is important because programs that focus on cash income alone may not generate improvements in certain measures of well-being such as nutrition, food security or health, which in turn may have significant long-term income effects.
- Improve evidence-based decision making: The results of evaluations strengthen and improve future program design and decision-making. In addition, MCC and MCAs may also be able to make necessary course corrections during implementation based on learning from evaluations.
- Contribute to global best practices: MCC evaluations are expected to contribute to the global understanding on what works in the development field. MCC makes the results of its evaluations publicly available to be used by other donors, partner countries, researchers, and non-governmental organizations.