March 20, 2013
MCC's Use of Evidence and Evaluation
The Millennium Challenge Corporation’s singular mission is to reduce poverty through economic growth. MCC accomplishes this by using evidence to guide decision-making from country selection, program selection and appraisal to monitoring and evaluation. Coupled with transparency and a culture of learning, this approach allows MCC to inform future investments and programming across the development community.
MCC is the only donor agency to base selection so heavily—and so transparently—on publicly available third-party data. MCC selects partner countries based on performance on indicators in three categories: Ruling Justly, Investing in People and Encouraging Economic Freedom. It applies hard hurdles in the areas of democratic governance and control of corruption. MCC publishes annual scorecards of country performance on its 20 selection criteria. This transparency allows countries to target reforms that matter for economic growth and allows stakeholders to hold MCC accountable for making selection decisions in accordance with its principles and mandate.
MCC uses economic analysis to inform investment decisions. Most poor countries have many development needs; stakeholders, both local and international, often struggle with setting priorities. MCC uses several tools to help identify investment opportunities that will be cost-effective and have the biggest impact in terms of raising local incomes and reducing poverty.
- MCC asks country partners to conduct a constraints analysis that identifies the main barriers to private investment and economic growth. Based on this analysis, and in broad consultation with civil society and the private sector, partner countries submit proposals for projects to overcome these barriers to growth.
- The program logic describes how a proposed investment is expected to reduce poverty through economic growth. It lays out the chain of activities and events by which a given project is expected to lead to increased household income and provides the foundation for project design, economic analysis and evaluation questions.
- Partner countries and MCC then use cost-benefit analysis to estimate the expected increase in household incomes of each proposed project. This helps MCC distinguish among projects with significant potential to spur growth and reduce poverty versus those that may be politically popular within a partner country but do not have sufficiently promising returns.
- MCC calculates ERRs before the compact, at compact closeout and during implementation if projects are modified. The cost-benefit analysis describes how the dollars spent on each activity will lead to higher incomes. It generates an economic rate of return (ERR) that reflects the fundamental economic viability of each proposed investment— that is, whether the expected results justify the costs.
- MCC and partner countries use a beneficiary analysis (BA) to assess how the project income gains estimated in the ERR analysis will be distributed across different income groups in a country or targeted region. The ERR and BA together inform project design and decisions about which country proposals to support in the MCC compact.
MCC is committed to delivering and measuring results throughout the entire lifecycle of its investments. From before investments begin to their completion and beyond, MCC’s robust and comprehensive “continuum of results” framework seeks to measure, collect and report on policy reforms associated with MCC compact eligibility and program investments, inputs and outputs that indicate whether projects are on track, interim outcomes as programs reach completion, and impacts as measured through independent evaluations. MCC’s continuum of results is designed to foster learning and accountability. Compact monitoring and evaluation plans transparently track project performance from processes and outputs at the beginning of a compact, to the high-level outcomes and impacts after a compact has finished in order to concretely assess whether the projects realized their economic returns, affected poverty and economic growth and the reasons why. MCC publishes quarterly reports of monitoring indicators against compact targets.
Leadership in Evaluation Practices
MCC uses independent evaluations for learning and accountability and to test assumptions about what works to reduce poverty. MCC independent evaluations build on monitoring to track and report on key input, output and some outcome indicators during compact implementation. It is common in the development community to focus on inputs (such as the funds dedicated to farmer training), outputs (such as the number of farmers trained) and increasingly on some intermediate outcomes (such as the rate of farmer adoption of improved techniques for cultivation).
However, MCC takes evaluation one step further to see if a link can be made between these indicators and an ultimate impact of increased household incomes. This is a difficult task and evaluations are the primary mechanisms for measuring whether or not that link occurred and why by testing the assumptions underlying the program logic. Results strengthen an evidence base to inform future investment decisions, both at MCC in the broader development community.
projects are modified. The cost-benefit analysis describes how the dollars spent on each activity will lead to higher incomes. It generates an economic rate of return (ERR) that reflects the fundamental economic viability of each proposed investment— that is, whether the expected results justify the costs.
MCC distinguishes between impact and performance evaluations.
- Performance evaluations are a valuable tool for estimating the contribution of MCC investments to changes in trends on outcomes, including household income. Performance evaluations are less rigorous and cannot attribute causal impact to MCC investments because they do not utilize a counterfactual. Performance evaluations serve an accountability purpose by comparing changes between the situation before the MCC investment occurred and after the MCC investment.
- Impact evaluations are the most rigorous form of evaluations because they estimate the causal impact of the MCC investment on key outcome indicators. They make it possible to know whether the observed impacts were caused specifically by an MCC investment or are the result of external factors. Impact evaluations compare what happened with the investment to what would have happened without it through use of a counterfactual. About 40 percent of MCC’s project portfolio is covered by impact evaluations.