One of the ways MCC is changing the conversation about foreign assistance is via its approach to monitoring and evaluation (M&E). This starts with tracking performance on processes and outputs at the beginning of a compact, and continues to tracking high-level outcomes and impacts at the end of a compact in order to concretely assess how a project’s activities reduced poverty and promoted economic growth. Economic rate of return analyses are used to estimate aggregate impacts, and MCC’s impact evaluations further measure the achievement of project results.
Water, Sanitation, and Irrigation Results
persons trained in hygiene and sanitary best practices
water access points constructed
of contracted water and sanitation works disbursed
All figures as of 3/10/17 2:44 pm.
Progress Towards Results Across Compacts To Date
Significant progress has been made towards cataloguing long-term results and impact. The latest results by country program and by sector demonstrate milestones achieved and beneficiaries reached. MCC tracks not only outputs but also outcomes and impacts across compacts. Those higher-level results materialize as compacts mature, and will be published as soon as they are available.
By Country Program
MCC aggregates results in key sectors to measure progress in those areas across Compacts.
|Land and Property Rights|
|Water, Sanitation, and Irrigation|
In all of these sectors, MCC plans to track not only outputs, but also outcomes and impacts across Compacts. Those higher-level results materialize as compacts mature and will be published as soon as they are available.
To fulfill MCC’s commitment to tracking results on poverty and economic growth, M&E is integrated into all phases of compact operations—from compact development through implementation. M&E also continues to engage after compact closure. The diagram below demonstrates how this takes place and how the results that are tracked become part of a feedback loop that is used to improve performance during a compact and to apply lessons learned to future compacts.
Integrating M&E in Compact Operations
Every country’s approach for assessing progress towards results is outlined in its compact M&E Plan, the central tool for monitoring and evaluating the compact’s activities. Development of the M&E Plan begins during compact development, and it is completed around the start of compact implementation. The M&E Plan is developed collaboratively by the Millennium Challenge Account, the local accountable entity responsible for implementing the compact, and MCC. The plan is regularly modified during the life of the compact to reflect changes in program circumstances or when new information is available.
Monitoring Compact Performance
As a federal agency, MCC is committed to the principles of performance measurement mandated under the Government Performance and Results Act (GPRA). The agency applies this same focus on results to its compact programming. MCC monitors progress towards compact results on a regular basis using performance indicators specified in the M&E Plan. The M&E Plan specifies indicators at all levels – process, output, outcome, and impact levels – so that progress towards final results can be tracked at every step. Lower level indicators (process and output) and their targets are typically drawn from project and activity work plans, with results estimated to materialize throughout years 1-3 of the compact life cycle. Higher level indicators (outcome and impact) and their targets are often, though not always, directly linked to the economic rate of return analysis that is conducted to estimate the impacts of the investment and are drawn from the benefit streams of that analysis. These results are estimated to materialize in years 3-5 of the compact life cycle for outcome indicators and after 7 or more years for impact indicators.
Every quarter, each MCA submits an Indicator Tracking Table (ITT) that shows actual performance of each indicator relative to the baseline and target levels that were established in the M&E Plan. MCC reviews this data every quarter to assess whether results are being achieved and integrates this information into project management decisions.
Data for performance monitoring and reporting comes from baseline and follow-up surveys, project implementers, and other entities. MCC strongly supports comprehensive, quality data collection conducted by local resources and frequently uses compact M&E funds to invest in surveys fielded both by private firms and national statistical agencies or other government entities.
All collected data, whether from surveys or implementers, undergoes regular data quality checks managed by the MCAs and monitored by MCC to ensure their integrity and accuracy.
Evaluating Compact Performance
Evaluation is a cornerstone of MCC’s focus on results and is key to measuring impacts during and at the end of a compact. While monitoring a compact’s performance against indicators and targets is important, it is not sufficient to fully assess a compact’s effectiveness, particularly on poverty and economic growth. MCC therefore supports a variety of evaluation types to complement performance monitoring and deepen the analysis of data gathered to measure results at higher levels. Outcome and impact indicators are often captured by such independent evaluations.
Generally, the following two types of evaluations are conducted.
Impact evaluations measure the changes in income, changes in other aspects of well-being, or both that are attributable to a defined intervention. Impact evaluations require a credible and rigorously defined counterfactual, which estimates what would have happened to the beneficiaries absent the project. Estimated impacts, when contrasted with total related costs, provide an assessment of the intervention’s cost-effectiveness.
Performance evaluations seek to answer descriptive questions, such as:
- what were the objectives of a particular project or program;
- what the project or program has achieved;
- how it has been implemented;
- how it is perceived and valued;
- whether expected results are occurring and are sustainable; and
- other questions that are pertinent to program design, management, and operational decision making.
MCC’s performance evaluations also address questions of program impact and cost-effectiveness.