2015 Report on Closeout ERRs

View as PDF

The main purpose of this report is to concisely document facts on ERRs that have been updated upon compact closeouts as of end-2015. The report‘s first three sections describe coverage, characteristics of all closeout ERRs and characteristics of closeout ERRs that fall below MCC’s ten percent threshold. Section IV presents original ERRs for open compacts using data available as of end-2015. The data used to estimate these closeout ERRs reflects final cost data as available at closeout. As the benefits stream for most projects may only start after compact closeout, these closeout ERRs remain an ex-ante estimate of economic return.

Authors
Sandra Ospina (Economic Analysis, Senior Director), Ospinasp@mcc.gov Marissa Block (Economic Analysis, Research Assistant)
Table 1
Overall Description of Data – Closed Compacts
Number Funds Committed Funds Disbursed
Total Closed Projects 122 5,482.10 5,283.80
Projects with Closeout ERR 76 3,059.28 3,232.71
Projects without Closeout ERR 46 2,422.81 2,051.09
Closeout ERR Pending 9 702.77 634.10
Projects for which no ERR will be calculated 37 1,720.04 1416.99
Completed, but no ERR planned 22 865.02 825.01
Cancelled Projects 15 855.03 591.98

Characteristics

Closeout ERRs are on average above MCC’s 10 percent threshold. ERRs are estimated at the lowest possible level of disaggregation. When reporting summaries for compact statistics—as we do here—ERRs are sometimes aggregated by project or by activity using funds as weights. The majority of reported ERRs, 59 out of 76 closeout ERRs, are calculated at the activity level while the remaining 16 are calculated at the project-level. Table 2 shows weighted and un-weighted average ERRs for different categories. The table shows no significant differences in these averages, with most ERRs fluctuating between 12 and 17 percent. The exceptions are in the compact category with Nicaragua’s at 2.10 percent—the lowest overall—Burkina Faso’s at 5.8 percent, and Lesotho’s at 8 percent. Morocco, El Salvador, Namibia, and Ghana’s weighted closeout ERR stand out at around 20 percent. The average ERR for the 76 projects weighted by disbursed funds at compact closeout is 15.2 percent. While it is not explicitly shown in Table 2, it should be noted that the total number of projects in Table 2 excludes projects for which no ERR will be calculated and projects with pending ERRs. Burkina Faso and Tanzania have five and four projects respectively with pending ERRs, and therefore the compact ERR will ultimately change. Additionally, Cabo Verde and Honduras have three and two projects respectively, but both consist of projects for which no ERR will be calculated, and are thus excluded from the table. See Table 7 and 8 in Annex II.
Table 2
All Projects with Closeout Data
Sector Weighted Closeout ERR Unweighted Closeout ERR Total Projects Funds Disbursed (US$M)
Agriculture and Irrigation 18.7% 18.0% 18 676.80
Energy 15.1% 13.7% 4 101.63
Finance, Invest., Trade 13.0% 11.3% 6 312.73
Health, Ed. & Community 15.7% 21.6% 14 432.64
Land 12.1% 13.8% 6 190.90
Other* 14.2% 15.3% 3 66.10
Transport 15.8% 12.5% 11 1022.84
WASH 11.1% 12.7% 14 429.08
Closeout Year Weighted ERR Unweighted ERR Total Projects Funds Disbursed (US$M)
2011 11.5% 9.9% 10 748.22
2012 20.1% 14.5% 15 668.82
2013 14.8% 15.9% 39 1537.57
2014 16.0% 21.2% 12 278.10
Size at Closeout (US$) Weighted ERR Unweighted ERR Total Projects Funds Disbursed (US$M)
< 50M 15.7% 15.9% 57 1255.01
50M – 100M 14.0% 14.5% 12 757.45
> 100M 15.6% 15.5% 7 1220.25
Country Weighted ERR Unweighted ERR Total Projects Funds Disbursed (US$M)
Armenia 12.3% 14.6% 2 36.90
Benin 12.8% 11.2% 3 278.13
Burkina Faso 5.8% 5.8% 1 66.20
El Salvador 20.3% 15.1% 8 417.10
Georgia 12.2% 8.0% 3 309.70
Ghana 19.8% 13.9% 7 251.72
Lesotho 8.0% 9.8% 6 309.95
Mongolia 12.4% 13.0% 7 223.42
Morocco 19.5% 19.2% 15 616.20
Mozambique 15.2% 20.7% 8 310.80
Namibia 19.2% 22.5% 11 211.90
Nicaragua 2.1% 2.1% 1 57.80
Tanzania 10.0% 6.8% 3 77.20
Vanuatu 10.3% 10.3% 1 65.69
All Projects 15.2% 15.7% 76 3232.71
*This includes Namibia’s tourism projects. Closeout ERRs have final cost data but do not include final benefits. Final benefits will in most cases take a number of years to be realized although it is possible at compact closure to have an updated estimation of the benefits. Notwithstanding that closeout ERRs are still not final ERRs, it is useful to compare closeout ERRs with their original estimates. As Chart 1a shows, the norm is that original project ERRs are above their corresponding closeout ERRs. Of the 70 projects for which there are original and closeout ERRs, the weighted average for original ERRs is 19.6 percent compared to the 15.1 percent for corresponding closeout ERRs.
Chart 1a
Chart 1b shows the difference in percentage points between closeout ERRs and original ERRs. The chart shows that for most projects the ERRs decreased, sometime exceeding a fall of 20 percentage points. The right hand tail in Chart 1b shows the 14 projects for which the closeout ERR increased relative to their original ERR (20% of all projects with original and closeout ERRs). The average of the absolute value of the gap (closeout ERR – original ERR) is 6.3 percentage points.
Chart 1b
Chart 1a
Chart 1b shows the difference in percentage points between closeout ERRs and original ERRs. The chart shows that for most projects the ERRs decreased, sometime exceeding a fall of 20 percentage points. The right hand tail in Chart 1b shows the 14 projects for which the closeout ERR increased relative to their original ERR (20% of all projects with original and closeout ERRs). The average of the absolute value of the gap (closeout ERR – original ERR) is 6.3 percentage points.
Chart 1b
More than one fourth of closeout projects have ERRs below MCC’s threshold and more than three-fourths of projects exhibit ERRs that decrease from EIF to close. Chart 1a reveals substantial variation across closeout ERRs. They range from -8 percent to 109 percent. Chart 1a identifies the 21 projects with an ERR below the threshold (nearly 30 percent of all projects) in red. Differences in ERRs range from a 67 percentage point decrease to a 29 percentage point increase as seen in Chart 1b.

Projects with ERRs below the threshold:

Projects with ERRs below the threshold are mostly small in size, concentrated in 2013, more prevalent in water and sanitation and transport projects, and evenly distributed across compacts. Table 3 disaggregates projects with closeout ERRs that fall below 10 percent in different categories. The average ERR for the 22 projects is 4.5 percent corresponding to approx. US$904M or 28 percent of disbursed funds. The table shows that more than two thirds (68 percent) of the projects are small projects (< US$50M). However, only a quarter of all small projects fall below the threshold, whereas 42% of all projects in the US$50-$100M category fall below the threshold. Roughly 30 percent of the disbursed funds correspond to two high value projects that sum up to US$271M. The two projects are Mozambique’s Rehabilitation/Construction of Roads Project (US$127.7M) and Lesotho’s Health Sector project (US$128.5M). Almost all sectors have at least one project with an ERR below the threshold. The water and sanitation sector has the most number of projects amounting to 40 percent of the value of the sector’s portfolio. The transport sector’s five projects amount to the highest value of disbursed funds, though this amounts to 30 percent of the value of the sector’s portfolio. Land projects that fall below the threshold account for 55 percent of the value of the sector’s portfolio, the highest of all sectors. Two of the four energy projects fall below the threshold and account for 36 percent of the value of energy’s portfolio. The three health, education, and community projects account for 50 percent of the US$432.6M for the sector’s total, and one of these projects, the Lesotho project on health, accounts for 33 percent of the sector’s total. On the other hand, less than 20 percent of agriculture and irrigation projects fall below the threshold. Their value corresponds to seven percent of the value of the sector’s portfolio.
Table 3
Closed Projects with Closeout ERR below 10% Threshold
Sector Weighted ERR Unweighted ERR Total Projects Funds Disbursed (US$M)
Agriculture and Irrigation 2.5% 1.7% 3 47.80
Energy 7.7% 8.2% 2 36.90
Finance, Invest., Trade -0.9% -0.9% 1 15.10
Health, Ed. & Community 6.5% 6.4% 3 218.75
Land 5.8% 5.8% 2 105.40
Other N/A N/A 0 0.00
Transport 5.8% 4.0% 5 307.61
WASH -0.8% -0.3% 6 172.43
Closeout Year Weighted ERR Unweighted ERR Total Projects Funds Disbursed (US$M)
2011 3.6% 4.1% 4 183.30
2012 0.9% 1.2% 3 77.01
2013 4.5% 2.2% 11 482.18
2014 7.3% 6.5% 4 161.50
Size at Closeout (US$) Weighted ERR Unweighted ERR Total Projects Funds Disbursed (US$M)
< 50M 1.5% 2.0% 15 309.34
50M – 100M 5.9% 5.4% 5 323.30
> 100M 6.2% 6.3% 2 271.35
Country Weighted ERR Unweighted ERR Total Projects Funds Disbursed (US$M)
Armenia N/A N/A 0 0.00
Benin 5.8% 5.8% 1 39.20
Burkina Faso 5.8% 5.8% 1 66.20
El Salvador 3.6% 3.6% 1 19.80
Georgia 3.6% 4.2% 2 86.30
Ghana 0.0% 0.0% 2 57.21
Lesotho 2.3% -1.4% 2 183.78
Mongolia 9.4% 9.0% 2 76.50
Morocco -0.9% -1.0% 2 18.70
Mozambique 5.4% 1.7% 3 171.70
Namibia 8.3% 6.7% 3 95.30
Nicaragua 2.1% 2.1% 1 57.80
Tanzania 2.9% 2.7% 2 31.50
Vanuatu N/A N/A 0 0.00
All Projects 4.5% 3.2% 22 903.99
Increased costs and reduced scope are the most important reasons behind closeout ERRs falling below the threshold. Table 4 identifies the reasons for reduced ERRs for every project below the threshold. The single most listed category is increased costs, followed by reduced scope. Other reasons include poor data at EIF and poor project design. The reasons listed are mainly provided by the compact-assigned lead economists. The reasons for four projects are taken from the “Summary of Findings” of their corresponding evaluation reports or from the reports themselves. Annex III provides excerpts of the available “Summary of Findings” for projects with evaluations. For the remaining projects in Table 4, seven evaluations are underway, and six evaluations are planned. Three projects currently do not have any evaluations planned.
Table 4
Major Drivers of Reduced ERRs at Closeout
Country Project/Activity Reduced Scope Increased Costs Reduced Beneficiaries Poor Data at EIF Low Carbon Credit Prices Government/ Third-Party Progress Project Design Additional Information
Benin Access to Land X X While the target number of land titles were surveyed, only a quarter of them were expected to be issued, greatly reducing project scope. The project also experienced moderate cost increases.
Burkina Faso** Rural Land Governance X
El Salvador WASH X X X The original model assumed water-borne disease would be completely eliminated by provision of clean water, which was not the case.
Georgia Regional Infrastructure Dev. X X Inadequate Water Supply
Georgia* Energy Rehabilitation: North/South Gas Pipeline X X Gas losses, the main anticipated benefit, were substantially reduced before the rehabilitation work began due to maintenance that was completed by the government after original economic analysis, but prior to the start of MCC-funded projects.
Ghana Irrigation X Initial assessments indicate that two components of this project are not adequately maintained and that the third has deteriorated considerably. Given the current state of the infrastructure, it is unlikely that the project will have a positive return.
Ghana Trunk Roads X X
Lesotho Health Sector X X X The ERR reduced after factoring in increased costs, borne by the Government of Lesotho, decreased benefits due to incorrect baseline assumptions, as well as adding other donor costs for the benefit streams not attributable to MCC funding alone.
Lesotho Rural Water Supply X Limited economic potential in target community
Mongolia Choir-Sainshand Road X Lower-than-expected traffic
Mongolia Wind Park X Government inaction; construction delays
Morocco* Enterprise Support X X X Costs were greater than originally expected, which was the primary reason the decision was made not to scale up. Additionally impacts on value-added growth and business survival rates are difficult to detect and measure after a single year.
Morocco* Artisan Production X Artisans were both reluctant to adopt the new kilns or unable to obtain financing for their 20% contribution to the purchase of the kiln. As a result, at the time of the evaluation, only two gas kilns were operational compared to the target of 144 kilns.
Mozambique Quelimane WASH X X The original model was estimated based on old data and the project was put on hold. Once reliable data was obtained, project was restarted with leftover funds, but the scope was greatly reduced.
Mozambique Roads X X The project was reduced from four road segments totaling 491 km to two segments totaling 253 km.
Mozambique Mocuba WASH X The project was significantly de-scoped in order to focus on an emergency activity targeted at a hospital.
Namibia National Training Fund X There were delays in launching the fund.
Namibia Improving the Quality of General Education X
Namibia Land Access and Management Activity and Livestock Activity X
Nicaragua* Transportation X The project was partially terminated and funding ceased to activities not already contracted. Of those activities that were funded, costs increased.
Tanzania Morogoro Water Supply X Due to construction delays, the treatment plants were completed after the compact by the Tanzanians, impacting the percentage of O&M costs recovered and the volume of water produced by the plants at compact end.
Tanzania Mafia Airport X
*Projects with completed evaluations. **No original ERR.

Original ERRs for Open Compacts

Original ERRs have been estimated for the majority of the open projects. Table 5 shows that 63 percent of the 35 open projects have an original ERR. They correspond to 73 percent of committed funds. The majority of the funds for which ERRs are not estimated correspond to the Indonesia compact (US$M 383). 1 The nine pending ERRs correspond to El Salvador II and to Liberia.
Table 5
Overall Description of Data – Open Projects
Number Funds Committed
Total Open Projects* 35 3,195.68
Projects with Original ERR 22 2331.28
Projects without Original ERR 13 864.40
Original ERR pending 10 475.40
Projects for which no ERR Will Be Calculated 3 389.00
* Total open projects is defined as all projects of compacts which have entered into force. Therefore, Liberia and El Salvador II are included, but Ghana II and Benin II are NOT included. Comparison between currently open and closed compacts reveals a downward trend in original ERRs per compact. Chart 2 shows that for open compacts, except Malawi, the per-compact average original ERRs are lower than 20 percent. For half of 18 closed compacts, the per-compact average original ERR exceeds 20 percent. Table 6 shows weighted ERRs of different categories for open projects, including the Moldova and the Senegal compact for which the closeout process is underway. The weighted average ERR for all 22 projects is 20 percent. If Malawi’s original ERR is replaced with 18.7 percent—the ERR revised one year later—the weighted average ERR falls to 16 percent.
Chart 2
Table 6
All Open Projects with Original ERRs
Sector Weighted ERR Unweighted ERR Total Projects Funds Committed (US$M)
Agriculture and Irrigation 14.9% 12.1% 4 232.39
Capacity Building 40.3% 40.3% 1 54.3
Energy 48.1% 48.1% 1 332.2
Health, Ed. & Community 12.8% 16.0% 6 245.6
Land 22.0% 22.0% 1 18.75
Other* 12.6% 12.6% 1 120
Transport 14.2% 14.2% 4 653.43
WASH 16.4% 16.0% 4 672.61
EIF Year Weighted ERR Unweighted ERR Total Projects Funds Committed (US$M)
2010 14.7% 13.1% 7 623.39
2011 16.4% 19.9% 5 711.74
2012 43.2% 28.0% 3 393.75
2013 15.9% 15.0% 2 486.3
2014 12.5% 16.8% 5 114.1
Size at EIF (US$) Weighted ERR Unweighted ERR Total Projects Funds Committed (US$M)
< 50M 7.8% 7.9% 7 372.34
50M – 100M 16.2% 16.9% 5 316.3
> 100M 20.9% 18.6% 10 1934.14
Country Weighted ERR Unweighted ERR Total Projects Funds Committed (US$M)
Cabo Verde II 16.4% 18.0% 2 61.55
Georgia II 12.7% 16.8% 5 116.1
Indonesia 13.0% 13.0% 1 131.5
Jordan 16.0% 16.5% 2 275.01
Malawi 48.1% 48.1% 1 332.2
Moldova** 18.2% 15.6% 3 220.86
Philippines 16.7% 22.2% 3 436.73
Senegal** 12.8% 11.2% 4 402.53
Zambia 17.0% 17.0% 1 354.8
All Projects 20.2% 17.7% 22 2331.28
*Other refers to Philippines KALAHI-CIDSS **Moldova and Senegal Compacts have closed, but at the time of this report they were still undergoing the closeout process. Two compacts—Senegal and Georgia II—have projects with original ERRs below the threshold. Chart 3 plots committed project funds and their corresponding original ERRs for open compacts, including Moldova and Senegal. Two projects, Senegal’s Podor Irrigation and Georgia II’s Education Infrastructure corresponding to 9.5 percent of all open projects or 2.7 percent of funds committed for all open projects, have original ERRs below 10 percent.
Chart 3

Annex I

2015 Updates to the Closeout ERR Report

The following list details the changes and additions of projects from the dataset used to calculate the statistics found in the 2015 Report on Closeout ERRs. Overall, the total number of closed projects increased from 94 to 122 and eighteen new closeout ERRs were added (11 for Namibia, 3 for Tanzania, 2 for Lesotho, and one each for Burkina Faso, and Nicaragua). The total number of open projects decreased from 45 to 35. Updates
  • The status of the Closeout ERR for Ghana’s Post Harvest Handling Activity is that no ERR will be calculated, not pending.
  • The closeout ERR for Ghana’s Irrigation Activity changed from 13% to 0%. The operation of the schemes is not active.
  • The original ERR for Ghana’s Irrigation Activity is 15%. It was previously listed as 22%, the activity-level ERR consisting of Land Tenure, Farmer and Enterprise Training, Irrigation, and Credit Sub-Activities.
  • The original ERR for Zambia is 17%, not 13.7%
  • The following changes were made to Lesotho’s Closeout ERRs:
    • Rural Water Supply & Sanitation: -7.9%
    • Health Sector: 5.2%
    • Wetlands and Restoration Conservation: None planned
  • Eleven Namibia projects were moved from open to closed and two additional projects are included. Two out of thirteen projects will not calculate an ERR.
  • Nine Burkina Faso projects were moved from open to closed. Two projects will not calculate an ERR, one project was cancelled, and one project has a closeout ERR. At EIF, the original ERRs of the roads project were disaggregated by road segments for a total of seven ERRs. At closeout, the ERRs will be calculated at a more aggregated level on primary roads, for a total of three ERRs. Including roads, there are five pending ERRs.
Additions
  • Mongolia’s Baynzurkh Bridge and UB-Nalaikh Road Activities were added to the list of projects. These two activities have an original ERR, but were ultimately cancelled and thus have no closeout ERR.
  • The closeout ERR for Nicaragua’s Transportation project was calculated by the evaluator and is now included.
  • The closeout ERRs for Lesotho’s Private Sector Development Project were added at the activity-level for the Civil Legal Reform Activity and Land Administration Reform Activity. The Credit Bureau and National ID activity and Payment and Settlement Systems activity are included, but both were cancelled. The Gender Equality in Economic Rights activity is now included, but no ERR is planned.
  • Tanzania’s Water & Sanitation Project was split into two activities. The original and closeout ERRs for the Morogoro Water Supply Activity and Lower Ruvu Plant Expansion Activity are now included.
  • The closeout ERR for Tanzania’s Mafia Airport is now included.
  • Four activities for El Salvador II are included in the open compact data. The compact has entered into force. The ERR status is pending.
  • Six activities for Liberia are included in the open compact data. The compact has entered into force. The ERR status is pending.

Annex II

Table 7 – Explanation for Projects without an ERR
Projects without Closeout ERR: 46 Projects for which no ERR will be Calculated: 37
Completed, but no ERR planned: 22
Country Project Activity Reason
Armenia Irrigated Agriculture Irrigation Infrastructure A revised ERR was calculated in the third year of the compact (2008). At closeout, it was intended that the evaluation would calculate a post-compact ERR. However, this did not occur.
Benin Access to Justice Insufficient data on benefits at closeout to calculate an updated ERR.
Burkina Faso Agricultural Development Diversified Agriculture The compact was approved without an ERR for this activity. At closeout, there was insufficient data and information to produce an ERR.
Burkina Faso Bright 2 Project Insufficient data and information to quantify benefits.
Cabo Verde Infrastructure Port of Praia An ex-post ERR will be calculated as part of the evaluation.
Cabo Verde Infrastructure Roads and Bridges Insufficient data and information; initial estimates at the time of closeout indicate that the ERR would be below the 10% threshold; however, an ERR is expected to be calculated as part of an ex-post evaluation.
Cabo Verde Watershed Management & Agricultural Support Insufficient data and information
Georgia Enterprise Development Agribusiness Development Insufficient data and information
Georgia Enterprise Development Georgia Regional Development Fund Insufficient data and information
Ghana Agricultural Development Farmer and Enterprise Training, Credit & Land Tenure There is little evidence of expansion and intensification of farming and significant restructuring of project components weaken the capacity to calculate a relevant closeout ERR.
Ghana Agricultural Development Post-Harvest Handling Insufficient data and information at closeout to calculate an updated ERR and significant redesign of the project make it difficult to calculate a relevant closeout ERR.
Ghana Rural Development Community Services: Electrification The project was significantly de-scoped and at closeout the budget was too small to calculate an ERR.
Ghana Rural Development Financial Services Insufficient data and information
Ghana Rural Development Procurement Activity was too small to calculate an ERR ($1.5M).
Ghana Transportation Ferry Insufficient data and information to calculate an updated ERR.
Honduras Rural Development Decision to not conduct an ERR was made prior to the current economist.
Honduras Transportation Insufficient data and information at closeout to calculate an updated ERR.
Lesotho Water Sector Wetlands and Restoration Conservation Activity was too small to calculate an ERR ($4M).
Lesotho Private Sector Development Gender Equality in Economic Rights Activity was too small to calculate an ERR ($1.04M).
Namibia Education Regional Study and Resource Centers There is no evidence that libraries can be linked to student outcomes.
Nicaragua Rural Business Development Decision to not conduct an ERR was made prior to the current economist
Tanzania Energy Kigoma Solar Power Insufficient data and information
Cancelled Projects: 15
Country Project Activity Reason
Burkina Faso Agricultural Development Access to Rural Finance The activity was terminated after MCC determined it had not made sufficient progress in achieving project targets.
Cabo Verde Private Sector Development Financial Sector Reform N/A
Lesotho Private Sector Development Credit Bureau and National ID The GoL officially requested removing the National ID card aspects of the National ID and Credit Bureau Activity from the compact, to align the activity with its own national e-passport initiative.
Lesotho Private Sector Development Payment and Settlement Systems N/A
Madagascar Agricultural Business Investment Compact was terminated in 2009 due to the military coup.
Madagascar Financial Services, Land Tenure, & Agri-business Investment Compact was terminated in 2009 due to the military coup.
Mali Airport Improvement Compact was terminated in 2012 due to the military coup.
Mali Alatona Irrigation Compact was terminated in 2012 due to the military coup.
Mali Industrial Park Compact was terminated in 2012 due to the military coup.
Mongolia Rail The Government of Mongolia requested to withdraw the Rail Project from the compact due to an inability to fulfill certain conditions for transparency by the Mongolian rail company’s Russian partners.
Mongolia Roads Bayanzurkh Bridge Project was cancelled due to two failed procurements and insufficient time remaining within the compact to initiate a third procurement.
Mongolia Roads UB-Nalaikh Road Project was cancelled due to two failed procurements and insufficient time remaining within the compact to initiate a third procurement.
Mozambique Water & Sanitation Nacala Urban Water Supply Cancelled due to under-performing contractor
Nicaragua Property Regularization Compact was partially terminated in response to actions by the Nicaraguan government inconsistent with MCC’s eligibility criteria; as a result, funding for this activity stopped.
Tanzania Energy Malagarasi Hydro (Energy) Cancelled due to environmental considerations.
Projects without Original ERR: 31 Open Projects without Original ERR: 13
Projects for which No ERR will be Calculated: 3
Country Project Activity Reason Closeout ERR Status
Georgia II Improving General Education Quality Education Assessment Support There is no literature or standard practice on methods to quantify the benefits of testing. It is too early in the compact to determine.
Indonesia Green Prosperity As two of the four activities with the project are grant facilities, the size and composition of the individual grants at EIF were unknown and it was not possible to calculate an original ERR. The two remaining activities are complementary to the grant facilities. As a grant facility, there will likely be no project-level closeout ERR. However, ERRs for individual grants will be calculated.
Indonesia Procurement Modernization Insufficient information and data to quantify benefits. There will likely be no closeout ERR for the same reason listed for the original ERR.
Projects Included in Other ERRs: 1
Country Project Activity Reason Closeout ERR Status
Georgia II Industry-led Skills & Workforce Development Strengthening Sector Policy and Provider Practice The costs are bundled together with the Competitive Program Improvement Grants activity ERR, as the two activities are complementary.
Closed Projects without Original ERR: 18
Country Project Activity Reason
Burkina Faso Agricultural Development Access to Rural Finance Compact approved without a cost-benefit analysis for this activity.
Burkina Faso Agricultural Development Diversified Agriculture Compact approved without a cost-benefit analysis for this activity.
Burkina Faso Rural Land Governance Compact approved without a cost-benefit analysis for this activity. In 2011 an ERR was calculated for the phasing decision at 10.8%.
Burkina Faso BRIGHT 2 Project Compact approved without a cost-benefit analysis for this activity. This project built on MCC’s 2005 threshold program.
El Salvador Productive Development Finance and Investment Support This was a grant facility, so there was no data ex-ante to construct an original ERR.
Ghana Rural Development Financial Services Activity was considered too small to construct an ERR ($24M).
Ghana Rural Development Procurement Activity was considered too small to construct an ERR ($2M).
Lesotho Water Sector Wetlands and Restoration Conservation Activity was considered too small to construct an ERR ($5M).
Lesotho Private Sector Development Payment and Settlement Systems Activity was considered too small to construct an ERR ($1.7M).
Lesotho Private Sector Development Gender Equality in Economic Rights Activity was considered too small to construct an ERR ($1.04M).
Madagascar Agricultural Business Investment N/A
Mali Industrial Park N/A
Projects were added after EIF: 5
Country Project Activity Reason
Mongolia Energy & Environment MC Energy Investment Fund This project is a result of funds being reallocated from the cancelled Rail project.
Morocco Artisan and Fez Medina Literacy & Vocational Training This project is the result of a reallocation of funds away from other projects.
Morocco Fruit Tree Productivity Catalyst Fund The activity was added after a re-scoping decision. The funds for the activity are included in the other Fruit Tree Productivity activity closeout ERRs.
Namibia Education Vocational and Skills Training, VGTF The activity was added to complement the National Training Fund.
Tanzania Energy Kigoma Solar Power This was added at the re-scoping decision following the cancellation of the Malagarasi Hydro Activity.
 
Table 8
Compact-Level Summary of Projects without a Closeout ERR
Country Cancelled Projects Projects with No ERR Planned Pending ERRs As a % of Total Projects As a % of Total Funds Disbursed
Armenia 1 33.3% 76.7%
Benin 1 25% 6.5%
Burkina Faso 1 2 5 89%* 84.6%*
Cabo Verde 1 3 100% 100%
El Salvador 0% 0%
Georgia 2 40% 14.4%
Ghana 6 46.2% 41.5%
Honduras 2 100% 100%
Lesotho 2 2 40% 2.3%
Madagascar 2 100% 100%
Mali 3 100% 100%
Mongolia 3 30% 0.20%
Morocco 0% 0%
Mozambique 1 11.1% 16%
Namibia 1 8.3% 8.2%
Nicaragua 1 1 66.7% 40.8%
Tanzania 1 1 4 66.7%** 85.9%**
Vanuatu 0% 0%
*This will become 33.3% of total projects and 23.1% of total funds disbursed when the five pending ERRs become available. **This will become 22.2% of total projects and 18.2% of total funds disbursed when the four pending ERRs become available.

Annex III

Excerpts from MCC’s Summary of Findings

I. Measuring Results of the Morocco Enterprise Support Project

Economic Rate of Return (ERR) Estimates and Decision Not to Scale-up the Enterprise Support Pilot As there was insufficient empirical evidence on the extent to which post-creation business training would increase revenues, productivity and survival rates of new enterprises, the Enterprise Support Project was structured as a two-year pilot program subject to a rigorous impact evaluation using a randomized design. Scale-up of the project was made contingent upon the results of the pilot program producing an estimated ERR above 10 percent for the scale-up project. The ERR prior to the implementation of the pilot program was estimated at 13.5 percent. Upon results one year after implementation, the overall ERR including scale-up phase projections was estimated at 1.21 percent while the ERR of the scale-up phase alone was estimated at 7.21 percent. 2 MCC decided not to proceed with scale-up of the Enterprise Support Project as the revised ERR estimate did not pass the hurdle-rate of 10 percent and was not estimated to generate sufficient benefits to justify the costs of the project. ERR at Closeout (Project-wide, not exclusive to Support to IGAs Activity) At closeout, the results of the impact evaluation became available and it was possible to revisit some of the benefit assumptions that could not be updated at the time the scale up decision was made. The impact evaluation found SME failure rates were reduced by 30%, rather than the 20% reduction assumed by the original ERR. However, this has essentially no significant impact on the ERR (an interesting problem of itself that we have yet to resolve). The only other significant impact the impact evaluation found was an improvement in value added for the AGR-INDH sub-activity of 30%, vs. 5% as was assumed in the original ERR. Making this change to the parameter increased the ERR for the AGR-INDH sub-activity from -3.5% to 4.4%, a significant improvement but still not higher than the 10% threshold. The overall closeout ERR was -0.9%.”

II. Measuring Results of the Artisan and Fez Medina Project Artisan Production and Promotion and Fez Medina Activities

“Even though potters were convinced of the relevance of replacing traditional kilns with less polluting gas kilns, this activity faced serious reluctance for many reasons: (i) the 20% contribution of potters to the imported gas kiln was beyond the financial capacity of potters; (ii) potters were reluctant to contract loans from banks for various reasons, among which the reimbursement conditions and (iii) Banks were not willing to finance the majority of potters operating informal, unregistered businesses; (iii) the maintenance cost of imported kilns were perceived as a long-term burden. The project design did not take into account the potential of the local gas kiln manufacturers and favored imported kilns from the start. The project should have considered developing a local kiln based on local usage and adapted to the needs and capacities of Moroccan potters.”

III. Measuring Results of the Nicaragua Transportation Project

“The calculation of the ex-post ERR utilized the Roads Economic Decision model (RED) developed by the World Bank. This approach captures the project’s flow of net benefits to road users defined as benefits minus costs which are discounted across time to yield a single value, using pre- and post-construction traffic counts and estimates of future traffic counts. The ex-post ERR estimated by the independent evaluator showed that the roads failed to meet the 10 percent hurdle rate. The average for the project, as a whole, was estimated to be 2.1 percent. Actual capital costs were, on average, 2.2 times greater than those estimated in the feasibility studies.”

IV. MCG Gas Pipeline Rehabilitation: Project Summary and ERR Model Revised

“Before the rehabilitation work was undertaken, the ERR was estimated to be 11.7% based on a forecast of a reduction in the amount of gas leakage from the pipeline and a assumed corresponding sale of carbon credits due to the reduction in methane emissions to the atmosphere. However, the high gas leakage rate was reduced before the rehabilitation work was undertaken, and it was not possible to sell carbon credits.”

Bibliography

Cronshaw, Mark, and Gustavson Associates. MCG Gas Pipeline Rehabilitation: Project Summary and ERR Model Revised. Rep. 23 June 2010. Web. Millennium Challenge Corporation “Measuring Results of the Morocco Enterprise Support Project”, MCC, Summary of Findings, 2015 Millennium Challenge Corporation “Measuring Results of the Artisan and Fez Medina Project Artisan Production and Promotion and Fez Medina Activities”, MCC, Summary of Findings, 2015 Millennium Challenge Corporation “Measuring Results of the Nicaragua Transportation Project”, MCC, Summary of Findings, 2014. Millennium Challenge Corporation. “Economic Rates of Return: Country Spreadsheets.” Millennium Challenge Corporation. Web. Ospina, Sandra, Aaron Mitchell, and Aaron Szott. 2014 Report on Closeout ERRs. March 2015.
Footnotes
  • 1. See Annex II for further explanation as to why an ERR was not originally calculated.
  • 2. Given the nature of post-creation business support, impacts on value-added growth and business survival rates are difficult to detect/measure after a single year. However, costs were greater than originally expected, which was the primary reason the decision was made not to scale up. While the results of the impact evaluation as summarized in this report further supports MCC’s final decision not to scale-up (as well as the fact that costs at closeout were even higher than estimated at the time of the scale up decision), previous interventions in post-creation business support would suggest that some benefits from support may accrue several years after implementation, longer than the total of two years given to the pilot program to be evaluated. Unfortunately, due to concerns about attrition rates (e.g. difficulty in following up with businesses) and possible provision of support services to control group businesses, further impact evaluation of this Project is not feasible.