Measuring Results of the Benin Access to Markets

Create PDF

The MCC compact with Benin was a five-year investment (October 2006 – October 2011) of $301.8 million. The $188.68 million Access to Markets component aimed to enhance the efficiency of Benin’s Port of Cotonou, reduce shipping costs to port users and increase the volume of goods flow.  The results of this component were the subject of an independent performance evaluation summarized here.

  • MCC’s investment logic relied on improved legal and institutional frameworks, construction of a new Wharf and jetty, and development of a revised port master plan that would increase the port’s cargo-handling capacity. And these improvements would increase efficiency and competitiveness, leading to increased use of port services – especially for exports – thereby contributing to national economic growth.
  • The evaluation confirmed that the port’s capacity, organization and use of modern equipment all improved as intended by the investment, leading to reduced operational costs and increased overall efficiency. It also confirmed increased port employment and significantly reduced petty corruption in port operations, also consistent with the investment logic.
  • However, the effects of these improvements on shipping market efficiency and competitiveness and their influence on broader economic outcomes could not be confirmed by the evaluation due to multiple factors:
    • Significant improvements in neighboring regional ports (primarily those in Togo, Nigeria, Ghana, and Cameroon) implied that the MCC-funded improvements surely helped protect Cotonou’s share of regional shipping, but did not increase its share, as the investment intended.
    • Shippers – ships paying directly to use port facilities – saw reductions in their port-related costs. But shipping lines – companies who charge for transporting cargo end-to-end – also saw cost savings, but did not pass these savings on to their customers, and therefore limited the potentially broader economic effect of reduced barriers to trade.
    • The evaluation has not been able to attribute changes in trade volumes to MCC investments as many other factors may have played a role
  • Both the design and the implementation of infrastructure projects like this would have benefitted substantially from better planning of the organizational and institutional changes needed to efficiently manage and support the investments.

This is the final evaluation. There are no further evaluation activities planned.

In Context

The MCC compact with Benin was a five-year investment (October 2006 – October 2011) of $301.8 million in 4 projects: Access to Financial Services, Access to Justice, Access to Land and Access to Markets. The Access to Markets Project included 4 major activities, Feasibility Studies/Assessment Activity, Port Institutional and Systems Improvements Activity, Port Security and Landside Improvements Activity, Waterside and Other Improvements Activity. The $188.68 million Access to Markets component is the subject of an independent performance evaluation released by MCC in 2018, the results of which are summarized here.  This component represents 62.5 percent of the total compact. The other components are the subject of their own independent evaluations.

Pie Chart of Benin Compact Spending

 

Program Logic

The Access to Markets Project was designed to improve port performance and security, expand capacity, and reduce port operating costs, thereby reducing overall transport costs which facilitate economic growth through increased specialization, product quality and trade.

Flow Chart of Program Logic

There were several key assumptions underlying the Access to Markets program logic during the design of the investment:

  • Improved Port infrastructure will reduce shipping costs to port users and enabling increased flow of goods through the Port facilitating real economic growth;
  • Improved Port infrastructure will decrease the average duration of stay of trucks at the Port;

For a more detailed version of the program logic, please refer to page 10 of the Benin M&E Plan, which can be found here: MCC/MCA-Benin Monitoring and Evaluation Plan.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources.  Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program.  However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention.  The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone.  Where estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

 

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

Indicators Level Baseline Actual Achieved Target Percent Complete 1
Volume of merchandise traffic passing through the Port of Cotonou Objective 4.1 mil MT

2004

8 mil MT

(09/2015)

6.3 mil MT 177.3%
Average duration of stay of trucks at Port Outcome 24 hours (2006) 15.33 hours

(06/2015)

7.0 hours 40%
Container ship waiting time at berth Outcome 2 days 1.30 days (09/2015) 1.0 day 65.0%
Container carriers waiting time at anchor Outcome 16 hours 56.88 hours (09/2015) 4 hours -338.7%  2
Average time to clear customs Outcome 3.80 days (2006) 2.07 days (06/2015) 1 day 64.5%
Annual number of thefts cases Outcome 40

(2006)

25

(09/2015)

20 75.0%
Execution rate of Training Plan Output 0

(2006)

96

(09/2011)

100 96%

Source: (Closeout ITT from September 2012 and Post-compact ITT from September 2015, which include data through implementation and post-compact periods, based on reporting from Benin UCF (Coordination and Formulation Unit)

 

The average completion rate of output and outcome targets is -13.0 percent; and in 14 of the 22 monitoring indicators, targets were met or exceeded. 3

               

Evaluation Questions

The evaluation was designed to answer questions such as:

  • On Competitiveness:
  1. How has the competitiveness of the Port evolved since 2006?
  2. Among the ports in the region, how has the competitiveness of the Port changed following completion of the works?

 

  • On Trade Volume:
  1. What is the relative change in the level of domestic and international traffic, volume of container and bulk maritime trade, value of trade (USD) and growth trends in relevant sectors before and after the improvements to the port?
  2. To what extent can changes in trade volume be attributable to MCC’s intervention?

 

  • Operational Efficiency:
  1. To what extent do the completed works mitigate/resolve observed constraints to port capacity and improve the efficiency of the operations as identified in due diligence and feasibility studies?
  2. How has the project affected the Port’s operational efficiency? What is the percentage change in the overall productivity of the port following completion of works?
  3. What percentage change in the port’s principal measures of operational efficiency can be observed following completion of the works?
  4. Has the level of congestion in the Port changed? If there has been change, what has caused the change?

 

  • On Costs:
  1. What percentage change in the port’s annual total direct costs (shipping, cargo handling and land transportation, etc.) can be observed following completion of the works?
  2. What is the relative change in the cost of doing business to importers, exporters, agents, transportation companies, and other businesses sensitive to port improvements?

 

  • Integration of Internal Markets
  1. To what extent has the port project contributed to achieving an overall compact objective of increasing the integration on internal markets?

 

  • On Employment:
  1. What net change can be observed in employment among the permanent and non-permanent employees in the port sector following completion of the works?

 

  • On Corruption:
  1. What has been the cost of corruption? Refer to evaluation methodologies developed by West African Trade Hub and World Bank?

 

  • On Unanticipated Impacts:
  1. What were unanticipated positive and negative impacts of port investments? What were unanticipated institutional, economic, et al. positive and negative impacts of port investments?

 

Evaluation Results

  The actual evaluation began in summer 2015, nearly four years after the closeout, providing ample time to detect effects of these operational changes. Because it was a single-site project, with no clearly identifiable counterfactual, MCC and the evaluators agreed to conduct a performance evaluation of the project implementation, focusing on a set of research questions grouped into seven categories: Competitiveness, Trade, Efficiency, Costs, Employment, Corruption and other Unanticipated Impacts. The analyses of these questions rely on:

  • Existing port activity records;
  • Private sector activity records;
  • Port Investment literature reviews;
  • Focus Group discussions with port users and dwellers; and
  • Key informant interviews (with Benin government officials and private sector actors).

A detailed evaluation design report is available on MCC’s evaluation catalog.

 
Evaluator             NORC (National Opinion Research Center) at University of Chicago with Nathan Associates and Agland
Impact or Performance? Performance
Methodology    Pre-post
Evaluation Period Program implementation concluded by October 2011. Final data collection was completed collection completed in September 2016
Outcomes

 

a.       Operational Efficiency (OE):

OE increased substantially at the South Terminal with the introduction of gantry cranes. Productivity reached 45 moves/hour in the first half of 2016, compared to 7-20 moves/hour prior to the investment: an improvement of between 125% and 542% in productivity. In comparison, during that period, Cotonou outperformed Lomé  and Tema (32 and 35 moves/hour, respectively)

Furthermore,   Benin Terminal’s crane productivity was 23 moves/hour for gantry cranes in the first half of 2016, marginally faster than its regional competitors (Tema at 22 and Abidjan at 21.5.)

 

b.      Costs:

Shippers experienced a reduction in import and export costs through Cotonou; a reduction comparable to other ports in the region with similar infrastructure improvements (Lomé and Tema).

The other main port users, the shipping lines 4 , also benefited from the improved facilities but their cost savings have not passed through the system due to the nature of the industry’s pricing practices.

 

c.       Competitiveness:

Larger ships are now able to call on Cotonou, as are gearless vessels (vessels without their own freight lifting facilities). Cotonou’s connectivity (as measured by the LSCI 5 ) has increased 61% since 2006. Traffic has increased, as have transshipment volumes. Cost for import/export goods has decreased and are competitive with other main regional ports. While some issues remain due to ongoing construction and piloting, overall the competitiveness of the port has improved, which is evident through its increased connectivity and traffic. Moreover, prior to the investment, Cotonou lagged behind its regional competing ports; it ranked worst out of competing ports in terms of fleet profile and connectivity. Today, Benin has caught up to its competitors. While it has not leaped ahead, and although the evaluation could not establish a strong attribution linkage, stakeholders’ estimate suggests that the PAC would have likely fallen further behind without MCC investment

 

d.      Employment:

The evaluation team found that port employment increased from 2006 to 2015. Most of the increases in employment were due to in-sourcing, which reflects an increase in the PAC’s in-house skillsets.  Notably, there was a 39% increase in PAC permanent employees and a doubling of top executives. Labor costs for the PAC increased by 55 percent, corresponding with a 57% increase in profit. Thus, while employment increased, it did not affect profitability. However, these increases did not correspond with an increase in productivity, since employment productivity decreased 30 percent as measured in employment productivity per metric ton from 2008 to 2014 due to the fact that increases in volumes did not keep up with increases in employment.

 

e.      Internal Markets:

Markets are not well developed for intra-regional trade within West Africa. The NORC evaluation team was not able to identify any significant area of specific advance that the MCC project affected with respect to formal trade or, indeed, with respect to trade enhancement of any kind between African producers and African consumers within the project market target area.  With that said, one area of potential project related gain involves regional transport market.   This source of market integration derives from the development of intermodal through transport services and through intermodal rates by one of the shipping lines that were a beneficiary of the project.  However the cause and effect relationship between these regional markets’ integrating initiatives and the project are indirect.

 

f.        Corruption:

Evidence suggests that the project resulted in significant reductions in petty corruption that formerly took place on the port terminal and to curbing petty theft that formerly occurred within the port terminal.  The MCC terminal monitoring system, improved lighting and extended fence protection were the proximate causes of these improvements.  However, ancillary investments made in the port pick up and drop off appointment management system probably accounted for the largest share of these gains.   Previously access to the port was informal and loosely controlled.  With the implementation of the new pick and drop off appointment system personnel, truckers, agents and other key participants with a legitimate need to enter the port could be much more closely controlled.

With that said, corruption with respect to off terminal trade logistics and with regard to cross border trading vis a vis Niger, Burkina and Nigeria remains a significant problem.   The community of service providers clustered in Cotonou continues to have a strong informal sector orientation and to do business via informal networks of trading partners which both diminishes the competitiveness of the PAC and limits the growth service sector growth potential of the entire Benin economy.

g.       Unanticipated Consequences:

The reassignment of valuable port terminal space which followed the investment created “winners” and “losers” among terminal operators and shipping agents.   Clearly the biggest winner was Bollore, which now possesses an exclusive capability of handling large capacity container ships with the port.   Just as clearly, the state-owned stevedoring company is left without advance cargo handling equipment, interim cargo storage capacity or modern systems with which to track cargoes and generate load plans, and therefore handling much less of the shipping volumes.

Perhaps the most significant unanticipated impact of the MCC project was its effect in accelerating the tidal degradation of the South Side of Cotonou City.  The breakwater in which the MCC invested has accelerated the sea related erosion of sections of the city which are densely inhabited and where numerous businesses are also located.

Objective-level Outcomes Trade Volumes:

The evaluation team found that after the opening of the South Terminal, the port handled 25,000 more containers than predicted in 2013, and 48,000 more containers than predicted in 2014. These increases may have been due to the increased competitiveness of the port, but they also may have been due to other factors. That said, after considering capacity constraints at the port, it is likely that much of the increases in traffic from 2012 to 2014 can be attributed to the increased port capacity due to MCC investment

Effect on household income attributable to MCC Not measured

 

Lessons Learned

  1. MCC’s flawed logic and assumptions suggest that investment in incremental port capacity should lead directly to positive economic returns measured in terms of job creation in Benin’s manufacturing, agri­business processing and non-port service sectors. An extensive literature sug­gests otherwise: investment in port infrastructure is a necessary but not sufficient condition for economic growth. MCC acknowledges that a more rigorous constraint analysis upstream could have determined that neither port capacity nor efficiency were binding constraints to growth.

 

  1. The M&E indicators should contain measures of port operational efficiency, which are key measures of port performance. The timing of the baseline data for evaluation should be linked to the timing of the program implementation.

 

  1. Engineering design plans need to be subject to several checks, including ones from shipping agents and major container lines. Engineering design plans should be finalized only after the concession is awarded.

 

  1. MCC acknowledges that the project design underestimated the need for a focus on private sector development. Both the IMF and the World Bank have underscored the need for the GoB to depend more on private sector funding for infrastructure development and less on public sector funding.

 

  1. MCC acknowledges that the plan for PAC to transform itself from service port into landlord port have never materialized. If such plans are essential to the program logic, MCC should ensure that such changes will occur.

 

Next Steps

This evaluation is complete and there are no planned next steps.

Footnotes
  • 1. It should be noted that monitoring targets were not based on rigorous economic analysis or well-documented assumptions, which might help to explain the perplexing range of completion rates in the above table of indicators.
  • 2. In contrast of the large negative achievement rate for this indicator, the evaluation report indicates a moderate decline in waiting time at anchor between 2007 and 2015 (from 2 days to 1.65 days)
  • 3. These figures are calculated using all non-evaluation indicators with targets in the Access to Markets Project.
  • 4. The expansion of the PAC’s South Terminal has made it possible for more large ships to call on it compared to before. However, cost savings stemming from the improved facilities (i.e. reduced time at the port and deployment of larger vessels) did not translate in significant reduction on ocean freight rates for shippers
  • 5. LSCI: Liner Shipping Connectivity Index