Summary of Findings

Measuring Results of the Benin Access to Justice Project

View as PDF

The MCC compact with Benin was a five-year investment (2006-2011) of $301.8 million. The $19.4 million Access to Justice (A2J) Project is the subject of an independent performance evaluation summarized here.

  • With the ultimate objective of improving the investment climate of Benin, the A2J project was designed to facilitate greater access to legal and judicial services, decrease the cost and time to resolve commercial disputes, and restore confidence in the justice sector. The project aimed to achieve these objectives by carrying out interventions in three activity areas: the Courts Activity led to the construction of a number of courthouses and other complementary interventions; the Arbitration Activity sought to establish and equip an arbitration center and build arbitration capacity in the justice sector; and the Business Registration Activity oversaw the rollout of a new business registration system.
  • The time, cost, and complexity of the business registration process has improved, and new business registrations have increased; courts were built, and there was a modest reduction in caseloads, though some challenges persist; and despite some negative perceptions about the arbitration center, the evaluators found a functioning center whose arbitrators are professional and respected.
  • The evaluation suggests that a better understanding of the baseline conditions would have been preferred and may have limited the extent to which interventions were abandoned, reduced in scope, or delayed significantly.
  • This evaluation is complete and there are no planned next steps.

In Context

The MCC compact with Benin was a five-year investment (2006-2011) of $301.8 million in four projects:  Access to Financial Services, Access to Markets, Access to Land and Access to Justice. The $19.4 million Access to Justice Project included three major activities: the $0.6 million Arbitration Center Activity, $1.2 million Business Registration Activity, and $17.6 million Courts Activity. The Access to Justice Project is the subject of an independent performance evaluation released by MCC in November 2018, the results of which are summarized here. This project represents 6.4% percent of the total compact. Other components of the compact are the subject of their own independent evaluations.

Pie Chart of Benin Compact Spending

The Access to Justice (A2J) Project is equivalent to 6.4% of the total compact investment.

These figures are based on MCC obligations as of March 2016.

Program Logic

The Access to Justice Project was designed to strengthen Benin’s inadequate justice system, which MCC determined to be substantially hindering the country’s economic growth. The project was expected to improve the functioning of the justice system, facilitate the registration of more businesses and faster and cheaper resolution of contract disputes, and increase the public’s confidence in the system. Ultimately, with increased capacity to enforce contracts, the country’s investment climate was expected to improve and thereby stimulate economic growth.

Flow Chart of Program Logic

There were several key assumptions underlying the Access to Justice program logic during the design of the investment:

 

  • The main compact-wide assumption was that the A2J’s objective, jointly with the Access to Financial Services project, the Access to Land project and the Access to Markets project, would lead to the goal of reducing poverty through economic growth, thanks to an improved confidence in the ability of the justice system to enforce contracts and resolve legal claims.

 

  • The Arbitration Activity was implemented under the assumption that an increased use of ADR would decrease the time and cost of resolving commercial disputes and thereby lead to a reduction in the average time required to enforce a contract, and that backlogged court cases would automatically be transferred to alternative dispute resolution; however, in practice both parties had to agree to such a transfer.

 

  • The Business Registration Activity was designed under the assumption that a marked reduction in the time, effort, and cost needed to register a business would lead to an increase in the number of micro, small, and medium enterprises and their value-added.

 

  • The Courts Activity was designed under the assumption that by increasing the number of courthouses, there would be a reduction in the time and the cost required to resolve disputes.

 

For a more detailed version of the program logic, please refer to page 5 of the Benin M&E Plan, which can be found here: https://assets.mcc.gov/documents/me_plan-Benin_8.pdf

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources.  Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program.  However, monitoring data is limited in that it cannot tell us whether improvements in the investment climate in Benin are attributable solely to the MCC-funded Access to Justice Project.  The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone.  Because estimating a counterfactual in the Benin Access to Justice Project was not possible, MCC invested in a performance evaluation, which compiled the best available evidence and assessed the likely impact of MCC investments on investment climate in the country.

 

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

 

Indicators Level Baseline Actual

Achieved (09/2011)

Target Percent Complete
Average time required to enforce a contract (days) Objective 720

(2006)

825 370 -30%
Average time required for TPIa to reach a final decision on a case (months) Objective 9.1

(2006)

4.9 3 69%
Average time required for TPI to reach a final decision on a case, from court case filing to court final decision (months) Objective 21.23 (2008) 7.3 8 105%
Average time required for Court of Appeals to reach a final decision on a case (months) Objective 22.6

(2006)

34.4 15 -155%
Average time required for Court of Appeals to reach a final decision on a case, from case entry at Court to final decision (months) Objective 27.66 (2008) 38.2 24 -287%
Percent of firms reporting confidence in the judicial system (percentage) Objective 35

(2008)

54 60 76%
Percent of cases resolved in TPI per year (percentage) Outcome 38.3

(2006)

32 50 -55%
Percent of cases resolved in Court of Appeals per year (percentage) Outcome 14.8

(2006)

12.74 24 -22%
Average distance required to reach TPI (km) Outcome 35.7

(2002)

13 13.2 100%
Number of enterprises registered through the business registration centers (no.) Outcome 1822 (2005-06) 14,819 4,722 448%
Average time to register a business (days) Outcome 37

(2006)

7.2 2 85%
Average time to register a sole proprietorship with CFE (days) Outcome 39

(2006)

6.2 2 89%
Number of magistrates trained (no.) Output 38b 100 100 100%
Number of court clerks trained (no.) Output 60 98 100 95%
Number of court inspections performed (no.) Output 10

(2006)

32 60 44%
Average number of inspections per court per year (no.) Output 1

(2006)

1.3 2 33%
Number of users of the legal information center (no.) Output 0 N/Ac 100 N/A

Source: Closeout ITT from 06/2012, which includes data through the end of the compact, based on reporting from the Directorate of Land, Registration, and Seals (Direction des Domaines, de l’Enregistrement et du Timbre), General Business Census (Recensement Général des Entreprises—RGE), Directorate of Planning and Forecasting (Direction de la Programmation et de la Prospective), Ministry of Justice, Center for Legal Documentation and information, Center for Arbitration, Mediation, and Conciliation, One-Stop Business Registration Office, World Bank.

a First Instance Courts

b For this and subsequent indicator, training had begun prior to MCC intervention with support from other donors. No baseline year available.

c Construction of the legal information center was completed within the last several months of the compact; as a result, no users were recorded during compact implementation. However, in the nine months following compact closure, the center saw 1,032 visitors, or 1,032% of the target.

 

The average completion rate of output targets is 68 percent and of outcome targets is 35 percent; in 4 of the 17 monitoring indicators, targets were met or exceeded. 1

 

It should be noted that monitoring targets were not based on rigorous economic analysis or well-documented assumptions, which might help to explain the range of completion rates observed in the above table of indicators.

 

Evaluation Questions

The evaluation was designed to answer three broad questions:

 

  • Were the project activities fully executed and did they result in meaningful changes in practice, knowledge, or attitudes?
  • Have the results proved sustainable?
  • What lessons can be drawn from the implementation of the Project activities?

 

The evaluation then focused on the following specific, activity-based questions, which are addressed below:

  • Courts Activity
    • Is the public experiencing easier access to courts?
    • Is the new electronic case management system being used, and if so, for what types of cases? If not, why not?
    • Are skills learned in training being used by judges and court staff?
    • Has confidence in the court system increased?
    • Are more of the public accessing the legal information center?
    • Has the cost of filing a contract case decreased?
    • Has the amount of time to resolve a contract case decreased?

 

  • Business Registration Activity
    • Are business owners experiencing improvements in the business registration system?
    • Has the time necessary to register a business been reduced as a result of the creation of the one-stop business registration office?
    • Has there been an increase in the number of businesses registered?

 

  • Arbitration Center Activity
  • Are more contract disputes being resolved through arbitration?
  • Are users satisfied with the ability to access arbitration?
  • Are arbitrators using the training provided?

 

Evaluation Results

The evaluation for the Access to Justice Project was conducted by Millennium Partners and Tetra Tech DPK (subcontractor). Key deliverables included an evaluation design report and a final evaluation report.  That final report is the subject of this summary and the source of the results presented below.

Courts Activity
Evaluator             Millennium Partners & Tetra Tech DPK
Impact or Performance? Performance Evaluation
Methodology Ex-post
Evaluation Period November 2014 – February 2016**
 

Outcomes: Courts Activity

Although only five of the planned nine court houses were built, at the time of this evaluation visit, the newly constructed courts were functional, albeit with a skeleton staff in some cases, and faced problems with electricity, internet, equipment, and maintenance. 84% of people surveyed reported that their access to courts has improved with the addition of the new courts. The percentage of firms reporting confidence in the judicial system increased from a baseline of 35% in 2006 to 54% in 2011.

Based on observations and interviews 2014, the electronic case management system appeared to be working fairly well in First Instance Courts (TPIs), though the usefulness and functionality varied by court and there were reports that electricity and internet outages continued to affect its day-to-day functioning. Magistrates and court staff had requested further training on how to use the system.

Some judges had little or no training while others judges participated in several one-off trainings in the absence of any comprehensive training program. Whereas all Beninese magistrates received training during the life of the A2J Project, most of those appointed since 2011 have had no training since they took up their positions. 29% of users surveyed reported that training of court staff would improve access to justice.

The A2J Project established a Center for Legal Documentation and Information (CDIJ) in order to expand access to legal information. Although CDIJ was extremely delayed and is not the hub for legal activity that provides information on pending services and legal advice as expected, it is functional and appreciated by users. CDIJ also included a comprehensive electronic database of legal information and documents, containing all laws, decrees, and regulations formulated in Benin from 1959 to April 2011. Lawyers and court officials reported finding the electronic database to be extremely useful, well-conceived, and thorough. However, while legal information is readily available in the database, CDIJ has not taken any active steps to disseminate information on court decisions nor update the database since 2011.

Due to insufficient data availability, the evaluators were unable to reliably assess cost savings to residents who file cases or retrieve documents from the new courts.

In terms of the time taken to resolve a dispute, most key indicators worsened during the Compact period.  For instance, the average time required to enforce a contract increased from 720 days in 2006 to 825 days in 2011; the average time to reach a final decision on a case in the Court of Appeals increased from 27.7 months in 2008 to 38.2 months in 2011; the percentage of cases resolved in the Court of Appeals per year decreased from 14.8% in 2006 decreased to 11% in 2011; and the percentage of cases resolved in TPIs decreased from 38.3% in 2006 to 31% in 2011. On the other hand, the average time required for a TPI to complete a case (from case filing to final decision) decreased from 21.2 months in 2008 to 7.3 months in 2011. However, since the courts were only completed in 2011, it is unclear whether any of the changes to these key indicators relate to the Compact investments.

 

 

Outcomes: Business Registration Activity

Although the decree establishing the One-Stop Business Registration Office (GUFE) was signed in November 2011, it only became operational in 2012, i.e., after the end of the Benin Compact.

According to A2J Project documentation, the average time to register a business was reduced only slightly from 32 days at the launch of the project in 2006, to 31 at the close of the project in 2011, whereas GUFE documentation reported that the time to register decreased from 14.7 days in 2007 to 4.3 days in 2010. Furthermore, after a set of 2014 reforms, it now takes approximately 2-3 days to register a business.

Although detailed data on the cost of starting a business is not available, the World Bank estimates that the cost of starting a business decreased from 172.3% of per capita income in 2006, to 149.9% of per capita income in 2011. The evaluators describe a number of reforms that likely helped reduce the cost of starting a business, including the following two major policy reforms:

·         New businesses are now exempt from paying certain taxes for the first year of operation

·         The previous minimum capital requirement of
one million in local currency (CFA) 2 has been eliminated.

New business registrations for the first three years of the compact were 3,269, 5,128, and 4,365 respectively, a sharp increase over the 1,608 new registrations recorded in 2006. The increase in new business registrations continued, with 5,679 new registrations in 2011, and over 6,000 new registrations per year in 2012 through 2014. The evaluation team acknowledged that the tax benefits mentioned above might create a perverse incentive for people to abandon existing businesses just before the one-year mark just to establish new businesses, which would overstate the number of truly new businesses established. Furthermore, the evaluation team was unable to determine whether this increase in business registration has led to expanded employment opportunities and increased incomes.

Outcomes: Arbitration Activity There was a modest reduction in caseloads; 92 cases were resolved between 2006 and 2011, which otherwise would have been handled in the courts. However, the assumption that more cases would be transferred to the Center for Arbitration, Mediation, and Conciliation (CAMeC) was complicated by the reality that both parties have to agree to the transfer.

While 92 cases only amounts to 61% of the Compact target, the evaluators emphasize that some of these cases are highly complex and important, and with CAMeC clauses routinely included in all contracts of the Ports Authority, further reductions in court cases can be expected.

Informants stated that most arbitrations and mediations at CAMeC are resolved within 6 months or less, as compared to court cases, which can be delayed for years or even decades.

Of the CAMeC users interviewed by the evaluators, 80% expressed their satisfaction with the services offered by the arbitration center, and parties appreciate the speed and discretion of the process.

Likewise, 80% of arbitrators and staff report being satisfied with the training received. However, the evaluation could not assess whether the training had the intensity or frequency needed. Furthermore, the evaluation team found that some arbitrators had never been trained and CAMeC staff were eager to improve their knowledge of mediation, arbitration, and conciliation procedures, and related skills.

Objective-level Outcomes: This evaluation does not allow causal inference about the A2J Project’s objective of improving the ability of the justice system to enforce contracts and reconcile claims, thereby fostering economic growth. However, despite some clear positive changes reported in the Activity-level outcomes above, given the overall mixed results, including the decision by MCC to suspend funding for a number of A2J components, it is safe to conclude that more progress can be made.
Effect on household income attributable to MCC N/A

** This represents the duration of the Millennium Partners/Tetra Tech DPK evaluation. NORC had previously been under contract starting in September 2011 to conduct this evaluation along with evaluations of the access to land and the access to financial services projects. In September 2015 MCC formally cancelled NORC’s evaluations of the justice and land projects.

 

Lessons Learned

Below are the key lessons learned from the project as identified by the evaluators. More detailed, Activity-focused lessons are included in the final evaluation report:

  • Need for a rigorous problem analysis and baseline study before launching activities. The A2J project began with solid baseline information for the Business Registration Activity, which likely played a role in the relative success of that activity. However, the project lacked sufficient data for the Arbitration Center and Courts Activities. The fact that some of the A2J interventions were abandoned, reduced in scope, or completed very late in the Compact may have been mitigated with more up-to-date information about the legal landscape, activities of other key donors, and social culture of Benin before launching project activities. For example, it apparently took some time after launch of the A2J Project before it was clearly understood that the EU and MCA were implementing very similar interventions, which resulted in lost time and misunderstandings until the two began coordinating to eliminate overlaps. Similarly, it was assumed that court case backlogs could be reduced by transferring cases to CAMeC. However, according to Benin law, this can only be done when the conflicting parties agree to seek resolution at CAMeC. This fundamental difference in a key assumption of the Arbitration Center Activity had implications for what results could be expected, and for actions the Compact might have taken to improve the likelihood of achieving those results.

 

  • Need to hold development partners accountable. Upon signing the Compact, the Government of Benin (GoB) committed to completing a variety of complementary interventions but did not follow through. For example, it did not recruit sufficient magistrates, construct prisons to accompany the new courts, or fully commit to financially supporting and staffing the new courts. It also delayed formulation of a list of books and documents to be acquired by the CDIJ. Several of these commitments were conditions precedent but the A2J Project was still able to proceed thanks to deferring or waiving of these CPs. If certain reforms and complementary interventions are considered necessary conditions to the success of an investment, it is critical to hold development partners accountable for meeting them.

 

  • Need for change management. For a project requiring significant organizational changes, the A2J project did not implement sufficient outreach or change management activities. If project implementers and other stakeholders had felt more ownership over the changes, which could have been encouraged by better communication and change management support, the Arbitration Center and Business Registration activities might have avoided some of the delays they experienced.

 

Next Steps

This evaluation is complete and there are no planned next steps.

Footnotes
  • 1. These figures are calculated using all non-evaluation indicators with targets in the Access to Justice Project.
  • 2. CFA or XOF stands for Communauté Financière d’Afrique (“Financial Community of Africa”) and is the currency used in Benin and seven other West African countries.