Summary of Findings

Measuring Results of the Benin Access to Financial Services Project

View as PDF

The MCC compact with Benin was a five-year investment (October 2006- October 2011) of $301.8 million. The $15.5 million 3 Access to Financial Services (A2F) Project is the subject of an independent performance evaluation summarized here. The key findings from the evaluation are summarized in the Lessons Learned section:

  • Improving financial services is not synonymous with improving access to capital for MSMEs: The evaluation also found that improving access to capital by MSMEs requires additional conditions to be satisfied, such as reducing the risks associated with lending to MSMEs, and adjusting interest rates so that they can be afforded by such enterprises. The key point here being the Project focused on addressing the supply side of financial services, but not on improving the quality and/or creditworthiness of MSMEs.
  • Assess the entire value chain: The evaluation recommends that, for any future MSME support interventions, the project should assess the components of the entire value-chain in the design phase. These components include MSMEs’ basic accounting and internal management capabilities, relationships with buyers and suppliers, access to local infrastructure (especially electricity), as well as the types of competitors and substitute products available in the market.
  • Hardware and financial supervision alone is not enough: The evaluation contends that MFI support must also include a policy dimension to strengthen MFI governance systems, through training and clarifying of internal controls, not just providing hardware and financial supervision.
  • Need to enhance MFI governance structure to address health and economic sustainability of MFI sector: The evaluation found that while the project contributed to the revival and growth of the MFI sector in Benin, especially after 2010 Pyramid-Scheme Crisis, it did not influence the governance structure. Common governance weaknesses continue to be a hindrance to the economic sustainability of the MFI sector.

In Context

The MCC compact with Benin was a five-year investment (October 2006- October 2011) of $302 million in 4 projects: Access to Justice, Access to Land, Access to Markets and Access to Financial Services. The Access to Financial Services Project included two major activities, the Financial Institution and Borrower Capacity Building Activity, and the Financial Enabling Environment Activity.  The $15.5 million Access to Financial Services Project is the subject of the independent performance evaluation summarized here and represents 5.1 percent of the total Compact budget. The remaining Projects are covered by separate independent evaluations.

Pie Chart: Benin Compact Spending

Program Logic

The Access to Financial Services Project (A2F) aimed to improve the efficiency of micro, small and medium-sized enterprises (MSMEs) to respond to opportunities by reducing costs and improving access to financial services. The Project was expected to improve financial services provided to MSMEs, increase microfinance institutions’ (MFIs’) self-sufficiency, decrease MFIs’ portfolio at risk, and increase loans guaranteed with land titles.

 

Financial Enabling Environment Activity:

  • Strengthen supervision of microfinance institutions;
  • Arrange stakeholder forums on expanding financial services;
  • Improve the financial sector regulatory environment;
  • Improve the microfinance credit information bureau

 

Financial Institution and Borrower Capacity Building Activity:

  • Create a grant facility to support MSMEs and to expand financial services; and
  • Help financial institutions increase lending based on land as collateral.
Flow Chart of Program Logic

There were several key assumptions underlying the Access to Financial Services program logic:

  • Enhanced institutional capacity among financial institutions will lower costs and these savings in transaction costs will be passed on to MSMEs in the form of less expensive credit and other financial services;
  • Improved scale and scope of financial services will meet MSMEs’ demand for services;
  • MSMEs will use increased credit and other financial tools productively, resulting in increased value-added to MSMEs;
  • Financial institutions increase use of land titles as collateral for credit; and
  • CRITICIAL ASSUMPTION: MSMEs were already creditworthy and it was mainly a supply-side issue.

 

For a more detailed version of the program logic, please refer to page 9 of the 2011 Benin M&E Plan, which can be found here: MCC/MCA-Benin Monitoring and Evaluation Plan.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped by monitoring and evaluation sources. Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program. However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention. The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone. Because estimating a counterfactual in the Benin Access to Financial Services Project was not possible, MCC invested in a performance evaluation, which compiled the best available evidence and assessed the likely impact of MCC investments.

 

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

Indicators Level Baseline Actual Achieved (09/2011) Target Percent Complete
Value of credits granted by MFI institutions (at the national level) (Millions of CFA) Objective 67,091

(2005)

130,543 88,697 294%
Value of savings collected by MFi institutions (at the national level) (Millions of CFA) Objective 38,269

(2005)

66,446 107,154 41%
Number of loan recipients of Micro Finance Institutions at the national level (Number) Objective 122,769

(2005)

615,067 153,461 1604%
Number of savers among Micro Finance Institutions at the national level (Number) Objective 46,947

(2005)

1,461,696 987,071 340%
Average portfolio-at-risk > 90 days of microfinance institutions participating in the Challenge Facility (Percentage) Outcome 5.9

(2005)

6.7 3 -27% 1
Average portfolio-at-risk > 90 days of microfinance institutions at the national level (Percentage) Outcome 11

(2005)

6.5 6 90%
Number of MFIs inspected by CSSFD Outcome 27

(2005)

102 50 326%
Average time for treating an application for MFI authorization (Days) Outcome 90

(2005)

41.8 30 80%
Rate of MFI applications authorized by CSSFD (Percentage) Outcome 96

(2005)

75.6 100 -510%
Number of new loans guaranteed with land titles Outcome 218

(2007)

1,989 1,118 197%

Source: Post-Compact ITT from 12/26/2014 provided by UCF (Unité de Coordination et de Formulation) based on data provided by former CSSFD (Cellule de Surveillance des Structures Financières Décentralisées, renamed ANSSFD (Agence Nationale pour la Surveillance de Systèmes Financiers Décentralisés), Ministry of Finances, and other implementing entities.

 

The average completion rate of output and outcome targets is 187 percent; and in 20 of the 22 monitoring indicators, targets were met or exceeded. 2

 

Evaluation Questions

The evaluation was designed to answer:

Access to Financial Services Project (General)

  • Were the higher level goals, the lower level objectives, the strategy to achieve the objectives, and the specific activities clearly defined and consistent with one another?
  • Did the goals, objectives, strategy and the activities designed initially change over time?
  • Was the project well understood by the local actors? If not, why not?
  • What were the reasons and logic for any changes that were introduced during project implementation and the consequences of those changes?
  • Did the local context where the Compact was implemented favor or hinder the Compact activities? What were the main local constraints that influenced project implementation?
  • Was the Compact’s organizational set-up effective for achieving its objectives?
  • How sustainable were the improvements introduced under the Compact?
  • What lessons can be learned from the Compact for the implementation of future projects in other developing countries?

 

Financial Enabling Environment Activity

  • To what extent has the regulator contributed to improved overall MFI loan portfolio quality and financial performance?
  • To what extent has regulator played a direct role in providing early warning against adverse developments in any MFI?
  • To what extent has regulator supervision contributed to capacity building in MFIs in the sense of capability for supervision preparedness?

 

Financial Institution and Borrower Capacity Building Activity

  • Are transactions costs lowered?
  • Do transactions take less time?
  • Does loan portfolio quality increase?
  • Do deposits and lending activity increase?
  • Do operating costs decrease and profits increase?

 

Evaluation Results

According to MCC’s Close-Out Memo, the Access to Financial Services Project is expected to raise the incomes of approximately 745,147 Beninese by 2026, 20 years after Compact Signing. The evaluation was originally designed as an impact evaluation to be carried out by NORC (National Opinion Research Center).  Due to challenges related to internal validity of the original evaluation methodology (including loss of representative control vs. treatment groups) as well as contract management issues, MCC and NORC agreed to modify the design to be a performance evaluation.  Official documents from line ministries and implementing entities were the main data sources. Additional Focus Group discussions with former program participants and key informant interviews contributed to data used for the evaluation. A detailed evaluation design report has been released and available on MCC’s evaluation catalog.

 

 
Evaluator NORC (National Opinion Research Center)
Type Performance
Methodology Ex-Post
Evaluation Period The evaluation contract ran from October 2011 to June 2017, however the actual evaluation covered October 2015 through June 2017
 

Outputs

 

The project adopted an in-house approach for implementing the Challenge Facility instead of the planned hiring of an external agency, after two failed procurements. This shift happened late in the project implementation period because of the high costs of hiring a third party. The consequent time and disbursement pressure meant that the project was implemented over a two-year period instead of the original five-year period.
Outcomes: Strengthened capacity of MFIs to deliver financial services – The majority of MFIs expressed a positive and supportive judgement regarding the role of the regulator in strengthening the MFIs in Benin. MFIs were assessed in terms of a composite index using a weighted average of five key ratios with individual benchmarks. These ratios measure capitalization, portfolio quality, efficiency, liquidity, and leverage. Altogether eight MFIs showed no change in score before and after the grant, and a total of seven MFIs had a deteriorating score compared to pre- grant situation. Only three MFIs showed an improvement of the score as compared to pre- grant situation. There is a significant correlation (98% significance) between the MFI score and other key indicators such as assets, loans, savings, the number of branches, and the rate of growth of assets. In other words, institutions with good performance had the fastest growth.

Increase financing available to MFIs and MSMEs for equipment and other procurement: In terms of procurement, the MFIs fared better than the MSMEs. More than four fifths of MFIs (81%) found the equipment procured to be of high quality, compared to only 51% of MSME grantees, while 72% of MFIs – and only 30% of MSMEs – said the equipment was delivered on time. About 70% of the MFIs confirmed that they were consulted during the procurement process while only 30% of the MSMEs gave a similar response.

Objective-level Outcomes Increased access to credit for MSMEs – Improving access to capital for MSME, still remains a major challenge. Although about 41% of MFIs improved their access to commercial finance, MFI access to capital did not necessarily translate to MSME access to capital. One conclusion in this regard is that improving financial services is not synonymous with improving access to capital by MSMEs which requires additional conditions to be satisfied such as reducing the risks associated with lending to MSMEs and interest rates that are affordable by such enterprises.

 

Growth in revenues, profitability for MFIs and MSMEs – The financial analysis of the grantee MSMEs reveals a split between the enterprises continuing to operate with various degree of profitability (about 50% of the enterprises) and those that have either closed or are heavily dependent on further subsidies for continued operations. Among the grantees were 5 private enterprises, 23 associations, and 14 cooperatives. Of the five private enterprises, four are doing relatively well and one intends to start operations in late 2016 (GATID). Most enterprises that were supported lacked and still lack basic management tools such as a double entry accounting system, operational manuals and procedures, market research expertise, quality control and customer data base management. No significant effort was made to address these soft skills for strengthening enterprise capacity. There was an association between failed or failing enterprises and those who could provide financial data.

Effect on household income The evaluation did not measure changes in household income as the focus was on the MFI and MSME-levels.

 

Lessons Learned

  • Further understanding of local context: The project suffered from insufficient preparation and baseline data to understand the situation on the ground before the project was designed and commenced. The evaluation suggests that the project could have capitalized and built upon existing potential in the finance sector. For instance, many small enterprise start-ups use personal and family funds to get off the ground and maintain operations by raising private and informal capital and the role of “Tontines” – informal savings and credit clubs – was poorly understood at the start of the project. Understanding each of these could have helped inform project design and MSMEs targeting.
  • Improving financial services is not synonymous with improving access to capital for MSMEs: The evaluation also found that improving access to capital by MSMEs requires additional conditions to be satisfied, such as reducing the risks associated with lending to MSMEs, and adjusting interest rates so that they can be afforded by such enterprises. The key point here being the Project focused on addressing the supply side of financial services, but not on improving the quality and/or creditworthiness of MSMEs.
  • Assess the entire value chain: The evaluation recommends that, for any future MSME support interventions, the project should assess the components of the entire value-chain in the design phase. These components include MSMEs’ basic accounting and internal management capabilities, relationships with buyers and suppliers, access to local infrastructure (especially electricity), as well as the types of competitors and substitute products available in the market.
  • Hardware and financial supervision alone is not enough: The evaluation contends that MFI support must also include a policy dimension to strengthen MFI governance systems, through training and clarifying of internal controls, not just providing hardware and financial supervision.
  • Need to enhance MFI governance structure to address health and economic sustainability of MFI sector: The evaluation found that while the project contributed to the revival and growth of the MFI sector in Benin, especially after 2010 Pyramid-Scheme Crisis, it did not influence the governance structure. Common governance weaknesses continue to be a hindrance to the economic sustainability of the MFI sector.

 

Next Steps

This evaluation is complete and there are no planned next steps.

Footnotes
  • 1. It should be noted that monitoring targets were not based on rigorous economic analysis or well-documented assumptions, which might help to explain the perplexing range of completion rates in the above table of indicators. Many baseline values and targets were based on data from other local and international institutions.
  • 2. These figures are calculated using all non-evaluation indicators with targets in the Access to Financial Services Project.
  • 3. The $15.5 million is the total disbursement amount according to MCC’s Administration and Finances’ records