Summary of Findings

Measuring Results of the Ghana Financial Services Activity

View as PDF

The MCC compact with Ghana was a five-year investment (2007-2012) of $547 million. The $30 million Financial Services Activity is the subject of an independent impact evaluation summarized here.

  • The Financial Services Activity was designed to strengthen and improve capacity of rural and community banks in Ghana to enable them to deliver sound financial services to rural communities in the country. The activity sought to improve financial service delivery, operations, and access to information at rural banks with the objective of enhancing the depth and value of rural financial services and widening access to savings services and cash transfers.
  • The evaluation found that (1) All rural banks report a reduction of the average transaction time from 15 minutes to less than 5 minutes; (2) The rural banks indicate that real time monitoring of branches through the connectivity system has greatly helped improving the real-time supervision; and (3) All the rural banks agreed that by being connected to the national check clearing platform (CCC) check clearing times reduced to one day as opposed to five days and even up to one month previously
  • The evaluation found the following lessons (1) Future projects should anticipate growth in the volume of data after implementation and (2) Operating costs, particularly related to communication, rose substantially, primarily because of the major devaluations (over 100%) of the Ghanaian currency, which resulted in doubling of fees associated with a US dollar-based contract.
  • This evaluation is complete and there are no planned next steps.

In Context

The MCC compact with Ghana was a five-year investment (2007-2012) of $547 million in three projects:  the Agricultural Project, the Rural Development Project, and the Transportation Project. The Rural Development Project included three major activities, i) the Procurement Capacity Activity, ii) Community Services Activity, iii) Financial Services Activity. The $30 million Financial Services Activity is the subject of an independent performance evaluation released by MCC in April 2018, the results of which are summarized here.  This component represents 5.4 percent of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.

 

Pie Chart: Ghana Compact I Expenditure

Program Logic

The Financial Services Activity was designed to strengthen and improve capacity of rural and community banks in Ghana to enable them to deliver sound financial services to rural communities in the country.  The activity sought to establish a computerized networking system among all rural banks and the APEX Bank Server, thereby improving financial service delivery, operations, and access to information at rural banks with the objective of enhancing the depth and value of rural financial services and widening access to savings services and cash transfers.

Program Logic: Flow Chart

There were several key assumptions underlying the Financial Services Activity program logic during the design of the investment:

  • As a direct result of the inputs, it was expected that the rural banks would experience the following short-term outcomes: (1) Higher transaction speed for depositing and withdrawing money; (2) Shorter client queuing time at the branch; (3) Faster check clearing times; (4) Faster and more accurate statement printout; (5) Improved supervision of branches due to real-time connectivity with the branches; and (6) Improved internal controls.
  • In the medium and long-term, it was expected this would lead to: (1) Increased volume of transactions at rural banks; (2) Increased levels of client confidence in rural banks; (3) Enhanced integration of the rural banks in the banking system; and (4) Increased transparency of the rural banks.

For a more detailed version of the program logic, please refer to page 4 of the Ghana M&E Plan, which can be found here.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources.  Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program.  However, monitoring data is limited in that it cannot reflect the full range of targeted outcomes and cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention.  The limitations of monitoring data is a key reason why MCC invests in independent evaluations to assess the achievement of a broader set of program outcomes.  When feasible, MCC supports impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone.  When estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

 

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

 

Indicators Level Baseline (2007) Actual Achieved (March 2012) Target Percent Complete
Number of inter-bank transactions Outcome 516,656 3,286,179 1,475,900 289%
Value of deposit accounts in rural banks Outcome $283,421,931 $497,534,104 $1,117,776,371 26%
Number of rural banks automated under the automation/computerization and interconnectivity of rural bank activity Output 0 130 121 107%
Number of rural banks connected to the WAN Output 0 134 121 111%

Source: (e.g. Closeout ITT from March 2012, which includes data through the end of the compact, based on reporting from the Millennium Development Authority (MiDA))

 

The average completion rate of output targets is 109 percent and targets were met or exceeded in 2 of the 2 output indicators.  The average completion rate of outcome targets is 157 percent and targets were met or exceeded in 1 of the 2 outcome indicators. 1

 

Evaluation Questions

The evaluation was designed to answer the following questions:

  • Did the activity improve the speed and reliability of transactions?
  • Did the activity improve accuracy and availability of accounts information?
  • Did the activity reduce transactions and check clearing times?

There was no economic analysis conducted of this activity. Details of the benefits measured by the evaluation are in the Evaluation Design Report here.

 

Evaluation Results

The evaluator used the longitudinal bank data to examine the effect of computerization and connectivity on bank outcomes. The rollout of the intervention across banks was staggered over a 20-month period.  The first rural banks made the transition to automation in June 2010, while the final set of rural banks completed the process in February 2012. The evaluation takes advantage of this staggered rollout of the intervention – namely, the fact that different banks transitioned to computerization at different times over a 20-month timeframe – to identify and measure the effects of the activity on outcomes of interest. The longitudinal bank data gathered from monthly returns covers a longer period than the 20-month implementation timeframe, spanning from January 2009 (16 months prior to computerization of the first set of rural banks) to December 2013 (almost two years after the intervention ended). This first approach takes the form of an OLS regression.

 

Rural banks were originally selected for computerization and interconnectivity according to a set of prioritization criteria (availability of correct and balanced data, infrastructure readiness, information security compliance, basic computer appreciation for staff, etc.); however, the original priority order is not available, and the evaluator learned that the priority order was altered because of different circumstances and that the migration order was basically random in practice.   However, the evaluator was concerned that the rural banks that went live earlier may be ‘superior’ in unobservable dimensions that could introduce bias in the estimates. If this is the case, it is possible that the estimation of the activity effect will be biased upwards. Therefore, the evaluator also estimated the effect of the activity by comparing each bank with their own earlier performance. By applying this methodology, they took into account possible selection issues, as the identification of effects is based solely on changes within banks due to the computerization.  Using the monthly data for each individual bank, the general model can be specified as a fixed effects regression model for panels.

 
Evaluator             NORC
Impact or Performance? Impact
Methodology    Mixed methods using a difference-in-differences fixed effects regression and an OLS regression for the quantitative analysis.  The qualitative study looked at the perceptions of the bank managers regarding the activity and performance trends with the aim of better explaining the impact of the activity.
Evaluation Period Banks received the treatment over a 20-month period.  The first rural banks made the transition to automation in June 2010, while the final set of rural banks completed the process in February 2012. The data spans from January 2009 to December 2013.  The qualitative study took place in August 2017.
Outcomes  

Improved speed and reliability of transactions

·         All rural banks report a reduction of the average transaction time from 15 minutes to less than 5 minutes and in some cases as low as 2 minutes.  The quantitative analysis indicates that the value of accounts and average balances per customer have increased due to the activity, although the number of customers was not affected- indicating a possible improvement in the customers’ satisfaction which could be linked to speed and reliability of bank services.

 

Improve accuracy and availability of accounts information

·         The rural banks indicate that real time monitoring of branches through the connectivity system has greatly helped improving the real-time supervision and getting more accurate reports as compared to before the project.  The quantitative analysis shows a positive effect of the activity on the value of accounts and average balances per customer, which is consistent with improvements in the customer experience. The evaluation also found a positive effect on bank net income; however, bank expenses per customer have not changed.

Reduced transactions and check clearing times

·         All the rural banks agreed that by being connected to the national check clearing platform (CCC) check clearing times reduced to one day as opposed to five days previously and even up to one month for out of station checks. The power generatorshelped to continue with the normal operation during power interruptions, reducing transaction times.

Objective-level Outcomes N/A
Effect on household income attributable to MCC N/A

 

Lessons Learned

Data Volume – Future projects should anticipate growth in the volume of data after implementation. The Data Center became obsolete 2 years after the end of the project and had to be replaced by a new server to accommodate the growth of data.

Operating Costs – Operating costs, particularly related to communication, rose substantially, primarily because of the major devaluations (over 100%) of the Ghanaian currency, which resulted in doubling of fees associated with a US dollar-based contract.

Product Development – The dependence of the rural banks on the APEX Bank for product development is a concern for two reasons. First, each bank has different needs that require local solutions. Their initiative for introducing modern technology for addressing their local issues is currently blocked since no new product can be introduced in a single bank without it being accepted and generalized for all the rural banks through the T24 software. Second, product development can be a major source of growth for rural banks. As such, any structural arrangements or relationships that stymy product development serve as an impediment to the growth of the rural banks. Future projects should incorporate into plans and stakeholder discussion a degree of autonomy for individual financial institutions.

 

Next Steps

This evaluation is complete and there are no planned next steps.

Footnotes
  • 1. These figures are calculated using all non-evaluation indicators with targets in the Rural Development Project/Financial Services Activity.