Summary of Findings

Measuring Results of the Ghana Education Sub-Activity

View as PDF

The MCC compact with Ghana was a five-year investment (2006-2012) of $536 million. The $9.4 million education component is the subject of an independent performance evaluation summarized here.

  • While a theory of change for the sub-activity was not explicitly stated, the evaluator found that MCC and MiDA were operating on the hypothesis that an investment in education infrastructure would result in improved access to schools and better learning environments, which would lead in turn to increased enrollment, reduced dropout, increased attendance, and higher completion rates.
  • In the five year compact timeframe, MCC constructed or rehabilitated 250 classroom blocks across 221 schools in Ghana. Schools rehabilitated by MCC were in better condition than schools not rehabilitated by MCC.  However, in many MCC schools that scored poorly on the school conditions survey, respondents highlighted problems with planning and design during implementation as a contributor to the current condition.  Respondents across MCC and non-MCC schools in all three zones perceived that investment in infrastructure had a positive effect on enrolment, attendance, completion, and learning.
  • There were two major lessons learned: (1) There was not a long-term maintenance strategy for the investment, causing many schools to fall into disrepair. Current MCC education infrastructure projects have included a greater emphasis on operations and maintenance, and MCC aims to deliver relevant learning from those evaluations.  (2) During implementation, there was poor monitoring of the construction and quality suffered as a result.  MCC now limits the locations for school construction so that better monitoring can take place.

This evaluation is now complete.

In Context

The MCC compact with Ghana was a five-year investment (2006-2012) of $536 million in 3 projects:  Agriculture, Rural Development, and Transportation. The Rural Development Project included 3 major activities: Procurement Capacity, Community Services and Financial Sector. The Community Services activity consisted of 3 components:  Education, Water and Sanitation, and Rural Electrification. The $9.4 million education component is the subject of an independent performance evaluation released by MCC in 2017, the results of which are summarized here.  This component represents 1.7% percent of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.

Pie Chart: Ghana Compact Spending

The Community Services Activity is equivalent to 58% of the Rural Development Project and 8% of the total Compact Investment.

*These figures are based on MCC obligations as of September 2016.

Program Logic

A theory of change for the sub-activity was not explicitly stated in the many documents reviewed by the evaluator. In a review of relevant documents, the evaluator found that MCC and MiDA were operating on the hypothesis that an investment in education infrastructure would result in improved access to schools and better learning environments, which would lead in turn to increased enrollment, reduced dropout, increased attendance, and higher completion rates.

These intermediate outcomes could then be linked to the overarching programmatic objective of an increase in long-term earnings by increasing the education level of community members. This underlying theory was most explicitly explored in the cost-benefit analysis where employment was linked with years of education—that is, completion of kindergarten (KG), primary school, junior high school (JHS), and secondary school. Although the linkage between education and economic growth was implied, the cost-benefit analysis did not specifically explore the linkage between education and an increase in agricultural productivity.

Flow Chart: Program Logic of Ghana Education Sub-Activity

There were several key assumptions underlying the Education Sub-activity’s program logic during the design of the investment:

  • For a more detailed version of the program logic, please refer to page 3 of the Ghana M&E Plan, which can be found here.  Note that this program logic is for the compact as a whole; as noted above, there was no specific program logic for this activity.The intervention would decrease dropout rates at both primary and secondary levels and more students would stay in school and complete secondary school.
  • The project schools would provide benefits to students and the community in terms of improved access to education over 20 years.
  • Maintenance of the schools would take place.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources.  Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program.  However, monitoring data is limited in that it cannot reflect the full range of targeted outcomes and cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention.  The limitations of monitoring data is a key reason why MCC invests in independent evaluations to assess the achievement of a broader set of program outcomes.  When feasible, MCC supports impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone.  When estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

Indicators Level Baseline

2007

Actual Achieved (03/2012) Target Percent Complete
Number of students enrolled in schools affected by education facilities sub-activity Outcome  

37,733

41,409 43,393 58% 1
Additional female students enrolled in schools affected by education facilities sub-activity Outcome 0 2,166 2,665 81%
Number of school blocks rehabilitated Output 0 44 35 126%
Number of school blocks constructed Output 0 206 253 81%

Source: (e.g. Closeout ITT from 07/2012, which includes data through the end of the compact, based on reporting from MiDA)

 

The average completion rate of output targets is 103.5 percent and targets were met or exceeded in 1 of the 2 output indicators.  The average completion rate of outcome targets is 69.5 percent and targets were met or exceeded in 0 of the 2 outcome indicators. 2

 

Evaluation Questions

The evaluation was designed to answer the following questions:

  • What are the current conditions of MCC investments made for the education sub-activity? How do the conditions of MCC investments compare to non-MCC-supported sites?
  • How did the implementation process and/or post-completion maintenance contribute to current conditions of MCC investments?
  • What other factors explain both perceived school-level outcomes and the current conditions of schools?
  • What are the perceived outcomes of the investments in school infrastructure?

The evaluation did not cover the following benefit streams that were modeled in the economic analysis of the program. More detail on this topic can be found in the Evaluation Design Report here.

  • Differential earnings based on years of education completed
  • Improve matriculation and schooling completion rates from primary to higher vocational education

 

Evaluation Results

To answer the evaluation questions, the evaluator supplemented existing data with two distinct but related data collection activities: a school conditions survey and cross-case studies.

The school conditions survey was a systematic examination of current school infrastructure conditions. This survey was completed in 221 MCC schools and 192 non-MCC schools where enumerators scored different aspects of school infrastructure. Ratings of condition were made on a three-point system—poor, average, and good—and each rating was followed up with a photograph of the object being rated. Using the ratings from the school conditions survey, the evaluator calculated an aggregate school conditions score for each school in the study.

Following the school conditions survey and preliminary analyses of the data gathered, the evaluator conducted nine district-level, in-depth case studies with cross-case analysis to answer evaluation questions about the processes that may have led to the current conditions of school infrastructure, and perceptions of key stakeholders on the relationship between the investments made and school-level outcomes such as enrollment, attendance, completion, and learning. Key informant interviews (KIIs), focus group discussions (FGDs), and community score cards (CSCs) were conducted with parents, students, teachers, school leaders or headmasters, district education officers, individuals responsible for operations and management, construction consultants and implementers, MiDA and MCC staff, and a representative from the Ministry of Education.

 

 
Evaluator             Social Impact
Impact or Performance? Performance
Methodology    School conditions survey with photographs to confirm condition, focus group discussion, key informant interviews, community score card
Evaluation Period The Education sub-activity took place in two phases. Phase 1 began between 2007 and 2008 and focused on rehabilitation including replacing roofs, doors, and windows; repairing cracks; reinstalling window and door locks; and providing classrooms with a new chalkboard and furniture.  Phase 2 of the sub-activity took place in 2009 and focused on construction.

 

Data collection took place in 2016, giving a 7-9 year exposure period.

Intermediate Outcomes In the five year compact timeframe, MCC constructed or rehabilitated 250 classroom blocks across 221 schools in Ghana.  MCC schools are in better condition than non MCC schools.  However, there is variability in the condition of MCC schools across and within the three zones (Northern Agricultural, Afram Basin, and

Southern Horticultural).

•  Poor planning and design at the implementation stage contributed to the current condition of school buildings across all zones and schools.

•  In the Northern and Afram zones, contractors sometimes subcontracted the work to others, in violation of their contract.  There was not enough oversight of the construction, and there was a perception that the subcontracting negatively affected the quality of construction.

•  There is no practice of routine or preventive maintenance across all zones and schools.

•  Lack of maintenance funding and community buy-in were cited as the top two barriers to maintenance. Low-scoring MCC schools had more maintenance issues than high-scoring ones.

Objective-level Outcomes ·   Perception across all zones in all study schools was that improvements in infrastructure positively affected enrollment, attendance, completion and learning.

·   Most respondents across MCC and non-MCC schools thought that the quality of teaching and learning at their school had improved. A higher percentage of respondents at high-scoring MCC schools felt this way than respondents at low-scoring MCC schools.

·   Enrollment has increased at both MCC schools and non MCC schools, with respondents in MCC schools citing much of their increase as being driven by students in poor-quality neighboring schools switching to MCC schools.

Effect on household income attributable to MCC N/A

Lessons Learned

  • A long-term maintenance strategy is necessary to maintain infrastructure investment. Lack of preventive or routine maintenance had a significant effect on the infrastructure condition of schools, permitting many to fall into disrepair and reducing buildings’ life and the potential economic and social gains that the schools could have brought to their communities. Making a specific plan for and supporting a culture of preventive or routine maintenance along with increased community ownership can significantly extend the life of school buildings.  The evaluation found that schools with more community ownership were better maintained.   Moreover, respondents were about four times more likely to cite a lack of maintenance financing as the driver of maintenance practices versus a lack of skills or training, highlighting that planning and financing should be a top priority for future investment rather than training.  Ongoing MCC projects in education infrastructure have included a greater emphasis on operations and maintenance, and MCC aims to deliver relevant learning from the evaluation of these components.
  • During implementation, there needed to be better monitoring of the construction; finished quality suffered as a result of poor monitoring. Subcontracting by local contractors—despite contractual specifications against it—was identified as a major challenge during implementation of the MCC intervention. To tackle this issue, GoG and donors investing in school infrastructure needed to review the system for monitoring and enforcing construction contracts to ensure the completion of schools which meet design specifications.  This was a function of the number of schools and the location.  MCC has taken this lesson and limited the regions in both Morocco and Cote d’Ivoire where schools are being constructed.  In these cases, MCA staff have been placed in the regions to better monitor implementation.  In this way, MCC has addressed the tension of expanding access to schools in remote regions, but in a way that is implementable.

 

Next Steps

This evaluation is complete and there are no planned next steps.

Footnotes
  • 1. Percent complete is affected by the fact that baseline and targets were set based on data from 327 classroom blocks, but actual recorded data is based on 250 classroom blocks constructed.
  • 2. These figures are calculated using all non-evaluation indicators with targets in the Rural Development Project /Community Services Activity/Education Sub-Activity.