Summary of Findings

Measuring Results of the Namibia Community Skills Development Centers

View as PDF

The MCC compact with Namibia was a five-year investment (2009-2014) of $304.5 million. The $16.8 million Community Skills Development Center (COSDEC) component is the subject of an independent performance evaluation summarized here.

  • The COSDEC sub-activity rehabilitated and constructed community-based vocational institutions to improve the employment and earnings prospects of clients from disadvantaged backgrounds.
  • 7 out of 7 COSDECs were completed and 15 out of 15 members of COSDEC management were trained.
  • About 85% of enrollees in the first cohort to benefit from the improved COSDECs reported that they had completed their COSDEC training.
  • One year after the end of training, only about 40% of enrollees reported that they were employed and approximately two-thirds of enrollees had no earnings from employment in the month prior to the survey. Despite having similar training outcomes, women were much less likely to be employed and were paid much less than their male counterparts.
  • A key lesson learned is that vocational training interventions need a credible approach for identifying skills gaps in the labor market.
  • This evaluation is complete and there are no planned next steps.

In Context

The MCC compact with Namibia was a five-year investment (2009-2014) of $304.5 million in three projects:  Education, Agriculture, and Tourism. The Education project sought to improve the quality of the workforce in Namibia by enhancing the equity and effectiveness of basic, vocational, and tertiary education. It included six major activities: Improving the Quality of General Education, Vocational Education and Skills Training, Improving Access to and Management of Textbooks, Investment in Regional Study and Resource Centers, Expanding and Improving Access to Tertiary Finance, and Cross-Project Support. The Vocational and Skills Training activity consisted of three sub-activities:  (i) establishment of a National Training Fund (NTF); (ii) competitive grants for high priority vocational skills training through the Vocational Training Grant Fund (VTGF); and (iii) expansion and improvement of Community Skills Development Centers (COSDECs) and training of the Community Skills Development Foundation’s management staff. The $16.8 million COSDEC component is the subject of an independent performance evaluation released by MCC in January 2018, the results of which are summarized here; the NTF and VTGF evaluations were released simultaneously.  The COSDEC component represents 5.5% percent of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.

Pie Chart: Namibia Compact Spending

*These figures are based on MCC obligations as of March 2016.

Program Logic

The Community Skills Development Centers (COSDEC) sub-activity was designed to address the quality of the country’s labor force through rehabilitating and constructing community-based institutions that provide basic levels of vocational training to clients from disadvantaged backgrounds – particularly out-of-school youth who lack access to the formal vocational training system – to improve their employment and earnings prospects. As illustrated below, the physical improvements to the COSDECs were expected to increase access to trainings and enable them to offer additional types of training, thus increasing overall enrollment. The technical support to the COSDECs and Community Skills Development Foundation (COSDEF) that supports and supervises the COSDECs was expected to result in improved management practices and adoption of accredited unit standards.  In addition, both the physical improvements and the improved pedagogical skills of instructors were expected to result in an improved quality of trainings.

In the intermediate term, the new infrastructure and tools, as well as management improvements, were intended to enable COSDECs to be formally registered and accredited, and offer officially accredited courses. More trainees were expected to complete training through the COSDECs and use the SME units to help start their own enterprises. In the long term, it was anticipated that this approach would increase training, employment, and earnings for enrollees—particularly among the disadvantaged—and contribute to the ultimate compact goals of decreased poverty and increased economic well-being.

 

Flow Chart: Program Logic


Acronyms: SME = Small- and medium-enterprises; TA = technical assistance

There were several key assumptions underlying the COSDEC program logic during the design of the investment:

  • Limited availability of qualified trainers will not constrain improvements in training quality and relevance.
  • Employers recognize and value the improved quality of the COSDECs.

For a more detailed version of the program logic, please refer to page 9 of the Vocational Training Evaluation Design Report, which can be found here.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources.  Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program.  However, monitoring data is limited in that it cannot reflect the full range of targeted outcomes and cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention.  The limitations of monitoring data is a key reason why MCC invests in independent evaluations to assess the achievement of a broader set of program outcomes.  When feasible, MCC supports impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone.  When estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on output indicators specific to the evaluated program.

Indicators Level Baseline (2009) Actual Achieved (09/2014) Target Percent Complete
COSDECs completed Output 0 8 8* 100%
COSDEC staff trained in management Output 0 15 15 100%

 

Source: Closeout ITT from December 2014, which includes data through the end of the compact, based on reporting from MCA-Namibia, the Namibia Training Authority.

*This total includes the seven COSDECs that were constructed or renovated and a warehouse at the Namibian College of Open Learning that was simply intended to allow the Namibian College of Open Learning to free up space for vocational training.

 

The average completion rate of output targets is 100 percent; and targets were met or exceeded in both of the output indicators. 1

Evaluation Questions

The evaluation was designed to answer the following questions:

  • To what extent did the COSDEC subactivity increase the availability of training?
  • How were the new and renovated COSDECs managed?
  • How did COSDEC training affect the employment outcomes of trainees?
  • How did COSDEC training affect the earnings and income of trainees?
  • Did the employment and earnings outcomes of COSDEC trainees vary by trainee characteristics?

 

Evaluation Results

The performance evaluation of the COSDEC Sub-activity integrates qualitative analysis and a quantitative outcomes analysis, the results of which are presented in separate tables below:

Qualitative analysis. The qualitative analysis explores implementation of the Sub-activity, how it evolved after the compact, and its sustainability. It relies primarily on two rounds of qualitative data collected from stakeholders through interviews and focus group discussions close to the end of the compact and again one year later. The evaluator used the data collected in each round to identify major themes related to the research questions by triangulating information from various data sources.

 

 
Evaluator             Mathematica Policy Research
Impact or Performance? Performance
Methodology    Other Performance
Evaluation Period ·         Implementation of Technical Assistance occurred from early 2012 to mid-2014; construction was finalized in mid-2014; and the first round of trainings in the new or renovated COSDECs began in July 2014

·         Round 1 qualitative data collected:  October-November 2014

·         Round 2 qualitative data collected:  November-December 2015

COSDEC Management ·         Most COSDECs increased the number of trainees and courses offered, although some increases were only in short courses.

·         Stakeholders report the new infrastructure provides a more conducive learning environment, improved perceptions of the COSDEC in the community, and enabled progress toward COSDECs’ meeting national registration and accreditation requirements, though the process is ongoing.

·         However, by the end of December 2016, communication with COSDEF revealed that none of the COSDECs was yet accredited or registered, although most were at an advanced stage with these processes.

·         COSDEC managers reported that their SME units were 80 to 100 percent full, although many had to waive user fees because trainees were not able to pay.

·         By the end of 2015, stakeholders reported that COSDECs could competently develop their annual budget plans, which play an important role in their funding approval.

They were also taking an active and collaborative role in strategic planning with the COSDEF, and had introduced a more inclusive management style.

·         In addition, the COSDECs continued to build on the instructor training provided under the Compact through off-site training aimed at further improving instructors’ pedagogical skills and the quality of instruction.

·         However, the technical assistance related to marketing of the COSDEC in the local community had not resulted in meaningful changes in marketing practices.

·         Funding for COSDECs primarily comes from the Namibian government, although they also collect tuition fees and revenue from other income-generating activities. Stakeholders were optimistic that government funding of COSDECs would continue at the required level ensuring sustainability, and are even focusing on increasing funding from the government to expand the COSDEC system.

Employment and Earnings N/A
Effect on household income attributable to MCC N/A

 

Outcomes analysis. The outcomes analysis seeks to describe the characteristics and outcomes of enrollees in the seven new or renovated COSDECs. It relies on a survey of COSDEC enrollees that collected information about their training and labor market outcomes about one year after the end of COSDEC training. The outcomes analysis is largely descriptive in nature and presents numerical descriptions of average outcomes for the full sample of enrollees and relevant subgroups. It is important to note that while these outcomes are important for understanding the current experience of participants, they cannot be interpreted as representing the direct impact of the intervention.

 

 
Evaluator             Mathematica Policy Research
Impact or Performance? Performance
Methodology    Ex-post
Evaluation Period ·         Implementation of Technical Assistance occurred from early 2012 to mid-2014; construction was finalized in mid-2014; and the first round of trainings in the new or renovated COSDECs began in July 2014

·         An outcomes survey of the first cohort of trainees to use the new or renovated COSDECS was conducted on a rolling basis 12 months after their training ended (from January to June 2016) to allow sufficient time for outcomes of interest to manifest

Training Completion ·         About 85% of enrollees in the survey analysis sample reported that they had completed their COSDEC training.

·         Trainees’ perceptions of training quality were generally very positive.

Employment and Earnings ·         At the time of the survey, one year after the end of training, about 40% of respondents reported being employed, and about 42% of respondents were productively engaged (defined as holding a paid job or being engaged in further vocational training).

·         Only about 13% of all respondents reported that they were employed in a job related to their vocational training. In addition, many who were employed held jobs that were temporary and/or with which they were dissatisfied.

·         Consistent with the low employment rate, approximately two-thirds of enrollees in the analysis sample had no earnings from employment in the month before the survey, and almost one-third had no individual income at all.

·         Female enrollees had similar training completion rates but significantly lower employment rates and earnings than male enrollees.

Effect on household income attributable to MCC N/A

 

Lessons Learned

  • Vocational training interventions need a credible approach for identifying skills gaps in the labor market. A key part of the Vocational Training Activity’s program logic involved shifting the Namibian vocational training sector from being supply driven to one that was driven by the needs of the labor market. The high unemployment rate among COSDEC graduates a year after they completed training suggests the need for a more effective process to align course offerings at the COSDECs with market demand; the evaluation of another sub-activity yielded a similar finding. Assessing market demand at a project level might include diagnostics of labor market trends and specific hiring challenges of key industries as well as an assessment of the feasibility and cost-effectiveness of supporting target populations to achieve the skills and experience profiles necessary to be hired into said positions. Further, at the provider level, vocational training providers can draw on broader national studies of market demand and also engage with local employers, going beyond arranging job attachments or internships.
  • Gender considerations may need to extend beyond immediate participation opportunities. Although women had similar training completion rates, they appear to face substantial challenges in the labor market relative to men. Moreover, these challenges were not related to differences in the types of courses women selected or other training characteristics. Therefore, where such labor market barriers are thought to exist, special attention could be given to enhancing the employment prospects of female trainees, such as linking them with female mentors in the community (for example, female-owned businesses) or undertaking affirmative action initiatives to provide direct and stronger support to female graduates.
  • Consider the timing of interventions and how to ensure sustainability. Given the limited capacity at the COSDECs, it would have been ideal to start technical assistance earlier to build in more of a time cushion within the compact timeframe. This would have provided time to oversee and reinforce some of the new practices which were not adopted by COSDECs, such as actively marketing their programs among potential students. Alternatively, funding and support for a few months into the post-compact period could have been coordinated through another donor.
  • Facilitating system-level behavior change in an organization, in this case the COSDECs and COSDEF, requires more than simply “training.” Much of the Technical Assistance provided to the COSDECs and COSDEF was classroom-style training and workshops.  In future investments, MCC should thoroughly assess the organization/institution as well as the key behaviors that must change in order to achieve project impact as a precursor to project design and use the information gleaned to inform the content of and vehicle for delivering interventions.

 

Next Steps

This evaluation is complete and there are no planned next steps.

Footnotes
  • 1. These figures are calculated using all relevant non-evaluation indicators with targets in the COSDEC sub-activity.