Measuring Results of the Namibia National Training Fund

The MCC compact with Namibia was a five-year investment (2009-2014) of $304.5 million. The $3.4 million National Training Fund (NTF) component is the subject of an independent performance evaluation summarized here.

  • This sub-activity established a fully operational NTF, which requires employers with a payroll over N$1 million (approximately US$70,000) to contribute 1% of their payroll costs to training, intended to serve as a sustainable source of funding for high priority vocational education and skills training in Namibia.
  • The goal of NTF funding was to develop a workforce whose skills are better matched to the needs of the economy, thereby increasing employment, earnings, and income.
  • All the structures necessary for the NTF to function have been established. Targets for employer registration were met, with over 80% of anticipated employers currently registered with NTF and collection targets surpassed for the first two years of operation.
  • Although considered financially sustainable, uncertainty exists about whether the existing capacity of the Namibian vocational training sector can absorb the funding and effectively create a more skilled workforce.
  • A key lesson learned is that interventions to achieve policy reform should build in time for testing and incorporating lessons learned.

This evaluation is complete and there are no planned next steps.

In Context

The MCC compact with Namibia was a five-year investment (2009-2014) of $304.5 million in three projects:  Education, Agriculture, and Tourism. The Education project sought to improve the quality of the workforce in Namibia by enhancing the equity and effectiveness of basic, vocational, and tertiary education. It included six major activities: Improving the Quality of General Education, Vocational Education and Skills Training, Improving Access to and Management of Textbooks, Investment in Regional Study and Resource Centers, Expanding and Improving Access to Tertiary Finance, and Cross-Project Support. The Vocational and Skills Training activity consisted of three sub-activities:  (i) establishment of a National Training Fund (NTF); (ii) competitive grants for high priority vocational skills training through the Vocational Training Grant Fund (VTGF); and (iii) expansion and improvement of Community Skills Development Centers (COSDECs) and training of the Community Skills Development Foundation’s management staff. The $3.4 million NTF component is the subject of an independent performance evaluation released by MCC in January 2018, the results of which are summarized here; the VTGF and COSDEC evaluations were released simultaneously.  The NTF component represents 1% percent of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.

Pie Chart: Namibia Compact Spending

*These figures are based on MCC obligations as of March 2016.

 

Program Logic

The NTF sub-activity within the Education project was designed to address the quality of the country’s labor force by supporting the Namibia Training Authority (NTA) in the creation and operationalization of the National Training Fund, a levy collection and distribution system that aims to provide a sustainable source of funding for high priority vocational education and skills training. NTF funds can be spent on (1) grants for training providers and recognition of prior learning (RPL) programs in key priority areas, as determined primarily by Industry Skills Committees (ISCs); and (2) grants to levy-payers for employer-sponsored training conducted. Direct outputs included establishment of the NTF council, development of regulations, piloting, and implementing all aspects of the levy collection, distribution, and reporting system (LCDRS) framework and the system itself. In addition, capacity-building support was provided to the ISCs, which have a critical role in identifying key priority areas for funding under the levy system. Together, these outputs were intended to result in a fully functioning levy system in the short-term, in which employers are fully interacting with the system by registering, paying the levy, and applying for grants for training conducted; ISCs are expected to identify key priority areas so training in these areas can be procured from registered and accredited providers; and RPL candidates will be assessed and certified. In the intermediate term, funds are continually disbursed to employers and training providers, and more people are trained through these mechanisms and receive RPL certificates. Combined, this is expected to result in a skilled workforce well matched to the needs of the economy, which in the long term is expected to increase employment and income, furthering attainment of the Compact’s ultimate goal:  poverty reduction through economic growth.

Flow Chart: Program Logic

Acronyms: NTF = National Training Fund; LCDRS = Levy collection, distribution, and reporting system; ISC = Industry Skills Committee; NTA = Namibia Training Authority; RPL = Recognition of Prior Learning

* Details for the RPL pilot and operationalization are described in the VTGF program logic

 

There were several key assumptions underlying the NTF program logic during the design of the investment:

  • NTA receives employer database from Inland Revenue Services, integrates it with the NTA database and updates it regularly.
  • Sufficient compliance in levy payment occurs after the LCDRS is in place.
  • ISCs identify key priority areas using information from the national plans, market surveys, and stakeholder input.
  • RPL certificate is valued in the same way as a traditional vocational training certificate.
  • Individuals eligible for RPL, i.e., have worked but lack certifications, are aware of the RPL system.
  • Training is of sufficient quality and may be improved to the extent that lower-quality providers are screened out of the system.
  • Registration and accreditation processes are fair, transparent, and effective.

For a more detailed version of the program logic, please refer to page 6 of the Vocational Training Evaluation Design Report, which can be found here.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources.  Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program.  However, monitoring data is limited in that it cannot not reflect the full range of targeted outcomes and cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention.  The limitations of monitoring data is a key reason why MCC invests in independent evaluations to assess the achievement of a broader set of program outcomes.  When feasible, MCC supports impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone.  When estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

Indicators Level Baseline (2009) Actual Achieved (09/2014) Target Percent Complete
Percent of NTF Levy funds collected that are awarded for skills training and related improvement of training facilities and equipment Output 0 Pending 80 0%
Vocational trainees assisted from the NTF levy Output 0 Pending 7,000 0%
Compliance rate for NTF levy Outcome 0 Pending 80 0%

Source: Closeout ITT from December 2014, which includes data through the end of the compact, based on reporting from MCA-Namibia and the Namibia Training Authority.

 

By the end of the Namibia compact, no progress had been made on the NTF output and outcome indicators. This is largely due to the delay in establishing an operational NTF levy collection system. While the NTF levy was targeted to be completed in 2013, it was not completed until April 1, 2014, only five months before the compact closed.

 

Evaluation Questions

The evaluation was designed to answer the following questions:

  • How is the NTF levy collection and distribution system operating in practice, compared with the specifications outlined in the regulatory framework?
  • What are stakeholders’ perceptions of the sustainability of the NTF?

The evaluation did not aim to isolate impacts on income, a benefit stream modeled in the program’s economic analysis. It was not feasible to isolate changes in income in order to attribute them to the NTF sub-activity. However, economic analysis of the NTF component was originally combined with that of the Vocational Training Grant Fund (VTGF), and the VTGF evaluation did measure impacts on income, amongst other outcomes.

 

Evaluation Results

The NTF final evaluation builds on a prior round of data collection and analysis in 2014 and primarily draws on qualitative data collected from a variety of sources between October and December 2015; this included 29 in-depth interviews, site visits, and reviews of documents provided by stakeholders and collected from the media. Interviews were conducted with NTA board and NTF council members, RPL certificate recipients, employers, ISC members, and other relevant stakeholders.

The evaluator and data collection subcontractor coded the data and then the evaluator triangulated information from multiple sources, recognizing that each respondent has his/her own perspectives, information, and interests. The team identified major themes that emerged from the data related to the research questions above. The resulting analysis accounts for similarities and differences in perspectives across different respondent groups. It also provides a comprehensive picture of the operation and evolution of the NTF.

 
Evaluator             Mathematica Policy Research
Impact or Performance? Performance
Methodology    Other (performance)
Evaluation Period Compact-funded Technical Assistance was provided from:  September 2010 to June 2014

Levy collection began:  April 2014

Round 1 qualitative data collected:  October-November 2014

Round 2 qualitative data collected:  November-December 2015

Functionality of NTF ·         All the structures necessary for the NTF to function have been established; all the key NTF positions have now been filled, but additional capacity might be necessary.

·         Most stakeholders agree that employer registration and levy payments have gone well. Over 80% of the anticipated number of eligible employers have registered and collection targets were surpassed for the first two years of operation. However, due to the lack of exhaustive employer databases, it is difficult to know the true number of eligible employers or how much they should contribute.

·         The NTF is allocating levy funds in accordance with the regulatory formula, though a lot of the funding has been left unspent and only half of the allotted amount has been necessary to administer the fund. It has been difficult to find enough qualified training providers to provide high priority skills training; and at the time of data collection, only around 15% of registered employers had requested reimbursement for in-service training. The employers interviewed for the evaluation noted that the reimbursement process was unclear and cumbersome.

·         The LCDRS IT system is operational and serves many purposes, but the system still faces several challenges, including the lack of integration between different branches of the system, such that information added in one part does not flow to other parts, and not all of the reporting system is operational.

·         The establishment of the accreditation and registration processes is considered effective for ensuring training quality. However, the two processes are seen as duplicative and cumbersome, creating a bottleneck to the provision and expansion of training in high-priority areas.

·         A vast majority of the respondents felt that the NTF was financially sustainable. However, they expressed uncertainty about whether the Namibian vocational training sector can absorb the funding and use it effectively to create a more skilled workforce.

·         Overall, most stakeholders, including employers, thought it was too soon to tell whether NTF would result in a more skilled workforce but they remained cautiously optimistic.

Employment and Earnings N/A
Effect on household income attributable to MCC N/A

 

Lessons Learned

  1. Improving the quality and relevance of TVET programming requires a package of interventions which are rooted in the key institutional problems identified in the given context. The Namibia Vocational Education and Skills Training Activity did not fully account for the complexities of reforming both the system which plans for and allocates resources (such as the ISCs) and the system of implementing new modalities of TVET programming.  To this end, MCC would benefit from conducting a more thorough assessment of the institutions and stakeholders involved in TVET systems in the early stages of Compact development. This could range from labor market supply/demand assessments to behavior change assessments to a detailed stakeholder mapping, which would inform the detailed design of the project.  This would allow the project to invest in the most urgent and proximal steps to achieving TVET reform and provide a foundation for sustained improvements.
  2. Interventions to achieve policy reform should build in time for testing new approaches and incorporating lessons learned. Passing the NTF policy took more time than anticipated, which had implications for the sequencing of follow-on activities.  In addition, components of the levy system were not fully tested nor was there an opportunity to make modifications to the levy system during the Compact, when MCA-Namibia and MCC were available to provide support.  Ensuring work plans build in a realistic amount of time to enact a policy, test/implement the policy, and incorporate lessons learned for continued implementation, needs to be a very important part of project design.
  3. Projects need to plan for all conditions that are necessary to achieve success. Establishing a levy to provide more training funds was necessary to transform the Namibia vocational training system into a demand-driven system but it was not sufficient. The project was designed to address several complementary conditions, like preparing the NTA to manage these new resources and establishing the ISCs to identify training needs, but not all were fully achieved.  Furthermore, a number of complementary conditions, such as ensuring vocational training providers were prepared to absorb the additional funding, were not sufficiently addressed by the project.  The absence of such complementary conditions can affect the longer-term ability of the levy funds to truly meet the skills needs of the private sector.  Therefore, it is critical to design projects with a clear logic that accounts for all of the conditions necessary and sufficient for achieving targeted results and to ensure that implementation decisions are considered within the context of their effect on targeted results.

 

Next Steps

This evaluation is complete and there are no planned next steps.