Measuring Results of the Namibia Vocational Education and Skills Training Activity

The MCC compact with Namibia was a five-year investment (2009-2014) of $304.5 million. The $28 million Vocational Education and Skills Training component was comprised of three sub-       activities: (i) establishment of a National Training Fund (NTF); (ii) competitive grants for high priority vocational skills training through the Vocational Training Grant Fund (VTGF); and (iii) expansion and improvement of seven Community Skills Development Centers (COSDECs) and training of the  Community Skills Development Foundation’s management staff. Collectively, these sub-activities invested a significant amount in standing up new operations and improving the management capacity of critical institutions in the vocational training sector. The three sub-activities are the subject of an independent qualitative evaluation, whose interim results are summarized here.

NTF

  • The NTF sub-activity was designed to support the Namibia Training Authority (NTA) in the creation and operationalization of the National Training Fund, a levy collection and distribution system that aims to provide a sustainable source of funding for high priority vocational education and skills training, enable individuals to receive skills that are needed in the economy, and ultimately increase employment and earnings.
  • The NTF sub-activity was significantly hindered by delays in establishing the NTF, posing challenges for some intermediate outcomes like disbursement of levy funds and development of a skilled workforce that meets the needs of Namibia’s economy. Perceptions of NTF sustainability in terms of organization and management were also mixed. However, the evaluation found positive early signs for several short-term outcomes, including employer registration and levy collection; NTA expected to exceed its annual target for the first year of levy collection.
  • A second round of qualitative data collection was conducted in late 2015, and focused on the longer-term evolution of the NTF after the compact ended, including assessing whether the potential barriers in using levy funds effectively had been addressed. This report is expected to be released in 2017.

 

VTGF

  • The VTGF sub-activity was designed to address the quality of the country’s labor force and increase employment and earnings by funding vocational skills programs in high-priority areas while the NTF was being established.
  • The qualitative analysis found that the NTA gained valuable experience in managing grants through the VTGF, and that trainees and employers both had positive perceptions of the sub-activity. However, it also highlighted that it was challenging to identify a sufficient number of providers to participate and meet initial training requests.
  • The VTGF sub-activity is also the subject of an impact evaluation using random assignment, which seeks to assess impacts on key outcomes, including increased completion of training in key priority skill areas, increased paid employment or self-employment, increased earnings, and increased job security and mobility. The evaluation will also try to measure the variation in impact by gender, income level, and language group. The VTGF final evaluation report is expected to be released in 2017.

COSDEC

  • The COSDEC sub-activity was designed to rehabilitate and construct community-based institutions that provide basic levels of vocational training to clients from disadvantaged backgrounds – particularly out-of-school youth who lack access to the formal vocational training system – to improve their employment and earnings prospects.
  • There are indications that COSDECs are being managed differently than in the past and also evidence of increases in types of training, number of training slots, and training quality. However, although the management support provided by the compact assisted COSDECs with the registration and accreditation process, at the time of data collection, no COSDEC had completed it. Additionally, SME units were not yet in use at the time of data collection.
  • A quantitative outcomes analysis is also planned for the COSDEC sub-activity. This analysis will be based on a tracer survey that seeks to measure key outcomes of the first cohort of COSDEC trainees to receive training after the renovations were complete. For example, the tracer survey will document training, employment and income outcomes of COSDEC trainees in the 12-month period since the completion of training. Additionally, a second round of qualitative data collection was conducted to follow up on the ongoing management of the COSDECs and help interpret the results of the outcomes analysis. A final evaluation report is expected to be released in 2017.

In Context

The MCC compact with Namibia was a five-year investment (2009-2014) of $304.5 million in three projects: Education, Agriculture, and Tourism. The Education project sought to improve the quality of the workforce in Namibia by enhancing the equity and effectiveness of basic, vocational, and tertiary education and of technical skills. It included six major activities: Improving the Quality of General Education, Vocational Education and Skills Training, Improving Access to and Management of Textbooks, Investment in Regional Study and Resource Centers (RSRCs), Expanding and Improving Access to Tertiary Finance, and Cross-Project Support. The Vocational and Skills Training activity consisted of three sub- activities: (i) establishment of a National Training Fund (NTF); (ii) competitive grants for high priority vocational skills training through the Vocational Training Grant Fund (VTGF); and (iii) expansion and improvement of nine Community Skills Development Centers (COSDECs) and training of the Community Skills Development Foundation’s management staff. The $28 million Vocational Training activity, including its three sub-activities, is the subject of an independent qualitative evaluation released by MCC on September 21, 2016, the interim results of which are summarized here. Given slightly different objectives and targeted populations, each sub-activity is being evaluated separately with different evaluation questions and methodologies; the results from the qualitative interim evaluations are presented in turn below. The Vocational and Skills Training activity represents 10 percent of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.

                                                  

National Training Fund Sub-activity

Program Logic

The NTF sub-activity within the Education project was designed to address the quality of the country’s labor force by supporting the Namibia Training Authority (NTA) in the creation and operationalization of the National Training Fund, a levy collection and distribution system that aims to provide a sustainable source of funding for high priority vocational education and skills training. Key pieces of the NTF sub- activity included developing the regulations required for fully operationalizing the NTF, establishing the NTF council, and piloting all aspects of the levy. Eligible employers are required to contribute to the NTF through a payroll-based vocational education and training levy. Levy funds are then disbursed to levy paying employers for training carried out and to training providers in order to conduct vocational training in the high-priority skill areas the NTA identifies.

 

                        

 

Acronyms: LCDRS = Levy collection, distribution, and reporting system; ISC = Industrial Skills Committee; RPL = Recognition of Prior Learning

* Details for the RPL pilot and operationalization are described in the VTGF program logic below

 

There were several key assumptions underlying the NTF program logic during the design of the investment:

  • NTA receives employer database from Inland Revenue Services, integrates it with the NTA database and updates it regularly.
  • Sufficient compliance in levy payment occurs after the LCDRS is in place.
  • ISCs identify key priority areas using information from the national plans, market surveys, and stakeholder input.
  • RPL certificate is valued in the same way as a traditional vocational training certificate.
  • Individuals eligible for RPL, that is, have worked but lack certifications, are aware of the RPL system.
  • Training is of sufficient quality and may be improved to the extent that lower-quality providers are screened out of the system.
  • Registration and accreditation processes are fair, transparent, and effective.

 

For a more detailed version of the program logic, please refer to page 6 of the Vocational Training Evaluation Design Report, which can be found here: https://data.mcc.gov/evaluations/index.php/catalog/90/download/388.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources. Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program. However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention. The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone. Where estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

 

Indicators

 

Level

Baseline (2009)

Actual Achieved (09/2014)

 

Target

Percent Complete

Percent of NTF Levy funds collected that are awarded for skills training and related improvement of training facilities and equipment

 

 

Output

 

 

0

 

 

Pending

 

 

80

 

 

0%

Vocational trainees assisted from the NTF levy

Output

0

Pending

7,000

0%

Compliance rate for NTF levy

Outcome

0

Pending

80

0%

Source: Closeout ITT from December 2014, which includes data through the end of the compact, based on reporting from MCA-Namibia and the Namibia Training Authority.

 

By the end of the Namibia compact, no progress had been made on the NTF output and outcome indicators. This is largely due to the delay in establishing an operational NTF levy collection system. While the NTF levy was targeted to be completed in spring 2013, it was not completed until April 1, 2014, only six months before the compact closed.

 

Evaluation Questions

The qualitative interim evaluation sought to address three important junctures in the evolution of the NTF’s levy collection, distribution, and reporting system. These are the system’s establishment, initial operations, and likely sustainability, as outlined in the following questions:

  • Was the establishment of the NTF levy collection, distribution, and reporting system implemented as planned?
  • How is the NTF levy collection and distribution system operating in practice compared to the specifications outlined in the regulatory framework?
  • What are the stakeholder perceptions of sustainability of the NTF levy collection and distribution system?

 

Evaluation Results

The first round of qualitative data collection, covered in this summary of findings and the associated evaluation report, was conducted shortly after the establishment of the NTF levy collection system and six months before levy disbursements were scheduled to begin. As such, this evaluation primarily focused on outputs, immediate outcomes and initial perceptions about longer-term effects. Specifically, the evaluation focused on the operationalization of the NTF, establishment of the levy collection, disbursement, and reporting IT system, staffing needed for operating the levy, the initial operations of the levy, the NTF’s ability to measure and respond to the demand for skilled labor, and perceptions about the NTF’s sustainability.

Evaluator

Mathematica Policy Research

Impact or Performance?

Performance

Methodology

Qualitative implementation analysis

Evaluation Period

Compact-funded Technical Assistance was provided from: September 2010 to June 2014

Levy collection began: April 2014

Round 1 qualitative data collected:  October-November 2014 Round 2 qualitative data collected:  November-December 2015*

 

*The results of this data collection effort are the subject of a separate report.

Short-term Outcomes

Early signs of employer registration and levy collection are encouraging. By the time of data collection, 2,200 employers had registered and paid the levy and the NTA expected to meet the annual target of N$250 million for the first year of levy collection. However, some concern has been expressed about effective enforcement of registration for all applicable employers and accurate levy payments.

Intermediate Outcomes

Disbursement of levy funds was not expected to begin until April 2015, and thus was not covered in the scope of the interim report. However, at the time of data collection, a fully functioning levy distribution system had not been developed. Additionally, the evaluation found the NTF may face some barriers in using levy funds, including difficulties in determining the key-priority skill areas in all of the industries, partially due to the fact that some Industrial Skills Committees were not fully functioning and because of a lack of training provider capacity in Namibia. Furthermore, perceptions of NTF sustainability in terms of organization and management were mixed, with respondents highlighting that NTA must have enough qualified staff to manage the NTF and disburse funds effectively and transparently.

Objective-level (long-term) Outcomes

Due to the timing of the data collection and overall scope of the evaluation, it was not possible to assess the objective-level or long- term outcomes of the NTF sub-activity.

 

Lessons Learned

  • Experience “learning by doing” should help the NTA manage the NTF post-compact. The NTA received valuable preparation for its management of the NTF by managing a large portion of the related VTGF sub-activity, which is described further below. Although this opportunity to “learn by doing” does not guarantee that the NTF will be successful, it does make it more likely that the process of awarding and managing grants will operate smoothly and effectively under the NTF. Specifically, the NTA will be able to use improved VTGF procedures and templates, and apply the lessons learned in evaluating the proposed costs of training to expedite the process of awarding grants. MCC compacts should be encouraged to use the compact period to provide implementing entities with similar hands-on experience, where feasible.
  • Key policy decisions that affect sustainability should be taken before MCC compacts end. To the extent possible, policy and regulatory hurdles should be identified before Entry into Force, with consultant support as needed, to ensure that partner governments can reap the benefits of intensive technical assistance to operationalize new government functions and increase the likelihood of sustainability. Important policy decisions were still to be determined at the end of the Namibia compact. For example, the process ISCs would use to identify priority skill areas and what constitutes vocational training were both unclear.  This lack of clarity could easily impede the NTA’s ability to identify the skills that are truly in demand and prevent employers from being reimbursed for sponsored trainings. It will be important to resolve these issues soon so that stakeholders maintain support for the system.

 

Next Steps

A second round of qualitative data collection was conducted in late 2015, and focused on the longer- term evolution of the NTF after the compact ended, including assessing whether the potential barriers in using levy funds effectively had been addressed. A final evaluation report is expected to be released to the public in 2017.

 

Vocational Training Grant Fund Sub-activity

Program Logic

The VTGF sub-activity within the Education project was designed to address the quality of the country’s labor force by providing funding for vocational skills programs in high-priority areas while the NTF was being established. VTGF included several components, the primary being awarding grants to training providers through a competitive bidding process. The grants were then used to award scholarships to eligible disadvantaged applicants. Additional components of the VTGF sub-activity included piloting a Recognition of Prior Learning (RPL) program, through which experienced employees who do not have formal qualifications receive vocational skills certifications, and funding the marketing of the VET levy to increase public awareness about the levy with the goal of increasing the likelihood of employer compliance. VTGF also intended to establish a framework by which Industrial Skills Committees (ISCs) would identify high priority skill areas; however, not all ISCs were functional during the compact period.

                                   

Acronyms: LCDRS = Levy collection, distribution, and reporting system; ISC = Industrial Skills Committee; RPL = Recognition of Prior Learning

Detailed levy development and processes are reflected in the NTF program logic above

 

There were several key assumptions underlying the VTGF program logic during the design of the investment:

  • Training providers are on the road to the NTA registration and Namibia Qualifications Authority (NQA) accreditation.
  • RPL certificate is valued in the same way as a traditional vocational training certificate.
  • Training is of sufficient quality.

 

For a more detailed version of the program logic, please refer to page 4 of the Vocational Training Evaluation Design Report, which can be found here: https://data.mcc.gov/evaluations/index.php/catalog/90/download/388.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources. Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program. However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention. The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone. Where estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

 

Indicators

 

Level

Baseline (2009)

Actual Achieved (09/2014)

 

Target

Percent Complete

Vocational trainees assisted through the MCA-N Vocational Training Grant Fund

 

Output

 

0

 

1,500

 

1,638

 

92%

Graduates from MCC-supported education activities

Outcome

0

541

1,392

39%

Namibia Qualifications Authority (NQA)-accredited and/or Namibia Training Authority (NTA)-registered vocational training providers

 

 

Outcome

 

 

25

 

 

64

 

 

60

 

 

111%

Source: Closeout ITT from December 2014, which includes data through the end of the compact, based on reporting from MCA-Namibia, the Namibia Training Authority.

 

The average completion rate of output and outcome targets is 81 percent; and in 1 of the 3 indicators, targets were met or exceeded. 1

 

Evaluation Questions

The qualitative portion of the VTGF evaluation was designed to answer questions such as:

  • Was the VTGF sub-activity implemented as planned?
  • How were the VTGF grants managed?
  • What were beneficiaries’ perceptions of the VTGF grants?
  • How did employers hire VTGF graduates, and what were their perceptions of the graduates?
  • Were the RPL and employer-provided training pilots implemented as planned? How did employers’ perceptions of and attitudes towards their RPL-certified employees change after they became certified? How did the perceptions of employees about their job security and mobility change?

These questions were designed to provide valuable evidence to inform the implementation of similar training mechanisms envisaged under the NTF. Further analysis that will be completed through an impact evaluation will seek to assess the impact of the VTGF sub-activity on its long-term outcomes.

 

Evaluation Results

The qualitative analysis found the VTGF sub-activity was primarily implemented as planned with some enhancements added during implementation. The first addition was capacity-building grants for training providers to improve infrastructure and equipment and to support instructor training. The grants were intended to make longer-term changes to the vocational training sector by expanding providers’ physical infrastructure in order to meet registration and accreditation requirements. They also served as an incentive to compete for VTGF grants. The second addition was a board and lodging allowance, which aimed to increase attendance and reduce the number of dropouts among VTGF-funded trainees.

Trainees and training providers alike agreed that it was an important addition to the basic training grants, although the sustainability of the allowance is of concern to NTA.

Evaluator

Mathematica Policy Research

Impact or Performance?

Performance

Methodology

Qualitative implementation analysis

Evaluation Period

  • VTGF grants awarded throughout the compact period, with the first grants awarded in the fourth quarter of 2010 and the last grant awarded in the third quarter of 2014
  • Round 1 qualitative data collected:  October-November 2014
  • Round 2 qualitative data collected:  November-December 2015*
  • The baseline survey was conducted on a rolling basis within 3 months of training start*
  • The follow-up survey was conducted on a rolling basis 12 months after training ended to allow sufficient time for outcomes of interest to manifest*

 

*The results of these data collection efforts are the subject of separate reports.

Short-term Outcomes

The qualitative analysis primarily focused on VTGF implementation and perceptions of participants and employers, and therefore did not measure many short-term outcomes directly. The evaluation did find, however, that the NTA gained valuable experience in managing grants through the VTGF, which suggests increased NTA capacity. In addition VTGF did expand the vocational training offerings. However, it was challenging to identify a sufficient number of providers to participate and meet initial training requests. Additionally, although the VTGF training grants were intended to be targeted at high- priority skill areas determined by market demand in Namibia, the process to determine market demand was not as scientific as  planned and, at the time of data collection, rigorous guidelines for prioritizing skill areas had not been produced. Also, the ISCs were not fully functioning for all industries by the end of the compact period.

Intermediate Outcomes

The qualitative analysis primarily focused on VTGF implementation and perceptions of participants and employers and therefore did not measure many intermediate outcomes directly. The evaluation did find the RPL program was implemented successfully and was viewed positively by participating recipients and employers, but notes that it was challenging to obtain participation from employers.

Objective-level (long-term) Outcomes

The qualitative analysis did not measure the objective-level or long- term outcomes of the VTGF component. This will be addressed in an ongoing impact evaluation, the report for which is expected to be released in 2017.

Effect on household income attributable to MCC

N/A

 

Lessons Learned

  • It is important to identify, understand and mitigate risks, e.g., barriers to entering the vocational training market. There are limitations in the number of training providers in Namibia and the courses and levels at which they offer training. This posed a challenge in meeting VTGF training targets and will be an important ongoing challenge in addressing high-priority skill areas through the NTF. This challenge could be especially important if the high-priority areas include new skills and higher levels of skill not currently available from Namibian training providers (which might be the reason for their being high priority in the first place). As an example of a specific barrier to entering the market or expanding offerings, the registration and accreditation process for training providers is widely viewed as lengthy, cumbersome, and unnecessarily duplicative since many of the requirements for registration and accreditation are similar. Future Compacts should identify such challenges when analyzing risks and seek ways to mitigate them, such as streamlining redundant procedures, mapping service provision, examining the possibility of partnering with providers in the region to fill gaps in quality provision (e.g., South Africa has a strong training market), and identifying and addressing weaknesses in the government’s quality assurance mechanisms.
  • Compacts must understand the participants they are targeting, what might interfere with their participation or adoption of new skills, and attempt to overcome such challenges, if appropriate. Training providers agreed that the board and lodging allowance was very helpful for decreasing the number of dropouts and increasing attendance, and trainees universally appreciated the allowance. A board and lodging allowance will likely be necessary for future programs like the NTF, even though it may impose a burden on funding of training in high-priority skill areas. If an allowance is offered, it should be included in the early planning phase rather than implemented later, which made administration of the allowance more challenging in the case of VTGF. Ideally, this allowance should be targeted at disadvantaged trainees, but it may be challenging to develop a mechanism to effectively target the allowance (or funding more generally) at this group.

 

Next Steps

The VTGF sub-activity is also the subject of an impact evaluation using random assignment, which seeks to assess impacts on key outcomes of the sub-activity, including increased completion of training in key priority skill areas, increased paid employment or self-employment, increased earnings, and increased job security and mobility. The evaluation will also try to measure the variation in impact by gender, income level, and language group. The VTGF final evaluation report is expected to be released to the public in 2017.

 

Community Skills Development Centers Sub-activity

Program Logic

The Community Skills Development Centers (COSDEC) sub-activity within the Education project was designed to address the quality of the country’s labor force through rehabilitating and constructing community-based institutions that provide basic levels of vocational training to clients from disadvantaged backgrounds – particularly out-of-school youth who lack access to the formal vocational training system – to improve their employment prospects. These COSDECs offer two main types of programs: (i) national programs, which typically last between seven and ten months and target out-of- school youth; and (ii) local programs, which typically are shorter and can be taught as center-based programs at the COSDEC itself or as outreach programs on an ad-hoc basis. MCC funded the construction or rehabilitation of seven COSDECs, provided new tools and equipment, and provided technical support for the management of the new or renovated COSDECs. Additionally, MCC helped establish small- and medium-enterprise (SME) units in four of the COSDECs.

                               

Acronyms: SME = Small- and medium-enterprises; TA = technical assistance; COSDEF = Community Skills Development Foundation

 

There were several key assumptions underlying the COSDEC program logic during the design of the investment:

  • Limited availability of qualified trainers will not constrain improvements in training quality and relevance.
  • Employers recognize and value the improved quality of the COSDECs.

 

For a more detailed version of the program logic, please refer to page 9 of the Vocational Training Evaluation Design Report, which can be found here: https://data.mcc.gov/evaluations/index.php/catalog/90/download/388.

 

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources. Monitoring data is collected during and after compact implementation and is typically generated by the program implementers; it focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program. However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention. The limitations of monitoring data is a key reason why MCC invests in independent impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone. Where estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on output and outcome indicators specific to the evaluated program.

 

Indicators

 

Level

Baseline (2009)

Actual Achieved (09/2014)

 

Target

Percent Complete

COSDECs completed

Output

0

8

8*

100%

Arts and Crafts Centers completed

Output

0

1

1

100%

COSDEC staff trained in management

Output

0

15

15

100%

Source: Closeout ITT from December 2014, which includes data through the end of the compact, based on reporting from MCA-Namibia, the Namibia Training Authority.

 

*This total includes the seven COSDECs that were constructed or renovated and a bulk store at the Namibian College of Open Learning that was simply intended to allow the Namibian College of Open Learning free up space for vocational training.

 

The average completion rate of output and outcome targets is 100 percent; and in all of the indicators, targets were met or exceeded. 2

 

Evaluation Questions

The qualitative portion of the COSDEC evaluation was designed to answer questions such as:

  • Was the COSDEC sub-activity implemented as planned?
  • How did COSDEC training affect the employment outcomes of trainees?
  • How were the new and renovated COSDECs managed?

 

Evaluation Results

The first round qualitative analysis of the COSDEC sub-activity focused on implementation and early evidence of changes in operations or perceptions of the COSDECs as a result of the intervention. The qualitative analysis found mixed results on the implementation of the COSDEC sub-activity. Although the construction and renovation of the COSDECs was largely implemented as designed, there was some discontent with the design of the new facilities, the equipping of the COSDECs with new tools encountered challenges, and additional support related to training of instructors and marketing of COSDECs is needed.

Evaluator

Mathematica Policy Research

Impact or Performance?

Performance

Methodology

Qualitative implementation analysis

Evaluation Period

  • Implementation of Technical Assistance occurred from early 2012 to mid-2014; construction was finalized in mid-2014; and the first round of trainings in the new or renovated COSDECs began in July 2014
  • Round 1 qualitative data collected:  October-November 2014
  • Round 2 qualitative data collected:  November-December 2015*
  • An outcomes survey of the first cohort of trainees to use the new or renovated COSDECS was conducted on a rolling basis 12 months after their training ended to allow sufficient time for outcomes of interest to manifest*

 

*The results of these data collection efforts are the subject of a separate report.

Short-term Outcomes

There are indications that COSDECs are being managed differently than in the past, but some changes are still underway. There is also evidence of increases in types of training, number of training slots, and training quality.

Intermediate Outcomes

Although the management support provided by the compact assisted COSDECs with the registration and accreditation process, at the time of data collection, no COSDEC had completed it. Additionally, due to the fact that the SME units had only recently been completed, they were not yet in use.

Objective-level Outcomes

Due to the timing of the evaluation period shortly after completing construction of the COSDECs and before the completion of the first cohort of trainees, it was not possible to assess the objective-level or long-term outcomes of the COSDEC sub-activity.

Effect on household income attributable to MCC

N/A

 

Lessons Learned

  • Key stakeholders should be consulted during project design in order to ensure investments address existing needs. The technical assistance component of the COSDEC sub-activity could have been better tailored to the Namibian context and the initial capacity of the COSDECs.

Additional consultation with key stakeholders such as the COSDEC managers and COSDEF early on could have helped in this regard. Similarly, for the future design of infrastructure development investments similar to the COSDEC sub-activity, it would be useful to consider engaging some of the stakeholders on the ground at an early stage. Such engagement will allow these stakeholders to assess how the design fits in their context, create an opportunity to address any practical concerns early on, and possibly establish greater buy-in from stakeholders about the investments.

  • Evaluation criteria for procurements should be considered carefully.  The tools purchased for the COSDEC sub-activity were procured from a supplier who won the procurement based on the lowest price. Unfortunately, the tools of very low quality that did not meet stakeholder expectations. For procurements such as this where quality can vary substantially, it is recommended to also consider quality when evaluating proposals.

 

Next Steps

A quantitative outcomes analysis is also planned for the COSDEC sub-activity. This analysis will be based on a tracer survey that seeks to measure key outcomes of the first cohort of COSDEC trainees to receive training after the renovations were complete. For example, the tracer survey will document the training outcomes, employment and income of COSDEC trainees in the 12-month period since the completion of training. Additionally, a second round of qualitative data collection was conducted to follow up on the ongoing management of the COSDECs and help interpret the results of the outcomes analysis. A final evaluation report is expected to be released to the public in 2017.

Footnotes
  • 1. These figures are calculated using all non-evaluation indicators with targets in the VTGF sub-activity.
  • 2. These figures are calculated using all non-evaluation indicators with targets in the COSDEC sub-activity.