Independent evaluators’ access to microdata collected by government ministries is essential to generate robust impact measurements; gleaning data from publicly available reports alone is no replacement for direct data sharing. While the Ministry of Education and Science (MES) published tracer survey study reports in 2019 and 2020, the evaluator was unable to obtain the microdata underpinning the reports, even after multiple requests to MES over a period of months. This microdata would have provided richer information pertaining to labor market outcomes (wages) in populations of interest (gender) for independent analysis. The existence of a high-capacity successor entity to the Millennium Challenge Account–Georgia did not obviate the risk of obtaining complete microdata in a timely manner from a partner. In the case of this evaluation, the inability to access microdata forced the evaluator to adapt their analysis: for example, they could not separate out the 12% of trainees who received project-supported Technical and Vocational Education Training (TVET) courses from the larger comparison sample of all TVET trainees, leading to a likely underestimate of project impact. The analysis was also unable to adjust their model to account for differences in demographic characteristics and course level. MCC is addressing this lesson by setting clear expectations with government agencies regarding data sharing early on in program implementation. In addition, highlighting MCC’s recently updated Guidelines for the Transparent, Reproducible, and Ethical Data and Documentation in interactions with data owners in Mozambique has proven effective in assuring them primary source data will be handled with careful consideration of privacy concerns.
Lesson Learned