To ensure the integrity of the evaluation, regular oversight and data quality checks are required of field data training and collection, including data collection funded by the MCA. Lesotho’s Land Administration Reform Activity baseline evaluation data collection had insufficient oversight during the initial randomized controlled trial design and related problems with the management of its enumerators. By the time the issues were realized by MCC, MCA and the independent evaluator, it was too late to recollect baseline data in the majority of areas. A new baseline was collected in 2013 as part of this difference in differences evaluation design. Although MCA hired a new local data collection firm, there were similar problems with oversight and quality checks of the local data collection firm’s work. This caused multiple issues for the integrity of the evaluation. First, the data entry was improperly completed leading to the re-entry of the data directly by MCC’s independent evaluator and a related redrafting of the baseline report. Second, although every household was supposed to receive a household head and a wives survey module, when the World Bank took over the evaluation from MCC’s original independent evaluator, it was found that only 138 wives’ questionnaires were administered in the entire sample of which only 17 could be matched to the household questionnaires. Essentially this led to the loss of data from women in the baseline. Modules on businesses, who had a separate listing exercise to ensure a large enough sample, were also missing. For the end line, the World Bank used quality checks throughout the data collection process to ensure these sorts of errors are not occurring. A quality control plan is now required of all MCC’s independent evaluators.
Lesson Learned