Successful and quality data collection requires collaboration between all stakeholders – including the Evaluator, MCA, both MCC and MCA M&E teams, and data collection agencies and contractors. In terms of the data on which the evaluation relied, the household survey contained many flaws which limited its usability, and the sample size of households did not allow for generalizable results. Grantee and non-grantee survey data collection experienced many problems as well, such as delays in developing and procuring baseline data. This resulted in data that was intended for a baseline being implemented after some grantees had already received grants. Furthermore, much of the collected data was not usable, which highlights the importance of a stronger, clearer role of evaluator oversight in data collection contracts. As the Evaluator was not a formal part of the data collection contract, they were left out of data collection fieldwork and encountered challenges in ensuring quality data. More collaboration between the Evaluator, the MCA, the M&E team, and other contractors (e.g. data collection agencies) could remedy this problem in future evaluations. In addition to the aforementioned Evaluation Management Committees, MCC has also begun standardizing contract language around Evaluators’ role in data collection oversight.
Lesson Learned