Effective qualitative data collection enhances an evaluation’s learning opportunities; it can even inform the structure of the evaluation itself. The evaluation team did not understand key dynamics that were both relevant for program implementation, as well as evaluation design and implementation. For example, early data collection focused on the household rather than “kraal managers” as the appropriate unit of analysis. Similarly, the evaluation team did not understand the full range of behaviors to investigate during quantitative data collection, the risks of community conflict, or the outside farmer poaching problem until they collected and analyzed qualitative data. Perhaps the implementer was aware of these dynamics and their project documentation simply did not reflect it, or perhaps these are issues the implementer also could have learned about in the process of their adaptive implementation approach as discussed above. Either way, understanding these dynamics increased the value of the subsequent quantitative data collection dramatically. It also helped build trust with the implementer, who welcomed updates about what the evaluator was learning and providing their feedback on this learning, and showed that the evaluator cared to really understand the intervention and not just pass judgment on it.
Lesson Learned