Qualitative data can help clarify and provide insights on unexpected findings. In MCC’s early evaluations, such as the evaluation of the Peri-Urban Rangeland Project (PURP), MCC employed largely household surveys with little qualitative analysis. As such, there was no way to obtain an understanding of some unexpected findings. More recently, MCC has used mixed method data collection methodologies, including impact and performance evaluation elements with triangulation of data from qualitative, quantitative and administration data sources. Allowing time for analysis of the household data sets to inform the qualitative instruments and add perspective on some of the results finding in household surveys could be beneficial as well.
Severe weather can create difficulties in evaluating aspects such as mortality rates and control of animal numbers. In Mongolia, the year of the baseline data collection (Phase 1-late 2010 and Phase 2-early 2012) followed a dzud (2009-2010). This left lower animal numbers and stocking rates on the land than there would have otherwise been and higher than normal mortality rates when measuring animal deaths over the few years prior to the baseline. Although control herders faced the same severe weather issues, it still was difficult to interpret some of the data findings and related causes. For example, animal numbers increased in all areas for both treatment and control herders between baseline and endline, but animal stocking rates of program herder groups remained largely within the carrying capacity of the land that was determined by the PURP implementers in 2009 during parcel mapping activities.