Land quality can be a key factor in investment and behavior change around land utilization. Land quality should be considered when establishing comparison groups and analyzing land-based outcomes. This is one of the first impact evaluations that attempted to capture measurements of land quality and land degradation. The Peri-Urban Rangeland Project (PURP) theory of change was based on the idea that granting private use rights over common use rangeland would encourage better rangeland management and decrease overgrazing of pastureland, especially around heavily populated areas around city and regional centers. In line with the project logic, the evaluation found four of the five PURP areas saw increases in perceptions of tenure security around ability to protect their land from overgrazing and expropriation, as well as some changes in rangeland management behavior and related increases in perceptions of land quality. For the fifth area (Choibalsan), the evaluation found that, it was not as overgrazed (18% of herder groups stocking above the carrying capacity of the land) as the other project areas and similarly that there were already high levels of perceived tenure security against expropriation of land and protection from overgrazing in that area. Choibalsan hence saw no significant changes in perception of tenure security and relatively larger increases in herd numbers than comparison households (the opposite of the expected effect). Although the lack of change in perception of tenure in Choibalsan could have been due to relatively high perception of tenure and perhaps leases being issued prior to the baseline, another explanation and one that would also explain the change in herd size, is that Choibalsan was not as degraded as the other peri-urban areas and already stocking animals below the carrying capacity of the land.
Beyond implications for outcomes, there is also a concern that establishing comparison groups on solely socioeconomic factors may not truly be comparable, as factors like land quality and characteristics of the land (slope, soil and water availability) can drive land utilization and investment behavior. In the evaluation of PURP for Phase 1, the two evaluators actually established different comparison groups. The socioeconomic survey by Innovations for Poverty Action used traditional matching methods. Alternatively, the land quality survey by USDA Agricultural Research Service Jornada used neighboring parcels with similar land characteristics noting that they did not believe the comparison areas selected by the socioeconomic survey had comparable land parcels, especially when measuring pastoral use and land degradation. A household may be similar in terms of income/education but if parcel sizes, land quality and land characteristics differ across treatment and comparison groups, the herders living on those parcels will not have similar land investment and herd management practices. MCC should consider when it makes sense to add land quality as a key evaluation variable. Due to the complexities in measuring land quality, MCC should ensure the necessary technical expertise is part of the required expertise in evaluator scope of work.