Evaluations tracking land transaction time risk failure without a detailed understanding of which land transactions will be tracked, what part of the transaction process at which institutions will be captured, whether tracking administrative or consumer time and the start/stop dates at each step along the process. The RSPS evaluation in Mongolia experienced two failed evaluation designs, which led to a baseline established only in the final year of the compact. The initial evaluation designs failed to 1) distinguish the type of transaction time collected (consumer, administrative or publicized time) and clarify the points in time being measured, which provided useless noncomparable data with wide variation in responses; and 2) incorrectly assumed people would go back to General Authority of State Registration (GASR) as soon as the transaction processed. In fact, people were automatically told to come back at the publicized transaction processing time by law (regardless of when the registration was processed), and for key transactions like mortgages, the person did not actually need to return to GASR. After understanding the complete context, the third and successful RSPS evaluation design and data capture relied on surveys of those accessing banking loans pre-post, key informant interviews, and data capture of historic and recent administrative data from GASR and the newly established electronic Property Registry System.
To avoid these failed evaluation efforts in the future, new compacts should obtain, early on, a complete understanding of the land transaction process for each of the key land transactions tracked and establish effective evaluation designs and related baseline data capture at the start of the compact. It is important to understand which type of transaction time is a key constraint (administrative or consumer) and to include those nuances within the logic and related evaluation design. In Mongolia, the processing of records was largely already done within the official publicized transaction time; however, the issue was consumer time getting requests into the formal system and obtaining related documents rather than the administrative time to process those transactions. For the baseline, review of paper records is usually required as the paper forms often contain the application dates and approvals (or stamped dates) which are used to measure processing time. There are cases when there is an existing digital system with scanned in records or digitally captured records, which can provide some historic transaction data like in Mongolia’s case. The steps, institutions, and type of records likely will differ pre/post project as MCC land interventions often streamline procedures and establish new systems and institutions.