Ensure sufficient statistical power. In the future, before investing significant resources into an evaluation, MCC should ensure that there will be sufficient statistical power for measuring realistic changes in key outcomes. Measuring many of the Samtskhe-Javakheti Road Rehabilitation’s intermediate outcomes and ultimate impacts proved difficult, given that impacts are generally disperse (i.e. covering large populations). With a better understanding of the expected impacts and a clearer identification of beneficiaries, a more precise alignment between an intervention’s program logic and its evaluation design can be achieved. In addition, the challenges with using the household data which was collected in measuring impacts suggest that data collection strategies can also be improved and tailored to the given evaluation methodology.
Many of these lessons are similar to those that MCC has learned from previous evaluations. As a result, MCC has already adjusted its evaluation practices to include a formal review process for evaluations, an evaluation risk assessment, and use of standardized evaluation templates. The new process also requires substantive review and clearance of key evaluation documents by sector specialists in order to incorporate feedback on the technical and factual accuracy of evaluation plans.