Be cautious when pursuing randomized rollout designs. The Lesotho Compact attempted two randomized rollout evaluation designs and both suffered delays that threatened their internal validity and compromised their statistical power. Ultimately, the Rural Water evaluation was preserved, while the Health evaluation was not. This challenge is not unique to Lesotho. Given MCC’s five-year implementation timelines, the time taken to prepare for implementation, and the realities that often occur on the ground, completing one phase of implementation and ensuring a sufficient lag before the beginning of another phase, can be very difficult and should be considered carefully and commitment of relevant parties secured before investing significant resources in this type of design.
Lesson Learned