When agreeing to an evaluation design, particularly impact evaluation, the team must adequately consider tradeoffs and agree to rules of implementation. In Jordan, contingency funds were not needed and the project decided to scale-up the water pipe works to new areas outside of the original work plan. However, the evaluation design was based on the original work plan and these areas were designated as “controls” in the evaluation design. The scale-up therefore had to be delayed and phased in after the evaluation’s final round of data collection to preserve the ability of the evaluation to rigorously estimate the impact of the program. By the time of this scale-up decision, significant resources had already gone into implementing an impact evaluation of the WNP, so this was the only resolution possible. This was perhaps one of the first instances where significantly more infrastructure (approximately 65 km of new sewage pipes) was implemented than was initially agreed in the Compact and it could only occur as infrastructure contingency funds were released and re-applied to additional works.
While there were real-time planning discussions with the M&E team regarding the location of these additional sewage lines, final decisions were made based on the country’s infrastructure need and the impact on M&E ‘control areas’ minimized as best as possible. If there had been a discussion upfront, during the evaluation design, that there was a possibility of contingency funds being used to expand works to specific areas, these areas could have been excluded from the evaluation design. More open discussions about work plans and more clarity about future plans or contingencies can avoid situations where evaluation designs limit options for project implementers.