Measuring Results of the Alatona Irrigation Project in Mali

Introduction

This Summary of Results outlines the background and results of the Interim Findings Report by Innovations for Poverty Action (IPA) who served as the MCC Independent Evaluator for the Alatona Irrigation Project (AIP). The Interim Report was based on data collected in mid-2012 and is not intended to assess intermediate or final impacts of the AIP since the data was collected immediately after project completion. Additional data collection and analysis is planned to assess the impacts of the AIP if  deemed feasible given recent turmoil in Mali.

In Context

The MCC compact with Mali was a five-year investment of $461 million (of which $435,628,223 was spent) in two projects: the Bamako International Airport Project and the AIP. The AIP included six major activities:

  1. Road upgrade ($44.8m): The project planned to pave 81 km of road between the towns of Niono and Goma Coura to improve farm-to-market access and general transportation. The project improved the roadbed and made a critical flood plain cross passable year round but only paved 16 km of road. 
  2. Irrigation Planning and Infrastructure ($159.8m): Under this activity, the project increased the capacity of the entire Office du Niger (ON) irrigation and canal sytem and added an additional 4,940 hectares (originally 16,000) of irrigated land in the Alatona perimeter. This was a heavy-construction exercise consisting of expansion of the main conveyance system, canal dredging, land clearing, and laying out new irrigated land parcels. It also included improvements to the existing Office du Niger (ON) water management system.
  3. Land allocation ($940k): Through this activity (i) new irrigated land parcels were surveyed, (ii) five-hectare farms were allocated in ownership to 954 beneficiary households, (iii) land recipients were educated about their rights and obligations, (iv) the local land registration system was upgraded, and (v) revenue from the sale of land was collected for future community needs (66 percent of the land was sold to project beneficiaries). 801 Project Affected Persons (PAP), 153 New Settler (NA) concessions and 63 women’s garden associations received land titles.
  4. Resettlement, Social Infrastructure and Social Services ($31.8m):  Each household displaced or affected by the irrigation infrastruture development received new housing, a full range of social benefits, and two hectares of irrigated land as compensation for their loss of access to land, as well as incurring the obligation to purchase an additional three hectares of irrigated land over 12 years. The social infrastructure included schools, health centers, and water holes with pumps. In addition the project built two warehouses for each of the new settlements, one for rice and one for horticultural crops. Finally, as part of the resettlement package, women of each PAP household were entitled to 0.05 ha (500 square meters) of market garden land situated in the perimeter's double-crop land. 
  5. Agricultural services ($15m): This activity assisted farmers in all 801 PAP concessions to improve their farming and business skills and work together in farmers' associations. In addition, farmers received seeds and other supplies to help them begin cultivating the land they received through the project.
  6. Financial services ($420K): This activity improved the capacity of local financial institutions to track loans, farmers to secure loans and repay them and manage their financial resources. It also provided PAPs with the funds they need to make an initial deposit to and open an account with a local MFI of their choosing. 

 

The $253 million AIP component of the Mali Compact is the subject Interim Evaluation Report by Innovations for Poverty Action (IPA). This AIP component represents approximately 53% of the total compact. Other components of the compact are the subject of forthcoming independent evaluations.

 

                                  

Program Logic

The AIP focused on increasing production and productivity, increasing farmer incomes, improving land tenure security, modernizing irrigated production systems and mitigating the uncertainty from subsistence rain-fed agriculture by introducing innovative agricultural, land tenure, and water management practices. In addition, the compact included policy and organizational reforms aimed at realizing the Office du Niger’s (ON) potential to serve as an engine of rural growth for Mali. There were two categories of beneficiaries in the AIP: (i) Project Affected Persons (PAPs) who were already living in the area and had limited to no farming experience; and (ii) New Settlers (NAs) who had previous farming experience and were allocated parcels of land in the Alatona perimeter through a lottery.

The primary outcomes targeted by the program included:

  • Greater irrigation access and reliability
  • Increase in farm yields
  • Increase in loans and investments on land
  • Increase in crop revenue and household income
  • Increase in women’s income

 

                             

Additionally there were several key assumptions identified in the project logic such as:

  • Funded bank accounts would be used as collateral in obtaining a loan from the micro-lending institutions operating within Alatona.
  • Land titles increased the willingness of farmers to secure loans.
  • Effective farming practices were combined with sufficient agricultural inputs (labor, fertilizer, seed, etc.) in order for the expected increase in yields.
  • Ascertain amount of integration between PAPs and NAs that took place in order for social diffusion of farming practices to take place.
  • The Office du Niger properly managed the flow of water from the Niger River enabling sufficient irrigation.
  • There was sufficient confidence in land titles and effective functioning of the land registry office.

Measuring Results

MCC has used multiple sources to measure results, which are generally grouped into monitoring and evaluation sources. For the AIP, monitoring data was collected during compact implementation by the program implementers; it focused specifically on measuring program outputs directly affected by the program. However, this monitoring data was limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC-funded intervention. The limitations of monitoring data was a key reason for why MCC invests in independent impact evaluations, which used a counterfactual to assess what would have happened in the absence of the AIP investment..

Monitoring Results

The following table summarizes performance on output indicators specific to the AIP.

 

Indicators

Baseline (2006)

Actual Achieved (7/2012)

Original Target

End of Compact Target

Percent Complete

Cultivation intensity during the dry season (%)

0

15

20

20

75%

Value of agricultural products sold by farmers (Millions of FCFA)

0

95

11.14

11.14

853% 1

Hectares under new irrigation 2

0

4,942

16,000

4,942

100%

Contracted irrigation construction works disbursed (%)

0

96

100

100

84.47%

Market garden parcels allotted to PAP or New Settler women

0

801

1,034

1,034

77%

Rural hectares formalized

0

2,995

16,000

4,942

61%

Net Primary School Enrollment Rate (Alatona Zone) (%)

1

56

60

60

93%

Work completed on the Niono- Goma Coura road (%)

0

27

100

100

27%

Functional producer organizations

0

30

30

30

100%

Hectares under production (rainy season)

0

3,274

4,942

4,942

66%

Hectares under production (dry season)

0

733

1,040

1,040

70%

OERS Established

0

9

17

17

53%

Active MFI clients

0

846

793

793

107%

Source: November, 2012 ITT (https://assets.mcc.gov/documents/MALI-Q20-KPI_1.pdf)

 

The average completion rate of these targets is 145% percent; and in 6 of the 12 indicators, targets were met or exceeded. When the completion rate for the indicator “Value of agricultural products sold by farmers” is removed this percentage drops to 68%.

Evaluation Questions

The evaluation was designed to test various hypothesis for each activity as outlined here:

Activity

Evaluation Hypothesis

Irrigation Planning and Infrastructure

The AIP will improve New Settler households’ well-being by increasing agricultural productivity, agricultural incomes, and household consumption.

 

Access to irrigation will increase agricultural production in PAP villages.

 

Distribution of the gains in agricultural income: Gains in agricultural income among PAP villages will be distributed among increases in consumption per capita, asset and livestock holdings, and input utilization.

Land Allocation

The provision of land titles will increase productivity in the Alatona perimeter by increasing household investment in their plots and access to credit to finance these investments.

Resettlement, Social Infrastructure and Social Services

The process through which New Settlers create social capital in their new villages will be important for consumption smoothing and the functioning of village associations.

 

The productivity of both New Settlers and PAPs will increase over time, possibly at different rates.

 

Social learning in the Alatona perimeter will complement formal extension services, leading to improved agricultural productivity.

Agricultural services

Access to irrigation will increase the demand for inputs (fertilizer and seed), agricultural capital, and household and hired agricultural labor.

 

AIP will increase women’s incomes in the Alatona perimeter and potentially influence women’s empowerment.

Financial Services

The provision of land titles will increase productivity in the Alatona perimeter by increasing household investment in their plots and access to credit to finance these investments.

 

Interim Evaluation Results

IPA’s Interim Report was based on a comparison of baseline and interim survey data of treatment and comparison groups. The baseline data were collected in 2008-2009, and interim data were collected in 2012. IPA’s interim survey was conducted soon after the New Settlers (NAs) were settled on their  plots. Because no project benefits could have yet accrued to these NAs, the interim analysis focused solely on impacts on the Project Affected Persons (PAPs) – local households who were displaced by the irrigation project and compensated with allocations of irrigated land, as well as training, technical and material support to facilitate a change of livelihoods to help them benefit economically from the project investments.

Agricultural production in the treatment area increased dramatically, growing by 15 metric tons (MT) per farmer, a ten-fold increase. This compares with total production per farmer in the comparison areas that was relatively stable at 5MT. IPA noted that these production increases were associated with corresponding increases in the amount of fertilizer applied by treatment farmers, which just matched the fertilizer they received in the project starter-kit grants. According to IPA, this intensive use of inputs    and corresponding production levels need to be tracked as the PAPs limited ability to finance these large fertilizer investments without the grants may constrain their future productivity to lower long-term levels.

Poverty, measured as changes in asset holdings plus real consumption expenditures, was reduced by 18 percentage points among the PAPs from a baseline of 40 percent.  The declines were not the result of improved food consumption patterns, which normally signals sustained real incomes; rather they were driven by increases in asset holdings and non-food consumption. The increased asset holdings were directly affected by the project, encompassing new housing, generous “starter-kit” grants of livestock and equipment, agricultural inputs and cash grants, all intended to facilitate transitions to irrigated agriculture. And the non-food consumption measures also reflected direct support to the PAPs in the form of improved community infrastructure such as water wells and schools.

These measures of significantly increased production and reduced poverty are only short-term indicators of the AIP impacts. Longer term impacts will be influenced by both the disappearance of the generous starter-kit grants, and by the PAPs ability to learn and improve upon their newly acquired irrigated agriculture skills and resources. IPA advocates that these poverty declines need to be monitored over a longer period to ascertain whether the declines are sustainable.

Evaluator

Innovations for Poverty Action

Methodology

Randomized controlled trial (RCT) approach for New Settlers and propensity score matching for Project Affected Persons

Evaluation Period

2007-2012

Immediate Outcome

Total amount of land cultivated by PAP households increased by 1.572 hectares largely due to an increase in land cultivated by men. Women’s land shows no significant change. However there is substantial increase in fertilizer usage but increase in other inputs is similar to that given to PAPs as part of the starter kits.

Intermediate Outcome

It is too early to assess the intermediate outcomes of the project, given the limited number of growing seasons completed by beneficiaries at time of data collection.

Ultimate Impact

It is too early to assess the ultimate impact of the project, given the limited number of growing seasons completed by beneficiaries at time of data collection.

Lessons Learned

IPA’s Interim Report was not meant to outline measurements at the time of project completion and before targeted materialized. Given that the NAs had not yet farmed an entire season, and that most of the PAPs had farmed a single, or zero, growing seasons, IPA noted that additional post-compact data collection was needed, if deemed feasible given conditions on the ground, to offer up lessons learned on intermediate and ultimate impacts. These intermediate and ultimate impacts have been articulated in the form of evaluation questions for this possible post-compact data collection and analysis to include:

  • Livelihood transition: Are the PAPs switching to farming? If so, is this transition happening in a successful/sustainable manner?
  • Role and status of water user associations: How well are they functioning and what does this mean for the sustainability of compact benefits? How effective was MCC in working with the water user associations?
  • Wo men’s Gar den Plo ts ap p ro ach : How has the garden affected the role of women in the household and community? How effective were the market gardens in meeting their objectives? Why or Why not?
  • Access to Finance / Savings: Do greater land use rights (rights and land size) influence their decisions to lend? How effective was the activity with respect to farmers taking out and repaying loans, type and amount of savings?
  • Land titling: What are the effects of the land titling activity?
  • Expansion of the Main Conveyance: What were the effects of the expansion of the main conveyance? Who were the beneficiaries of this expansion and how were they affected?
  • Overall effect of starter kits: What was the effect of the starter kits in achieving outcomes? How are the assets of the starter kits being sustained? For the starter kit inputs like seed and fertilizer, are farmers acquiring these inputs after the starter kit supply was depleted? How do these results compare with those that only received improved and more reliable access to water as a result of expansion of the main conveyance?
  • Yields, Prices and Value Chain: Have there been changes in the (i) yields for the rainy season and the dry season by crop type; (ii) crop prices; (iii) access to inputs; and (iv) access to markets – both inside and outside the perimeter (main conveyance beneficiaries)?
  • Integration of PAPs/New Settlers and social learning: How/are these groups integrating? What is the role of community associations and how/is social learning occurring? What are the reasons behind any changes or lack of change?
  • Labor demand: Is there an increased demand for labor and if so, how is this demand being met?
  • Approaches to agricultural development: Was the farmer field school approach an effective way to provide extension services to the PAPs? Should the project have considered alternate approaches?
  • Farmer diversification: What are the PAPs planting in the dry season? Have livestock numbers changed, and if so why?
  • Division of labor: Does the labor burden on women and children included in the treatment areas differ from the comparison group?
  • Farming practices: Have farming practices changed over time to adopt the use of more advanced technology (i.e. motoculteur). If so, why and if not, why not?

Next Steps

With the 2012 Coup and residual turmoil in Mali, IPA was forced to suspend data collection before the entire sample frame was surveyed and leave the country. Given this further evaluation activities have been postponed but an Independent Evaluator is soon being commissioned to: (i) perform an evaluability assessment; and (ii) if feasible, develop and implement an evaluation designed to answer the      questions outlined above. The IPA evaluation was designed to evaluate the outcomes of the activities    as a whole, meaning as a bundled treatment. IPA was not able to distinguish the effects from the financial services activity from the market gardens activity for example because all the activities took place at once for the same beneficiaries. The same will hold true for any future evaluation.

However, as demonstrated in the post-compact evaluation questions outlined above, MCC is focused on generating more useful information about the effects of the individual AIP activities. While any post- compact evaluation will not be able to statistically distinguish the effect of these activities from the AIP as a whole, mixed methods could generate useful information on what is and is not working and why.

Footnotes
  • 1. Such a large increase in value of agricultural products sold by farmers is mostly because there was a zero baseline given these were beneficiaries that were new to farming and also the compact target is considered to be very low.
  • 2. It is important to note that in 2009 the project was re-scoped as costs were greater than the estimated budget and provided 4,942 hectares of new irrigation instead of the original target of 16,000 hectares.