Measuring Results of the Burkina Faso Rural Land Governance Project

The MCC Compact with Burkina Faso (2009-2014) invested $58 million in a Rural Land Governance (RLG) Project, which was designed to increase investment in land and rural productivity through improved  land tenure security and land management. RLG was divided into two phases and took place in 47 communes, including 17 pilot communes in Phase 1 (2009-2012) and 30 additional communes in Phase 2 (2012-2014). RLG successfully established 78 land administration offices, trained over 60,000 stakeholders on conflict resolution and land management, and formalized land rights for almost 4,800 households.

To evaluate the impact of RLG, an independent evaluator is conducting an impact evaluation using difference-in-differences methodology. The evaluator collected baseline data in Phase 1 areas in 2010 and interim data in 2012, which was an input into the decision to expand to Phase 2 areas. Local land institutions had just been established in pilot communes and no land use certificates (APFRs) had yet been provided to households. Although very early in project implementation, analysis of interim data found an increase in some perceptions of land tenure security. No statistically significant change was measured for the number of land conflicts resolved or agricultural productivity; however, this is to be expected from interim data collected too early to measure these longer-term effects. Endline data is planned for collection in Phase 1 and Phase 2 areas in 2017 and should provide a more comprehensive view of results.

In Context

The MCC Compact with Burkina Faso was a five-year investment (2009-2014) of $480.9 million in four projects: Rural Land Governance (RLG) Project, Agriculture Development Project, Roads Project, and BRIGHT II Schools Project. This memorandum focuses on the RLG Project, which represented 12 percent of the Compact ($58 million) and included three major activities: Legal and Procedural Change and Communication, Institutional Development and Capacity Building, and Site-Specific Land Tenure Interventions. RLG took place in two phases across 47 communes. Implementation of Phase 1 covered 17 communes in years one and two of the Compact (2009-2012), with extension to 30 additional communes in Phase 2 areas in the final three years of the Compact (2012-2014).

 

                                         

Program Logic

The Rural Land Governance Project was designed to increase investment in land and rural productivity through improved land tenure security and land management. The project logic follows below:

                                   

There were several key assumptions underlying the Rural Land Governance program logic during the design of the investment:

  • There was tenure insecurity within the 47 commune areas, including perceptions of weak tenure security and occurrence of conflicts;
  • Existing customary systems were unable to effectively resolve conflicts;
  • Land investment was constrained by tenure insecurity;
  • Once land institutions and regulations were in place, residents would demand land services, including formalization of land rights through APFRs; and
  • Land institutions and knowledge would be sustainable post compact, including through financing of these institutions by the government.

Measuring Results

MCC uses multiple sources to measure results, which are generally grouped into monitoring and evaluation sources. Monitoring data is collected during and after compact implementation and is typically generated by the program implementers. This data focuses specifically on measuring program outputs and intermediate outcomes directly affected by the program. However, monitoring data is limited in that it cannot tell us whether changes in key outcomes are attributable solely to the MCC- funded intervention. The limitations of monitoring data is a key reason why MCC invests in independent

 

impact evaluations, which use a counterfactual to assess what would have happened in the absence of the investment and thereby estimate the impact of the intervention alone. Where estimating a counterfactual is not possible, MCC invests in performance evaluations, which compile the best available evidence and assess the likely impact of MCC investments on key outcomes.

Monitoring Results

The following table summarizes performance on key output indicators specific to the evaluated program according to the compact monitoring data. 1

 

Indicators

 

Level

Baseline (2010)

Actual Achieved (07/2014)

 

Target

Percent Complete

Communal land cover maps completed

Output

0

47

47

100%

Land rights formalized 2

Output

0

4,793

 

 

Number of Rural Land Possession Certificates (APFR) approved by the local government

 

Output

 

0

 

2,167 3

 

6,000

 

36%

Number of households receiving Rural Land Possession Certificates (APFR)

Output

0

403

3,000

13%

Rural Hectares (Ha) formalized in new zone of Di, targeted under the Agriculture Development Project

 

Output

 

0

 

207 4

 

2,240

 

9%

Number of parcels formalized in the new zone of Di

Output

0

418 5

 

 

Number of parcels of irrigated land leased to households or legal entities in the existing irrigation zones (Zones Amenagees)

 

Output

 

0

 

0 6

 

6,000

 

0%

Number of plots registered in the Ganzourgou pilot project

Output

0

15,905

11,000

145%

Number of households that benefited from parcels in the Ganzourgou pilot project

 

Output

 

0

 

3,695

 

 

Rural hectares (Ha) formalized in the Ganzourgou pilot project zone

Output

0

47,531

50,000

95%

Legal and regulatory reforms adopted

Output

0

54

15

360%

Stakeholders trained

Output

0

61,057

8,452

722%

Number of functioning CORS stations

Output

0

9

9

100%

Number of geodetic control points

Output

0

700

700

100%

Land administration offices established or upgraded

Output

0

78

78

100%

Number of municipal buildings constructed

Output

0

47

47

100%

Source: (July 2014 Closeout ITT, based on reporting from various contractors and surveyors, including LTP-5, LTP- 35 and LTP-45. Data specific to Di were taken from report by AD-4.9 and AD-7.)

 

In 8 of the above 13 indicators with targets, the targets were met or exceeded.

 

Evaluation Questions

The evaluation was designed to answer the following key questions:

1.    Impacts of project activities in Phase 1 and Phase 2 communes:

  • What is the impact of the project on (1) perceptions of land tenure security; (2) the frequency and types of land conflicts; and (3) producers’ investment decisions (e.g., by increasing investment levels, encouraging farmers to make more fixed investments, etc.) that increase incomes?
  • Subgroup analysis: Did land tenure (and the other outcomes) for women improve as a result of the project activities?

2.  Review of legal and policy changes and national impacts:

  • How effective was the MCA-BF in reforming land laws and regulations?
  • Specifically, the evaluation will review the adoption of the Rural Land Law (Loi No. 034/2009) and its primary application decrees, as well as the amendments and revisions to the Law on Reorganization of Agriculture and Land (RAF), including whether communes outside of the 47 treatments made any changes based on legal and policy changes.

3.  Institutional development and performance:

  • What are the effects of RLG Project on land governance and administration, including development, performance and sustainability of new institutions that have been established by the project, including Services Fonciers Ruraux (SFRs) and Commissions de Conciliation Foncière Villageoise (CCFVs)?

Evaluation Results

This interim evaluation report focuses on early results from Phase 1 of the project, during which legal and procedural changes were conducted at the national level and institutional development, capacity building, and site-specific land tenure interventions were just beginning to be conducted in 17 pilot Phase 1 communes.

The results of the interim evaluation on the impact of Phase 1 on perceptions of land tenure security, the number of conflicts, and agricultural productivity can be summarized as follows:

  • A general improvement in land tenure security perceptions were observed in both treatment and comparison areas for all perception measures. This result may indicate the effectiveness of national outreach and legislative efforts.  As outreach and legislative changes were national in nature, there was not a good counterfactual to test this theory.
  • When comparing treatment and comparisons, there is statistically significant evidence that the RLG Project Phase 1 improved certain types of farmers’ perception of land tenure security, but did not improve all types of perceptions of tenure security. The evaluation showed statistically significant evidence of RLG Project Phase 1 effects on farmers’ perception as measured by concerns over the arrival of new people to exploit land and the return of previous villagers to reclaim land.  However, there is no evidence that the RLG Project Phase 1 activities had impacts on the perception of land tenure security as measured by perceptions on inheritance disputes, animal damage or farmers’ restrictions on areas for livestock.
  • No statistically significant site-specific impact on the number of land conflicts was observed.
  • No statistically significant evidence of impacts on land conflicts resolution or farmers’ confidence in Land Chiefs, Village Development Councils, and the High Court as land conflict resolution institutions was observed.
  • There is no overall evidence of increasing agricultural revenues as a result of RLG Phase 1 activities between 2010 and 2012 for major crops in Burkina Faso.

The findings suggest encouraging interim results regarding changes in perceptions of land tenure security. This is consistent with expected results because the project logic expected early changes in perceptions of land tenure security followed in the longer-term with related changes in agricultural investment and production. The endline data collection and analysis scheduled in 2017 should provide an opportunity for more comprehensive understanding of long-term RLG impacts.

Evaluator

Impaq International

Methodology

Difference-in-Differences using two stage cluster sampling

Evaluation Period

Late 2010-Early 2012

Immediate Outcome

Findings suggest encouraging short-term results in regards to initial changes in perceptions of tenure security for both treatment and control groups. However, no statistically significant impact on the number of land conflicts or farmers’ confidence in land conflict resolution institutions has been measured. This is not surprising considering the limited time between project implementation and interim data collection. Land institutions had been operating for a very limited time period.

Intermediate Outcome

Thus far, there has been no evidence of changes in conflicts or agricultural revenues as a result of Phase 1 activities. However, the project logic did not expect such changes in the short-term and IMPAQ notes the lack of findings on these variables could be the result of the interim survey carried out too early to measure these results, which was ahead of the bulk of implementation, including the lack of provision of APFRs prior to interim data collection.

Ultimate Impact

Ultimate impact could not be assessed within the timeframe of this round of data collection for the impact evaluation. 2017 endline data will provide final results.

Lessons Learned

The experience of the evaluation provided a few important lessons learned:

  • Considering the short 5 year time period of MCC Compacts, the decision to extend a pilot phase of a land project should not depend on evaluation results. The timeline of less than two years between baseline and follow-up was too short to see results, especially when considering the timeline necessary to implement legislative and regulatory changes, establish new land offices and facilitate site specific formalization, as well as the time necessary for land results to develop after implementation is complete.
  • The more detailed and nuanced questions on perceptions of land tenure and types of conflict used in the Burkina questionnaire allowed a better understanding of project impacts on land tenure and related potential effects on productivity.  These more detailed questions on sources of tenure insecurity and levels of conflict should be considered in future evaluation instruments. .

Next Steps

Endline data collection is planned for 2017 and will focus on longer-term results from Activities 1 and 2, as well as early to mid-term results of RLG activity 3 (site specific) interventions in Phase 1 and Phase 2 communes.

Footnotes
  • 1. This table reflects data from the final approved MCA-Burkina Faso Indicator Tracking Table (ITT) and captures progress through the final date of the compact, July 30, 2014. Please note that although additional work was conducted during the closeout period, the progress made during that period is not included here.
  • 2. In Burkina Faso, this includes the number of households receiving: APFRs; formal land rights in Di; APFR-like rights  [Anchor] in Ganzourgou Province; and land leases in existing irrigation zones.
  • 3. Although the “APFRs approved” target was not met, RLG by Compact end had introduced application files of 13,447 APFRs.
  • 4. During the three-month compact closeout period (August-November 2014), the number of hectares secured by a title on the Di perimeter reached 637, and the number of hectares of irrigated land leased to households by the state in the Di perimeter was 1,198.
  • 5. All 418 parcels that were formalized in Di during the compact were for land titles and no parcels of irrigated land leased were formalized until after compact end date. The number of land titles delivered in the Di perimeter reached 1,306 during the compact closeout period (August-November 2014) following the compact end. All titles went to project affected parties (PAPs). During the same closeout period, the number of parcels of irrigated land leased to households in the Di perimeter reached 1,061.
  • 6. The number of parcels of irrigated land leases to households or legal entities in the Zones Amenages (old perimeters) remained at 0 in the closeout period (August-November 2014); however, the number of leases approved by the State in the Zone Amenage (old perimeters) reached 3,443.