Summary of Findings

Measuring Results of the Artisan and Fez Medina Project

Artisan Production and Promotion and Fez Medina Activities

View as PDF

In Context

Pie chart: Percentage of Compact Funding to Each Project

The MCC compact with Morocco was a five-year investment (2008-2013) of $666.4 million in five projects:  Fruit Tree Productivity; Small Scale Fisheries; Artisan and Fez Medina, which included the Functional Literacy and Vocational Training (FLVT) activity; Financial Services; and Enterprise Support.

 

This $84 million Artisan and Fez Medina Project represented 13% of the total compact and aimed to stimulate economic growth by leveraging the links between the craft sector, tourism, and the Fez Medina’s rich cultural, historic and architectural assets. The Artisan and Fez Medina project consisted of the Artisan Production and Promotion Activities, the Fez Medina Activity, and the Functional Literacy and Vocational Training Activity. 

 

This performance evaluation focuses on two components: (i) the Artisan component ($16.8) of the Project, which represents 3% of the total compact and was comprised of two activities: Artisan Promotion and Artisan Production. This component provided resources to develop handicrafts and architectural heritage by allowing artisans in Fez and Marrakech to improve the quality of their products through access to technical training of production techniques and business management. (ii) The Fez Medina Activity ($35.5 million), which aimed to support the design and reconstruction of historic sites within the Fez Medina, including,  (i) the design and reconstruction of Place Lalla Ydouna (PLY) and three 14th- and 15th-century fondouks (large, multi-story structures surrounding a central courtyard); (ii) an international design competition for the PLY reconstruction and re-design to better serve local residents and attract visitors to Fez; and (iii) the development of a production zone at Ain Nokbi for the resettlement of copperware workers affected by the rehabilitation of PLY.

Program Logic

The Artisan component of the Artisan and Fez Medina Project (AFM) was designed to stimulate growth in the artisanal sector, through the creation of the necessary conditions for increasing productivity and competitiveness of mono-artisans and SMEs, with the goal of increasing economic growth and reducing poverty in the Fez and Marrakech regions.  To achieve this, the Artisan Promotion Activity carried out a major marketing campaign for artisans and SMEs in the sector; this included promotional events and expos, participation in trade fairs, and business networking, as well as training to build artisans’ capacity in marketing and management.  To complement these efforts, the Compact provided technical assistance to develop a national label branding Moroccan handcrafts, and tourism circuits in Fez and Marrakech were rehabilitated, improved and in some cases newly created to increase foot traffic along artisan routes.  The Artisan Production Activity trained potters in these regions in improved production techniques and safety standards and also provided a 40% subsidy towards the acquisition of new gas kilns. 

The program logic for the first activity is described in the following figure:

Inputs

Outputs

Immediate

Outcomes

Intermediate

Outcomes

Goal

MCC Project funded:

       

Support to pottery  production

-Training of potters on production, marketing and management techniques;

-A subsidy to potters to purchase gas kilns to replace traditional and highly polluting ones

 

 

-Potters in Fez and Marrakech are trained ;

-Potters in Fez and Marrakech have access to subsidized gas kilns

 

 

-Modern techniques in pottery production are adopted by trainees in the fields of production, social and environment security, marketing and management as well as in terms of using the gas kilns

 

 

-Pottery product quality is improved

-Loss in pottery cooking is reduced

-Pottery production capacity is increased

 

 

-Potters’ income is improved in Fez and Marrakech

-Environment degradation caused by traditional kilns  is reduced

Promotion of artisans and artisan SMEs

-A promotion campaign to promote artisan products

-Creation of a national label for Moroccan Handicraft

-Creation/Rehabilitation of  touristic circuits

 

 

 

 

-Artisans participated to shows held in Morocco

-SMEs of handicraft participated in international shows, 

-Touristic circuits are created/ rehabilitated

-A national label of handicraft is created

 

 

 

-SMEs of handicrafts appended the national label on their products

 

 

 

-Sales of artisans and SMEs of handicrafts  increased both on the domestic market and to export

 

 

 

-Income of artisans increased

-Value added of SMEs of handicraft increased

Fez Medina :

Rehabilitation of historic sites

 

-Historic sites are rehabilitated

 

  • Number of tourists visiting Fez increased
  • Length of stay in hotels increased

 

-Tourist spending increased

 

 

The main assumptions underlying the Artisan project program logic were that:

  • Potters would have sufficient incentive to acquire new gas kilns, as well as the financial capacity and/or sufficient access to finance to cover their 20% share of the cost for new gas kilns;
  • Potters would have the technical means necessary to put into practice the lessons in best practices received through the training financed by the project;
  • Banks would be willing and able to provide potters with loans adapted to their specific conditions;
  • There would be favorable economic conditions with respect to tourism development and exports of handicrafts;
  • There would be sufficient interest and capacity by artisans and handicraft SMEs to comply with required standards and procedures to participate in the national label program; and
  • There would be sufficient government funding to ensure expansion and sustainability of the national label.

 

Measuring Results

 MCC uses multiple sources to measure results. Monitoring data is used during compact implementation.  Independent evaluations are generally completed post compact. Monitoring data is typically generated by the program implementers, and specifically covers the program participants who received treatment through the compact. MCC conducts performance evaluations to assess whether the program was adequately designed to meet the needs of the program beneficiaries and how the program was implemented. In this performance evaluation, the evaluator used a mixed method approach including quantitative survey of beneficiaries, key informant interviews with project implementers, focus group discussions with beneficiaries, and a document review to assess what tools had been employed in data collection.

 

Monitoring Results

The following table summarizes the performance of output and outcome indicators specific to the evaluated projects.

Artisan Production and Promotion

Indicators

Level

Baseline

Actual

Target

Completion rate

Artisan production

 

 

 

 

 

Number of potters trained

Output

0

2,332

3,905

60%

Number of gas kilns delivered to artisans

Output

0

20

144

14%

Loss rate in pottery production 1

Outcome

30%

5%

5%

100%

Artisan Promotion

 

 

 

 

 

Number of touristic circuits created/upgraded by the Project

Output

0

11

7

157%

Number of artisans participating in promotion events

Output

0

312

200

156%

Number of SMEs participating in promotion events

Output

0

118

130

91%

Number of SMEs that append the label on their products

Outcome

0

0

30

0%

 

The average completion rate of the output and outcome targets is 82.6%; and the targets have been met or exceeded in 3 out of 7 indicators.

 

Fez Medina

As of the end of the compact, work on the three fondouks reached about 35% completion, and the construction work on Place Lalla Ydouna had not yet begun.

 

Indicators

Level

Baseline

Actual

Target

Completion rate

Number of artisans benefitting from Ain Nokbi

Process

0

1283

NA

NA

Number of rehabilitated and/or constructed sites

Output

0

1

5

20%

Number of tourists visiting Fez

Outcome

341,076

286,840

n/a

 

Average length of stay in hotels of Fez

Outcome

2.20

1.96

n/a

 

Average daily expenditures per tourist

Outcome

165.0 2

No data

n/a

 

 

Evaluation Questions

The final performance evaluation for the Artisan and Fez Medina project was designed to answer evaluation questions related to relevance, coherence, effectiveness, efficiency, impacts and sustainability. The main questions are summarized below:

  • Was the financial package, including the project’s gas kiln subsidy, sufficient to incentivize the acquisition by potters of gas kilns? Were the mechanisms established by the project sufficiently effective for potters to acquire gas kilns?
  • Did the potters adopt production and marketing techniques on which they were trained through the project? What was the impact on the quality of the potters’ products? On the prices that they received for improved products? On their income?
  • Did the promotional campaign for the single-product artisans and handicraft SMEs improve the sales of these groups?
  • Did the new and improved tourism circuits lead to more foot traffic along artisan routes?  Did they lead to increases in sales?
  • To what extent did the Fez Medina activity contribute to achieving the expected results of the project?
  • What are the positive and negative, the expected and unexpected consequences of the implementation of Resettlement Action Plan on the affected parties?

 

Evaluation Results

Although most targets were met for the output indicators for the Artisan Production and Promotion activities, useful measurement of the impact of these activities was not feasible under this evaluation. The promotion campaign was completed two months before Compact end date. The financial capacity of potters turned out to be insufficient to provide the 20% personal contribution for purchase of subsidized kilns. Only 20 potters could afford to purchase the new gas kilns to replace the traditional, highly polluting one.

 

The project created the “national label of handicrafts” to promote the authenticity of Moroccan handicrafts. Thirty handicraft SMEs were supported to achieve compliance with the environmental, social and technical requirements of the national label and were declared eligible to append the label to their products. However, as the Pilot Phase of the National Labeling effort had not yet been completed as of the Compact end date, none of these SMEs had yet begun to append the label to their products. 3

 

As for the Fez Medina activity, no output was completed except the construction of a new workshop at Ain Nokbi where the copperware artisans, initially operating in Place Lalla Ydouna, were resettled. According to a survey, 60% of the affected parties by the project declared that the Resettlement Action Plan (RAP) had a positive impact on their social and economic situation. For 54% of them, the project presented them with new socio-professional opportunities.

 

In the new fondouk constructed at Ain Nokbi, working conditions substantially improved. At Place Lalla Ydouna, the air used to be so polluted that the workers there were called “the black men”. Today, the situation has greatly improved thanks to air extraction and filtering systems. Women, in particular, expressed what a positive impact the improved working conditions had had on their lives.

However, the project had some unexpected adverse effects on neighboring populations that were economically linked to Place Lalla Ydouna, but were not beneficiaries of the RAP. When the project was conceived, the risk of adverse effects on this group was considered to be minimal because of the short duration of these effects. However, the delays sharply exacerbated the adverse impacts on many businesses.

 

 

 

Evaluator            

Consortium TRANSTEC & Attitude Conseil

Methodology   

Pre post

Evaluation Period

From April 15th to November 15th 2013

Artisan Promotion Campaigns

Outputs (other than those already documented in project “Monitoring Results”, above)

  • 16 events organized for  independent artisans and SMEs: campaigns targeting national and international tourists
  • 200 artisans participated in at least one trade fair (national or international) or business trip
  • 30 events organized for SMEs
  • 130 SMEs participated in at least one trade fair (national or international) or business trip
  • 30 SMEs were declared eligible to begin branding their products with the national label for promotion of handicrafts, yet no enterprise was able to append the label to their products as of the Compact end date

Intermediate outcomes

While the below results are the perceptions of beneficiaries measured in the evaluator’s “Survey of Artisan Beneficiaries” and aren’t meant as attributable impacts, the following results are helpful indications of beneficiaries’ perceptions of outcomes and impacts:

  • 7% of the participants saw no improvement in their sales, 54% increased revenue by 10% or less, 29% increased them by15 to 40% and only 10% increased them by more than 50%.
  • 9.5% of participants in international trade fairs and marketing missions doubled their export sales. 78.5% of the participants had an average increase of 32% in export sales
  • 7% of the participants in national trade fairs had no increase in their sales; 28% of participants increased sales by 10%; 12% increased sales by 20%; and 5% doubled their sales.
  • 33% of beneficiaries of tourist circuits declare that their sales increased by less than 10%, 16% declare that their sales increased by 10% or more;
  • 95% of the participants in promotional campaigns said that the project enabled them to acquire a better negotiating position and to have more confidence in the future

Ultimate Impact

No impact is measurable at this stage

4

Support to Artisan Production

Intermediate Outcomes

  • 29% of the potters said that after the training they came up with new products on their own
  • 30% of the potters reported employing new marketing practices after attending trainings
  • 27% of the potters reported that they modified the management of their enterprise
  • 36% of the potters reported that they modified their work routines in order to better guarantee their safety and that of their employees
  • 45% of the participants reported an increase in sales of less than or equal to 10%; and 8% said their sales increased by 10% to 25%.
  • No increase in sales was noted for exports

Ultimate Impact

  • On average, potters who benefitted from training perceived that their income had increased by 9.6%.
  • At the time of evaluation, it was too early to assess the impact of acquiring gas kilns on income since only two gas kilns were operational at the time of evaluation.

Fez Medina Activity

Intermediate Outcomes

  • The work conditions were improved for copperware artisans resettled from Place Lalla Ydouna to Ain Nokbi

Ultimate Impact

  • No impact was detected at the end of the Compact
 

 

 

Lessons Learned

The qualitative findings of this performance evaluation for the Artisan Production, Artisan Promotion, and Fez Medina components of the AFM project yielded some lessons learned that should be taken into consideration during the design of future projects of this nature. In particular:

 

Artisan Promotion

  • Replicability of promotion campaigns will be difficult in the future due to the gratuity trap. The free participation by beneficiaries in promotional campaigns is very uncommon in these types of projects, for which, in general, stronger participation is required of enterprises;

 

Moreover, this quasi-free participation by artisans and SMEs in commercial events could over time induce resistance among artisans to participate in future commercial events that do not offer the same conditions. Therefore, artisans should be financially involved in financing promotional activities

 

  • In addition to the promotion activities, the project should assist SMEs and cooperatives to upgrade their capacities in order to be able to develop a product range that is truly adaptable to the target markets in terms of design, price and specific promotion tools.

 

Artisan Production

 

  • The project provided classroom training for potters. Even though the potters largely praised the way the training was conducted, the sessions were still considered too theoretical. The diversity of the participants, including some illiterate artisans, created some challenges for training providers. The participation of some artisans at multiple sessions of the same module demonstrates the difficulty in assimilating the lessons learned but also demonstrates artisans’ strong interest in these teachings. The pedagogical approach should have been tailored to the beneficiaries’ capacities and should include more practical training sessions. The project should have provided technical assistance to potters to help them put into practice what was learned.

 

  • Even though potters were convinced of the relevance of replacing traditional kilns with less polluting gas kilns, this activity faced serious reluctance for many reasons: (i) the 20% contribution of potters to the imported gas kiln was beyond the financial capacity of potters; (ii) potters were reluctant to contract loans from banks for various reasons, among which the reimbursement conditions and (iii) Banks were not willing to finance the majority of potters operating informal, unregistered businesses; (iii) the maintenance cost of imported kilns were perceived as a long-term burden. The project design did not take into account the potential of the local gas kiln manufacturers and favored imported kilns from the start. The project should have considered developing a local kiln based on local usage and adapted to the needs and capacities of Moroccan potters.

 

Fez Medina

  • Fez Medina was a very challenging activity because of the very peculiar space of intervention (e.g. buildings dating back to 14th and 15th centuries, dense population, etc.), a complex resettlement plan, and other social and environmental issues that needed to be addressed before works begin, in addition to a time constraint of a 5-year Compact. Such a complex activity should be implemented through an organizational scheme that clearly defines the roles and responsibilities of the relevant parties (APP and Project Management Unit). The project team should have the managerial capacities to tackle the challenges. The decision process should be flexible and very reactive to take appropriate measure to recruit skilled and capable human resources. The PMU members, especially ADER staff, should not be appointed by the implementing entity, but recruited based on performance. APP and MCC should ensure adequate leverage to be able to restructure the project team in case of a performance deficit.

 

  • ADER, the implementing entity, had good experience in rehabilitating old buildings, such as fondouks, but needed more assistance in managing complex social issues such as resettlement and the creation of a business plan design for rehabilitated sites that would ensure sustainability. Such assistance should be mobilized in a timely manner.
Footnotes
  • 1. Evaluation report (page 61) cites a 5% loss rate as the expected rate for those potters using gas kilns, though the timeline of the evaluation (with few kilns at full operation) did not allow for a statistically representative measurement of loss rate.
  • 2. Ibid.
  • 3. As of February 2015, the Ministry of Handicrafts continues to pursue the goal of 100 SMEs appending the label to their products.
  • 4. Due to the limited number of gas kilns purchased through project support, exacerbated by the delay in this activity, the evaluation was unable to assess the impact of gas kilns on incomes or revenues.