EMMANUEL DEVELOPMENT ASSOCIATION

IMPACT ASSESSMENT REPORT

THE IMPACT OF EDA PROGRAMMING ON THE LIVES OF WOMEN IN AKAKI-KALITY, ETHIOPIA

Universit degli studi Roma Tre (Rome, Italy) October 2010

TABLE OF CONTENTS 1. EMMANUEL DEVELOPMENT ASSOCIATION BACKGROUND 2. AKAKI-KALITY PROGRAMME 3. ALTERNATIVE BASIC EDUCATION 4. MONITORING AND EVALUATION 5. EMMANUEL DEVELOPMENT ASSOCIATION SUCCESSES 6. RESEARCH METHOD 7. IMPACT ASSESSMENT RESULTS 8. A NOTE ON FOOD SECURITY 9. OPPORTUNITIES 10. EDA SUCCESS STORY: CHILD SPONSORSHIP 3 4 5 6 7 8 10 16 18 19

1. Emmanuel Development Association Background The Emmanuel Development Association Ethiopia (EDA) is an indigenous, nongovernmental organization (NGO), established in 1996 with the aim to improve the socio-economic situation of children and youth among Ethiopias most vulnerable and disadvantaged communities.1 EDA has grown from its humble beginning with one staff member and a 10,000 birr ($1,200 USD) personal contribution of its founder, Ato Tessema Bekele, to a thriving locally-based NGO with a qualified and diverse staff of 191 working at head and project offices, and a total revenue of 26,368,987.17 birr ($1, 757,872 USD) and expenditure of 17,271,265 birr ($1,151,417 USD).2 The most significant effect of this growth, however, is not the number EDA staff or the amount of revenue and expenditure, but rather, the impact over the past fourteen years that EDA programs have had on the lives and wellbeing of over 80,000 people directly and 400,000 indirectly in disadvantaged communities throughout Ethiopia. It is attempting to understand this impact that is the focus of this study. EDAs guiding vision is to bring positive change and protection to Ethiopian children through empowering their families and communities, and throughout its nine project site areas, EDA works with poor communities specifically to enhance their wellbeing, through targeted interventions.3 From its overall goal to improve the wellbeing of disadvantaged children and community groups of urban and rural dwellers in a sustainable way through an integrated community based approach and active stakeholders participation, EDA has four strategic objectives guiding its programming over the next five years: (1) to increase early childhood development, Child Protection Program, and access to basic services for disadvantaged children and community groups, (2) to improve household food security, (3) to increase income opportunities of youth and community groups, (4) to increase organizational capabilities of community structures as well as EDA itself.4 These program objectives are all interlinked, and achieving one necessarily contributes to reinforce the achievement of the others. All four of these strategic objectives lead towards the central overall goal of improving the wellbeing of children. Thus, as per EDAs own stated goals and objectives, it is necessary to assess and evaluate the impact that EDA has had on the communities in which it works, not only on the quantifiable evidence of beneficiaries of and stakeholders within EDA programs, but also to attempt to evaluate the impacts that EDA programs have made to the less tangible but equally important qualitative aspect of peoples and the communities wellbeings. It is that task which is both the objective as well as the aspiration of this impact assessment study.

1 2

EDA, (2009), Annual Report, introduction. EDA, (2009), Annual Report, p. 23. 3 EDA, (2010), Five-Year Strategic Plan: 2010-2014, p. 11. 4 EDA, (2010), Five-Year Strategic Plan: 2010-2014, p. 20.

2. Akaki-Kality Programme Akaki-Kality is a sub-city of the bustling urban Ethiopian capital, Addis Ababa. It is located approximately 25kms southwest of Addis in the industrial outskirts of the capital, where approximately 60% of all of the industries of Addis are found. AkakiKality covers an area of approximately 6,143.4 hectares, and has a population of approximately 220,740 (approximately 114,095 female and 106,645 male).5 Poor infrastructures such as schools, health institutions along with a high prevalence of poverty-induced socio-economic problems such as HIV/AIDS, prostitution, begging are common throughout Akaki-Kalitys 8 kebeles (neighbourhoods), and Akaki-Kality is characterized by both deeply impoverished urban slums, as well as extensive rural farming areas. Further, Akaki-Kality has an HIV prevalence rate of 9.1%, compared to the approximate national prevalence rate of 3.5%, which estimates suggest range from 0.7% in rural areas to 5.5% in urban centres. As a result of this, the ratio of HIV/AIDS orphans is also higher in Akaki-Kality than in other communities throughout Ethiopia, and these children represent a population particularly vulnerable to extreme poverty, food insecurity, and a lack of a social safety net. It was therefore within this context that EDA began working in Akaki-Kality fourteen years ago. The Akaki-Kality programs have expanded to include programs and projects in Alternative Basic Education, improving Formal (government) Education, Youth Training and Livelihoods, Community Health and HIV/AIDS, Early Childhood Development (Child Sponsorship), Water Sanitation and Hygiene (WASH) initiatives, Womens Empowerment and Livelihoods, Capacity-building of Community-Based Organizations, Urban Agriculture (Food Security), and Smallholders Sustainable Development. All programs maintain the target groups of vulnerable women, youth and children, and function within the framework of promoting self-initiated community development initiatives and enabling self-sufficiency in order to achieve sustainability in development programs.

FSCE Ethiopia 2006.

3. Alternative Basic Education The Alternative Basic Education (ABE) program was EDAs first major project undertaken, and Akaki-Kality was the first project location. In line with the second United Nations Millennium Development Goal, to achieve Education for All by 2015, the ABE program focuses on providing quality basic primary school education for marginalized and out of school children. The class schedules of ABE schools are explicitly designed to work in harmony with the community in regards to school locations and class scheduling, particularly allowing for flexible schedules to meet the needs of children who must participate in the income-generating or asset-guarding [such as watching the goats and sheep] activities of their families. The ABE school system acknowledges the reality that many of these children must work during normal school hours, and thus attempts to embrace and work within this reality by offering flexible scheduling, cost-free6. The pedagogical methodology of ABE schools is participatory, and students are encouraged to ask questions and to take ownership of their learning. This methodology is especially empowering for Ethiopian girls, who might otherwise not be encouraged to develop their own sense of self, voice and confidence. As Kalkidan Abu, a 10-year-old girl attending ABE school in Akaki-Kality, states, now I feel more confident, at the ABE schools I am less scared to respond in class.7 It is this qualitative sense of self and confidence that will have a longlasting impact on the community of Akaki-Kality, as more girls are able to participate in schooling and gain confidence in themselves. The flexible scheduling and participatory methodology of ABE schools prepares students to be graduated after three years of ABE schooling into the fifth year of the public schooling system. Over the fourteen years of its existence and programming in this area, EDA has supported more than 33,000 students (60% girl-students) through the ABE program and graduated them on to secondary school.8 Currently, EDA has 11 ABE schools and 40 facilitators. 8 of these schools are located in Akaki-Kality.

Although all public schools in Ethiopia are tuition free, the cost of purchasing notebooks, textbooks, writing utensils and uniforms, let alone the loss of the labour-hours children in poor families provide towards their families livelihoods, makes attending school too costly and therefore unaccessible for too many children throughout Ethiopia. 7 EDA, (2010), Emmanuel Development Association: Success Stories, p. 5 8 EDA, (2009), Annual Report.

4. Monitoring and Evaluation EDA believes that development is a process, which begins with identifying a community problem and then doing something about it. Monitoring and Evaluation is one of the management tools that can help to improve this process. M&E focuses on: process of EDAs operation; performance of those involved in the development process; progress/results achieved due to EDAs program/projects; resources used; overall impact of EDAs program on the lives of children and poor communities. On top of the above, qualitative social aspects including degree of community participation, question of equity and distribution of benefits, and gender differentials and changing attitudes of communities will be monitored during the strategic planning period. It is within this process of evaluation of the qualitative social aspects and impacts of EDA programs that this Participatory Impact Assessment is based. The goal of this report is to understand how EDA programmes have IMPACTED the lives and wellbeing of participants (and particularly, the target groups of vulnerable women and children) EDA staff at head office and project level in cooperation with communities and other stakeholders will continuously engage in monitoring of the progress of program interventions, and further oversee periodic review of each activity, at every level of implementation.

5. Emmanuel Development Association Successes: Over the fourteen years of its existence as an organization, EDA has contributed to numerous tangible impacts. Some of these include: Direct support for 80,000 people and indirect support for 400,000 40 first cycle primary schools currently operating Over 33,000 marginalized children have graduated primary school and joined secondary schools (60% girls) (*from attendance records) 1 medium health clinic serving over 25,000 people (2008 Report) 8 junior secondary schools constructed to build the capacity of formal schools 2,387 youth trained through Business Skills workshops 8,250 orphans and vulnerable children (OVC) supported in health, education and nutrition Multipurpose youth training centre established, operational and is a learning centre and demonstration centre for 8,923 community members 877 smallholder farmers supported with agricultural imputs Constructed one Biogas plant for sanitation improvement and alternative energy sources in the community (Akaki-Kality)

Organizational Awards: World Bank Award for local initiative (2003) UNESCO Award for MDG in Quality Education best practice (2005) USAID Certificate of Achievement (2005) Millennium Award from Regional Governments in Ethiopia (2007/08) o Tarmaber woreda administration (region): Among many NGOs working in the district, EDA is nominated for the award not only by goodwill but for its best performance and tangible achievements, we thank EDA again and again. o This Impact Assessment will examine and highlight not only the tangible achievements of EDA in Akaki-Kality, so widely recognized by the international donor community and the National and Regional governments, but also investigate some of the less tangible but equally valuable impacts such as the increasing wellbeing, confidence and capabilities of individuals particularly women and children involved in EDA programming. National Award for Addressing Vulnerable Communities (OVCS) National Award for Best Practice from H.E. President Girma, FDRE (March 2010)

6. Research Method The main component of this impact assessment was a qualitative evaluation. The qualitative component of the methodology was designed to employ a combination of participatory tools including focus group discussions which sought depictions of types EDA programming, level of involvement of individuals and households, and the effects participation in programmes had on the participants conception of their own well-being in terms of their changed levels of functioning and the (potentially) resulting expansion of their self-identified capabilities. The process of this impact assessment was designed within the overall framework of Alkires 2002 Valuing Freedoms impact assessment, with adapted tools from standard impact assessments as suggested by Roche (Oxfam) and Bird (ODI), and, during the focus group discussions and workshops, using some participatory approaches and tools as defined by SEAGA (Socio-Economic And Gender Analysis tools from FAO). In an attempt to understand the impacts of EDA programming on the individual well-being of women and children and their own self-identified changes in functioning and capabilities, it seemed necessary to gain individual participant responses, information and views, as opposed to simply proxy indicators of increased household incomes, asset and educational levels. We hope that this small initial and exploratory study will provide a baseline of qualitative impacts for future programming and evaluation tools, and will demonstrate that EDA has had a significant impact on the well-being of beneficiaries participating in their programming. The voices of women and children were chosen to be the subjects of this project, as marginalised women and children are the target groups of EDA programming.

Ethical Considerations To ensure accessibility to all participants in EDA programmes, we decided to pursue multiple avenues of data collection, including focus group discussions (short, concise, over the lunch time period), semi-structured interviews questionnaires (also short, administered during focus group discussions or at the school or workplace), as well as holding two workshops: one, for women on the overall theme of leadership and capabilities and one for children and youth on child rights and capabilities. as well as ensuring that all participants were explained the purpose of the survey (to understand the impacts of EDA programming in order to improve and expand future programming), and explicitly stating that there was no obligation to participate, nor was there any threat to the individuals future participation in any EDA program through any of their anonymous responses.

Summary of Sample Subjects and Techniques

No. Subjects / Samples 1. 2.

Sample Sampling Size Techniques Random Sampling Random Sampling

Methods of Data Collection Semi-structured Interview Questionnaire Semi-Structured Interview Questionnaire Semi-structured Interview Semi-Structured Interview Focus Group Discussion Focus Group Discussion Review of EDA Documents; other NGOs; UN organizations

Quantitative Data Sources Youth Participants / 19 Beneficiaries Adult Participants / Beneficiaries Qualitative Data Sources Key Informant Interview (Govt) Key Informant Interview (ED) Youth Participants / Beneficiaries Adult Participants / Beneficiaries Secondary Sources Review 40

3. 4. 5. 6. 7.

1 1 1 (group) 1 (group)

Purposive sampling (one interview) Purposive sampling (one interview)

7. Impact Assessment Results: Figure 1: Respondent Ages (Adult)

The 40 respondents ages ranged from 27 to 70, with an average age (mean) of 39.9 years old. Although basic, this sort of demographic information helps to contextualise how the respondents might value the impacts of various qualitative aspects of their lives, and also how they might define or assess their change in well-being. Further, this information helps EDA to determine which variety of participatory activities, projects or programmes they might pursue, according to the need in the community, as there are changing needs and values at different stages of life.

10

Figure 2: Respondent Level of Education

This graph demonstrates the respondents current level of education. assigned to each level of education is valued as follows: 1 = No Formal Education 2 = Some Primary School 3 = Primary School Completed 4 = Some Secondary School 9 = Other

The number

This education graph demonstrates that the majority of respondents in this project are representative of one of the main target groups (marginalised women) of EDA. This information is useful in determining the type of future programmes offered by EDA, and also as a potential baseline for more standard impact assessment evaluations on tangible changing levels of education. After describing the qualitative aspects impact graphs, we will examine a comparison between education and overall qualitative impact which is a much more illustrative comparison.

11

Qualitative Aspect Impact Rankings Development projects have not only benefits as seen through cost-benefit analyses, but also a number of non-quantifiable effects, including the acquisition of useful knowledge, the cultivation of friendships amongst each other etc. Sabina Alkire further suggests that the ethical reason for studying these components is central: just as health and knowledge may be valued as ends as well as means to increasing productivity, so the abilities to enjoy other functionings (pertaining perhaps to participation, religion, art, human relationships) can be considered intrinsically valuable whether or not they are also instrumentally valuable.9 Alkires study in India was the direct inspiration for this project in Ethiopia; in particular, her methodology, which included a ranking evaluation on specific aspects of an individuals life was the most dominant inspiration of this short impact assessment. It is the methodology of this research, which this dissertation report now turns. The following information was gathered during the semi-structured interview when each participant was asked to give a ranking of 1 (no impact) to 4 (large impact) of the impact of participating in a particular EDA programme on the following qualitative aspects of their lives: Overall Life Personal Health Knowledge Friendships Family Role in the Community Happiness Confidence Although the question was supposed to elicit a simple single number answer, during the process of the interview each respondent started speaking in detail about his or her experienced change in functioning of each aspect. It very quickly became evident that it was an important as part of the participatory process (potentially contributing to expanding capabilities even through the impact assessment process itself) that we acknowledge, listen, and record each of the participants comments. During the data analysis, a value of 5 was given to any aspect in which a respondent had valued 4 and had spoken in great length or detail about (usually only 1 or 2 aspects per respondent). This resulted in a maximum impact level of 40 (8 aspects, each with a possible maximum impact value of 5). The following chart summarizes each respondents maximum impact level.

Alkire, 2002, p. 204.

12

Figure 3: Qualitative Aspect Rankings

Figure 4: Respondent Total Impact Value

13

As is evident on the chart above, the total impact per respondent ranges from a level of 14 to a level of 35 (out of 40). What is particularly interesting in this context is the comparison between the average (mean) total impact per respondent, and their personal level of education. The following pivot table demonstrates this connection: Table 1: Respondent Level of Education and (Avg.) Total Impact Average of Total: Level of Education 1 2 3 4 9 Grand Total Total 30 28 28 25 22 27

What we observe on the table above is that there is a correlation between a lower level of education and a higher total impact value. Stated another way, this table directly suggests that participants who have a lower level of education, ranked their participation in an EDA programme as having a higher impact on the various qualitative aspects of their lives than participants who had a higher level of education.

This information suggests that EDA has more impact on the lives of those who are their direct target group: marginalised and vulnerable (women).
Further, this information provides a key benchmark in terms of an ex-post, improving impact assessment, as it suggests that what we can learn from this sample, is that

EDA is doing excellent work targeting their programmes and should continue in this direction.

14

Moving away from a respondent-centric analysis of impacts towards a qualitative aspect analysis, the following graph shows each aspect with its corresponding total value (from all respondents):

Figure 5: Qualitative Aspect Total Impacts

What we observe here is that the most impacted aspect of the respondents lives were their Role in the Community (with 152 points), and their Friendships (with 151 points). These total values are further corroborated by the mode of each aspect, in which Role in the Community received an average ranking of 5, and Friendships an average ranking of 4. Further, the comments that many of the women made during the semi-structured interview process were particularly illustrative. As mentioned above, almost all of the respondents (93% women) wanted to discuss how these various qualitative aspects of their lives had been affected by participation in EDA programs (such as Microcredit group, Livelihoods support, Child sponsorship, Alternative Basic Education etc.), rather than simply assign each aspect a number value.

15

8. A Note on Food Security Food Security exists when all people at all times have physical and economic [and social] access to sufficient, safe and nutritious food to meet their dietary needs and food preferences for an active and healthy life. (World Food Summit, Rome, 1996). The four main dimensions of food security are: Physical Availability of food: Food availability addresses the supply side of food security and is determined by the level of food production, stock levels and net trade. Economic and physical Access to food: An adequate supply of food at the national or international level does not in itself guarantee household level food security. Concerns about insufficient food access have resulted in a greater policy focus on incomes, expenditure, markets and prices in achieving food security objectives. Food Utilization: Utilization is commonly understood as the way the body makes the most of various nutrients in the food. Sufficient energy and nutrient intake by individuals is the result of good care and feeding practices, food preparation, diversity of the diet and intra-household distribution of food. Combined with good biological utilization of food consumed, this determines the nutritional status of individuals.

Stability of the other three dimensions over time: Even if your food intake is adequate today, you are still considered to be food insecure if you have inadequate access to food on a periodic basis, risking a deterioration of your nutritional status. Adverse weather conditions, political instability, or economic factors (unemployment, rising food prices) may have an impact on your food security status. Source: FAO, Food Security Information for Action, Practical Guides To put into socio-economic context, the average amount earned each month (in the womens households) was approximately 200 399 birr per month (or 10 to 20 Euros, by current exchange rates). The average amount spent on food each month was also between 200 and 399 birr (10 to 20 Euros) per month. In terms of the womens reactions to the question, most of the women (32 out of 40 respondents) stated something to the equivalent that however much we earn [in a given month] our household spends on food.10 The more a household earned in a given month, they more they spent on food. Further, through our discussions with the participants in EDA programs (mostly rural women who earned their livings through either selling injera and participating in a co-

16

operative that EDA started), food issues, and being able to provide enough nutritious food for their families, was a constant concern, and participating in EDA programming on womens empowerment projects had a very large impact on the quality of their lives. In this regard, EDA programming is contributing to the access dimension of food security.

17

9. Opportunities Based on the findings and analysis of this initial study of a capability approach to impact assessment, there are many potentialities for further research and opportunities in this area. Some of the main opportunities are: Identifying a minimally accepted Impact Value (both individually as well as per aspect) in order to pursue future capability approach impact assessments and / or monitoring and evaluation of EDA programming. Including space for participant definitions of well-being (and how that well-being is impacted by any particular programming) during the process of an impact assessment Pursuing further research on the connection between Level of Education and Individual Total Impact Triangulating more research information by including information on the counterfactual and / or an outside control group (one within a similar socioeconomic reality as the organization participants but not organization participants)

EDA should continue their successes with targeting of marginalised and vulnerable women and children, as both their qualitative and quantitative impacts were significantly higher

18

10. EDA Child Sponsorship Program Success Story: Lohme Medeksa (Mother) Quiet and reserved yet beaming with life, Lohme Medeksa is an inspiration. Living with two of her four children in a one room thatched roof hut, she proudly shows her two prized possessions: a cow and its calf both animals also living with Lohme and her two daughters inside their hut. Her youngest child, Abiti Medeksa, 12 years old, is part of the Child Sponsorship program with Emmanuel Development Association. As a participant in this program Abiti receives educational materials, clothes, hygiene items and school fees covered. Through Child Sponsorship, Abitis family also has access to health care and medicines through the local EDA clinic. Perhaps the biggest change in the lives of Abiti and Lohme and their family, however, has been the cow Lohme received through a lottery of Child Sponsorship participants. Before receiving the cow, Lohme was a daily labourer, and she describes life as being very difficult. Her daily wages were barely enough to feed her children. After her husband passed, two of Lohmes children went to live with other relatives because she could no longer afford to support them. After receiving the cow through a Child Sponsorship lottery programme, Lohme says her life changed: She used the money she made from selling the milk not for building a bigger house or purchasing household goods, but instead, saved the money to ensure the health of her cow in order for her cow to produce a health calf. Now, Lohme hopes to transfer a legacy for her children. She no longer worries about her income but rather about the future, and the future for her children. Her goal is to continue growing her calf and save enough income from selling milk to have all of her children living together with her again. Lohmes far-sighted determination to keep her cow and not sell it or its calf will definitely help her reach her goal. Meeting Lohme Medeksa and listening to her story was an incredibly humbling and inspiring experience. Her wisdom, inner strength and capacity to reach her goals are all shining, and she praises God, EDA and CCFC for having a role in this incredible change in her life.

19