Professional Documents
Culture Documents
Contents
1. Introduction 2. Summary of Key Themes 3. How Root Capital & Our Clients Create Impact Root Capitals Mission, Vision & Strategy The Challenge: Heterogeneous Clients & Impacts Root Capitals Impact on Clients Our Clients Impacts on Farmers Our Clients Impact on the Environment Understanding the Global Positioning System (GPS) Approach to Social & Environmental Impact 4. Building the GPS for Impact Evolution of Metrics & Evaluation at Root Capital Impact Terminology: Outputs, Outcomes & Impacts Expanded Social & Environmental Scorecards Social & Environmental Scorecard Contents Lessons Learned: 2011 Portfolio Analysis Growing Businesses, Increasing Impact 5. Evaluating Impact From Outcomes to Impact Piloting Evaluation Methods Impact Case Study: COOPCAB Impact Ethnography: Fruiteq Learning to Evaluate Impact on the Ground Mobile-Enabled Evaluation: Acopio Mvil 6. Looking Ahead Client-Centric Impact Evaluation Rolling Out Our Approach 1 2 4 4 6 6 7 9 10 11 11 13 14 15 16 18 19 19 20 21 23 26 27 28 28 28
1. Introduction
Root Capitals mission is to grow rural prosperity by investing in agricultural businesses that build sustainable livelihoods in Africa and Latin America. This Roadmap lays out our evolving approach to measuring our impactour success in fulfilling our mission.
For twelve years, William (Willy) Okumu lived in a refugee camp while the Lords Resistance Army ravaged his community in northern Uganda. After the fighting stopped, Okumu returned to his land to rebuild. Today, he is the lead farmer of the Panyabono Organic Farmers Association, a group of 300 members who joined together to sell traceable, organic-certified cotton to Gulu Agricultural Development Company (GADC), a Root Capital client. With proceeds from last years cotton harvest, Okumu bought his second son an ox plow to prepare his own land and to hire out to neighbors. A few years back, virtually every school in the district was empty, he said, but now most of the farmers in his group are able to afford school fees to send at least some of their children back to school. Conditions are still hard, but access to export markets through GADC is helping Willy and his family to rebuild their livelihoods. Stories like Willy Okumus are what attract Root Capitals stakeholdersfrom employees to donors, investors, value chain partners, and even clients. And yet, such stories raise as many questions as they answer. How many of the 600,000+ farmers that Root Capital has reached over the past decade have similar stories? How do we know whether Root Capitals loan made a difference for them? And, most importantly, how can Root Capital, as an impact-focused agricultural lender, refine our business practices to maximize positive impacts while building a financially sustainable operation? This document represents our roadmap as we seek answers to these questions, charting our path to-date and our future direction. Specifically, it lays out the intended impacts that guide our strategy; describes challenges to measuring these impacts and our approach to overcoming these challenges; and presents some initial findings. Having piloted several approaches to impact evaluation over the past year, we now seek to replicate these approaches with other clients even as we continue to experiment with new approaches. This document also comes at an inflection point for Root Capital as a whole. In the past year, we completed our Scaling Impact Plan, our strategic plan for 2012-2016. In addition to setting ambitious goals for Root Capitals growth, the Scaling Impact Plan outlines our strategy for generating systemic change that goes far beyond the direct impact of our lending and financial training. We aspire to catalyze a financial market to serve agricultural businesses that connect small-scale farmers to markets across the developing world. Most importantly, we aspire to lead that market towards impact by maintaining an unwavering focus on underserved businesses and small-scale farmers, and by setting the standard for sustainable social and environmental practices over the long term. To do so, we seek to demonstrate that agricultural businesses drive long-term rural prosperity and promote environmentally sustainable practices, and to track whether we and our peers are continuing to reach the most under-served businesses and farmers as the market evolves. This document consolidates our thinking and activities todate and serves as a foundation for our impact agenda going forward. While the document can be read straight through, we have structured it so that readers can focus on the sections in which they are most interested. Section 2 provides a summary of our impact assessment priorities, philosophy, and approach. Section 3 provides an introduction to Root Capital and our clients, articulates our intended impacts that we test through evaluation, and describes some of the common challenges of evaluating impact. Section 4 traces the evolution of our approach to impact and metrics and provides initial data and analysis on our performance in 2011. Section 5 describes the evaluations with specific clients that we have piloted in the past year, and Section 6 looks ahead to our plans to replicate these pilots.
1. Introduction
A case study of the livelihood impacts of COOPCAB, a coffee cooperative in Haiti, conducted by a Root Capital financial management trainer during his field visit (see COOPCAB, page 21); and Several participatory consulting engagements with impact research embedded into the project design (being launched in summer 2012).
Our Vision
A thriving financial market serving agricultural businesses that generate long-term social, economic, and environmental sustainability for smallscale farmers and their communities around the world.
Our Strategy
We pursue our mission and vision through a three pronged strategy: Finance: Expand access to credit by lending to earlystage agricultural businesses whose financial needs are not being met and then accompany their growth over time. A majority of our loans are short-term trade credits for purchasing a harvest or for processing and delivering product to market; these loans are typically renewed annually. A smaller but growing portion of our portfolio consists of longer-term loans to finance the equipment and infrastructure needs of our clients.1 Advise: Deliver financial training to build management capacity so that agricultural businesses can realize their potential to grow and improve livelihoods. Catalyze: Amplify the demonstration effect of our Finance and Advise strategies among our peer social lenders, commercial financial institutions, and value chain actors to shape a new financial market serving agricultural businesses and the small-scale farmers they reach.
1 This report is focused on the impact of our Finance strategy, and in particular on our Sustainable Trade Fund, which houses loans to enterprises that export natural products such as coffee, cocoa, and nuts. Our newer Frontier Portfolios include loans in markets we are just beginning to enter, such as staple crops for domestic consumption. As the Frontier Portfolios scale, we will develop metrics and evaluations for them. Performance measurement is embedded within our Advise and Catalyze programs and we will report on these in future reports.
Households
FINANCE IMPACT
ADVISE
Businesses
Regional Growth
Se
r vi
ng
500
m il li o n
ar m s m a ll - s c a l e f
er s
Anatrans, a private enterprise in Burkina Faso that, in addition to purchasing cashews at premium Fair Trade prices from 4,000 small-scale farmers, recently built a factory that employs 800 peopleprimarily womento process these cashews for export. RAMSA, a private company that has created 330 high-quality jobs providing reforestation and timber management services in a region of the Peruvian Andes where legal and illegal logging has deforested over 30 percent of the original rainforest.
Nearly all of our clients support higher and more stable producer incomes. Beyond this, Root Capital and our clients have multiple types of impact that are difficult to compare and cannot be reduced to a single metric. This does not mean, however, that we should resign ourselves to drifting in a sea of relativism, in which all impacts are equal and any impact is good enough. We simply need tools to help us sort through the complexity.
Our impact is greatest when we support clients that are growing rural prosperity and could not do so without access to finance and / or training from Root Capital. Social financiers refer to this concept as additionality. Additionality means that we are providing products and services that our clients would not otherwise have rather than replacing credit and training others are currently providing.
If our nascent financial market is to meet the full demand for credit of agricultural businesses, we and our peers must continuously expand to underserved geographies and sectors, even as we scale up our current lending. Only by doing so can we avoid the current situation in microfinance, in which microfinance institutions increasingly compete with each other to serve 200 million people, while over two billion people living on less than $2/day continue to lack access to microfinance. For this reason, Root Capital seeks to lead the way in underserved markets, reaching clients that remain too small for even other social lenders and expanding in geographies where other lenders are least active. These are the clients trapped in the missing middle of rural agricultural finance described in Section 3. To further promote the importance of additionality in social finance, we will release an Issue Brief on the topic in late 2012.
3. Increasing Stability of Producer Incomes by offering producers a reliable route to market throughout volatile market conditions; by offering forward contracts or guaranteeing to purchase a specified volume; or by offering credit, savings, or in-kind loans that help producers to smooth their consumption over the year and to make long-term plans and investments. In return, the value that farmers provide to agricultural SGBs and to upstream exporters, processors, retailers, and consumers is stable supply of high-quality and sustainably-produced agricultural products. Farmers often have a choice of buyers of their crop, including one or more agricultural businesses and a variety of local middlemen and traders, some of whom use exploitative practices (for instance, making cash advances early in the season to secure crop at a fraction of its value at harvest). Farmers that perceive a high degree of value from the enterprise, and that trust the enterprise to balance its interests with farmers interests over the short- and long-term, sell much or all of their harvest to that enterprise. They thus enable the enterprise to successfully fulfill its own export contracts. Farmers that do not perceive value in the enterprise, or that do not trust the enterprise, are more likely to side-sell their harvest to other enterprises or local middlemen. Thus, key to understanding Root Capitals impact is understanding the value provided by agricultural SGBs to farmers, in the form of pricing, productivity, and stability, and understanding the value provided by farmers to agricultural SGBs in the form of quantity of product (e.g., absence of side-selling) and quality of product.
2 An example along these lines would be an organization that provides vaccinations to infants where the measures of success are clear the number of vaccinations relative to the costs and comparability with other entities providing similar services is straightforward.
Junn region
PERU
Root Capital Client Location
Lima
Women in Agriculture
Half of all farmers in developing countries and 75 percent in subSaharan Africa are women. Women carry out much of the work producing cash crops like coffee and cocoa and play an even greater role growing staple crops in the most food insecure regions of the world. Yet women farmers often lack access to credit. For example, a 1990 study in five African countries found that even though women are more likely than men to repay loans, they receive less than 10 percent of the credit available to small farmers.4 One of the reasons why women have difficulty securing credit is their lack of assets: only 2 percent of the titled land in the world belongs to women.5 However, research suggests that women spend a greater portion of their income on family expenses like school fees and nutritious food than men do. Supporting the livelihoods of women may therefore be a particularly effective way to grow rural prosperity. Root Capital launched our Women in Agriculture Initiative in early 2012 to promote businesses that, by generating economic opportunities for women, might have disproportionately large social impacts. We began by reviewing our existing portfolio. We assess each enterprise based on the following criteria: the percentage of its producers and employees who are female; whether women own or lead the businesses; and whether the business offers community programs that benefit women and girls (e.g., an education program for girls, womens health services). In 2011, we reached approximately 160,000 women in total through our lending, and we identified 60 clients, representing 30 percent of our portfolio, who had particularly gender inclusive policies or structures. In 2016, we plan to finance 200 such businesses and reach 690,000 women through our clients.6 To better understand the challenges women face as farmers, laborers, care-givers, and home-makers, we will include gender in our client impact evaluations, and conduct an ethnographic study focused specifically on gender.
3 4 5 6
Instituto Nacional de Estadstica e Informtica, http://www.inei.gob.pe/. Food and Agriculture Organization. Gender and Food Security. Rome: Food and Agriculture Organization. Steinzor, Nadia. Womens Property and Inheritance Rights: Improving Lives in a Changing Times. International Center for Research on Women: Washington DC, 2003. In 2011, our clients purchased crops from 60,000 women and sold drought-resistant seeds to an estimated 100,000 female farmers. In 2016, we project that our clients will purchase crops from 200,000 women and sell agricultural inputs to 490,000 women.
2.
Climate-smart agriculture is an approach to growing rural prosperity with three components: (i) increasing productivity and incomes, (ii) enhancing the resilience of livelihoods and ecosystems to climate change, and (iii), reducing and removing greenhouse gas emissions. Source: Food and Agriculture Organization, Climate-Smart Agriculture for Development: Managing Ecosystems for Sustainable Livelihoods. 2011, p. 3. Sustainable intensification is defined as producing more output from the same area of land while reducing the negative environmental impacts and at the same time increasing contributions to natural capital and the flow of environmental services. Source: Pretty, Toulmin, and Williams, Sustainable Intensification in African Agriculture, International Journal of Agricultural Sustainability, Volume 9, Issue 1, 2011.
Understanding the Global Positioning System (GPS) Approach to Social & Environmental Impact
Our metrics and evaluation activities focus on the following questions relating to the impact themes described above: 1. What value does Root Capital provide to our clients? 2. What value do Root Capitals clients provide to the small-scale farmers with whom they work, and what value do those farmers provide to Root Capitals clients? 3. What impacts do our clients have on rural communities and ecosystems? 4. What impacts do our clients have on regional economies within countries? In answering these questions, we find that data availability is the binding constraint for the reasons described above, plus additional ones: operational feasibility and cost. For instance, the most direct way of statistically demonstrating impact on producer incomes would be to launch large-scale quantitative household surveys of producers reached by our clients and of a control group of producers to establish the counterfactual. However, given the sizes and operational models of Root Capital and our clients, we have not yet found a context in which: 1) an appropriate control group is available; and 2) the sample
size would be large enough to ensure statistical significance. Moreover, such surveys cost in the hundreds of thousands of dollars per client, and we could not necessarily generalize these findings across our portfolio due to the diversity of contexts, business models, and impacts among our clients. For this reason, we believe that proof that lending to agricultural businesses grows rural prosperity will come not from a single, decisive study, but rather from the accumulation of compelling evaluations that target different types of impact and linkages in the chain from Root Capital to farmers, across a wide range of contexts. Building this base of evidence will require skill not only in the design and implementation of individual evaluations, but in triangulation and synthesis across evaluations. We have been experimenting with new and cost-effective ways to generate information about impact and to embed them into our long-term relationships with clients. In parallel, we seek to improve the rigor and accuracy with which we synthesize information from multiple imperfect but practical data sources and methodologies into meaningful and actionable findings. We liken this process of triangulation to that of a Global Positioning System (GPS), which combines the individual satellite readings from multiple satellites to triangulate a fairly precise estimate of ones geographic position. The results from each impact methodology, like the signals from GPS satellites, may be individually imperfect, but we can combine, cross-check, and synthesize them to improve their collective accuracy. (For more on this topic, see our article A GPS for Social Impact in the fall 2012 issue of Stanford Social Innovation Review.)
10
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
Speaking the Same Language: The Impact Reporting and Investment Standards (IRIS)
In 2009, the Rockefeller Foundation invited us to participate in an ambitious new initiative to standardize output metrics across organizations working in the impact investing industry. By agreeing with our peers on common definitions of the output metrics on which we report, we render those metrics more transparent and comparable. The outcome of this initiative was the Impact Reporting and Investment Standards (IRIS). Managed by the Global Impact Investing Network (GIIN), IRIS is a catalogue of terms and definitions that can be used to describe the social, environmental, and financial performance of an organization. Root Capital is an original member of the IRIS Taxonomy Committee responsible for defining these standards, as well as a member of the Agriculture Working Group.
Table 1: Root Capitals Core Social & Environmental Metrics
Metric Number of Farmers Reached Purchases from Rural Producers ($m) Total Revenue of Rural SGBs ($m) Hectares Under Sustainable Management
IRIS ID 2009 2010 2011 2012
*Includes both producers reached directly (i.e., producers who sold their crop to the enterprise) and producers reached indirectly (i.e., producers who purchased agro-inputs or postharvest handling services from the enterprise). Until 2010 Root Capital only lent to enterprises that reached producers directly. In 2010, we began lending to enterprises that provide agro-inputs and post-harvest handling services through our Innovation Portfolio and reported those metrics separately. Therefore, figures for 2010 - 2012 differ from those in our Quarterly Performance Reports, which do not include producers reached indirectly. The dramatic increase in number of farmers reached in 2012 reflects a large increase in the number of producers reached indirectly. Revenues and payments to producers are not expected to increase proportionately with our number of clients in 2012 because commodity prices reached historic highs in 2011 and are significantly lower in 2012.
Social & Environmental Metrics: From our earliest days, we tracked measures of our outputs and our clients outputsthat is, basic measures of the scale of our activities. (See Impact Terminology: Outputs, Outcomes, and Impacts below for a definition of the term outputs.) We call these our core Social & Environmental Metrics and report on four metrics using definitions that are compliant with the Impact Reporting and Investment Standards (IRIS, described in the box above) in our Quarterly Performance Reports. We record over a dozen more IRIS metrics for internal purposes. These core output metrics are a starting point. They provide an indication of the scale of our impact across our portfolio and a way for us to hold ourselves accountable for impact internally and externally. And yet, these metricslike most output metricsare one-dimensional. They mean little without further context. Over the past decade, Root Capital has reached more than 600,000 farmer households, but is that a lot or a little given the human and financial resources we have invested?
More importantly, the four metrics in the previous table leave us wondering: what does it mean that we reached a farmer? In what ways did the lives of these farmers and their families change? How important were these changes to them? And how do we know that it was our work that caused the changethat farmers lives would not have improved anyway? Unable to answer these difficult but fundamental questions, mission-driven organizations may be tempted to settle for the unsatisfying over-simplicity of pairing an estimated number of people reached with an anecdote about a few of those people. In 2010, with our core metrics in place, we adopted a two-pronged approach to begin to address these questions: 1. To get a better indication of which of our clients are generating which types of outputs, we expanded the Social & Environmental Scorecards to cover a wider range of client practices. 2. To move beyond measuring outputs to evaluating outcomes and impacts, in 2011 we began piloting a variety of deeper impact studies with selected clients. These efforts are described in Section 5, Evaluating Impact (page 19).
12
Outcomes
The effect of our activities on our intended beneficiaries
Impact
Proof that our outputs caused the outcomes; degree of attribution versus other causes
Root Capitals outputs include training sessions, first-time loans to clients that lack access to finance, and renewal loans to existing clients. Our clients outputs include payments to farmers for their harvests.
Root Capitals outcomes include the growth and stability of our clients businesses.
To prove Root Capitals impact on enterprise growth, we would need to show that clients would not have grown their businesses without Root Capitals loan and/or training. To prove our clients impact, we would need to show that farmers incomes would not have increased if not for our clients.
Our clients outcomes include higher and more stable incomes for small-scale farmers.
These terms are particularly important for Root Capital because we work through agricultural businesses to support farmers livelihoods. As such, we must consider not only our outputs and outcomes on our clients, but also our clients outputs and outcomes on small-scale farmers, their communities, and the environment.
To evaluate impact, we must go one step further and compare these outcomes to what would have happened in the absence of our interventionwhat economists call the counterfactual. We describe our efforts to do so in Section 5, Evaluating Impact below.
13
Specifically, the scorecards include 19 types of practices, grouped into four themes (See Figure 3 for more detail): Higher and More Stable Producer Incomes Producer and Employee Rights and Treatment Community-Level Practices Environmental Practices
Thus, the scorecards function as both a negative screen, filtering out undesirable practices, and a threshold test in which the loan officer must affirmatively identify how the clients business practices can be expected to generate social and environmental impact. Our impact team synthesizes these ratings to categorize our portfolio of loans by level of impact and to refine the client selection criteria moving forward. In early 2012, our Impact team began to aggregate and analyze the first year of data from the revised Social Scorecard. As some of the information is subjective and / or self-reported by Loan Officers, we are now assessing the quality of the data by cross-checking with other information sources where possible; we are also conducting a process audit of the way loan officers use the scorecard to ensure consistency. We presented initial results internally at our regional retreats during the summer of 2012 to inform discussions of lending strategy and client evaluation criteria. As we verify findings from this initial analysis, we look forward to sharing them externally.
14
Pre-investment Financial Management Training Accompanying Clients Growth Through Long-Term Relationships Client Enterprise Growth Renewal Rate Post-investment Financial Management Training Higher and More Stable Incomes Enterprise Pays Higher Price than Other Local Buyers Enterprise Raises Local Market Prices for All Producers Enterprise Wages and Benefits to Employees
Enterprise Enhances Producer Productivity in Existing Activities Enterprise Creates New Income-Generating Activities Producer and Employee Rights and Treatment Occupational Health and Safety Rights of Association of Employees and Producers Community-Level Practices Enterprise Provides Public Goods (e.g., health, educational programs) Enterprise Provides Credit to Producers Gender Inclusive Practices Environmental Practices
Environmental Management Systems Land Use and Biodiversity Conservation Agrochemical Use Soil Management Water Management Solid Waste Management Energy Usage
15
Overall, Root Capital was the primary lender to 67 percent of our clients in 2011.
9 An additional 19 clients received loans in earlier years that were still active in 2011, but did not take out any new loans in 2011, bringing our total number of clients reached in 2011 to 193. This analysis will focus on the 174 clients for whom we originated loans in 2011.
16
One component of our Scaling Impact Plan is to continue to extend multiple loan types to our renewal clients to meet a broader range of their financing needs.
We conducted further analysis on the growth of our renewal clients businesses (See Figure 4.) by compiling enterprise-level metrics for all of our borrowers for all years in which data is available. We found that 79 percent are growing at a rate of 10 percent or more and 30 percent are growing at a rate of 50 percent or more.11 Overall, 63 percent of 2011 renewal clients are fast-growing gazelles. (i.e., grew revenues at a compound annual growth rate of 20 percent or greater for all years for which data is available). Gazelle is a term coined by researchers to describe the subset of small businesses that are rapidly growing, as these businesses have been found to drive the majority of job creation in many economies.12 This analysis is purely descriptive. It demonstrates that Root Capitals clients, in general, are rapidly growing businesses. To assess the extent to which Root Capitals work contributed to this growth would require similar data about a comparison group of businesses to which Root Capital did not lenddata which is not readily available and would be difficult and costly to obtain. We are seeking partnerships with external researchers to explore these issues further.
Figure 4: Distribution of Compound Annual Growth Rates (CAGR) of 2011 Renewal Clients
25 63% of renewal clients are gazelles growing at a rate of 20% or more.12 20 30% of renewal clients are growing at a rate of 50% or more. 15
10
0 from <-40% -40% -30% -20% -10% -40% -30% -20% -10% 0% to
0 10%
10% 20%
20% 30%
30% 40%
40% 50%
50% 60%
60% 70%
70% 80%
80% 90%
90% 100% 110% 120% 130% 140% 150% 100% 110% 120% 130% 140% 150% >150%
10 The renewal rate is defined as the percentage of clients that received a loan in 2010 that subsequently received a loan in 2011 or 2012. 11 This analysis is based on the financial statements provided by our clients during their credit application, and does not control for inflation or exchange rate fluctuation. 12 Though there is no agreed-upon precise definition of the term, most researchers consider a business that sustains growth of 20 percent over the research period to be a gazelle. For more information, please see Small and Growing Businesses: Investing in the Missing Middle for Poverty Alleviation, Aspen Network of Development Entrepreneurs, March 2012, pages 10-12: http://www.aspeninstitute.org/sites/default/files/content/docs/ande/ANDE%20Literature%20Review%20-%20FINAL.pdf
17
average, 74 percent of their revenues out to producers. Some pay more, such as cooperatives whose charter is to maximize payments to producers, while others pay less, perhaps because they engage in additional value-added processing, or perhaps because they are simply not as efficient as others. The close clustering around the trendline indicates that these deviations are small relative to the general pattern of our clients paying roughly three-quarters of their revenues out to producers.
$9M
$6M
$3M
$0
$3M
$6M Revenue
$9M
$12M
$15M
18
5. Evaluating Impact
From Outcomes to Impact
Despite the growth of Root Capitals lending and the many positive outputs tracked in our Social and Environmental Scorecardsthe question remains: Can we really prove Root Capitals impact? This is a fair question and one that makes many a mission-driven organization squirm, due to the difficulty of measuring impact described in Section 3. Piecing together the information necessary to manage towards impact requires us to make many assumptions. The fundamental assumption is that the outputs we are measuring (e.g., payments to producers) are truly causing the outcomes we desire (e.g., higher and more stable incomes). Were measuring outputs, but how do we know that those outputs are driving outcomes and impact? Lets be honest: we cant be sure. The reality on the ground in any given community of small-scale farmers is complicated, and our ability as outsiders to understand that reality is limited at best. Managing toward impact in real time requires us to make educated assumptions about what is actually happening on the ground. We then perform impact evaluations to check those assumptionsto move beyond measuring outputs to measuring outcomes and impact. To evaluate impact, we must compare outcomes at the business, household, community, and ecosystem levels with what would have happened in the absence of our intervention the counterfactual. Thus, evaluating impact requires us to meet an even higher bar than measuring practices, outputs, or outcomes. To prove impact requires that we demonstrate how the world would have been different in the absence of our work. For instance, showing that farmer incomes increased isnt enough; we have to show that our work caused incomes to increase, netting out the influence of all other factors. After all, maybe our client would have been able to get a loan elsewhere if they hadnt gotten one from Root Capital, and the farmers would have achieved the same incomes. Economists and statisticians have developed a variety of research designs and techniques to assess causality, and careful qualitative research can also shed light on causal mechanisms. In evaluating impact, we keep this idea of the counterfactual top-of-mind, regardless of which research method we use.
+
Counter-factual outcomes
How fast would the client have grown without our loans and trainings? What would farmers incomes have been?
5. Evaluating Impact
19
In addition, we will consider large-scale quantitative experimental studies of products and services offered by rural SGBs to farmers in cases where 1) the results will not only establish impact at a particular client, but also prove a principle that can be generalized to other parts of our portfolio and the field at large; and 2) the client itself views the study as an opportunity to address a business challenge. We will conduct some evaluations ourselves and engage third parties to conduct others. Evaluations conducted by existing staff incur lower incremental cost than those conducted by external researchers. On the other hand, external researchers can bring independent perspectives and specialized expertise. As independent third parties, such researchers also have a better chance of learning about unintended consequences of our work and our clients work. If from the clients region, their language fluency and familiarity with local cultures give them additional insight into impact on the ground. For these reasons, we engaged a research anthropologist in our first round of pilot evaluations to conduct a case study at Fruiteq, a private enterprise that purchases mangoes from small-scale farmers in Burkina Faso. In addition to accomplishing our immediate goal of learning about Fruiteqs impact on farmers and their communities, we learned a great deal about the challenges of impact research from Batamaka Som, our experienced Burkinab anthropologist. (See Learning to Evaluate Impact On the Ground below.) Learning from this study continues to inform our approach to impact evaluation more broadly. Finally, a word on randomized control trials (RCTs), which many consider to be the gold standard of impact studies. We are evaluating the possibility of conducting RCTs, but have not yet found a context in which we are able to disburse loans to a randomly chosen selection of clients, nor a client that is willing to purchase the harvest of a randomly selected group of farmers. We do, however, see opportunity for collaborating with clients and researchers to conduct RCTs in situations where businesses are, for example, expanding their sourcing to a new region and may be willing to randomly select which farmers participate in year one of the pilot expansion and which enter in year two. In short, while RCTs are not as practical to implement with our clients as in, say, the field of microfinance, we are interested in identifying potential applications of this important methodology.
Mobile-enabled quantitative evaluations of our clients social and environment impact, often in combination with participatory mixed-method studies. Mobile information platforms that enable clients to generate information that is useful for impact evaluation and also enables our clients to manage their businesses more effectively. (See Mobile-Enabled Evaluation: Acopio Mvil, page 27.) Mobile survey data collection tools.
Portfolio data analysis to quantitatively test hypotheses about Root Capitals impact on clients. For instance, analysis of clients revenue growth before and after their first loan. In 2011 and early 2012, we compiled historical financial and social metrics for our clients (350+ enterprises and 1100+ loans) into one consolidated database; this will house data for new loans going forward. (Previously, client data were in separate databases for various uses.) Initial analyses are presented in Section 4.
20
5. Evaluating Impact
5. Evaluating Impact
21
Aging coffee trees. Declining coffee yields are widespread in Haiti, partly due to its aging stock of coffee trees. In the past two years, COOPCAB has distributed 250,000 coffee tree seedlings to its members. These seedlings are helping to rejuvenate coffee fields and accelerate the reforestation of Haitis deforested hillsides. Despite these positive strides, full rejuvenation of coffee fields would cost an estimated $2,000 per hectare of land, an investment that would take approximately five years to recoup and is cost prohibitive for the vast majority of Haitian coffee farmers. Lack of information. Because the coffee value chain is not always transparent and predictable, farmers can be exploited by middlemen who collect coffee opportunistically and supply it to exporters. To help farmers take control of their buyer relationships, COOPCAB holds information sessions to educate farmers about how the coffee market works, empowering them to be vigilant in their transactions with middlemen. Inconsistent coffee quality. To maintain consistency in their coffee, COOPCAB has a cupping and testing lab responsible for ensuring that product meets customer requirements. To maintain high quality as the coffee moves up the value chain, COOPCAB technical assistants monitor the process and advise farmers on best practices.
benefit of a counterfactual and although we believe COOPCAB is not solely responsible for this marked price increase the available information strongly suggests that the market access afforded by COOPCAB has contributed positively to the volume coffee farmers are able to sell and the price they are paid. Comparing the prices COOPCAB offered with those of its competitors in the most recent coffee season is even more informative. In the 2011-2012 season, Dominican middlemen paid farmers $1.46 per pound of coffee, while other middlemen paid $1.35 per pound, and the large conglomerate paid $1.25 per pound. All but COOPCAB pay their farmer suppliers in full at the time of delivery, with no second payment at the end of the season. Hence, while COOPCAB offers a lower upfront payment at the time of delivery, it provides a higher price overall. COOPCABs price premium of $0.10 to $0.30 per pound over the prices offered by other local buyers results in an estimated $40 to $250 of extra income per farmer. Given that average annual income per capita in Haiti is $660 (and likely lower in rural areas), and assuming a rural household size of five (a recent World Bank report estimated 4.613), these premiums translate to incremental income per person of $8 to $50, or 1 percent to 8 percent of annual income per capita. For instance, $40 is enough to buy a small goat; $250 is enough to feed a rural family of five for well over a month. In the past, the amount of coffee COOPCAB could purchase was constrained by its limited access to finance, leaving farmers to sell their harvest to middlemen. Prior to 2010, COOPCAB borrowed roughly $100,000 to $150,000 each season from a local credit cooperative, not enough to meet its financing need. Two cycles of Root Capital loans (including a $200,000 loan in 2011) in combination with financing from a local credit cooperative have enabled COOPCAB to purchase coffee from an additional 1,000 farmers in the 2011-2012 season. In aggregate, we conservatively estimate that access to credit from Root Capital enabled COOPCAB to generate at least $200K in incremental income to Haitian coffee farmers in the 2011-2012 season.
COOPCAB farmers have several other options when selling their coffee, ranging from a Haiti-based agro-industrial conglomerate to fly-by-night buyers who smuggle coffee from Haiti to the Dominican Republic, market it as Dominican coffee, and sell it at a rumored 500-600 percent mark-up. COOPCAB offers a long-term relationship to farmers and engenders loyalty by paying fair prices and sponsoring a series of agronomic and social programs. Our study of the 2011-2012 coffee season found that, compared with other local buyers, COOPCAB offers the highest purchase price ($1.56 per pound), divided into two payments ($1.04 at delivery and $0.52 after COOPCAB receives payment from its buyers). Before the creation of COOPCAB, farmers earned less than $0.60 per pound of coffee, and as low as $0.22 per pound in the late 1990s. Although we do not have the
13 Social Resilience and State Fragility in Haiti, World Bank, April, 2006.
22
5. Evaluating Impact
5. Evaluating Impact
23
The Importance of Trust and Loyalty Between A gricultural Businesses and Small-Scale Farmers
Strong enterprises play a critical role in buffering farmers from the effects of market shocks common in Burkina Faso and elsewhere. Fruiteqs financial strengthand its relationship with farmershelped both to survive a major crisis in 2011. Political upheaval in neighboring Ivory Coast caused a closure of the border between the two countries, blocking Fruiteqs access to the Ivorian port of Abidjan via trains equipped with cold storage. Many exports from landlocked Burkina Faso halted, and Fruiteq had a decision to make: either to skip the 2011 export season, wreaking havoc on farmers livelihoods and jeopardizing its reputation and trustworthiness with both farmers and European buyers, or to find an alternative means to ship the mangoes. Fruiteq choose to transport the mangoes by truck to a port in Ghana that is over 850 miles of bumpy roads away from Fruiteqs headquarters. At significant cost, Fruiteq searched across West Africa for trucks with cold storage for the journey, as there were none available in Burkina Faso. Fruiteqs farmer suppliers responded in-kind to Fruiteqs commitment to keep the business going by voting at their general assembly meeting to reduce the payments they earned that year by more than 10 percent from the initially agreed upon price.
The remaining 70 percent of farmers harvests are sold to the local market at $0.04 per kilogram.
Root Capitals first loan to Fruiteq supported its ability to increase mango purchases from farmers by 340 percent and to increase its annual revenue by 235 percent between 2009 and 2010. Prior to partnering with Fruiteq, the Burkinab farmer cooperative UPPFL produced mangoes for domestic consumption. Fruiteqs connection to the European market gave UPPFL access to premium export mango prices, increasing its revenues and enabling it to offer value-added processing services to farmers that increase prices. It also offers agronomic assistance to combat the devastating fruit flies, which leads to higher yields. Fruiteq has used the premiums from its sales of Fair Trade mangoes to sponsor the following community projects: A house for the village midwife, who previously lived 15 kilometers from Takaledougou, was built. There have been fewer deaths of infants and mothers from complications during childbirth now that the midwife lives within the community. Potable water wells in the villages of Banfora and Takaledougou bring drinking water to 3,000 villagers who previously drank river water as their main source. Prior to having access to the wells, women had to walk three to five kilometers to fetch potable water during the dry season. A small diesel engine that produces electricity to power food processing equipment such as grinding mills and vegetable or nut oil presses was installed in Takaledougou. This is a particularly useful time- and energy-saving device for women who typically do this work manually.
5. Evaluating Impact
24
An ambulance, available to 50,000 people, was purchased for Banfora and the surrounding communities. The ambulance enables sick and injured people in the community to be transported to health centers in Bobo-Dioulasso and Ouagadougou to get higher quality medical care than is available in Banfora. A pharmacy in Takaledougou has cut down on the use of counterfeit drugs and therefore has improved patient care in the village.
Beyond the impact that Fruiteq has on its farmer suppliers, the company also plays an important role as an employer in the region, providing jobs to 74 women for four months of each year. These seasonal employees earn roughly $2.00 per day, compared with the $1.20 per day they could earn at an area sugar cane company. Workers and mango farmers alike use their earnings for a variety of household activities. Some purchases, such as motorcycles, plow oxen and carts, and investments in petty trade, are made to increase earning power. Others, such as hard brick house construction, food purchases, and health care fees satisfy basic family needs. Still others, particularly related to childrens educational expenses, speak to the intergenerational investments made possible by incrementally higher incomes today.
Outcomes
Increasing Fruiteq revenues by 235 percent
Impacts / Counterfactual
This study did not seek to establish a counterfactual. However, given Fruiteqs limited access to finance, it is unlikely that Fruiteq would have received this high level of revenue growth without Root Capitals loan. In 2011, farmers earned 43% more income selling to Fruiteq (and the differentiated markets Fruiteq facilitates) than they would if they sold the same mangoes into the local market only.
F ruiteq enables farmers to sell different quality mangoes into three price-differentiated markets (local market, processed mango market, and export market) F armers earn 3-5 times the local market price for their highest-quality mangoes sold to Fruiteq for export
5. Evaluating Impact
25
A woman thought we were doing a survey to bring assistance to the village poor. When asked if she owns a cell phone, she answered no, but her 6-7 year old son challenged her answer by saying Mom, what about your cell phone in your handbag? She cursed the child and put him on time out.
Batamaka Som, October 2011
Here you come to ask us the same silly questions that you go sell to aid sponsors. Now when the aid comes you keep it for yourself. I dont want to answer any question. Go take the answers for the ones we provided last yearYoure all crooks of the same family. Youll ask me my name, my family size, the kind of goods I have, and so on and so on. I am tired of all this and I am not answering a question, nor will anyone else in this family.
Mango Farmer, October 2011
Sometimes humorous, sometimes searing, anecdotes such as these caution us to be thoughtful and respectful as outside researchers seeking to understand local realities. They remind us of the importance of approaching farmers and our clients not as objects of research, but as fellow human beings that bring their past experiences, perspectives, and goals to the research interaction. Even regarding something as objective as mobile-phone ownership, these subjective factors are of value in and of themselves and can dramatically influence the research findings. As Som wrote in his report, the understanding of many local people, whom it would be fair to call development brokers, is that development has become a business for which only the big shots of
development project management win. Anytime they see a researcher, they understand (rightly or wrongly) that the surveyor is a messenger of development. Thus, they provide the answers that can best market and sell their poverty and attract development perks for themselves or their village. Root Capital is in the early stages of experimenting with approaches to impact evaluation that do not conform to these perceptions of the development business. There are no one-size-fits-all solutions, but our guiding principle is to design impact evaluation projects that benefit the communities that participate, even while generating the information we need about impact. (See A Client-Centric Approach to Impact Evaluation below.)
26
5. Evaluating Impact
The data can be accessed in real time by cooperative managers to assist them with cash flow and inventory management, enabling them to manage their business more effectively. At the same time, this data provides critical information for Root Capital in our loan monitoring, enabling us to track the harvest to assess whether we will be repaid on time or if our clients are having trouble meeting their contract obligations. Finally, our impact assessment team can use this data to estimate the price premiums that farmers receive from our clients above and beyond what they would receive from other local buyers, a metric that is central to our impact but very difficult to capture. Acopio Mvil embodies the principles of our client-centric approach. Originally conceived as an efficient way to collect data solely for our own impact assessment work, we quickly realized that the project filled a critical data gap for our clients. By working with our clients to understand how they use data, we have been able to offer a data collection solution that can be used for multiple purposes by a range of stakeholders. As the 2011-2012 harvest and first Acopio Mvil trial comes to an end at COOMPROCOM, we will launch a second set of pilots with COOMPROCOM and another client in Mexico. In a related project, as part of our Root Link program in Guatemala, Honduras, and Nicaragua, our Financial Advisory Services team will be training 18 clients during 2013 on management of internal credit programs (i.e., funds housed within a cooperative targeting the delivery of microloans to producer members). This service enhances member loyalty and reduces the threat of side-selling (selling outside the coop), provides another income stream for cooperatives, and provides smallscale farmers with affordable credit throughout the year. Replacing paper-based management with electronic management will increase the efficiency of the internal credit program while also capturing data on producer-level financial activity that informs our impact assessment.
Our current system wasnt able to collect information about the harvest and we struggled because of that. Your system uses new technology that we believe will help us be more efcient in the years to come.
Ervin Calixto Miranda, Manager of COOMPROCOM
6. Looking Ahead
27
6. Looking Ahead
Client-Centric Impact Evaluation
Reflection on our first round of evaluation pilot projects, in particular, our studies with Fruiteq and Acopio Mvil, led us to articulate what we call a Client-Centric approach to impact evaluation, which seeks to generate the data we need about impact on small-scale farmers, while creating value both for the farmers and for the enterprise. Often, this means generating the data we need by first working with clients and their producers to generate the information that they need. Sometimes our information needs are fully aligned. In other cases, we seek information that is not directly useful for our client. In such cases, we prioritize studies that address a business challenge they are confronting and incorporate our research priorities along the way. We find that, in addition to strengthening our client relationships, a client-centric approach to impact evaluation improves the quality of the impact data that we obtain. Specifically, it: Focuses study design on issues that are of true importance to the client and producer community and thereby central to the impact experienced by that community; Increases the commitment level of the participants; Is more likely to elicit their honest responses; Helps the business operate more effectively or increase its social impact; and Identifies opportunities for further engagement that are relevant to the client. Our Client-Centric approach is less objective than experimental evaluations, and we will present the results with appropriate caveats. For some clients, however, experimental evaluations are simply not possible. A client-centric approach enables us to engage clients that might not otherwise participate in evaluations, and to help them increase their impact. In cases where experimental evaluations are possible, a client-centric approach maximizes the chance that a potentially interested client will feel comfortable proceeding. Our aspiration is that impact evaluation becomes a valued service to our clients and a comparative advantage of Root Capital in our nascent financial market.
It is not unusual to adopt a client-centric approach to evaluating our impacts on clients with whom we interact directly. Many organizations do so. However, we are charged with showing that our clients in turn have an impact on the incomes and livelihoods of large numbers of farmers with whom Root Capital staff only have very sporadic interactions. The standard way to do this is to launch large-scale surveys that are costly, burdensome, and sometimes repetitive to the client and to the farmers themselves, given the number of organizations conducting projects with certain organizations and communities of small-scale farmers. We may conduct quantitative surveys in cases where the results would demonstrate a general principle and the client sees clear value as well, but our approach is centered on practical and low-cost ways to partner with our clients to generate data on farmer incomes and livelihoods that is not a burden for them, but will rather help them make better business decisions.
28
6. Looking Ahead
We will consider five criteria when prioritizing impact evaluations: 1. Client Value: The evaluation must address a business or impact challenge of the client and, as a result, generate client buy-in and enthusiasm for the project. 2. Practical significance: The evaluation must produce high- quality data about one or more of our impact themes. 3. Internal validity: The evaluation must generate data that we can be confident in. At a minimum, the study must capture data on outcomes. If the project is intended to determine impact, it must establish an appropriate counterfactual. 4. External validity: We should have a perspective on which specific findings might be generalizable under what circumstances, and why. 5. Cost: Each evaluation should be the most cost-effective way to generate a given set of data. The concept of external validity is particularly salient for a social financier managing a portfolio of heterogeneous clients and impacts. Our clients, their contexts, and their impacts are sufficiently different that proving impact with one client might give a positive indication of impact with a subset of other, similar clients, but would not prove impact with our other 200+ clients. To avoid making exaggerated claims of impact, we assume that findings from one client are not generalizable unless we have a specific reason to think that they are. This is one reason why we choose to experiment with smaller-scale evaluations with multiple clients rather than investing a large proportion of our evaluation budget on a large-scale project intended to prove impact definitively with one client.
Given the difficulty of generalizing our evaluation results, we will prioritize a set of evaluations that collectively represent the bulk of Root Capitals portfolio along the following dimensions: Major regions in which Root Capital operates Major industries in which Root Capital operates Range of types and sizes of Root Capital clients
Thus, the clients chosen will neither be cherry-picked to demonstrate strong impact, nor will we focus on clients with particularly weak impacts in the hope of helping them to increase their impact. We anticipate that the clients chosen will form a broadly representative cross-section of Root Capitals portfolio, though there may be a bias towards clients sufficiently interested in impact to partner with us for an evaluation. Over time, we aspire to generalize from these studies to larger groups of Root Capital clients and agricultural businesses more broadly. Our ultimate goal is to demonstrate that lending and financial training for agricultural businesses can transform livelihoods for small-scale farmers while sustaining the natural environment upon which they and we depend. As we develop this base of evidence, we will share our approaches and findings through issue briefs, impact studies, and articles in external publications. Through this work, we seek to catalyze a financial market to serve agricultural businesses that connect small-scale farmers to markets across the developing world and to maximize the social and environmental impact of that market.
www.rootcapital.org