Thursday, November 21, 2013

BACK TO BASICS
VIOLA DESSANTI, SENIOR ANALYST, RESEARCH AND EVALUATION, FOR THE ONTARIO TRILLIUM FOUNDATION REFLECTS ON WHAT IS MEANT BY MEASURING IMPACT

I
VIOLA DESSANTI Senior Analyst, Ontario Trillium Foundation

f you work at a not-for-profit organization or charity, any level of government, or a public or private foundation, you probably have heard of the word impact. I am guessing you hear the word impact multiple times every day. I know I do. And with all the talk, I cant help but wonder what are we all talking about? Has impact become an elusive concept, the holy grail of philanthropy? Dont get me wrong - I believe that working to solve intractable complex social problems requires a steadfast focus on a long term vision. At the same time, I feel that we often talk about measuring impact without fully understanding the implications. In fact, I believe measuring impact will help us reach our collective goals, but only if we are clear about what it means. I do think it is important to demystify impact and break it down to where we can all wrap our heads around it. Heres my take: Impact is about a change. Impact is change that takes a long time to materialize. Therefore, measuring impact is a long-term proposition. At the Ontario Trillium Foundation, we continuously ask ourselves: what do we mean by impact? What does it look and feel like in communities? How can we measure it? Most importantly, how can we work with our grantees, meeting them at their capacity level, so that we are building our collective capacity to be able to say, This is the change we were able to achieve. Our work in impact measurement is rooted in the principle that what we measure, and what we require our grantees to measure, should benefit both the Foundation and the organizations we invest in. In the last couple of years, I have been fortunate to work alongside those in the Foundation charged with helping us better define the change we want to see and how to measure it. We are trying a number of different approaches to identify and demonstrate the impact of the Foundations investments: Define it: The OTF Strategic Framework explains what we are trying to accomplish through the investments we make in four sectors. We have identified four priorities for each sector. For each priority, we have defined one or two outcomes (or changes) we hope to achieve with our investment. To measure the changes, each outcome is associated with indicators that tell us when we are getting closer. For example, in the arts and culture sector, we know that if our investments result in More and better activities that celebrate the diverse cultures of the province we will be part of achieving one of the four arts and culture priorities to Preserve, promote and celebrate diverse cultures and heritages. Align it: We have started to identify the links between our theories of change and external evidence of community health and vibrancy. The Canadian Index of Well-Being provides a complete picture of wellbeing by incorporating a comprehensive set of the key social, health, economic, and environmental factors contributing to overall quality of life. Understanding the alignment between our granting investment strategy and the CIW will make it possible to demonstrate how the work of our grantees is making Ontario communities healthier and more vibrant. 1

HAS IMPACT BECOME AN ELUSIVE CONCEPT, THE HOLY GRAIL OF PHILANTHROPY?

Continued...

BACK TO BASICS
Refine it: We continue to refine our theories of change to ensure they reflect new evidence and impacts that are being achieved either through our own investments or those of others trying to accomplish the same things. Quantify it: Go beyond number of workshops held or clients served. We have explored ways of quantifying the economic value of investing in the sector. We have even dabbled with Social Return on Investment (SROI) on some of our grants to get a better sense of the methodology and how it could help towards measuring impact. Learning from it, as we go: Using developmental evaluation for the Future Fund program, we have been able to bring a complementary approach to measurement to ensure we are learning along the way and adapting to complex environments. Emergent learning practice has also helped us understand how to better bring story-telling into our research and evaluation work. These are just some of the avenues weve explored over the last three years. So far, I have learned two important lessons through this work: It is important to right-size the information we request of grantees to meet the capacity of the organization and our investment in it. Approaches to measuring impact are most successful when they bring something new to the person/organization doing the measuring and the person/organization receiving the information. OTF asks grantees to report on outcomes and tell us the change that they want to see, so that we can report on the change we are affecting. We do ask them to compile significant amounts of information in order to even receive our funding. We are very aware that we place a high burden of information on grantees and applicants. But it can also be an opportunity. Our opportunity as a Foundation is to collect all that wealth of information, analyse it in a broader context, and feed it back in the aggregate to the organizations and communities in Ontario. I believe this will create knowledge. This will create understanding. This will help us build a shared vision for the impact we want to see. It will also contribute to shared learning about what works and what does not. And as a personal side benefit, it will help me better understand what we all mean when we say we are investing for impact!

Sourcing bright ideas, fresh perspec tives and personal opinions on transformational change from Ontario and around the world

www.otf.ca/sic
2