The Social Impact Dilemma: When measuring support gets complicated
It’s an ever-relevant question for organisations working on social change — how do we know that we are making a difference? My focus on this concern only seems to grow, and it seems I am not alone. No matter where I am in the world, or which specific group of social changemakers I am with, we inevitably turn to the question of impact and how to measure it. As critical thinkers working on complex and sensitive issues, it makes sense to reflect on the positive change we intend to be creating.
Impact, or social change, is difficult to measure for so many reasons . The world is a messy place, and tracking influence, or change over time — especially over grant cycle timelines — is a daunting task. Within an organisation, limited measurement can be due to a limited amount of in-house learning, monitoring, and evaluation skills, or a general lack of time and resources to do thoughtful learning. At the project level, it can be due to varied and vague understandings of objectives, complex and constantly changing problems and approaches to tackling them, or uncertainties about how to handle ethical considerations.
This challenge is compounded for support organisations — organisations who assist other organisations to bring about change — like The Engine Room. We are interested in measuring our impact not simply by tallying outputs or ‘number of organisations supported,’ but by understanding the impact we have on the impact our partners have. You can get dizzy just thinking about it.
As we begin to unpack the action chain to impact, the process of learning and measuring change becomes increasingly complicated.
We the “Intermediary”: Our unique approach to supporting partners
When I joined The Engine Room in March, I saw the team capture stories about impact and communicate them clearly and thoughtfully. It was an exciting time to join as we committed to growing our learning function to support both our internal operations and our external support to change agents across the globe.
Learning feels especially important in our line of work — data and technology — where good intentions may be pervasive, but so are subpar, and even harmful, outcomes. For us, this begs a sense of urgency to develop systems that help us effectively learn about our influence and effects. And we don’t want to stop there; we also want to offer these ways of learning to our peers and partners.
What excites me about our support model at The Engine Room is that we focus on strengthening others’ impact and on working hand-in-hand with partners. In my relatively short time on the team, I’ve seen this manifest in a number of ways. I’ll give you three quick examples:
- When there is a regional funding call for project proposals our first instinct is to encourage the lesser known local groups to apply, and to provide light-touch project design support to local organizations seeking it.
- When we do research to produce global public goods, you may not even know that we participated in their production. Why? Because we see public goods as: “for us, by us” – take a look at our responsible data materials.
- Finally, many of the projects we work on are collaborative efforts, and we value working in diverse groups (see for example, DatNav and Alidade).
The Future of Learning: What we will do, and how we will do it
We do think it is possible to answer difficult learning questions, and we are continually inspired by peers and partners who are working hard to understand the effects of their own support work and to facilitate learning more generally.
I am amazed and humbled by the work we do, but the real reason I say all of this is to highlight how measuring impact at The Engine Room will be hard — we support others to drive impact, so what is our impact? Thanks to the generous support from the Ford Foundation, we have spent this year designing and testing a learning approach for our light touch support program called LiTS (schedule a call with us to learn more about this kind of support). This is where we are starting. Over the course of the year, we figured out a few pieces of the puzzle, which we will share in an upcoming post.
We hope that you will join us in our journey. If you have useful resources or experiences to help guide us, we would love to hear from you.