For the past year and a half, we’ve been working on a project with Internews to support organisational security practitioners working with human rights organisations. One aspect of the project focuses on measuring impact, and we learned a lot about this issue in the context of organisational security that did not fit into the monitoring and evaluation framework we created. We saved those learnings for this blog post, which we hope will help practitioners, allies and funders as they reflect on impact and make use of the framework.
Before creating the framework, we spent a lot of time trying to understand the experiences of practitioners working in different contexts. We participated in regional practitioner workshops, conducted one-on-one interviews in person and remotely, sent out surveys, engaged people at events and read reports, guides, articles and blog posts. We spoke to people who worked in relative isolation, others who worked for support organisations and practitioners who were part of networks. We explored different human rights defender contexts in Eastern Europe, Latin America, Southeast Asia and the Syrian diaspora, and we reached out to traditional, holistic and feminist practitioners.
Great need for funding
One common theme in the research was that there are circumstances outside of the practitioner’s control that limit impact. A major factor is funding. People we spoke to reported a general unwillingness among funders to cover tech equipment and supplies costs for human rights organisations. Unfortunately, organisational security alone cannot address outdated computers that are unable to do necessary software updates. Funding limitations also lead organisations to rely on relatively affordable pirated software, to share work devices and to use their personal devices, all of which increase risk.
Likewise, many organisations cannot afford a full-time IT person or a long-term relationship with a practitioner, forcing them to reach out for one-off emergency assistance rather than working on long-term behaviour change. If funders want to support these vulnerable organisations in serving their communities, they should expand what their funding covers.
From findings to principles
Recognition of this problem led us to put together principles that guided our approach to the framework. The following principles reflect themes that popped up repeatedly in our research as factors critical to the success of organisational security:
- Organisational security involves a holistic approach. This means addressing digital, physical and psychosocial security.
- The duration of org sec support affects the quality of the impact and the ability to measure that impact. Longer engagement leads to longer-term change.
- Organisational culture plays a significant role in the success of org sec. Every level of staff must be willing to make time to learn.
- Each organisation has unique needs, and org sec must respond to changing contexts. Both support plans and evaluation plans must suit political, cultural, sectoral and internal contexts.
When we presented the framework during the 2020 Internet Freedom Festival Organisational Security Village hosted remotely by Internews, some practitioners were surprised by the holistic approach but were able to make sense of it by thinking about the problems stress causes in human rights work and the ripple effect that can have on digital security. Addressing a digital security incident by only thinking about the digital security aspect will not necessarily build resilience for other digital threats or manage the stress such threats cause. Our aim with the framework is for organisations to be healthy, creating a safer environment for human rights work..
New ways of viewing impact
Keeping in mind that there are many ways to think about impact, we want to emphasise that this framework is a starting point. Practitioners should adapt it to suit their needs, and the guide that goes along with the framework offers tips for how to do that, particularly around adding or changing indicators. This framework focuses on long-term behaviour change related to organisational strengthening. We learned that some practitioners are used to ticking boxes because they have focused—understandably—on actions like whether or not an organisation got their data back or installed certain software, but this framework is about showing the evolution of an organisation over time.
Furthermore, impact is not only about what people in organisations do; it’s also about what they think and feel. We found that some practitioners without much experience in monitoring and evaluation were uncertain about the indicators that address how people feel. Whether it’s feeling safer or feeling more confident in an area of security, this is a legitimate way of measuring change. We encourage practitioners to ask people they support how they feel, whether through conversation or simple surveys—emoji surveys can be effective tools!—and to think about changes in attitude as fundamental to long-term changes in behaviour. This way of thinking about impact also circles back to the holistic approach by addressing positive psychosocial change.
The organisational security community is in the early stages of a much-needed conversation on measuring impact effectively. We hope this framework launches all kinds of approaches and tools, and we invite anyone who uses the framework to share their adaptations, any tools they created to collect information and their experiences. We all have a lot to learn from each other.
To dive into the full toolkit that we developed alongside the M&E framework, take a look here. You can learn more about what went into creating this toolkit here.
Illustrated by Matilde Salinas.