Measuring Social Change: Tools And Strategies For Impact Measurement

The task of measuring social impact often leaves people stumped. Even those who have identified well-defined outcomes can find measuring the change itself challenging. However, it doesn’t need to be complicated; simplicity is key. There are a few key considerations though, which we’ll cover in this post. The previous post in this series concerned itself with how to find out what we should be measuring and this post will cover how to measure change, and some of the methods and tools we might need along the way.

Measuring change

Change is the process through which something becomes different, involving the transition from one state, condition or phase to another.

We previously left off at the point where we had identified what the outcomes are that are achieved by or for the people who are affected by our activities. Before we can measure the change against those outcomes though, we need to identify what the state is that we are changing from. This provides us with a baseline to measure against, so we can show that there was a change from the original state to the new state. If we want to find out how things were before our intervention, it is best to engage beneficiaries or other people who might be affected prior to their engagement with our activities, services, etc. This will provide a clear picture of how things are or were for them before engagement.

If our activities or services are well-established and new people are not likely to engage, one way of assessing the pre-engagement baseline is through retrospective self-reporting by beneficiaries and others. It's worth noting that when asking people after the fact about how they felt before engaging, it can be difficult for them to remember clearly and accurately, so be sure to account for this in the analysis.

To understand what change has occurred against our specific outcomes, we must then measure where beneficiaries and other people affected are at, across all the different indicators we have identified for those outcomes. Rating on a scale can be an effective way of capturing this and we can compare responses from before and after the activity to see what amount of change has occurred. If enough change has happened across enough of our indicators for an outcome, we can state that it has been achieved, and to what level.  The converse may also be true of course.

Remember that behavioural change and broader societal change may take a long time to materialise and factors other than our specific intervention may have contributed to the change, so impact should never be overstated.

Methods and tools

With the right mix of methods and tools, we can uncover valuable insights that guide our efforts and amplify positive change. There is no one right way to gather the data we need to understand the impact we are creating through our activities; each organisation and intervention is unique and a different blend of methods and tools will be best suited to each.

Qualitative methods are particularly relevant if only a small number of people are affected. They also add contextual understanding and may uncover unanticipated outcomes. Many qualitative methods exist, but the most common ones for impact measurement are:

1. Interviews Conducting one-on-one interviews helps gather in-depth insights into peoples' experiences and perceptions. These can be structured, semi-structured, or unstructured.

2. Focus groups Bringing together various people to discuss specific topics allows for a range of perspectives and can uncover nuanced information in a group setting.

3. Case studies Detailed examinations of individual or group experiences provide a comprehensive understanding of specific examples of social impact creation.

4. Document reviews Analysing existing documents, such as reports, records, and media articles, helps in understanding the broader context of social initiatives.

Quantitative methods are best for capturing standardised and measurable data and offer the best options for statistical analysis; they really come into their own with larger numbers of responses.

1. Surveys Structured questionnaires with closed-ended questions can collect data from a large number of respondents, making it easier to analyse trends and patterns statistically.

2. Secondary data analysis Analysing existing datasets, such as the Census, health records or economic reports, to extract and interpret quantitative information relevant to impact measurement efforts.

3. Content analysis Quantifying the presence of certain words, themes, or concepts within textual data to identify patterns and trends.

All of the methods that involve engaging with the people affected can be done online, over the phone, or in-person; so we can choose the format that best suits each group and budget.  Surveys can be conducted using free tools like Google Forms or SurveyMonkey, and online interviews or focus groups can be hosted on Zoom or MS Teams. There are myriad options for paid software that will support larger datasets and more complex analysis. A professional researcher or consultant can provide support if skills are lacking in-house.

In conclusion

Measuring social impact doesn't have to be a daunting task. Whether you're looking to publish an impact report or simply understand the effectiveness of your initiatives, there are flexible and accessible ways to capture meaningful data. From establishing a clear baseline to employing a mix of qualitative and quantitative methods, you can gather valuable insights that demonstrate the true impact of your efforts. Remember, there is no one-size-fits-all approach. Tailor your methods and tools to suit your unique context, and don't hesitate to seek support if needed. With the right approach, you can not only measure change but also amplify the positive effects of your work, driving meaningful and lasting social impact.

About the author

Beca Wistreich, Director at The Outcome, is an accredited Advanced Practitioner of Social Value, and a member both of Social Value Ireland and Social Value International.