Social impact work is a complex, long-term endeavour, as is managing and measuring it. Only in the past few decades have serious attempts been made to explicitly measure social impact. Monitoring and evaluation (M&E) approaches, initially introduced by global donor agencies through logical frameworks, gradually made their way into the corporate sphere. Since then, the theory of change (ToC) has become a dominant approach in corporate M&E as it offers a clear pathway to the impact goal. In this article, Judith Matthis, Trialogue’s head of social impact advisory, considers the common pitfalls associated with applying the ToC and how to overcome them.
The Logical Framework Approach (LFA), developed by the United States Agency for International Development (USAID) in the late 1960s, is a structured planning and management tool used to design, implement and evaluate development programmes. It organises key components – such as objectives, activities, outputs, outcomes and assumptions – into a matrix format, helping teams clarify logic, track progress and ensure accountability throughout the project lifecycle.
This logic underpins the ToC, which, according to the Centre for Theory of Change, is a comprehensive description and illustration of how and why a desired change is expected to happen in a particular context. In effect, it starts with the end in mind by including causal thinking into an area that, by its nature, sometimes sways towards reactive responses.
Based on the ToC, an indicator framework can be created to track outputs, outcomes and ultimate impact. This indicator framework typically follows a logic model structure. So, while the ToC outlines the conceptual change required, the process to achieve that change will follow the logic model structure as evidenced by the indicator framework. The interdependent, dynamic nature of these two frameworks (the ToC and indicator framework) means that when things go wrong with one framework, it affects the other.
Avoiding common ToC pitfalls
Because the theory of change is a complex tool to develop and manage, it is sometimes met with scepticism or criticised as too theoretical and rigid. In our experience, this is more often a matter of how it is applied and managed, rather than an inherent flaw in the framework itself. Below some of the common ToC pitfalls, together with thoughts on how to avoid them.
1. Timing of application
Too often, the ToC is only considered once a strategy or approach is already well-entrenched and implementing partners are reporting on their funded work. This means that the ToC is developed to retrofit the strategy or approach, rather than being used to think through the problem and possible solutions from first principles. The ToC is not designed as a reflective or retrospective framework and can be cumbersome, even flawed, when applied only after a programme is already running. When used retrospectively, its application becomes much narrower than intended and the opportunity to conceptualise the problem statement holistically by considering optimal solutions to achieve the end state may be missed. This can also lead to mistaken assumptions.
The obvious solution is to develop the ToC and indicator framework at the project planning phase. However, this is not always possible or practical and in many cases, companies apply it once the programme is underway. In these cases, it makes sense to keep the underpinning indicator framework very basic, with the intention of revising and refining it in the following strategic round. What the ToC development process helps with in these circumstances is aligning stakeholders’ expectations in terms of programme outcomes and data monitored.
2. Rigidity of application
The causal linear nature of the ToC is valuable because it structures thinking around the means to the end; however, when applied too rigidly, it can stifle early course correction, potentially leading to a case of the cart driving the horse. Since the framework should be developed before the programme’s operational intricacies are understood, it is built on assumptions. This linearity does not fully account for contingent externalities, which could impact project outcomes. Social development work, by its nature, operates in complex, non-linear systems.
Instead, the framework should be viewed as dynamic. Keep an open mind about how the causal path may evolve and build flexibility into the programme implementation and monitoring process. For example, hold periodic programmatic ToC review sessions to update and revise the framework.
3. Assumed exclusivity
There is an often-held belief that the ToC needs to be exclusive in its application as a framework. This may be an assumption or practical reasoning, given the time invested in creating the framework. Whatever the reason, it can lead to a rigid application of the framework without periodic strategic reflection and an overly quantitative approach to work that may yield more qualitative insights.
In some cases, programmes gain valuable insights by accommodating other perspectives or frameworks that can work in conjunction with the ToC. Examples of these are the Value Creation Framework and community-participatory-focused frameworks.
Capitec Foundation applies the ToC together with the Value Creation Framework
The Capitec Foundation aims to enable access to quality education in South Africa, with a particular focus on improving mathematics outcomes for learners. Through its Whole School Approach, the Foundation supports all levels of the school to improve maths outcomes.
Every year, the Foundation conducts an external evaluation of its Whole School Approach. Initially based on the ToC, the Foundation incorporated the Value Creation Framework into the evaluation to provide a more holistic focus on the community and network benefits generated by programme activities over time. The framework emphasises the process of generating value within communities and networks through programme interventions, thereby providing a broader, more qualitative approach focused on the perspectives and experiences of all those involved in the programme. This dual framework approach has been successfully employed for two years to enhance the effectiveness of the Whole School Approach.
Avoiding common indicator framework pitfalls
As with the ToC, the application and management of an indicator framework are key to its effectiveness. Outlined below are some common challenges encountered when applying the indicator framework, together with recommendations for overcoming them.
1. Neglecting partner buy-in
When CSI programmes impose indicator frameworks and related data collection requirements on their partners without understanding the context in which those partners are working, it invariably affects the quality of the data collected. Once a decision has been made about what to track in a project, it very often means more work for already overstretched implementing partner teams. If partners don’t understand how the data benefits their programme, it is natural for them to struggle with the added workload, which can often lead to poor-quality data collection and reporting.
A consultative and collaborative approach helps nurture openness to change. Many successful M&E processes build on existing stakeholder relationships, are collaborative in approach and are able to demonstrate the value of the data to those collecting it and most affected by it.
Change management, along with establishing a transparent feedback process where all parties have visibility of the results, is critical for the effective implementation of the indicator framework. Data visualisation tools are a powerful way of sharing findings and ensuring that they are acted upon in an engaging and accessible manner for a broad audience of stakeholders.
2. Degree of ongoing management
Owing to the dynamic nature of the indicator framework, the change management required and the long-term nature of the results, it is most effective when actively managed by CSI programme staff who possess a deep understanding of the programme’s objectives and the purpose of the indicators being tracked. The frameworks are most effective when applied throughout the life cycle of a programme. They tend to fail when not actively managed or if data is not continually interrogated and assessed. This necessitates ongoing engagement with the indicator framework and a continual flow of data.
Ideally, programme monitoring according to the indicator framework should be an integral function of any CSI programme manager’s role. However, teams are often capacity-constrained and multi-hatting. Outsourcing this function to an experienced third party is another way to ensure active and effective management.
3. Short-term focus
Social impact takes time to materialise. CSI staff often do not stay in their positions long enough to witness the impact. The programme changes as roles or strategic direction change, which means the long-term impact may not materialise. This can discredit the framework, making it seem hypothetical and unable to demonstrate real impact.
This can be remedied by setting clear programme timeframes and identifying lead indicators of progress or success. Managing executive or board expectations about the long-term nature of the programme can also help.
4. Detail orientation
The development of an indicator framework can quickly become highly complex and in-depth, often requiring extensive measurement for various indicators. For the detail-oriented manager, the attraction of unearthing as many meaningful indicators as possible to paint an intricate picture of a programme may be irresistible. Unfortunately however, the challenges and drudgery involved in gathering accurate and credible data often do not align with the expectations of a basket of rich indicators.
Instead, a more sustainable approach begins with a few simple, practical and useful indicators that partners can easily collect to tell a straightforward story. Once data collection processes are well understood and working effectively, additional detail can be introduced in a way that retains the integrity of the process and the information.
5. Overemphasis on numbers
Since the framework is inherently more quantitative than qualitative, it can create the illusion that there is no place or need for qualitative programmatic information and reporting. This can lead to a skewed view of programmes. For a full perspective, both quantitative and qualitative information are important.
Ensure that the underlying monitoring processes are designed to capture the more human aspects of programmes, such as case studies, stories and photographs. The indicator framework does not need to replace qualitative reporting; instead, it can supplement it.
6. Insufficient communication
The indicator framework becomes very powerful once there is long-term data available to tell an impact story. However, this story will go unheard if the data is not presented in a credible, meaningful and compelling way.
Companies that establish effective data visualisation processes are able to regularly draw insights about their programmes and tell a convincing story to decision-makers and other stakeholders. It is important to design data collection processes with visualisation in mind, ensuring that systems integrate smoothly, rather than treating it as an add-on once the framework is already in place.
Like any tool, the effectiveness of a ToC or indicator framework depends on how well it is used. When applied thoughtfully and flexibly, these frameworks serve as powerful tools for strategic planning and ongoing monitoring. However, without proper use, they risk becoming abstract exercises with little real-world impact. To maximise value, these frameworks should be seen as tools for managing programmes strategically – with M&E supporting that strategy, not driving it.
