Monitoring and evaluation play an important role in a programme or project’s success. This article discusses some key monitoring and evaluation definitions and concepts.
What is monitoring and why is it important?
Monitoring is a routine, continuous process of diligently collecting and analysing data relating to a project’s inputs, activities, and outputs. By measuring these against expectations and targets, it helps stakeholders to make more informed and strategic decisions. It can function as an early warning system and is a good means of course correction.
Monitoring takes place while a programme is being implemented – it consists of collecting and analysing data to measure progress, scrutinising inputs (funds, resources), activities, and outputs (the resulting products or services) along the way. One generally weighs these against plans and goals, ensuring that they comply with various standards. In short, monitoring tracks money spent, actions carried out, and results achieved.
Monitoring helps one to track both the implementation and results of a programme. It is systematic and transparent, so that both funders and beneficiaries have access to the information. It is typically carried out internally, with no independent parties involved at this stage.
What is the difference between quantitative and qualitative monitoring?
Quantitative monitoring looks at numbers and percentages, for example the rise in school attendance, the drop in incidences of disease, the rollout of water pumps, or the publication of school textbooks. Qualitative monitoring looks at judgements, opinions, and beliefs, for example, observations regarding childhood development, focus groups providing their views on disciplining children, and so on.
What monitoring tools can we use?
Some of the tools of monitoring include surveys, formal reports, project status reports, project schedule charts, project financial status reports, graphs, interviews, focus groups, and informal reports. Monitoring reveals trends and patterns that can prove useful, as well as suggest which corrective actions may be necessary.
What is evaluation and why is it important?
People have different definitions of evaluation, so one has to be careful not to make assumptions when discussion evaluation. Different types of evaluation can be done before, during and after a programme. Evaluations are used for different purposes, and the process of evaluation is a tool for inspecting an area of interest. Evaluations can be used formatively (to make certain improvements), summatively (to help make decisions about an intervention), for learning purposes, to inform strategy, and to link to the strategic intent of M&E activity. Its function is broad since it looks at how best to improve educational or other outcomes.
Evaluation is about gathering evidence to assess the design, implementation, and results of a programme, looking at both positive and negative factors, and weighing them up in terms of relevance, effectiveness, impact, and ultimately sustainability. Using various performance indicators and metrics, it collects and analyses information, with the aim of determining where improvements can be made to a project, and whether a project should be continued, scaled, or halted.
It is usually an external process, requiring an objective assessment from an independent party, although self-evaluation is also possible. It allows a donor to assess return on investment – a crucial factor in determining how or whether to work with a beneficiary.
Evaluation is not just an end process. It can take place at any stage of the programme – as with process evaluation – but is usually understood in terms of whether a programme’s goals have been achieved or not (this is where confusion about evaluation can creep in). Evaluation guides decision-making and allows programme managers to learn from their failures as well as their successes. It allows them to reflect on lessons learnt and how to design more impactful programmes.
It is a good idea to plan for evaluations as they can be costly. An implementation evaluation can fix problems in a programme, but this can be avoided by good programme design upfront.
What are the similarities and differences between monitoring and evaluation?
Monitoring allows us to measure progress towards goals, but it does not calculate impact, or show the extent to which results have been achieved. It is not a time-bound procedure and involves frequent measuring and tracking. By contract, evaluation is infrequent and can take place both during and after an intervention. It is also able to show causality – which interventions have played a role in reaching the target – something monitoring cannot do.
What are the similarities between research and evaluation?
Although evaluation and research are somewhat similar, as they employ similar tools and methods, they are differentiated disciplines. Although research is helpful in terms of expanding scientific knowledge, evaluation provides concrete feedback on specific programmes with the aim of influencing those programmes. It enlarges an understanding of the findings in a field-specific, rather than a theory-dependent, manner.
Evaluation should be treated as an independent discipline, although it must meet certain rigorous standards and criteria. Evaluation does not draw broad conclusions that support general laws or theories, and it is useful for projects only.
What is a theory of change?
A theory of change is a methodology that is adopted during the planning stages of a programme to understand how actions within an intervention contribute to outcomes. It helps to establish the chain of events or causality at the heart of a programme’s success or failure. Why did the project not work? What assumptions were made that contributed to the failure of the project?
Social development programmes are complex and multi-dimensional. Some assumptions are made but poorly articulated, if at all. There is a lack of clarity about how changes unfold, and what sequences of changes contribute to events. A theory of change looks at how change happens within different contexts and clarifies people’s roles in contributing to change. It helps a programme manager to define as well as test the critical assumptions made. It provides a roadmap to help a programme manager get to the destination.
An example might be assuming that all teachers at a rural school would be able to attend a workshop in the nearest town, whereas they might not be able to do so if they do not have money for a taxi on the day in question. Clarifying underlying assumptions changes how one approaches a programme, as one looks for supporting evidence that the programme will be able to go ahead as planned.
A theory of change is very useful when it comes to M&E, allowing for a rigorous review of the roadmap, and enquiring whether it needs refinement or redrawing. A good M&E framework makes explicit the theory of change.
How do we design an M&E framework?
When designing an M&E framework, you should consider:
- Value-based decisions – your strategic intent, educational philosophy, or development strategy
- Evidence-based decisions – what research says about what works best
- Resource-based decisions – do you have the necessary resources to achieve your goals?
References:
- Demystifying Monitoring and Evaluation Jennifer Bisgard, co-founder of M&E consultancy Khulisa Management Services, explains why M&E needs more rigorous design and consistent implementation.
- Evaluation and Research: Differences and Similarities This 2003 article in The Canadian Journal of Program Evaluation provides a useful distinction between evaluation and research, indicating that the two are different disciplines and should be independent of one another.
- Are organisations learning from their M&E? Is increased investment in M&E feeding back into projects to improve them? This was debated in a breakaway session at Trialogue’s CSI Conference 2015.
- In Conversation with Lettie Miles and Lauren Fok on Conducting M&E at Zenex Foundation The value of monitoring is that we can better understand the context and beneficiaries in which we implement projects. We can see the project in action and identify gaps in implementation and capacity issues. Lettie Miles and Lauren Fok, Programme Managers at the Zenex Foundation, explain how monitoring can help to manage risks within a project.
- Monitoring, Evaluation and Learning Systems Robert Worthington, director at social enterprise Kwantu, introduces Monitoring, Evaluation and Learning (MEL) Systems, the importance thereof, the different types that exist, and what organisations should consider when deciding on MEL systems.