By Dr Fatima Adam
Why are education evaluations becoming increasingly important?
There is no doubt that improving education is a high priority for all key stakeholders in South Africa. This is evident through the huge financial investment and numerous initiatives underway to improve education delivery and outcomes. The South African Government allocated R320 billion for education in 2017. This is approximately 6.4% of its Gross Domestic Product (GDP), which is higher than other developing countries. In addition, there has been a significant increase in private sector support for education development, estimated at R25 billion.
Despite these huge investments and efforts, education outcomes are not improving at the expected rates. This is evident by the poor performance of South African learners on a range of international benchmarking mathematics and language tests. South African learners continue to perform at the lower end of the spectrum even when compared to their counterparts in other developing countries.
This has led to the widespread increase in interest and support for undertaking evaluations of education projects/interventions in order to better understand what works/doesn’t work, under what conditions, as well as how and why things are working or not working. This is leading toward a path of using evidence to drive decision making in the design and implementation of education improvement initiatives.
Why are evaluations important for Zenex?
The Zenex Foundation has always been committed to undertaking evaluations of its education initiatives. Over the past 20 years, the Zenex Foundation has commissioned over 70 evaluations. These ranged from the process to outcomes as well as impact evaluations. The results were used extensively to inform Zenex’s own work as well as share its learnings with others. Through the journey, we have also recognised the importance of carefully curating and managing the commissioning of evaluations.
How do we add value through commissioning processes?
For Zenex, it is not about whether we should commission evaluations but rather about how we commission them so that they are better able to inform our strategies and programmes as well as inform education policies and practices.
Whilst the Foundation relies on expert evaluators to undertake project/ programme evaluations, it believes that quality evaluations start at the point of commissioning. High-quality evaluations are generally underpinned by high quality commissioning processes. Although commissioning doesn’t require the same expertise as evaluators, commissioners should have a basic knowledge of the following:
- evaluation and research processes and design;
- research methods; and
- types of evaluations.
Without this basic knowledge, it is very difficult for commissioners to develop a scope of work and effectively manage the evaluation contracts.
In addition to this basic knowledge, commissioning good evaluations requires consideration of four critical factors. These factors form the scope of work to evaluators.
The first is to develop a clear and explicit description of the project, including its theory of change. The second is ensuring a clear purpose for commissioning the evaluation. Evaluations can be commissioned for various purposes, ranging from improving programme designs and implementation, informing decision-making, to establishing outcomes and impacts, and contributing to knowledge. The third is developing clear questions that are aligned with the purpose. The fourth is knowledge of the costs of the different type of evaluations. For instance, evaluations that test young learners in diverse languages, are geographically spread and in rural areas are more costly.
In the Zenex context, the scope of work informs the call for proposals from evaluators who are interested in undertaking the work. Zenex then short-lists and selects their preferred provider based on an assessment of their proposals against a set of criteria. Once the evaluator is appointed, Zenex has an engagement with the evaluators on whether the project is valuable, whether the questions can be answered and how the evaluator proposes to answer these questions. Often engagement between the evaluator and Zenex involves a robust discussion about sampling, design and methods and the trades offs that need to be made based on cost and feasibility.
Throughout the evaluation process, we remain clear about our role as a commissioner and this guides our level of engagement. Zenex typically has extensive upfront involvement in design and methods and then later involvement in understanding findings and engaging in dissemination and utilisation.
Solving development problems is an increasingly complex endeavour, which calls for a methodical approach of deciding on the appropriate course of action. This approach must be driven by systematic evidence-based approaches to programme design and delivery. This is achieved by implementing a systematic monitoring and evaluation strategy and plan to inform all aspects of the organisation’s work.
Source: Lessons and Reflections about Implementing M&E in South Africa: An anthology of articles by Zenex staff, M&E experts and NGOs. http://www.zenexfoundation.
