Since the CSR Law came into force in 2013, there has been a lot of conversation on spending across companies, social issues that have got the maximum funding and attention and industry trends in spending. While financial measurement is a great indicator of CSR uptake, the purpose of CSR – social impact, needs to equally be measured to understand how much of a difference the programs made on the ground.
At Sattva last week, we hosted a webinar on a topic we think about and work on a lot – Monitoring and Evaluation (M&E) of Social Programs. Between 2 panelists, Rathish Balakrishnan and Pratyush Panda, who collectively bring over 2 decades of experience in working on social program design, implementation and M&E and a lot of interesting audience questions, there were many useful takeaways. Here are 5 that resonated with me the most, especially since I am in the middle of designing an M&E framework for a healthcare/WaSH social project we are working on and we faced many of these questions as a team.
M&E: 5 tips to do it better (from the Sattva webinar)
1. The aim is not to Prove, but to Improve
“M&E is not a process for proving impact for stakeholders, but an ongoing tool to improving the impact of programs. It is a steering model for the leadership of an organization to make decisions based on the right data. Is my program creating value? What should be done to improve performance and achieve results? This should be the purpose of M&E.”
2. 90% of the battle is knowing what to measure
M&E is not an operational conversation; it is a leadership level conversation. It is critical to determine the 3-4 key indicators that are most important for your program, to understand if the program is working on the ground. This involves plotting
1.The long-term impact you want to make
2.Mid-terms outcomes which will get you there
3.Short-term outputs of your program
Once the outputs, outcomes and impact are discussed, M&E becomes a question of implementation which is a lot easier. Hence, the first step towards designing an effective M&E is a rigorous process to arrive at what you want to measure in relation to what you want to create.
3. Follow the LIDO rule: Less is better
LIDO is a budget supermarket chain which operates on a policy of fixed inventory. If something comes in, something has to go out. There is a cost to every data collection and you need to know what you are going to do with it. For every indicator that you are looking to measure, there is cost of data collection, cost of analysis and leadership bandwidth to see if this is making sense or not.
The webinar panelists recommended that M&E practitioners look at 5-6 key indicators. Outputs might be more in number, but outcomes have to be limited in number in order to make your M&E manageable and effective. Outputs are the activities that a program undertakes – eg: number of people trained on skill, number of health camps conducted.
Outcomes on the other hand are the results of those activities done over a sustained period of time. Eg: reduction in disease, increase in incomes etc.
4. Qualitative vs quantitative measurement
It is important to quantify progress through numbers that directly indicate the value being delivered or the situation being improved. Today, there are standards available for everything one can measure quantitatively – in education, healthcare, skill development and so on.
However, Qualitative assessments help us to really get to the bottom of the quantitative date and understand these numbers.
Have an ongoing rigorous system for collecting quantitative data. Once in a quarter or so, structure your operational reviews such that qualitative inputs help contextualize the quantitative data.
5. Leverage existing standards of measurement
There are existing global standards to measure all Human Development Indicators (HDIs). There are 17 UN Sustainable Development Goals with 169 targets. There are several National standards followed by the Government in India to assess the quality of schools, PHCs, anganwadis and other institutions. It is a good practice to look at standard indicators for measuring social parameters. For eg: There is a fixed and well accepted theory on reduction of IMR. If your program is looking at IMR reduction, not only is it going to be easier to relate to the accepted standard, but you will also find the district/State level data on IMR measurement directly usable for your program.
It is a good practice to align all your indicators to standard terms of measurement.