N R Narayana Murthy, the iconic founder of Infosys is famously known to have said, “In God we trust. Everyone else bring data to the table.”
Data is critical to informed decision making and social programs are no different.
A strong Monitoring & Evaluation system is vital to building effective and impactful social programs. M&E can help track progress and outputs systematically, measure outcomes and help you steer your social program in the right direction.
M&E is essential across the social program ecosystem: for business leadership, CSR managers and teams, NGO partners, program planners, implementers and funders, to acquire the information and understanding they need to make informed decisions about programmes.
However, we understand that social impact practitioners, be it CSRs or non-profits, have many challenges to overcome in setting up and deploying an effective M&E system, from making decisions on indicators that have to be monitored to cost of evaluation and ensuring quality of data collection.
Our webinar panelists Pratyush Panda, Head, CSR, ACC Limited and Rathish Balakrishnan, co-founder and Managing Partner, Sattva addressed many of these concerns for practitioners from the wealth of experience they have in designing and implementing M&E.
Watch the webinar on M&E:
Here are 12 key takeaways that emerged from the 1-hour discussion on M&E:
1. Not to prove, but to Improve
“M&E is not a process for proving impact for stakeholders, but an ongoing tool to improving the impact of programs. It is a steering model for the leadership of an organization to make decisions based on the right data. Is my program creating value? What should be done to improve performance and achieve results? This should be the purpose of M&E.”
2. 90% of the battle is knowing what to measure
M&E is not an operational conversation; we say it is a leadership level conversation. It is critical to determine the 3-4 key indicators that are most important for your program, to understand if the program is working on the ground. This involves plotting
- The long-term impact you want to make
- Mid-terms outcomes which will get you there
- Short-term outputs of your program
One the outputs, outcomes and impact are discussed, M&E becomes a question of implementation which is a lot easier. Hence, the firs step towards designing an effective M&E is a rigorous process to arrive at what you want to measure in relation to what you want to create.
3. Follow the LIDO rule: Less is better
LIDO is a budget supermarket chain which operates on a policy of fixed inventory. If something comes in, something has to go out. There is a cost to every data collection and you need to know what you are going to do with it. For every indicator that you are looking to measure, there is cost of data collection, cost of analysis and leadership bandwidth to see if this is making sense or not.
We recommend that you look at 5-6 key indicators. Outputs might be more in number, but outcomes have to be limited in number in order to make your M&E manageable and effective.
4. Qualitative vs quantitative
It is important to quantify progress through numbers that directly indicate the value being delivered or the situation being improved. Today, there are standards available for everything one can measure quantitatively – in education, healthcare, skill development and so on.
However, Qualitative assessments help us to really get to the bottom of the quantitative date and understand these numbers.
Have an ongoing rigorous system for collecting quantitative data. Once in a quarter or so, structure your operational reviews such that qualitative inputs help contextualize the quantitative data.
5. Leverage existing standards of measurement
There are existing global standards to measure all Human Development Indicators (HDIs). There are 17 UN Sustainable Development Goals with 169 targets. There are several National standards followed by the Government in India to assess the quality of schools, PHCs, anganwadis and other institutions. It is a good practice to look at standard indicators for measuring social parameters. For eg: There is a fixed and well accepted theory on reduction of IMR. If your program is looking at IMR reduction, not only is it going to be easier to relate to the accepted standard, but you will also find the district/State level data on IMR measurement directly usable for your program.
It is a good practice to align all your indicators to standard terms of measurement.
6. Make the M&E system your single source of truth
One of the pitfalls we have seen in organisations is this – there are very detailed, wonderful M&E systems designed and implemented – and no one uses it. To avoid this, there is a simple rule we recommend – the operational review document MUST be the M&E framework. The M&E data is what must be presented and discussed at all levels of strategy and operational decision making in the organization.
7. Align people and process before technology
Technology is a very strong lever to enable M&E systems to work effectively. But before you bring in technology, you need to firm up the process and align your people around it. This is key for quality data collection. Once people and process are aligned, tech automatically becomes a very good enabler.
8. Measure the impact in money-terms
M&E has been given reasonable importance in the Section 135 of the Indian Companies Act. It is also true that the Corporate sector looks to understand returns on any kind of investment it makes, and CSR is no exception. This makes M&E that much more critical in a CSR program – you need to justify the result of your spending not just to the Government but also to all your stakeholders. How does the CSR impact the bottom-line of the company is something the leadership may well be looking to understand.
9. Start with a Logical Framework Analysis from Day 1 of your project
The Logical Framework Approach is a commonly used tool for design, implementing and monitoring development projects. The LFA lays out what are called – OVIs – Objectively Verifiable Indicators. Our recommendation is that you have 3 to 5 such OVIs in each of your projects. Make it 40% quantitative, 60% qualitative for a good mix.
10. Engage an independent party for your social audit process
Pratyush Panda underscores the importance of having an independent assessment done of your projects every year or so. Hire a third party expert to conduct an unbiased and objective analysis of your CSR performance, he recommends.
11. Getting your (non-tech) field team to execute M&E through simple tech
There are several tools available in the market today in India to enable the process of effective data collection including mobile apps, forms and survey tools. Right from google forms to more sophisticated tools like SocialCops, there is a range of products that help organisations gather data. An easy-to- understand and easy-to-use tool for data collection can help your field staff execute M&E operations well enough for the right metrics to come in.
Where technology and expertise become important is during the process of data analysis and reporting.
“Align your team on why they are collecting the data – this can significantly increase the quality of output.”
“My operations team was more comfortable with a handheld method for gathering data. Converting our M&E into an app made the cycle of data collection easier for us. However, finalise your tools in the first year of your program, so that you could gather good data over the years.” – Pratyush Panda
12. Determine the ideal duration for you to start evaluation
Monitoring is an ongoing activity and has to begin from day 1 of the program. Evaluation of a program however depends on sector and issue. For example, in skill development programs, a cycle from training to job placement could well complete in 3 months, which means that your M&E should be a lot more agile. For sectors like health and education, you might have to wait longer to see results.
“If you want to measure impact, keep a 3-5 years horizon with an interim evaluation of the project once a year.”
Have a question about M&E? We could get it answered for you – write to us at email@example.com
Join our mailing list to get to know about upcoming webinars and events.