From customer support and appointment reminders to service updates, Voice AI evolved far beyond traditional IVR (Interactive Voice Response) systems. Today’s conversational agents can hold natural conversations, switch languages, handle interruptions, and respond in real time. As these capabilities mature, we began exploring whether Voice AI could also transform how organisations engage with communities at scale.
Can Voice AI make monitoring, evaluation, and learning more effective while remaining practical and responsible in community settings?
Community conversations are rarely linear, they are conversational, unstructured, and require patience. In our first interactions with Voice AI, the agent struggled with interruptions, responded out of context, and often rushed through the interaction. The agent did not wait long enough before responding. If someone paused for a couple of seconds to think, the agent would jump in with “Samajh gayi” (Understood), assuming the respondent had finished speaking. The actual response only emerged when the agent followed up to understand what they meant. This led us to redesign several questions with guided follow-up prompts rather than relying only on predefined response options.
We provided the Voice AI agent with as much programme context as possible so it could respond more effectively to non-standard or unexpected questions. Giving respondents a little more time before the agent responded made conversations feel considerably more natural, especially for open-ended questions. We revisited the questionnaire with programme teams to identify the questions that mattered most. We introduced skip question logic so respondents were not asked irrelevant questions.
After several rounds of iteration, we deployed our first Voice AI pilot to conduct an impact assessment for a scholarship programme. The objective was to understand how the scholarship had influenced students’ lives. Working with our implementation partner, we informed students about the calls in advance and scheduled them over weekends based on their availability.
The agent reached out to 100 scholarship recipients. Of these, 79 answered the call and 51 completed the full 15-question survey, with conversations lasting an average of six minutes. More encouraging than the completion rates was the quality of the responses. Students shared thoughtful reflections on how the scholarship had shaped their education and career choices. In one conversation, a student explained that they had financed their education through both personal savings and the sale of family land. While the Voice AI agent captured the reference to the land sale, it missed the contribution from personal savings.
Encouraged by these results, we expected our next deployment to perform similarly. In an apprenticeship programme, we again reached out to 100 respondents. Around 40 answered the calls, but only 13 completed the survey.
One of the biggest learnings from the pilots was that programme context shapes participation. In the ongoing scholarship programme, students remained connected to the implementation partner, remembered the programme clearly, and had a stronger incentive to share their experiences. In contrast, the apprenticeship programme had ended six months earlier; respondents had weaker recall, fewer touchpoints with the implementing partner, and less incentive to stay on the call.
What we learned across pilots
Over the last few months, we have tested Voice AI across multiple impact assessments monitoring and verification exercises. Every deployment has taught us something new. With each pilot, we have refined the questionnaire, improved the conversational flow, and incorporated feedback from respondents, implementation partners, and our project teams. One advantage of working with LLM-based systems is that they improve with every additional layer of context.
One of the biggest learnings from this journey is that the success of a Voice AI deployment is shaped by three interconnected factors: the capability of the Voice AI technology, the programme context, and the operational design of the deployment.
Across our pilots, we have now made over 9,000 calls and completed over 1000+ conversations across impact assessments, monitoring and verification calls. Completion rates have ranged from 10% to 65%, depending on the nature of the programme and the deployment approach. One of our recent uses of Voice AI involved verifying outcomes from a large skilling programme. We reached out to 6,000 candidates, about 4,000 respondents answered the call, and 806 completed the full survey.
Based on our pilots, Voice AI capabilities helped us listen to communities at scale while generating insights to support stronger programme management:
1. Faster data collection: The ability to reach thousands of beneficiaries simultaneously enabled us to collect structured feedback within two days, an exercise that would typically take several weeks through manual calling.
2. Programme-level outcome verification: Verified training completion, job offers, placements, and retention for 800+ candidates across 20 training partners. The data helped programme teams identify differences in outcomes across training partners and where additional support may be required.
3. Multilingual deployment and learning: Expanded deployments across multiple regional languages Marathi, Kannada, Tamil, and Telugu, with each pilot helping refine language and conversational performance.
Reflections from our Journey
The next phase is to understand how Voice AI performs across different community segments, such as farmers and women entrepreneurs. We are also exploring whether it can support longer, more in-depth conversations, or enable two-way communication with communities through reminders and updates. This opens up several possibilities for non-profits. Voice AI could help organisations stay connected with communities at scale through regular check-ins, programme nudges, and sharing relevant programme learnings back with communities. We are also beginning to explore multimodal approaches that combine Voice AI with channels such as messaging platforms and web-based surveys. Giving respondents the flexibility to choose when and how they respond could improve accessibility, participation, and the overall community engagement experience.
Want to explore how Voice AI can help you listen to communities at scale? Get in touch with us at getimpactinsights@sattva.co.in to explore how Voice AI can strengthen your data collection and community engagement.
Disclaimer: Sattva follows a cloud-first, secure data processing model using approved SaaS platforms with encryption, access controls, and logging safeguards. All new tools undergo internal data and information security evaluation before deployment. We obtained explicit consent from each respondent before proceeding with the interaction, and follow all applicable data privacy protocols.
