Talking Book user feedback is a valuable source of qualitative data—allowing our partners to learn from communities
What do you do with lots of qualitative data? This is the question the Amplio team has posed in an effort to amplify community voices in monitoring and evaluation efforts.
The Amplio Talking Book's built-in microphone allows users to record their feedback. Community members can share their questions, comments, and opinions for program staff to hear. Depending on the program, Talking Books can collect thousands of user feedback recordings. Each message offers insights into how people are living, what aspects of the program are working, and what challenges communities are facing.
How user feedback amplifies community voices
With user feedback, our partners can hear directly from community members in their own words and voices. Over the years of implementing Talking Book programs, we have seen our partners apply the user feedback function in a variety of ways:
In northern Ghana, UNICEF Ghana's communication for development (C4D) program taught community members how to record user feedback and encouraged them to do so. Participants recorded their questions and comments in response to UNICEF's Talking Book content. Many people also described their personal challenges and experiences. Some were inspired to record their own songs about new knowledge, attitudes, and behaviors they were learning. User feedback gave UNICEF a deeper understanding of the specific needs of communities, including barriers to behavior change, and supported their C4D strategy. (You can read a case study here.)
For CARE Ethiopia’s Act With Her project, community facilitators used the Talking Book user feedback function to record what they did for the day (their program activities) and track attendance numbers for the listening sessions they conducted.
Other partners have included questions at the end of each Talking Book message, prompting listeners to record their thoughts about what they were learning.
Lisa Zook explains a new system for processing Talking Book user feedback.
Designing a user feedback processing application
User feedback can amplify community voices for participatory monitoring and evaluation. But listening to, translating, transcribing, and making sense of thousands of recordings is a major undertaking. It requires time and resources. To address this, Amplio recently revamped its approach to processing user feedback. Our goal is to help Talking Book partners more easily and efficiently sample, listen to, sort, and process user feedback, so they can put it to use for evidence-informed programming decisions.
We are currently piloting a web-based application that allows multiple analysts to log in simultaneously to access and process pieces of user feedback. The application gives each analyst a randomly selected user feedback recording to listen to, transcribe, and answer a few questions about. After answering the questions, you simply press ‘submit’, the page refreshes, and you can process the next piece of user feedback.
A new application will help analysts track and process Talking Book user feedback.
User feedback processing features
We’ve built in some fun features to make the process more efficient.
You can speed up or slow down a recording. You can also create a loop—i.e, select a segment of the recording and have it play over and over. This is helpful when you're transcribing or trying to understand an accent or capture a point.
A tracker at the top of the webpage tracks the number of messages that you have processed while also tracking the collective team’s progress.
You can customize the analyst questions to align with your program's monitoring and evaluation strategy. This can help identify issues and trends. For example, you could align the questions to key indicators or areas of interest.
A tracker at the top of the webpage tracks the number of messages that you have processed while also tracking the collective team’s progress.
Data visualizations are linked to the results, which helps program teams interpret and more easily act upon the information.
Questions for Talking Book user feedback analysis can be customized.
Partnering to pilot and refine the application
In collaboration with our partners, we piloting the user feedback processing application. With every new partnership, we are asking the following set of questions:
What is your goal for user feedback?
What do you want to learn? How will you use this information?
How will you encourage program participants to record user feedback?
Who is available to analyze the recordings?
What other resources might you need?
Ongoing collaboration, testing, and feedback from our partners will help us design an appropriate approach to processing user feedback. We would be remiss not to point out that qualitative data is often a great resource precisely because it does not fit in a pre-described indicator table and isn’t constrained by structured questions.
Balancing the task of processing large quantities of recorded user feedback with the desire to hear directly from participants, in their own words and voices, is something we are still exploring. We are excited that several new Talking Book partners are taking on this challenge alongside us, recognizing the value of this data source.
Lisa Zook is Amplio's Monitoring, Evaluation, and Learning Advisor. She is also the director of research and impact for InformED International. Lisa presented a lightning talk about our new user feedback processing system at the 2021 Global Digital Health Forum.