Aman Khullar
ABSTRACT
Voice-based discussion forums where users can record audio messages which are then published for other users to listen and comment, are often moderated to ensure that the published audios are of good quality, relevant, and adhere to editorial guidelines of the forum. There is room for the introduction of AI-based tools in the moderation process, such as to identify and filter out blank or noisy audios, use speech recognition to transcribe the voice messages in text, and use natural language processing techniques to extract relevant metadata from the audio transcripts. We design such tools and deploy them within a social enterprise working in India that runs several voice-based discussion forums. We present our findings in terms of the time and cost-savings made through the introduction of these tools, and describe the feedback of the moderators towards the acceptability of AI-based automation in their workflow. Our work forms a case-study in the use of AI for automation of several routine tasks, and can be especially relevant for other researchers and practitioners involved with the use of voice-based technologies in developing regions of the world.
- Vani Viswanathan Aaditeshwar Seth. 2020. ‘What Covid-19 Means To Us’ Voices from the Indian Hinterland. https://www.theindiaforum.in/article/what-covid-19-means-usGoogle Scholar
- Dipanjan Chakraborty, Mohd Sultan Ahmad, and Aaditeshwar Seth. 2017. Findings from a civil society mediated and technology assisted grievance redressal model in rural India. In Proceedings of the Ninth International Conference on Information and Communication Technologies and Development. 1–12.Google ScholarDigital Library
- Dipanjan Chakraborty, Akshay Gupta, and Aaditeshwar Seth. 2019. Experiences from a mobile-based behaviour change campaign on maternal and child nutrition in rural India. In Proceedings of the Tenth International Conference on Information and Communication Technologies and Development. 1–11.Google ScholarDigital Library
- Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You can’t stay here: The efficacy of reddit’s 2015 ban examined through hate speech. Proceedings of the ACM on Human-Computer Interaction 1, CSCW(2017), 1–22.Google ScholarDigital Library
- Eshwar Chandrasekharan, Mattia Samory, Shagun Jhaver, Hunter Charvat, Amy Bruckman, Cliff Lampe, Jacob Eisenstein, and Eric Gilbert. 2018. The Internet’s hidden rules: An empirical study of Reddit norm violations at micro, meso, and macro scales. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–25.Google ScholarDigital Library
- Kate Crawford. 2016. Can an algorithm be agonistic? Ten scenes from life in calculated publics. Science, Technology, & Human Values 41, 1 (2016), 77–92.Google ScholarCross Ref
- Kate Crawford and Tarleton Gillespie. 2016. What is a flag for? Social media reporting tools and the vocabulary of complaint. New Media & Society 18, 3 (2016), 410–428.Google ScholarCross Ref
- Andrew Cross, Nakull Gupta, Brandon Liu, Vineet Nair, Abhishek Kumar, Reena Kuttan, Priyanka Ivatury, Amy Chen, Kshama Lakshman, Rashmi Rodrigues, 2019. 99DOTS: a low-cost approach to monitoring and improving medication adherence. In Proceedings of the Tenth International Conference on Information and Communication Technologies and Development. 1–12.Google ScholarDigital Library
- Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. 2020. A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.Google ScholarDigital Library
- Theodoros Giannakopoulos. 2015. pyaudioanalysis: An open-source python library for audio signal analysis. PloS one 10, 12 (2015), e0144610.Google ScholarCross Ref
- Theodoros Giannakopoulos. 2020. PyAudioAnalysis audio features. https://github.com/tyiannak/pyAudioAnalysis/wiki/3.-Feature-ExtractionGoogle Scholar
- Google. 2021. Dialog Flow. https://cloud.google.com/dialogflowGoogle Scholar
- Google. 2021. Speech To Text. https://cloud.google.com/speech-to-textGoogle Scholar
- Robert Gorwa, Reuben Binns, and Christian Katzenbach. 2020. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society 7, 1 (2020), 2053951719897945.Google ScholarCross Ref
- Guodong Guo and Stan Z Li. 2003. Content-based audio classification and retrieval by support vector machines. IEEE transactions on Neural Networks 14, 1 (2003), 209–215.Google Scholar
- Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, 2017. CNN architectures for large-scale audio classification. In 2017 ieee international conference on acoustics, speech and signal processing (icassp). IEEE, 131–135.Google Scholar
- Map My India. 2021. Map My India. https://www.mapmyindia.com/Google Scholar
- Mira Johri, Sumeet Agarwal, Aman Khullar, Dinesh Chandra, Vijay Sai Pratap, Aaditeshwar Seth, and the Gram Vaani Team. 2021. The first 100 days: how has COVID-19 affected poor and vulnerable groups in India?Health Promotion International (05 2021). https://doi.org/10.1093/heapro/daab050 arXiv:https://academic.oup.com/heapro/advance-article-pdf/doi/10.1093/heapro/daab050/37949360/daab050.pdfdaab050.Google Scholar
- Zahir Koradia, Piyush Aggarwal, Aaditeshwar Seth, and Gaurav Luthra. 2013. Gurgaon idol: A singing competition over community radio and IVRS. In Proceedings of the 3rd ACM Symposium on Computing for Development. 1–10.Google ScholarDigital Library
- Cliff Lampe and Paul Resnick. 2004. Slash (dot) and burn: distributed moderation in a large online conversation space. In Proceedings of the SIGCHI conference on Human factors in computing systems. 543–550.Google ScholarDigital Library
- Honglak Lee, Peter Pham, Yan Largman, and Andrew Ng. 2009. Unsupervised feature learning for audio classification using convolutional deep belief networks. Advances in neural information processing systems 22 (2009), 1096–1104.Google Scholar
- Lie Lu, Hong-Jiang Zhang, and Hao Jiang. 2002. Content analysis for audio classification and segmentation. IEEE Transactions on speech and audio processing 10, 7(2002), 504–516.Google ScholarCross Ref
- Meghana Marathe, Jacki O’Neill, Paromita Pain, and William Thies. 2015. Revisiting CGNet Swara and its impact in rural India. In Proceedings of the Seventh International Conference on Information and Communication Technologies and Development. 1–10.Google ScholarDigital Library
- Brian McFee, Alexandros Metsai, Matt McVicar, Stefan Balke, Carl Thomé, Colin Raffel, Frank Zalkow, Ayoub Malek, Dana, Kyungyun Lee, Oriol Nieto, Dan Ellis, Jack Mason, Eric Battenberg, Scott Seyfarth, Ryuichi Yamamoto, viktorandreevichmorozov, Keunwoo Choi, Josh Moore, Rachel Bittner, Shunsuke Hidaka, Ziyao Wei, nullmightybofo, Darío Hereñú, Fabian-Robert Stöter, Pius Friesch, Adam Weiss, Matt Vollrath, Taewoon Kim, and Thassilo. 2021. librosa/librosa: 0.8.1rc2. https://doi.org/10.5281/zenodo.4792298Google Scholar
- Gram Vaani Community Media. 2020. Lockdown Chronicle: The story of a Migrant workers’ platform across India’s lockdown. https://drive.google.com/file/d/1ViL56UlX5g-AGddriF4N2bXAJi4LHXNM/viewGoogle Scholar
- Aparna Moitra, Vishnupriya Das, Gram Vaani, Archna Kumar, and Aaditeshwar Seth. 2016. Design lessons from creating a mobile-based community media platform in Rural India. In Proceedings of the Eighth International Conference on Information and Communication Technologies and Development. 1–11.Google ScholarDigital Library
- Preeti Mudliar, Jonathan Donner, and William Thies. 2012. Emergent practices around CGNet Swara, voice forum for citizen journalism in rural India. In Proceedings of the Fifth International Conference on Information and Communication Technologies and Development. 159–168.Google ScholarDigital Library
- David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes 30, 1 (2007), 3–26.Google ScholarCross Ref
- Government of India. 2011. Census Data. https://censusindia.gov.in/2011-common/censusdata2011.htmlGoogle Scholar
- Neil Patel, Deepti Chittamuru, Anupam Jain, Paresh Dave, and Tapan S Parikh. 2010. Avaaj otalo: a field study of an interactive voice forum for small farmers in rural india. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 733–742.Google ScholarDigital Library
- Karol J. Piczak. [n.d.]. ESC: Dataset for Environmental Sound Classification. In Proceedings of the 23rd Annual ACM Conference on Multimedia (Brisbane, Australia, 2015-10-13). ACM Press, 1015–1018. https://doi.org/10.1145/2733373.2806390Google ScholarDigital Library
- Polyglot. 2021. Polyglot. https://polyglot.readthedocs.io/en/latest/Google Scholar
- Agha Ali Raza, Mansoor Pervaiz, Christina Milo, Samia Razaq, Guy Alster, Jahanzeb Sherwani, Umar Saif, and Roni Rosenfeld. 2012. Viral entertainment as a vehicle for disseminating speech-based services to low-literate users. In Proceedings of the Fifth International Conference on Information and Communication Technologies and Development. 350–359.Google ScholarDigital Library
- Marietje Schaake and Rob Reich. 2021. Election 2020:Content Moderation and Accountability. https://fsi-live.s3.us-west-1.amazonaws.com/s3fs-public/hai_cyberpolicy_election_3_v1.pdfGoogle Scholar
- A Seth, A Gupta, A Moitra, D Kumar, D Chakraborty, L Enoch, O Ruthven, P Panjal, RA Siddiqi, R Singh, 2020. Reflections from Practical Experiences of Managing Participatory Media Platforms for Development. In Proceedings of the 2020 International Conference on Information and Communication Technologies and Development. 1–15.Google ScholarDigital Library
- Gram Vaani. 2021. COVID-19 response services. https://gramvaani.org/?p=3631Google Scholar
- Gram Vaani. 2021. Gram Vaani. https://gramvaani.org/Google Scholar
- Gram Vaani. 2021. ‘Mobile Vaani. http://mobilevaani.inGoogle Scholar
- Aditya Vashistha, Edward Cutrell, Gaetano Borriello, and William Thies. 2015. Sangeet swara: A community-moderated voice forum in rural india. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 417–426.Google ScholarDigital Library
- Aditya Vashistha, Abhinav Garg, and Richard Anderson. 2019. Recall: Crowdsourcing on basic phones to financially sustain voice forums. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
- Aditya Vashistha, Pooja Sethi, and Richard Anderson. 2017. Respeak: A voice-based, crowd-powered speech transcription system. In Proceedings of the 2017 CHI conference on human factors in computing systems. 1855–1866.Google ScholarDigital Library
- Aditya Vashistha, Pooja Sethi, and Richard Anderson. 2018. BSpeak: An accessible voice-based crowdsourcing marketplace for low-income blind people. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.Google ScholarDigital Library
- Aditya Vashistha and William Thies. 2012. {IVR} Junction: Building Scalable and Distributed Voice Forums in the Developing World. In 6th USENIX/ACM Workshop on Networked Systems for Developing Regions ({NSDR} 12).Google Scholar
- Sida I Wang and Christopher D Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 90–94.Google Scholar
- Deepika Yadav, Mayank Gupta, Malolan Chetlur, and Pushpendra Singh. 2018. Automatic annotation of voice forum content for rural users and evaluation of relevance. In Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies. 1–11.Google ScholarDigital Library
Index Terms
- Experiences with the Introduction of AI-based Tools for Moderation Automation of Voice-based Participatory Media Forum
Index terms have been assigned to the content through auto-classification.
Recommendations
Moderation Challenges in Voice-based Online Communities on Discord
Online community moderators are on the front lines of combating problems like hate speech and harassment, but new modes of interaction can introduce unexpected challenges. In this paper, we consider moderation practices and challenges in the context of ...
Moderation Visibility: Mapping the Strategies of Volunteer Moderators in Live Streaming Micro Communities
IMX '21: Proceedings of the 2021 ACM International Conference on Interactive Media ExperiencesVolunteer moderators actively engage in online content management, such as removing toxic content and sanctioning anti-normative behaviors in user-governed communities. The synchronicity and ephemerality of live-streaming communities pose unique ...
The Unsung Heroes of Facebook Groups Moderation: A Case Study of Moderation Practices and Tools
CSCWVolunteer moderators have the power to shape society through their influence on online discourse. However, the growing scale of online interactions increasingly presents significant hurdles for meaningful moderation. Furthermore, there are only limited ...
Comments