Yiwen Dong
Yuyan Wu
Jesse R Codling
ABSTRACT
Crowd monitoring involves tracking and analyzing the behavior of large groups of people in large-scale public spaces, such as sports games. In sports stadiums, understanding audience reactions to the games and their distribution around the public facilities is important for ensuring public safety and security, enhancing the game experience, and improving crowd management. Recent crowd-crushing incidents (e.g., Kanjuruhan Stadium disaster, Seoul Halloween Stampede) have caused 100+ deaths in a single event, calling for advancements in crowd monitoring methods. Existing monitoring approaches include manual observation, wearables, video-, audio-, and WiFi-based sensing. However, few meet the practical needs due to their limitations in cost, privacy protection, and accuracy.
In this paper, we introduce GameVibes, a novel method for crowd behavior monitoring using crowd-induced floor vibrations to infer audience reactions to the game (e.g., clapping, stomping, dancing) and crowd traffic (i.e., the number of people entering each door). The main benefits of GameVibes are that it allows continuous, fine-grained crowd monitoring in a cost-effective and non-intrusive way and is perceived as more privacy-friendly. Unlike monitoring an individual person, crowd monitoring involves understanding the overall behavior of a large population (typically more than 1,000), leading to high uncertainty in the vibration data. To overcome the challenge, we first establish the game and facility association to inform the context of crowd behaviors, including 1) game associations (temporal context) between the crowd reaction and the game progress and 2) facility associations (spatial context) between the crowd traffic and facility layouts. Then, we formulate the crowd monitoring problem by converting the conceptual graph of the audience-game-facility association into probabilistic game/facility association models. Through these models, GameVibes first learns the latent representations of the game progress and facility layout through neural network encoders, and then integrates heterogeneous game/facility information and vibration data to estimate crowd behaviors. This mitigates the estimation error due to the uncertainty in vibration data. To evaluate our approach, we conduct 6 real-world deployments for NCAA Pac-12 games at Stanford Maples Pavilion. Our results show that GameVibes achieves a 0.9 F-1 score in crowd reaction monitoring and 9.3 mean absolute error in crowd traffic estimation, which correspond to 10% and 12.2% error reduction, respectively, compared to the baseline methods without context-specific information.
- Ali M Al-Shaery, Shroug S Alshehri, Norah S Farooqi, and Mohamed O Khozium. 2020. In-depth survey to detect, monitor and manage crowd. IEEE Access 8 (2020), 209008–209019.Google Scholar
Cross Ref
- Majd Alwan, Siddharth Dalal, Steve Kell, Robin Felder, 2003. Derivation of basic human gait characteristics from floor vibrations. In 2003 Summer Bioengineering Conference, June. 25–29.Google Scholar
- Majd Alwan, Prabhu Jude Rajendran, Steve Kell, David Mack, Siddharth Dalal, Matt Wolfe, and Robin Felder. 2006. A smart and passive floor-vibration based fall detector for elderly. In 2006 2nd International Conference on Information & Communication Technologies, Vol. 1. IEEE, 1003–1007.Google ScholarCross Ref
- Maria Andersson, Joakim Rydell, and Jorgen Ahlberg. 2009. Estimation of crowd behavior using sensor networks and sensor fusion. In 2009 12th International Conference on Information Fusion. IEEE, 396–403.Google Scholar
- Reza Bahmanyar, Elenora Vig, and Peter Reinartz. 2019. MRCNet: Crowd counting and density map estimation in aerial and ground imagery. arXiv preprint arXiv:1909.12743 (2019).Google Scholar
- Amelie Bonde, Jesse R. Codling, Kanittha Naruethep, Yiwen Dong, Wachirawich Siripaktanakon, Sripong Ariyadech, Akkarit Sangpetch, Orathai Sangpetch, Shijia Pan, Hae Young Noh, and Pei Zhang. 2021. PigNet: Failure-Tolerant Pig Activity Monitoring System Using Structural Vibration. In Proceedings of the 20th International Conference on Information Processing in Sensor Networks (Co-Located with CPS-IoT Week 2021). ACM, Nashville TN USA, 1–13. https://doi.org/10.1145/3412382.3458902Google ScholarDigital Library
- Amelie Bonde, Shijia Pan, Mostafa Mirshekari, Carlos Ruiz, Hae Young Noh, and Pei Zhang. 2020. OAC: Overlapping office activity classification through IoT-sensed structural vibration. In 2020 IEEE/ACM Fifth International Conference on Internet-of-Things Design and Implementation (IoTDI). IEEE, 216–222.Google ScholarCross Ref
- Maria Moitinho de Almeida and Johan von Schreeb. 2019. Human stampedes: An updated review of current literature. Prehospital and disaster medicine 34, 1 (2019), 82–88.Google Scholar
- Yiwen Dong, Amelie Bonde, Jesse R Codling, Adeola Bannis, Jinpu Cao, Asya Macon, Gary Rohrer, Jeremy Miles, Sudhendu Sharma, Tami Brown-Brandl, 2023. PigSense: Structural Vibration-based Activity and Health Monitoring System for Pigs. ACM Transactions on Sensor Networks (2023).Google Scholar
- Yiwen Dong, Jonathon Fagert, and Hae Young Noh. 2023. Characterizing the variability of footstep-induced structural vibrations for open-world person identification. Mechanical Systems and Signal Processing 204 (2023), 110756.Google ScholarCross Ref
- Yiwen Dong, Joanna Jiaqi Zou, Jingxiao Liu, Jonathon Fagert, Mostafa Mirshekari, Linda Lowes, Megan Iammarino, Pei Zhang, and Hae Young Noh. 2020. MD-Vibe: physics-informed analysis of patient-induced structural vibration data for monitoring gait health in individuals with muscular dystrophy. In Adjunct proceedings of the 2020 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2020 ACM international symposium on wearable computers. 525–531.Google ScholarDigital Library
- Jonathon Fagert, Mostafa Mirshekari, Shijia Pan, Linda Lowes, Megan Iammarino, Pei Zhang, and Hae Young Noh. 2021. Structure-and sampling-adaptive gait balance symmetry estimation using footstep-induced structural floor vibrations. Journal of Engineering Mechanics 147, 2 (2021), 04020151.Google ScholarCross Ref
- Victoria Filingeri, Ken Eason, Patrick Waterson, and Roger Haslam. 2017. Factors influencing experience in crowds–the participant perspective. Applied ergonomics 59 (2017), 431–441.Google Scholar
- Renee Glass. 2005. Observer response to contemporary dance. Thinking in four dimensions: Creativity and cognition in contemporary dance (2005), 107–121.Google Scholar
- Sabrina Haque, Muhammad Sheikh Sadi, Md Erfanul Haque Rafi, Md Milon Islam, and Md Kamrul Hasan. 2020. Real-time crowd detection to prevent stampede. In Proceedings of International Joint Conference on Computational Intelligence: IJCCI 2018. Springer, 665–678.Google ScholarCross Ref
- Guido R. Hiertz, Dee Denteneer, Sebastian Max, Rakesh Taori, Javier Cardona, Lars Berlemann, and Bernhard Walke. 2010. IEEE 802.11s: The WLAN Mesh Standard. IEEE Wireless Communications 17, 1 (Feb. 2010), 104–111. https://doi.org/10.1109/MWC.2010.5416357Google ScholarDigital Library
- Nicholas Jarvis, John Hata, Nicholas Wayne, Vaskar Raychoudhury, and Md Osman Gani. 2019. Miamimapper: Crowd analysis using active and passive indoor localization through wi-fi probe monitoring. In Proceedings of the 15th ACM International Symposium on QoS and Security for Wireless and Mobile Networks. 1–10.Google ScholarDigital Library
- Wang Jingying. 2021. A survey on crowd counting methods and datasets. In Advances in Computer, Communication and Computational Sciences: Proceedings of IC4S 2019. Springer, 851–863.Google ScholarCross Ref
- Burak Kantarci and Hussein T Mouftah. 2014. Trustworthy sensing for public safety in cloud-centric internet of things. IEEE Internet of Things Journal 1, 4 (2014), 360–368.Google ScholarCross Ref
- Ellis Kessler, Vijaya VN Sriram Malladi, and Pablo A Tarazaga. 2019. Vibration-based gait analysis via instrumented buildings. International Journal of Distributed Sensor Networks 15, 10 (2019), 1550147719881608.Google ScholarCross Ref
- Brian F Kingshott. 2014. Crowd management: Understanding attitudes and behaviors. Journal of Applied Security Research 9, 3 (2014), 273–289.Google ScholarCross Ref
- Ven Jyn Kok, Mei Kuan Lim, and Chee Seng Chan. 2016. Crowd behavior analysis: A review where physics meets biology. Neurocomputing 177 (2016), 342–362.Google ScholarDigital Library
- Ajay Kumar 2021. Crowd behavior monitoring and analysis in surveillance applications: a survey. Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12, 7 (2021), 2322–2336.Google Scholar
- Sonu Lamba and Neeta Nain. 2017. Crowd monitoring and classification: a survey. In Advances in Computer and Computational Sciences: Proceedings of ICCCCS 2016, Volume 1. Springer, 21–31.Google ScholarCross Ref
- Saurabh Maheshwari and Surbhi Heda. 2016. A review on crowd behavior analysis methods for video surveillance. In Proceedings of the Second International Conference on Information and Communication Technology for Competitive Strategies. 1–5.Google ScholarDigital Library
- Mostafa Mirshekari, Jonathon Fagert, Shijia Pan, Pei Zhang, and Hae Young Noh. 2020. Step-level occupant detection across different structures through footstep-induced floor vibration using model transfer. Journal of Engineering Mechanics 146, 3 (2020), 04019137.Google ScholarCross Ref
- Mostafa Mirshekari, Shijia Pan, Jonathon Fagert, Eve M. Schooler, Pei Zhang, and Hae Young Noh. 2018. Occupant localization using footstep-induced structural vibration. Mechanical Systems and Signal Processing 112 (2018), 77–97. https://doi.org/10.1016/j.ymssp.2018.04.026Google ScholarCross Ref
- Michael S Molloy, Zane Sherif, Stan Natin, and John McDonnell. 2009. Management of mass gatherings. Koenig and Schultz’s Disaster Medicine: Comprehensive Principles and Practices; Schultz, CH, Koenig, KL, Eds (2009), 265–293.Google Scholar
- Shijia Pan, Mario Berges, Juleen Rodakowski, Pei Zhang, and Hae Young Noh. 2019. Fine-grained recognition of activities of daily living through structural vibration and electrical sensing. In Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation. 149–158.Google ScholarDigital Library
- Shijia Pan, Tong Yu, Mostafa Mirshekari, Jonathon Fagert, Amelie Bonde, Ole J Mengshoel, Hae Young Noh, and Pei Zhang. 2017. Footprintid: Indoor pedestrian identification through ambient structural vibration sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1–31.Google ScholarDigital Library
- Yves Reuland, Sai GS Pai, Slah Drira, and Ian FC Smith. 2017. Vibration-based occupant detection using a multiple-model approach. In Dynamics of Civil Structures, Volume 2: Proceedings of the 35th IMAC, A Conference and Exposition on Structural Dynamics 2017. Springer, 49–56.Google ScholarCross Ref
- Nikitas M Sgouros. 2000. Detection, analysis and rendering of audience reactions in distributed multimedia performance. In Proceedings of the eighth ACM international conference on Multimedia. 195–200.Google ScholarDigital Library
- Avinash Sharma, Brian McCloskey, David S Hui, Aayushi Rambia, Adam Zumla, Tieble Traore, Shuja Shafi, Sherif A El-Kafrawy, Esam I Azhar, Alimuddin Zumla, 2023. Global mass gathering events and deaths due to crowd surge, stampedes, crush and physical injuries-lessons from the Seoul Halloween and other disasters. Travel medicine and infectious disease 52 (2023).Google Scholar
- Catherine J Stevens, Emery Schubert, Rua Haszard Morris, Matt Frear, Johnson Chen, Sue Healey, Colin Schoknecht, and Stephen Hansen. 2009. Cognition and the temporal arts: Investigating audience response to dance using PDAs that record continuous data during live performance. International Journal of Human-Computer Studies 67, 9 (2009), 800–813.Google ScholarDigital Library
- Mohammad Yamin, Abdullah M Basahel, and Adnan A Abi Sen. 2018. Managing crowds with wireless and mobile technologies. Wireless Communications and Mobile Computing 2018 (2018).Google Scholar
- Gde Yogadhita and Widiana Agustin. 2023. Football Stampede in Kanjuruhan Stadium from the Perspective of Disaster Preparedness on Mass Casualty Incident: A Case Study of Mass Gathering Event. Prehospital and Disaster Medicine 38, S1 (2023), s78–s79.Google ScholarCross Ref
- Kathryn M Zeitz, Heather M Tan, M Grief, PC Couns, and Christopher J Zeitz. 2009. Crowd behavior at mass gatherings: a literature review. Prehospital and disaster medicine 24, 1 (2009), 32–38.Google Scholar
- Guijuan Zhang, Dianjie Lu, and Hong Liu. 2018. Strategies to utilize the positive emotional contagion optimally in crowd evacuation. IEEE Transactions on Affective Computing 11, 4 (2018), 708–721.Google ScholarDigital Library
Index Terms
- GameVibes: Vibration-based Crowd Monitoring for Sports Games through Audience-Game-Facility Association Modeling
Recommendations
Crowd-based urban characterization: extracting crowd behavioral patterns in urban areas from Twitter
LBSN '11: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Location-Based Social NetworksThe advent of location-based social networking sites provides an open sharing space of crowd-sourced lifelogs that can be regarded as a novel source to monitor massive crowds' lifestyles in the real world. In this paper, we challenge to analyze urban ...
An Experiment in Rule-Based Crowd Behavior for Intelligent Games
ICCIT '09: Proceedings of the 2009 Fourth International Conference on Computer Sciences and Convergence Information TechnologyToday video games and animated movies and robotics involve individual and collective character behavior. To this end, behavior algorithms are studied to identify underlying principles of how information flows between the characters and research studies ...
Urban area characterization based on crowd behavioral lifelogs over Twitter
Recent location-based social networking sites are attractively providing us with a novel capability of monitoring massive crowd lifelogs in the real-world space. In particular, they make it easier to collect publicly shared crowd lifelogs in a large ...
Comments