Temitayo Olugbade
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Movement dataset reviews exist but are limited in coverage, both in terms of size and research discipline. While topic-specific reviews clearly have their merit, it is critical to have a comprehensive overview based on a systematic survey across disciplines. This enables higher visibility of datasets available to the research communities and can foster interdisciplinary collaborations. We present a catalogue of 704 open datasets described by 10 variables that can be valuable to researchers searching for secondary data: name and reference, creation purpose, data type, annotations, source, population groups, ordinal size of people captured simultaneously, URL, motion capture sensor, and funders. The catalogue is available in the supplementary materials. We provide an analysis of the datasets and further review them under the themes of human diversity, ecological validity, and data recorded. The resulting 12-dimension framework can guide researchers in planning the creation of open movement datasets. This work has been the interdisciplinary effort of researchers across affective computing, clinical psychology, disability innovation, ethnomusicology, human-computer interaction, machine learning, music cognition, music computing, and movement neuroscience.
Supplemental Material
Available for Download
zip
csur-2021-0820-supp.zip (5.6 MB)
Supplementary materials
- [1] . 2014. Datasets for action, gesture and activity analysis. In 2nd International Conference on Intelligent Systems and Image Processing.Google Scholar
Cross Ref
- [2] . 2011. Action dataset—A survey. In SICE Annual Conference. IEEE, 1650–1655.Google Scholar
- [3] AniAgeProjectdataset. (????). Traditional Dances Download. Retrieved from https://www.euh2020aniage.org/testthaidancedownload.Google Scholar
- [4] . 2010. Fall detection with multiple cameras: An occlusion-resistant method based on 3-D silhouette vertical distribution. IEEE Trans. Inf. Technol. Biomed. 15, 2 (2010), 290–300.Google ScholarDigital Library
- [5] . 2020. Supporting the voice of people with disabilities in Kenya, Uganda and Jordan. In Rethinking Giving Voice Workshop, Vol. 2020. Association for Computing Machinery.Google Scholar
- [6] . 2020. Moving in unison after perceptual interruption. Sci. Rep. 10, 1 (2020), 1–13.Google ScholarCross Ref
- [7] . 2017. The theory of constructed emotion: An active inference account of interoception and categorization. Soc. Cog. Affect. Neurosci. 12, 1 (2017), 1–23.Google Scholar
- [8] . 2015. Interoceptive predictions in the brain. Nat. Rev. Neurosci. 16, 7 (2015), 419–429.Google ScholarCross Ref
- [9] . 2008. Walking, cycling, and obesity rates in Europe, North America, and Australia. J. Phys. Activ. Health 5, 6 (2008), 795–814.Google ScholarCross Ref
- [10] . 1995. The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychol. Bull. 117, 3 (1995), 497.Google ScholarCross Ref
- [11] . 2012. Skill Training in Multimodal Virtual Environments. CRC Press.Google ScholarDigital Library
- [12] . 2017. Impulse radio ultra-wideband communications for localization and tracking of human body and limbs movement for healthcare applications. IEEE Trans. Ant. Propag. 65, 12 (2017), 7298–7309.Google ScholarCross Ref
- [13] . 2018. Musical genre-dependent behavioural and EEG signatures of action planning. A comparison between classical and jazz pianists. Neuroimage 169 (2018), 383–394.Google ScholarCross Ref
- [14] . 2021. Bridging the gap between emotion and joint action. Neurosci. Biobehav. Rev. (2021).Google Scholar
- [15] . 2017. RGB-D datasets using Microsoft Kinect or similar sensors: A survey. Multim. Tools Applic. 76, 3 (2017), 4313–4355.Google ScholarDigital Library
- [16] . 2018. VGGFace2: A dataset for recognising faces across pose and age. In 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG’18). IEEE, 67–74.Google ScholarDigital Library
- [17] . 2019. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Patt. Anal. Mach. Intell. (2019).Google Scholar
- [18] . 2015. “But, I don’t take steps.” Examining the inaccessibility of fitness trackers for wheelchair athletes. In 17th International ACM Conference on Computers & Accessibility. 193–201.Google Scholar
- [19] . 2021. A low-cost stand-alone platform for measuring motor behaviour across developmental applications. iScience (2021), 102742.Google ScholarCross Ref
- [20] . 2013. A survey of video datasets for human action and activity recognition. Comput. Vis. Image Underst. 117, 6 (2013), 633–659.
DOI: Google ScholarDigital Library - [21] . 2017. DeepBreath: Deep learning of breathing patterns for automatic stress recognition using low-cost thermal imaging in unconstrained settings. In 7th International Conference on Affective Computing and Intelligent Interaction. IEEE, 456–463.Google ScholarCross Ref
- [22] . 2020. Interpersonal entrainment in music performance: Theory, method, and model. Mus. Percept.: Interdisc. J. 38, 2 (2020), 136–194.Google ScholarCross Ref
- [23] . 2018. IEMP North Indian Raga.
DOI: Google ScholarCross Ref - [24] . 2019. Towards automatic screening of typical and atypical behaviors in children with autism. In IEEE International Conference on Data Science and Advanced Analytics (DSAA).
DOI: Google ScholarCross Ref - [25] . 2018. Sensor-based datasets for human activity recognition–a systematic review of literature. IEEE Access 6 (2018), 59192–59210.Google ScholarCross Ref
- [26] . 2017. Home camera-based fall detection system for the elderly. Sensors 17, 12 (2017), 2864.Google ScholarCross Ref
- [27] . 2018. Linguistic Bodies: The Continuity between Life and Language. The MIT Press.Google ScholarCross Ref
- [28] . 2010. Tracking benchmark databases for video-based sign language recognition. In European Conference on Computer Vision. Springer, 286–297.Google Scholar
- [29] . 2019. Context in human emotion perception for automatic affect detection: A survey of audiovisual databases.
DOI: Google ScholarCross Ref - [30] . 2015. What can music tell us about social interaction? Trends. Cog. Sci. 19, 3 (2015), 111–114.Google ScholarCross Ref
- [31] . 2016. From pose to activity: Surveying datasets and introducing CONVERSE. Comput. Vis. Image Underst. 144 (2016), 73–105.Google ScholarDigital Library
- [32] . 2015. Convenience sampling, random sampling, and snowball sampling: How does sampling affect the validity of research? J.Vis. Impair. Blind. 109, 2 (2015), 164–168.Google ScholarCross Ref
- [33] . 2016. RGBD datasets: Past, present and future. In IEEE Conference on Computer Vision and Pattern Recognition Workshops. 19–31.Google ScholarCross Ref
- [34] . 2007. Before and below “theory of mind”: Embodied simulation and the neural correlates of social cognition. Philos. Trans. Roy. Societ. B: Biol. Sci. 362, 1480 (2007), 659–669.
DOI: Google ScholarCross Ref - [35] . 2018. Datasheets for datasets. arXiv (2018).Google Scholar
- [36] . 2000. Neural aspects of cognitive motor control. Curr. Opin. Neurobiol. 10, 2 (2000), 238–241.Google ScholarCross Ref
- [37] . 2021. A critical examination of virtual reality technology in the context of the minority body. In CHI Conference on Human Factors in Computing Systems. 1–14.Google Scholar
- [38] . 2022. Ethics in the mining of software repositories. Empir. Softw. Eng. 27, 1 (2022), 1–49.Google ScholarDigital Library
- [39] . 2021. Phenotype discovery from population brain imaging. Med. Image Anal. 71 (2021), 102050.Google ScholarCross Ref
- [40] . 2018. AVA: A video dataset of spatio-temporally localized atomic visual actions. In IEEE Conference on Computer Vision and Pattern Recognition. 6047–6056.Google ScholarCross Ref
- [41] . 2006. Creating and annotating affect databases from face and body display: A contemporary survey. In IEEE International Conference on Systems, Man and Cybernetics, Vol. 3. 2426–2433.
DOI: Google ScholarCross Ref - [42] . 2015. Bodily expression for automatic affect recognition. Emot. Recog.: Patt. Anal. Appr. (2015), 343–377.Google Scholar
- [43] . 2013. A critical review of action recognition benchmarks. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 245–250.
DOI: Google ScholarDigital Library - [44] . 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770–778.Google ScholarCross Ref
- [45] . 2010. The weirdest people in the world? Behav. Brain Sci. 33, 2-3 (2010), 61–83.Google ScholarCross Ref
- [46] . 2017. Global participation in sport and leisure-time physical activities: A systematic review and meta-analysis. Prevent. Med. 95 (2017), 14–25.Google ScholarCross Ref
- [47] . 2005. Grasping the intentions of others with one’s own mirror neuron system. PLoS Biol. 3, 3 (2005), 0529–0535.
DOI: Google ScholarCross Ref - [48] . 2015. Public Involvement in Research: Values and Principles Framework.
Technical Report .Google Scholar - [49] . 2018. IEMP Uruguayan Candombe.
DOI: Google ScholarCross Ref - [50] . 2013. Body movements for affective expression: A survey of automatic recognition and generation. IEEE Trans. Affect. Comput. 4, 4 (2013), 341–359.Google ScholarCross Ref
- [51] . 2014. Rhythm in joint action: Psychological and neurophysiological mechanisms for real-time interpersonal coordination. Philos. Trans. Roy. Societ. B: Biol. Sci. 369, 1658 (2014), 20130394.Google ScholarCross Ref
- [52] . 2000. Tracking people in presence of occlusion. In Asian Conference on Computer Vision, Vol. 5. Citeseer.Google Scholar
- [53] . 2019. Ecological validity of manual grasping movements in an everyday-like grocery shopping task. Experim. Brain Res. 237, 5 (2019), 1169–1177.Google ScholarCross Ref
- [54] . 2014. The Data Revolution: Big Data, Open Data, Data Infrastructures & Their Consequences. London.
DOI: Google ScholarCross Ref - [55] . 2013. Affective body expression perception and recognition: A survey. IEEE Trans. Affect. Comput. 4, 1 (2013), 15–33.Google ScholarDigital Library
- [56] . 2018. Patient and Public Involvement in Health and Social Care Research: A Handbook for Researchers.
Technical Report . Retrieved from https://www.rds-london.nihr.ac.uk/wpcms/wp-content/uploads/2018/10/RDS_PPI-Handbook_2018_WEB_VERSION.pdf.Google Scholar - [57] . 2020. ARBEE: Towards automated recognition of bodily expression of emotion in the wild. Int. J. Comput. Vis. 128, 1 (2020), 1–25.Google ScholarDigital Library
- [58] . 2020. The role of respiration audio in multimodal analysis of movement qualities. J. Multim. User Interf. 14, 1 (2020), 1–15.Google ScholarCross Ref
- [59] . 2021. Scan once, analyse many: Using large open-access neuroimaging datasets to understand the brain. Neuroinformatics (2021), 1–29.Google Scholar
- [60] . 2022. Reproducible brain-wide association studies require thousands of individuals. Nature 603, 7902 (2022), 654–660.Google ScholarCross Ref
- [61] . 2015. Tagging firefighter activities at the emergency scene: Summary of AAIA’15 data mining competition at knowledge Pit. In Federated Conference on Computer Science and Information Systems (FedCSIS). IEEE, 367–373.Google ScholarCross Ref
- [62] . 2012. The influence of an audience on performers: A comparison between rehearsal and concert using audio, video and movement data. J. New Mus. Res. 41, 1 (2012), 67–78.Google ScholarCross Ref
- [63] . 2018. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med. Res. Methodol. 18, 1 (2018), 1–7.
DOI: Google ScholarCross Ref - [64] . 2008. Open data in science. Serials Rev. 34, 1 (2008), 52–64.
DOI: Google ScholarCross Ref - [65] . 2018. Survey on emotional body gesture recognition. IEEE Trans. Affect. Comput. (2018).Google Scholar
- [66] . 2019. Benchmark RGB-D gait datasets: A systematic review. In ECCOMAS Thematic Conference on Computational Vision and Medical Image Processing. Springer, 366–372.Google ScholarCross Ref
- [67] . 2019. How can affect be detected and represented in technological support for physical rehabilitation? ACM Trans. Comput.-Hum. Interact. 26, 1 (2019), 1–29.Google ScholarDigital Library
- [68] . 2018. IEMP Malian Jembe.
DOI: Google ScholarCross Ref - [69] . 2018. Recent trends in machine learning for human activity recognition–A survey. Data Mining Knowl. Discov. 8, 4 (2018), e1254.Google Scholar
- [70] . 2014. A survey of datasets for human gesture recognition. In International Conference on Human-computer Interaction. Springer, 337–348.Google ScholarCross Ref
- [71] . 2021. Re-imagining algorithmic fairness in India and beyond. In ACM Conference on Fairness, Accountability, and Transparency. 315–328.Google ScholarDigital Library
- [72] . 2021. “Everyone wants to do the model work, not the data work”: Data cascades in high-stakes AI. In CHI Conference on Human Factors in Computing Systems. 1–15.Google Scholar
- [73] . 2013. Toward a second-person neuroscience 1. Behav. Brain Sci. 36, 4 (2013), 393–414.Google ScholarCross Ref
- [74] . 2011. The Primacy of Movement.
(Advances in Consciousness Research , Vol. 82.)John Benjamins Publishing Company, Amsterdam.DOI: Google ScholarCross Ref - [75] . 2020. Recent evolution of modern datasets for human activity recognition: A deep survey. Multim. Syst. 26, 2 (2020), 83–106.Google ScholarCross Ref
- [76] . 2019. Video benchmarks of human action datasets: A review. Artif. Intell. Rev. 52, 2 (2019), 1107–1154.Google ScholarDigital Library
- [77] . 2021. The bodies of TEI—Investigating norms and assumptions in the design of embodied interaction. In International Conference on Tangible, Embedded, and Embodied Interaction. 1–19.Google ScholarDigital Library
- [78] . 2017. Automatic affect perception based on body gait and posture: A survey. Int. J. Soc. Robot. 9, 5 (2017), 617–641.Google ScholarCross Ref
- [79] . 2015. Neuromechanical principles underlying movement modularity and their implications for rehabilitation. Neuron 86, 1 (2015), 38–54.Google ScholarCross Ref
- [80] . 2018. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann. Internal Med. 169, 7 (2018), 467–473.
DOI: Google ScholarCross Ref - [81] . 2016. Comparing two sampling methods to engage hard-to-reach communities in research priority setting. BMC Med. Res. Methodol. 16, 1 (2016), 1–11.Google ScholarCross Ref
- [82] . 2002. Databasing fMRI studies—Towards a “discovery science” of brain function. Nat. Rev. Neurosci. 3, 4 (2002), 314–318.Google ScholarCross Ref
- [83] . 2017. Rhythms of the body, rhythms of the brain: Respiration, neural oscillations, and embodied cognition. Conscious. Cogn. 56 (2017), 77–90.Google ScholarCross Ref
- [84] . 2010. Open data and the social contract of scientific publishing. BioScience 60, 5 (2010), 330–331.
DOI: Google ScholarCross Ref - [85] . 2012. A threefold dataset for activity and workflow recognition in complex industrial environments. IEEE MultiM. 19, 03 (2012), 42–52.Google ScholarDigital Library
- [86] . 2008. Body shape in American and British adults: Between-country and inter-ethnic comparisons. Int. J. Obes. 32, 1 (2008), 152–159.Google ScholarCross Ref
- [87] . 2019. Disability, Bias, and AI.
Technical Report .Google Scholar - [88] . 2019. Cyborg perspectives on computing research reform. In CHI Conference on Human Factors in Computing Systems. 1–11.Google Scholar
- [89] . 2020. Emotion recognition from gait analyses: Current research and future directions. arXiv preprint arXiv:2003.11461 (2020).Google Scholar
- [90] . 2000. Developmental features of rapid aiming arm movements across the lifespan. J. Motor Behav. 32, 2 (2000), 121–140.Google ScholarCross Ref
- [91] . 2014. Automatic emotion recognition based on body movement analysis: A survey. IEEE Comput. Graph. Applic. 34, 6 (2014), 35–45.Google ScholarCross Ref
- [92] . 2016. RGB-D-based action recognition datasets: A survey. Patt. Recog. 60 (2016), 86–105.Google ScholarDigital Library
- [93] . 2020. A comprehensive survey on transfer learning. Proc. IEEE 109, 1 (2020), 43–76.Google ScholarCross Ref
- [94] . 2020. Predicting intentions from motion: The subject-adversarial adaptation approach. Int. J. Comput. Vis. 128, 1 (2020), 220–239.Google ScholarDigital Library
Index Terms
- Human Movement Datasets: An Interdisciplinary Scoping Review
Recommendations
A survey of video datasets for human action and activity recognition
Highlights Description of datasets for video-based human activity and action recognition. 68 datasets reported: 28 for heterogeneous and 40 for specific human actions. Useful data, such as web for dowloading, published works or ground truth, are ...
Video benchmarks of human action datasets: a review
Vision-based Human activity recognition is becoming a trendy area of research due to its wide application such as security and surveillance, human---computer interactions, patients monitoring system, and robotics. In the past two decades, there are ...
Tagging with Movement: Somatic Strategies for Image Classification
MOCO '14: Proceedings of the 2014 International Workshop on Movement and ComputingMovement researchers in HCI have begun to utilize theories from the field of Somatics to support their investigations into tangible, ubiquitous, and wearable computing systems. Despite recent advances in integrating movement theory within HCI, the ...
Comments