Wenbo Huang
Skip Abstract Section
Abstract
During the past decade, human activity recognition (HAR) using wearable sensors has become a new research hot spot due to its extensive use in various application domains such as healthcare, fitness, smart homes, and eldercare. Deep neural networks, especially convolutional neural networks (CNNs), have gained a lot of attention in HAR scenario. Despite exceptional performance, CNNs with heavy overhead is not the best option for HAR task due to the limitation of computing resource on embedded devices. As far as we know, there are many invalid filters in CNN that contribute very little to output. Simply pruning these invalid filters could effectively accelerate CNNs, but it inevitably hurts performance. In this article, we first propose a novel CNN for HAR that uses filter activation. In comparison with filter pruning that is motivated for efficient consideration, filter activation aims to activate these invalid filters from an accuracy boosting perspective. We perform extensive experiments on several public HAR datasets, namely, UCI-HAR (UCI), OPPORTUNITY (OPPO), UniMiB-SHAR (Uni), PAMAP2 (PAM2), WISDM (WIS), and USC-HAD (USC), which show the superiority of the proposed method against existing state-of-the-art (SOTA) approaches. Ablation studies are conducted to analyze its internal mechanism. Finally, the inference speed and power consumption are evaluated on an embedded Raspberry Pi Model 3 B plus platform.
- [1] . 2019. An ultra-low energy human activity recognition accelerator for wearable health applications. ACM Trans. Embed. Comput. Syst. 18, 5s (2019), 1–22.Google Scholar
Digital Library
- [2] . 2021. Two-stream convolution augmented transformer for human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence. 286–293.Google ScholarCross Ref
- [3] . 2021. XGest: Enabling cross-label gesture recognition with RF signals. ACM Trans. Sensor Netw. 17, 4 (2021), 1–23.Google ScholarDigital Library
- [4] . 2021. Facilitating human activity data annotation via context-aware change detection on smartwatches. ACM Trans. Embed. Comput. Syst. 20, 2 (2021), 1–20.Google ScholarDigital Library
- [5] . 2019. A fog-based application for human activity recognition using personal smart devices. ACM Trans. Internet Technol. 19, 2 (2019), 1–20.Google ScholarDigital Library
- [6] . 2022. Channel-Equalization-HAR: A light-weight convolutional neural network for wearable sensor based human activity recognition. IEEE Trans. Mob. Comput. Early access.
DOI: Google ScholarCross Ref - [7] . 2022. Real-time human activity recognition using conditionally parametrized convolutions on mobile and wearable devices. IEEE Sensors J. 22, 6 (2022), 5889–5901.Google ScholarCross Ref
- [8] . 2021. Semisupervised human activity recognition with radar micro-Doppler signatures. IEEE Trans. Geosci. Rem. Sens. 60 (2021), 1–12.Google Scholar
- [9] . 2021. Shallow convolutional neural networks for human activity recognition using wearable sensors. IEEE Trans. Instrum. Measur. 70 (2021), 1–11.Google Scholar
- [10] . 2015. Deep learning. Nature 521, 7553 (2015), 436–444.Google ScholarCross Ref
- [11] . 2021. A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Trans. Neural Netw. Learn. Syst. Early access.
DOI: Google ScholarCross Ref - [12] . 2018. Recent advances in convolutional neural networks. Pattern Recog. 77 (2018), 354–377.Google ScholarDigital Library
- [13] . 2018. RF-based fall monitoring using convolutional neural networks. Proc. ACM Interact., Mob., Wear. Ubiq. Technol. 2, 3 (2018), 1–24.Google ScholarDigital Library
- [14] . 2018. Understanding and improving recurrent networks for human activity recognition by continuous attention. In Proceedings of the ACM International Symposium on Wearable Computers. 56–63.Google ScholarDigital Library
- [15] . 2016. Human activity recognition with smartphone sensors using deep learning neural networks. Exp. Syst. Applic. 59 (2016), 235–244.Google ScholarDigital Library
- [16] . 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In Proceedings of the 24th International Joint Conference on Artificial Intelligence.Google ScholarDigital Library
- [17] . 2016. Deep learning for human activity recognition: A resource efficient implementation on low-power devices. In Proceedings of the IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE, 71–76.Google ScholarCross Ref
- [18] . 2015. Human activity recognition using wearable sensors by deep convolutional neural networks. In Proceedings of the 23rd ACM International Conference on Multimedia. 1307–1310.Google ScholarDigital Library
- [19] . 2020. Interpretable and accurate convolutional neural networks for human activity recognition. IEEE Trans. Industr. Inform. 16, 11 (2020), 7190–7198.Google ScholarCross Ref
- [20] . 2018. Real-time human activity recognition from accelerometer data using convolutional neural networks. Appl. Soft Comput. 62 (2018), 915–922.Google ScholarCross Ref
- [21] . 2019. Adaptive fusion and category-level dictionary learning model for multiview human action recognition. IEEE Internet Things J. 6, 6 (2019), 9280–9293.Google ScholarCross Ref
- [22] . 2016. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.Google ScholarCross Ref
- [23] . 2016. Pruning filters for efficient convNets. arXiv preprint arXiv:1608.08710 (2016).Google Scholar
- [24] . 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. arXiv preprint arXiv:1510.00149 (2015).Google Scholar
- [25] . 2018. Exploring linear relationship in feature map subspace for convNets compression. arXiv preprint arXiv:1803.05729 (2018).Google Scholar
- [26] . 2018. Optimizing multi-sensor deployment via ensemble pruning for wearable activity recognition. Inf. Fusion 41 (2018), 68–79.Google ScholarDigital Library
- [27] . 2021. Pruning by explaining: A novel criterion for deep neural network pruning. Pattern Recog. 115 (2021), 107899.Google ScholarCross Ref
- [28] . 2018. A novel ensemble ELM for human activity recognition using smartphone sensors. IEEE Trans. Industr. Inform. 15, 5 (2018), 2691–2699.Google ScholarCross Ref
- [29] . 2018. SCSP: Spectral clustering filter pruning with soft self-adaption manners. arXiv preprint arXiv:1806.05320 (2018).Google Scholar
- [30] . 2020. Filter grafting for deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6599–6607.Google ScholarCross Ref
- [31] . 2018. On attention models for human activity recognition. In Proceedings of the ACM International Symposium on Wearable Computers. 100–103.Google ScholarDigital Library
- [32] . 2019. Attention-based convolutional neural network for weakly labeled human activities’ recognition with wearable sensors. IEEE Sensors J. 19, 17 (2019), 7598–7604.Google ScholarCross Ref
- [33] . 2019. AttnSense: Multi-level attention mechanism for multimodal human activity recognition. In Proceedings of the International Joint Conferences on Artificial Intelligence. 3109–3115.Google ScholarCross Ref
- [34] . 2018. Deep mutual learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4320–4328.Google ScholarCross Ref
- [35] . 2021. A systematic review of smartphone-based human activity recognition methods for health research. Nat. Partn. J.: Digit. Med. 4, 1 (2021), 1–15.Google Scholar
- [36] . 2017. Ensembles of deep LSTM learners for activity recognition using wearables. Proc. ACM Interact., Mob., Wear. Ubiq 1, 2 (2017), 1–28.Google ScholarDigital Library
- [37] . 2012. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In Proceedings of the International Workshop on Ambient Assisted Living. Springer, 216–223.Google ScholarDigital Library
- [38] . 2010. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the 7th International Conference on Networked Sensing Systems (INSS). IEEE, 233–240.Google ScholarCross Ref
- [39] . 2017. UniMiB SHAR: A dataset for human activity recognition using acceleration data from smartphones. Appl. Sci. 7, 10 (2017), 1101.Google ScholarCross Ref
- [40] . 2012. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of the 16th International Symposium on Wearable Computers. IEEE, 108–109.Google ScholarDigital Library
- [41] . 2011. Design considerations for the WISDM smart phone-based sensor mining architecture. In Proceedings of the 5th International Workshop on Knowledge Discovery from Sensor Data. 25–33.Google ScholarDigital Library
- [42] . 2012. USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the ACM Conference on Ubiquitous Computing. 1036–1043.Google ScholarDigital Library
- [43] . 2021. Attention induced multi-head convolutional neural network for human activity recognition. Appl. Soft Comput. 110 (2021), 107671.Google ScholarDigital Library
- [44] . 2020. The layer-wise training convolutional neural networks using local loss for sensor-based human activity recognition. IEEE Sensors J. 20, 13 (2020), 7265–7274.Google ScholarCross Ref
- [45] . 2021. Attend and discriminate: Beyond the state-of-the-art for human activity recognition using wearable sensors. Proc. ACM Interact., Mob., Wear. Ubiq 5, 1 (2021), 1–22.Google ScholarDigital Library
- [46] . 2021. Sequential weakly labeled multiactivity localization and recognition on wearable sensors using recurrent attention networks. IEEE Trans. Hum.-Mach. Syst. 51, 4 (2021), 355–364.Google ScholarCross Ref
- [47] . 2021. Latent independent excitation for generalizable sensor-based cross-person activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence. AAAI, 11921–11929.Google ScholarCross Ref
- [48] . 2022. Multi-ResAtt: Multilevel residual network with attention for human activity recognition using wearable sensors. IEEE Trans. Industr. Inform. Early access.
DOI: Google ScholarCross Ref - [49] . 2021. Learning disentangled representation for mixed-reality human activity recognition with a single IMU sensor. IEEE Trans. Instrum. Measur. 70 (2021), 1–14.Google Scholar
- [50] . 2021. NOHAR-NOvelty discrete data stream for human activity recognition based on smartphones with inertial sensors. Exp. Syst. Applic. 166 (2021), 114093.Google ScholarCross Ref
- [51] . 2020. A comparison of unidirectional and bidirectional LSTM networks for human activity recognition. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 1–6.Google ScholarCross Ref
- [52] . 2019. Gait activity authentication using LSTM neural networks with smartphone sensors. In Proceedings of the 15th International Conference on Mobile Ad-Hoc and Sensor Networks (MSN). IEEE, 456–461.Google ScholarCross Ref
- [53] . 2022. CapsGaNet: Deep neural network based on capsule and GRU for human activity recognition. IEEE Syst. J. Early access.
DOI: Google ScholarCross Ref - [54] . 2021. Meta-HAR: Federated representation learning for human activity recognition. In Proceedings of the International World Wide Web Conferences (WWW). ACM, 912–922.Google ScholarDigital Library
- [55] . 2020. Human activity recognition based on dynamic active learning. IEEE J. Biomed. Health Inform. 25, 4 (2020), 922–934.Google ScholarCross Ref
- [56] . 2020. Deep ConvLSTM with self-attention for human activity decoding using wearable sensors. IEEE Sensors J. 21, 6 (2020), 8575–8582.Google ScholarCross Ref
- [57] . 2021. Time series change point detection with self-supervised contrastive predictive coding. In Proceedings of the Web Conference. ACM, 3124–3135.Google ScholarDigital Library
Index Terms
- Deep Ensemble Learning for Human Activity Recognition Using Wearable Sensors via Filter Activation
Recommendations
Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
MM '15: Proceedings of the 23rd ACM international conference on MultimediaHuman physical activity recognition based on wearable sensors has applications relevant to our daily life such as healthcare. How to achieve high recognition accuracy with low computational cost is an important issue in the ubiquitous computing. Rather ...
Human activity recognition with smartphone sensors using deep learning neural networks
This paper proposes a deep convolutional neural network for HAR using smartphone sensors.Experiments show that the proposed method derives relevant and more complex features.The method achieved an almost perfect classification on moving activities.It ...
Human Activity Recognition by Using Different Deep Learning Approaches for Wearable Sensors
AbstractWith the spread of wearable sensors, the solutions to the task of activity recognition by using the data obtained from the sensors have become widespread. Recognition of activities owing to wearable sensors such as accelerometers, gyroscopes, and ...
Comments