Deepak Nagaraj
ABSTRACT
With recent advances in various hardware technologies, human motion capturing (MoCap) has gained importance in the fields such as computer vision, computer animation, gesture recognition in gaming, and most importantly in bio-mechanical analysis. In this direction, human motion is being captured using various kinds of sensors. Correspondingly, many model-based and data-based techniques have been developed in order to decode sensor readings into information understandable by a person. Given that the current technologies still lack applicability in real-world scenarios considering cost and ease of information gathering, leaves substantial room for improvement. This article focuses on the development of a novel machine learning based proof of concept for real-time human pose estimation using data collected from sparse inertial measurement units (IMU) system which is cost-effective and least intrusive in the scope of skilled crafts domain. Training diverse bi-directional recurrent neural networks (bi-RNN) with variable window size and building an ensemble of these models to estimate human pose in terms of human-joints' angles more accurately and robustly is discussed.
- Abdullatif Alwasel, Karim Elrayes, Eihab Abdel-Rahman, and Carl Haas. 2011. Sensing Construction Work-Related Musculoskeletal Disorders (WMSDs). Proceedings of the 28th International Symposium on Automation and Robotics in Construction, ISARC 2011. https://doi.org/10.22260/ISARC2011/0027Google Scholar
Cross Ref
- Sheldon Andrews, Ivan Huerta, Taku Komura, Leonid Sigal, and Kenny Mitchell. 2016. Real-time Physics-based Motion Capture with Sparse Sensors. In Proceedings of the 13th European Conference on Visual Media Production (CVMP 2016) (CVMP 2016). ACM, New York, NY, USA, Article 5, 10 pages. https://doi.org/10.1145/2998559.2998564Google ScholarDigital Library
- Leo Breiman. 1996. Bagging predictors. Machine learning 24, 2 (1996), 123--140.Google Scholar
- Gavin Brown, Jeremy Wyatt, Rachel Harris, and Xin Yao. 2005. Diversity creation methods: A survey and categorisation. Journal of Information Fusion 6 (2005), 5--20.Google ScholarCross Ref
- Maximilian Derouet, Deepak Nagaraj, Erik Schake, and Dirk Werth. 2019. Towards a Digitized Understanding of the Skilled Crafts Domain. In Business Information Systems Workshops, Witold Abramowicz and Rafael Corchuelo (Eds.). Springer International Publishing, Cham, 435--444.Google ScholarCross Ref
- Yu Guan and Thomas Plötz. 2017. Ensembles of Deep LSTM Learners for Activity Recognition using Wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (Jun 2017), 1--28. https://doi.org/10.1145/3090076Google ScholarDigital Library
- Sepp Hochreiter. 1998. The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 6, 2 (April 1998), 107--116. https://doi.org/10.1142/S0218488598000094Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735--1780. https://doi.org/10.1162/neco.1997.9.8.1735Google ScholarDigital Library
- Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael J. Black, Otmar Hilliges, and Gerard Pons-Moll. 2018. Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time. ACM Trans. Graph. 37, 6, Article 185 (Dec. 2018), 15 pages. https://doi.org/10.1145/3272127.3275108Google ScholarDigital Library
- Siddharth Joshi and Stephen Boyd. 2009. Sensor Selection via Convex Optimization. Trans. Sig. Proc. 57, 2 (Feb. 2009), 451--462. https://doi.org/10.1109/TSP.2008.2007095Google Scholar
- Yuki Kawana, Norimichi Ukita, Jia-Bin Huang, and Ming-Hsuan Yang. 2018. Ensemble convolutional neural networks for pose estimation. Computer Vision and Image Understanding 169 (2018), 62--74. https://doi.org/10.1016/j.cviu.2017.12.005Google ScholarCross Ref
- Ludmila I. Kuncheva and Christopher J. Whitaker. 2003. Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy. Mach. Learn. 51, 2 (May 2003), 181--207. https://doi.org/10.1023/A:1022859003006Google ScholarDigital Library
- K. Manohar, B. W. Brunton, J. N. Kutz, and S. L. Brunton. 2018. Data-Driven Sparse Sensor Placement for Reconstruction: Demonstrating the Benefits of Exploiting Known Patterns. IEEE Control Systems Magazine 38, 3 (June 2018), 63--86. https://doi.org/10.1109/MCS.2018.2810460Google Scholar
- Liam Paninski. 2005. Asymptotic Theory of Information-Theoretic Experimental Design. Neural Computation 17, 7 (2005), 1480--1507. https://doi.org/10.1162/0899766053723032 arXiv:https://doi.org/10.1162/0899766053723032Google ScholarDigital Library
- G. Pons-Moll, A. Baak, J. Gall, L. Leal-Taixé, M. Müller, H. Seidel, and B. Rosenhahn. 2011. Outdoor human motion capture using inverse kinematics and von mises-fisher sampling. In 2011 International Conference on Computer Vision. 1243--1250. https://doi.org/10.1109/ICCV.2011.6126375Google ScholarDigital Library
- Daniel Roetenberg, Henk Luinge, and Per J. Slycke. 2009. Xsens MVN: Full 6DOF Human Motion Tracking Using Miniature Inertial Sensors.Google Scholar
- Robert E Schapire. 1990. The strength of weak learnability. Machine learning 5, 2 (1990), 197--227.Google Scholar
- Paola Sebastiani and Henry Wynn. 2000. Maximum Entropy Sampling and Optimal Bayesian Experimental Design. Journal of the Royal Statistical Society Series B 62 (02 2000), 145--157. https://doi.org/10.1111/1467-9868.00225Google Scholar
- Diana Trojaniello, Andrea Cereatti, and Ugo Della Croce. 2013. Comparative Evaluation of Gait Event Detection Methods Based on a Single IMU: Error Sensitivity Analysis to IMU Positioning. In Converging Clinical and Engineering Research on Neurorehabilitation, José L Pons, Diego Torricelli, and Marta Pajaro (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 741--745.Google Scholar
- T. v. Marcard, G. Pons-Moll, and B. Rosenhahn. 2016. Human Pose Estimation from Video and IMUs. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 8 (Aug 2016), 1533--1547. https://doi.org/10.1109/TPAMI.2016.2522398Google ScholarCross Ref
- T. von Marcard, B. Rosenhahn, M.J. Black, and G. Pons-Moll. 2017. Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs. Comput. Graph. Forum 36, 2 (May 2017), 349--360. https://doi.org/10.1111/cgf.13131Google ScholarDigital Library
- Xuesu Xiao and Shuayb Zarar. 2018. A Wearable System for Articulated Human Pose Tracking under Uncertainty of Sensor Placement. In IEEE RAS/EMBS Int. Conf. Biomedical Robotics and Biomechatronics (BioRob).Google ScholarCross Ref
- Xsens. 2019.. https://www.xsens.com/products/Google Scholar
Index Terms
- An RNN-Ensemble approach for Real Time Human Pose Estimation from Sparse IMUs
Recommendations
Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generation
SA '22: SIGGRAPH Asia 2022 Conference PapersReal-time human motion reconstruction from a sparse set of (e.g. six) wearable IMUs provides a non-intrusive and economic approach to motion capture. Without the ability to acquire position information directly from IMUs, recent works took data-driven ...
Capturing Human Body Dynamics Using RNN Based on Persistent Excitation Data Generator
CBMS '14: Proceedings of the 2014 IEEE 27th International Symposium on Computer-Based Medical SystemsHuman body walking movement involves both single and double support phases and is considered difficult to model. The aim of this study was to develop a method to capture human body dynamics during walking using Recurrent Neural Networks (RNN). In ...
Optimized Motion Capture System for Full Body Human Motion Capturing Case Study of Educational Institution and Small Animation Production
DMDCM '11: Proceedings of the 2011 Workshop on Digital Media and Digital Content ManagementMotion capture system or MOCAP is a set of devices used for capturing moving objects. In addition to had used in the scientific community, Medical, Engineering, MOCAP is currently being used extensively in film and animation industry to create realistic ...
Comments