Abstract
It is of vital importance to accurately monitor post-stroke and one common way is to use wearable sensors to collect data and apply Convolutional Neural Networks (CNNs) to process it. In this paper a pipeline for TS classification is evaluated on a complex dataset which comprises 18 different ADLs collected from inertial measurement units sensors. The pipeline involves imaging the segmented TS data by employing three encoding techniques namely: Gramian Summation Angular Fields (GASF), Gramian Difference Angular Fields (GADF) and Markov Transition Fields (MKV). These encoding techniques were originally designed for univariate time-series, one contribution of this work is to propose a way to adapt it to multivariate TS by imaging each axis of the sensors separately and fusing them together to create multi-channel images. Another limitation comes from the fact that the resulting image size equals the sequence length of the original TS, we tackle this by employing a linear interpolation on the TS sequence to increase or decrease it. A comparison of the performance accuracy with respect to the employed encoding technique and the image size has been done. Results showed that GASF and GADF performed better than MTF encoding, besides fusing the images together and increasing the image size to a certain limit improved the accuracy from 87.5% for the state-of-the-art results to 91.5%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bobin, M., Amroun, H., Boukalle, M., Anastassova, M., Ammi, M.: Smart cup to monitor stroke patients activities during everyday life. In: 2018 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp. 189–195. IEEE (2018)
Boukhennoufa, I., Zhai, X., McDonald-Maier, K.D., Utti, V., Jackson, J.: Improving the activity recognition using GMAF and transfer learning in post-stroke rehabilitation assessment. In: 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 000391–000398. IEEE (2021)
Boukhennoufa, I., Zhai, X., Utti, V., Jackson, J., McDonald-Maier, K.D.: A comprehensive evaluation of state-of-the-art time-series deep learning models for activity-recognition in post-stroke rehabilitation assessment. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 2242–2247. IEEE (2021)
Boukhennoufa, I., Zhai, X., Utti, V., Jackson, J., McDonald-Maier, K.D.: Wearable sensors and machine learning in post-stroke rehabilitation assessment: A systematic review. Biomedical Signal Processing and Control 71, 103197 (2022)
Brillinger, D.R.: Time Series: Data Analysis and Theory. SIAM (2001)
Capela, N.A., Lemaire, E.D., Baddour, N.: Feature selection for wearable smartphone-based human activity recognition with able bodied, elderly, and stroke patients. PLoS ONE 10(4), e0124414 (2015)
Chaeibakhsh, S., Phillips, E., Buchanan, A., Wade, E.: Upper extremity post-stroke motion quality estimation with decision trees and bagging forests. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4585–4588. IEEE (2016)
Crichton, S.L., Bray, B.D., McKevitt, C., Rudd, A.G., Wolfe, C.D.: Patient outcomes up to 15 years after stroke: survival, disability, quality of life, cognition and mental health. J. Neurol. Neurosurg. Psychiatry 87(10), 1091–1098 (2016)
Centers for Disease Control and Prevention: Outpatient rehabilitation among stroke survivors-21 states and the district of Columbia, 2005. MMWR Morb. Mortal Wkly Rep. 56(20), 504–507 (2007)
Feigin, V.L., et al.: Global and regional burden of stroke during 1990–2010: findings from the global burden of disease study 2010. Lancet 383(9913), 245–255 (2014)
He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., Li, M.: Bag of tricks for image classification with convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 558–567 (2019)
Kaku, A., Parnandi, A., Venkatesan, A., Pandit, N., Schambra, H., Fernandez-Granda, C.: Towards data-driven stroke rehabilitation via wearable sensors and deep learning. arXiv preprint arXiv:2004.08297 (2020)
Mast, B.T., Azar, A.R., MacNeill, S.E., Lichtenberg, P.A.: Depression and activities of daily living predict rehospitalization within 6 months of discharge from geriatric rehabilitation. Rehabil. Psychol. 49(3), 219 (2004)
Mayo, N.E., Wood-Dauphinee, S., Côte, R., Durcan, L., Carlton, J.: Activity, participation, and quality of life 6 months poststroke. Arch. Phys. Med. Rehabil. 83(8), 1035–1042 (2002)
Pound, P., Gompertz, P., Ebrahim, S.: Illness in the context of older age: the case of stroke. Sociol. Health Illn. 20(4), 489–506 (1998)
Quinn, T.J., Langhorne, P., Stott, D.J.: Barthel index for stroke trials: development, properties, and application. Stroke 42(4), 1146–1151 (2011)
Shamay, N.S., William, T.W., Patrick, K.W., Philip, T.T., Jefferry, W.C.: Sensorimotor impairments of paretic upper limb correlates with activities of daily living in subjects with chronic stroke. S. Afr. J. Physiother. 67(1), 9–16 (2011)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, inception-ResNet and the impact of residual connections on learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Vrigkas, M., Nikou, C., Kakadiaris, I.A.: A review of human activity recognition methods. Front. Robot. AI 2, 28 (2015)
Wang, Z., Oates, T.: Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, vol. 1 (2015)
Weiss, G.M.: WISDM smartphone and smartwatch activity and biometrics dataset. UCI Machine Learning Repository (2019)
Wiener, J.M., Hanley, R.J., Clark, R., Van Nostrand, J.F.: Measuring the activities of daily living: Comparisons across national surveys. J. Gerontol. 45(6), S229–S237 (1990)
Acknowledgements
This work is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) through grants EP/R02572X/1, EP/P017487/1, EP/V000462/1, EP/X52590X/1 and EP/V034111/1.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Boukhennoufa, I., Zhai, X., Utti, V., Jackson, J., McDonald-Maier, K.D. (2022). Encoding Sensors’ Data into Images to Improve the Activity Recognition in Post Stroke Rehabilitation Assessment. In: El Yacoubi, M., Granger, E., Yuen, P.C., Pal, U., Vincent, N. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2022. Lecture Notes in Computer Science, vol 13364. Springer, Cham. https://doi.org/10.1007/978-3-031-09282-4_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-09282-4_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09281-7
Online ISBN: 978-3-031-09282-4
eBook Packages: Computer ScienceComputer Science (R0)