Real-time forecasting of exercise-induced fatigue from wearable sensors

Highlights

• Forecasted wearable sensor data and associated fatigue progression online.

• The proposed model outperformed the state-of-the-art for fatigue recognition.

• Spatiotemporal attention blocks captured the dependency between timesteps and features.

• Model performances were evaluated in a user-independent approach.

Abstract

Although a number of studies attempt to classify human fatigue, most models can only identify fatigue after fatigue has already occurred. In this paper, we propose a novel time series approach to forecasting wearable sensor data and associated fatigue progression during exercise. The proposed framework consists of spatio-temporal attention-based Transformer with an auxiliary critic and a fatigue classifier. The Transformer network is used to analyze the person-independent pattern underlying the past kinematic sequence obtained from wearable sensors and generate short term predictions of the human motion. Adversarial training is employed to regularize the Transformer and improve the time series forecasting performance. A fatigue classifier is used to estimate person-independent fatigue levels based on the forecasted wearable sensor data from the Transformer model. The proposed approach is validated with simulated and real squat datasets which were collected from young healthy participants. The proposed network can accurately forecast a time horizon of up to 80 timesteps for motion signal forecasting and fatigue classification. In terms of fatigue prediction, an accuracy of 83% and a Pearson correlation coefficient of 0.92 were achieved on forecasted motion data with unseen participant data. The experimental results show that our model can predict fatigue progression and outperforms other state-of-the-art techniques, achieving 95% correlation compared to 83% for the best performing baseline method. Successfully predicting fatigue progression can help a patient or athlete monitor and adjust their exercise session to prevent overexertion and fatigue-induced injury.

Introduction

Exercise-induced fatigue monitoring is used to characterize the physical and mental fatigue accumulated over time due to repeated movement. This is particularly important for athletes, coaches, rehabilitation patients, therapists and other clinicians, since high levels of fatigue can hinder adaptation during training or rehabilitation. In rehabilitation, patients experiencing high levels of fatigue are at greater secondary risks of injury [1], [2], [3]. In sport training, high levels of fatigue can hinder adaptation during training, inhibit performance and lead to injury [4]. Therefore, actively monitoring fatigue status during exercise can provide immediate feedback needed to adjust the training strategy in order to prevent injury and maximize overall performance [2], [4].

Although traditionally defined as a reduction in muscle force output [5], exercise-induced fatigue more broadly involves the perception of fatigue as well as changes in performance [6]. Detecting or forecasting exercise-induced fatigue is a difficult task, because it is challenging to find a single metric (e.g., velocity loss or altered range of motion or muscle activity) that can accurately identify fatigue progression. Muscle electrical activity or human movement between individuals and even for the same individual in repeated trials can be variable and may contribute to conflicting findings in the literature [7]. This variability makes person-independent fatigue recognition challenging.

Several studies of exercise-induced fatigue and recovery have used statistical and data mining techniques from different sensor data [8], [9]. For example, Karg et al. [8], Karvekar et al. [10] and Zhang et al. [9] have achieved the classification of fatigued and non-fatigued states with an accuracy of over 90% by using various sensors, including optical motion capture, Inertial Measurement Units (IMU), and surface electromyography (sEMG). Papakostas et al. [2], Wang et al. [11] and Huang [12] also conducted extensive experiments to identify the onset of physical fatigue during rehabilitation using sEMG or heart rate variability (HRV) and demonstrated low error (around 4.3%) in fatigue detection during rehabilitation therapy. However, these models are reactive, since detection and potential intervention are contingent on participants reporting or being labeled as fatigued. There are only a few studies that have attempted to forecast the physiological variations and associated outcome ahead of time [13], [14]. It is optimal to detect fatigue onset as early as possible and alert individuals before performance is impaired or an injury occurs.

This paper focuses on forecasting future motion to predict future fatigue instead of classifying fatigue status from data that has already been observed. Fatigue progression forecasting can be considered as a multivariate multi-step wearable sensor time series forecasting problem and fatigue classification problem. A basic assumption behind multivariate time series forecasting is that its variables are dependent on one another [15]. Therefore, it is important to consider both the spatial and temporal dependencies especially for user-independent forecasting problems. We develop a person-independent framework (see Fig. 1) to predict both the human motion and fatigue status by combining forecasting and fatigue estimation. It should be noted that fatigue is estimated on a gradual scale rather than using binary fatigue vs. non-fatigue detection. Motivated by the advancements of attention mechanisms, a new spatiotemporal attention model is proposed to jointly model the spatial and temporal dependencies within IMU signals forecasting networks for accurate prediction. To alleviate the effect of error accumulation, we add adversarial training followed by attention-based encoder–decoder model. A fatigue classifier [7] is employed to classify future fatigue levels based on the forecasted IMU sensor time series. The proposed time series forecasting model is compared to other common time series forecasting approaches (e.g., LSTM, and CNN). Experiments with leave-one-out cross-validation are conducted to determine whether the proposed approach for generalization across participants is superior.

Overall, our main contributions are as follows:

• We develop a novel deep learning based system that predicts the person-independent kinematic characteristics and associated fatigue status on-line during exercise performance.

• We propose a novel spatiotemporal attention block to capture the dependency between timesteps and features.

• We introduce an adversarial training mechanism adapted to the fatigue prediction task.

• Extensive experiments on an IMU dataset show that our motion predictor significantly outperforms state-of-the-art methods.