Multimodal fusion for functional rehabilitation at home

Conventional musculoskeletal rehabilitation consists of a therapy consultancy, an exercise assignment, and an execution task with or without assistance of the therapist. This classical approach consumes plenty of the patient's time, money and effort, and especially those of medical staff. Serious games have been studied as an aided tool for clinical and home-based rehabilitation, with patient autonomy in the exercise execution but monitored by the therapist staff. In order to estimate joint angles, most of these systems used a Kinect camera for musculoskeletal rehabilitation motion but the estimation accuracy represents the principal drawback. Therefore, in this work, which is under development, we have proposed a data fusion system based on accelerometers devices (Shimmer) and Kinect camera. Figure 1 shows that wearable sensors can be used to closely monitor Parkinson's disease motor fluctuations and predict clinical scores with high accuracy. Exploiting a quaternion representation of orientation, we propose to estimate joint angle from Shimmer sensors using a gradient descent algorithm, which is based on accelerometer and magnetometer signals (magnetic distortion has been taken into account). The estimated orientation will be fused with the calculated orientation using angular rate signal. The weighted sum fusion algorithm is used. The output of the fusion algorithm is used to continuously correct Kinect camera output. Below, in Figure 1, we describe the general workflow of the proposed system. The system will be able to inform the user in real time the successful completion of the rehabilitation exercises.