RehaBEElitation: the architecture and organization of a serious game to evaluate motor signs in Parkinson’s disease

Introduction

The use of serious games (SG) has received considerable attention in the health domain (Adcock et al., 2020). SG are digital games developed to entertain players and also to achieve an additional goal, such as acquiring learning or improving the users’ health (Dörner et al., 2016; Ritterfeld, Cody & Vorderer, 2009). Because exercises through games are perceived by the individuals as entertainment rather than therapy, these instruments promote more motivation to patients and reinforce adherence to the therapeutic process, contributing to the maintenance and/or improvement of the quality of life of the individuals (Tarousi et al., 2021). In addition to allowing movement training for motor rehabilitation purposes, which is the most commonly found application in the literature for SG (de Oliveira et al., 2021), patient monitoring and motor symptom assessment of a wide range of diseases, such as Parkinson’s disease (PD), can be performed using SG (Wilkinson et al., 2018).

PD is a neurodegenerative disorder that affects approximately 2% of the population over 60 years old (Singh & Ganley, 2021) and is characterized by the loss of dopaminergic neurons, which promote the appearance of motor symptoms, such as bradykinesia and tremor (Balestrino & Schapira, 2020). Due to the variety and severity of the symptoms of this disease, the importance of implementing technological resources such as SG as a valuable tool for clinical assessment and patient-centered monitoring is highlighted (de Oliveira et al., 2021; Tarousi et al., 2021). When used for clinical assessments, SG allows health professionals to provide more personalized follow-up to patients while also assisting them in disease monitoring (Tarousi et al., 2021).

The objective assessment of motor signs of PD using wearable sensors has been widely studied. de Oliveira Andrade et al. (2020) observed that it is possible to study resting hand tremor and postural tremors using a single inertial measurement unit (IMU) placed on the back of the hand. Rabelo et al. (2017) discriminated groups of PD patients from healthy elderly by an objective assessment of bradykinesia using inertial sensors. However, objectively assessing such symptoms using SG may be an efficient and promising alternative, due to the advantages provided by these instruments (Dias et al., 2016; Tarousi et al., 2021).

One of the reasons that explain the scarcity of tools effectively used in clinical practice, not limited to controlled environments, is the lack of projects that actually meet the needs of professionals and patients. It is precisely in order to fill this gap that this study presents the design and implementation of a serious game, so-called RehaBEElitation. The game was designed considering a long experience of our research group in the evaluation of motor signs of PD (de Oliveira Andrade et al., 2020; de Oliveira et al., 2021; Escudeiro et al., 2021; Folador et al., 2021b; Folador et al., 2021a; Luiz et al., 2021; Machado et al., 2016; Peres et al., 2021; Rabelo et al., 2017).

A literature review conducted by our group in 2021 showed that most studies related to the development of SG for PD evaluation (de Oliveira et al., 2021) do not concurrently address usability issues and game architecture design. Thus, this research presents in a detailed way the game architecture design through business process model notation (BPMN), which uses a set of graphical representations to facilitate the understanding and implementation of the game architecture (Carvalho et al., 2015). In addition, once the game architecture is defined, it is relevant to validate it still at a design stage. This is an important step that we found to be neglected in most studies (Andrade Ferreira et al., 2020; Avola et al., 2018; Cai et al., 2021; Oña et al., 2018; Sánchez-Herrera-Baeza et al., 2020; van de Weijer et al., 2019).

Thus, we proposed using the heuristic evaluation based on the Nielsen-Shneiderman heuristics to evaluate the system’s usability (Zhang et al., 2003). Heuristic evaluation is typically used after the system has been fully developed. In this study, we did an evaluation while the design is still in progress to predict and avoid usability issues. This is especially important in the development of SG for severe diseases such as PD, because a lack of understanding of potential usability issues may affect the patient’s experience, contributing to clinical follow-up failure.

Proving that a serious game is accessible is also crucial for ensuring the quality of the game design and for enhancing the player experience. Accessibility generally refers to the elimination of barriers that prevent individuals with various disabilities from accessing or utilising a product, such as a serious game (Fortes et al., 2017). According to Garber (2013), approximately 2% of the population, or approximately 6.3 million individuals, are unable to play computer games due to a disability. Consequently, this attribute was also considered in this study.

In relation to other studies (de Oliveira et al., 2021) the developed serious game is innovative in the following aspects: (i) detailed provision of the game architecture through business process model diagrams; (ii) usability evaluation at the design stage through heuristic evaluation; (iii) alignment between clinical and game design requirements; (iv) design of many events that enable the evaluation of clinical signs of PD, such as tremor and bradykinesia, at specific moments of the game; (v) development of a human machine interface (HMI) device (glove) specific for this application; (vi) proposal of a game narrative that meets the efforts required to deal with the challenges of PD; (vii) possibility of longitudinal evaluation of the disease, which ensures long-term follow-up of patients; and (viii) incorporation of the most relevant movements for clinical evaluation of hand motor signs in PD into the game.

Background

To determine the current state of research in this field, a literature review was conducted, which included recent studies on the use of SG for rehabilitation or motor sign monitoring of individuals with PD. A table showing the current studies has been elaborated (Attachment S1), the following characteristics are presented: the objective of the game, the required player movements, the HMI used, the symptom/characteristic evaluated, the instrument used to evaluate the game’s usability and information on the game architecture design.

No study focused on the monitoring of any motor sign of Parkinson’s disease through the use of the serious game. In addition, the studies did not describe the incorporation of game components that would allow objective assessment of the disease’s motor symptoms. Thus, the PD symptoms were not evaluated using the serious game, but rather other instruments and/or subjective scales (e.g., Box and Blocks Test (BBT), Purdue Pegboard Test (PPT), Berg Balance Scale (BBS), Parkinson’s Disease Questionnaire (PDQ-39), MDS-UPDRS and Hoehn and Yahr (HY) scale). Our proposal was to develop a serious game that can be used for both rehabilitation and monitoring symptoms of PD through game interaction and recording of inertial data.

No study developed a proprietary HMI for serious game user interaction. The development of a specific instrument, as done in this study, can be beneficial in several ways, including: efficient detection of the game’s movements, ease in solving calibration problems, greater customization capacity (e.g., size, colours, etc.) to meet various demands, marketability, and a reduced cost HMI device. Several instruments were used in the studies, such as Leap Motion Controller (Avola et al., 2018; Cemim et al., 2022; Fernández-González et al., 2019; Foletto, d’Ornellas & Prado, 2017; Oña et al., 2020; Oña et al., 2018; Sánchez-Herrera-Baeza et al., 2020; Shah et al., 2019), Microsoft Kinect (Avola et al., 2018; Cikajlo et al., 2018; Nuic et al., 2018; Silva et al., 2017), HTC Vive (Chen et al., 2020; Stanica et al., 2020), Oculus Rift (Oña et al., 2020; Sánchez-Herrera-Baeza et al., 2020), Beyond Your Motion (Blanc et al., 2022; Leblong et al., 2017), Myo Gesture Control Armband (Stanica et al., 2020), Inter Real Sense camera (Bevilacqua et al., 2021), head mounted display (Avola et al., 2018), tablet (Dauvergne et al., 2018) and stepping mat (Yuan et al., 2020).

Some studies did not detail the usability assessment instrument, while others used questionnaires developed by the research team. No mention is made of whether such instruments were tested and validated, and no study evaluated game usability during the design stage.

Most studies did not present information related to the game architecture. Some studies mentioned the main modules and components of the game, but did not present detailed and schematized information about the architecture and organization of the system. The lack of illustrative tools, such as diagrams and schemes, may hinder the understanding about the software framework as a whole. In this study, the system was developed in a modularized way in order to implement parallel and simultaneous programming at different levels. The system modules are completely independent from each other, allowing the fluidity of the game, the control of the interface elements in real time, the inclusion of other elements and the expansion of the game functionality without changing its basic characteristics. These game architecture features avoid its obsolescence and enable future advances.

An important consideration is that, with the exception of Shah et al. (2019), no reviewed studies have assessed the accessibility of the game. This is a relevant assessment as it checks whether the disabled player is able to play the game similarly to a healthy individual, i.e., it assesses whether the game can be played at a competent level by anyone.

In summary, the main contributions and novelties of this study are: (i) development of a serious game capable of assessing PD in an objective way, i.e., the game itself presents features that allow the assessment of motor symptoms of the disease. Thus, the game is not only useful for rehabilitation, but also for the evaluation of motor symptoms of the disease; (ii) evaluation of the game usability at a design stage, which can contribute to a significant reduction of errors and failures in the system. This can accelerate the game refinement stage after a pilot test; (iii) careful detailing of the game architecture in the form of easy-to-understand diagrams, in order to facilitate the comprehension of the developed software framework; (iv) implementation of the system in a modularized way in order to allow future inclusion of processes and features and/or changes in the software; and (v) evaluation of the accessibility of the game to ensure that individuals with PD are able to interact with the game.

Materials & Methods

Results

The architecture of the RehaBEElitation serious game

The general BPM describing the interaction between the actors of the system (patient/player and therapist/researcher) and the game processes is shown in (Attachment S7). Figure 1 shows the BPM for the calibration of the game. Figures 2 to 5 depicts the BPM from Phase 1 to 4 of the game. Figure 6 shows the BPM related to the data collection stage.

Discussion

The serious game RehaBEElitation was created to meet a real demand in the evaluation of upper limb motor signs in PD. The hand plays a fundamental role in our functional activities and its function can be severely affected in early stages of the disease, being of great relevance to evaluate motor signs in this region. Although there are initiatives in the literature for the assessment of upper limbs in PD using SG, there is a lack of development of methods that insert protocols from consolidated clinical tools, such as the MDS-UPDRS, in the game design. This is undoubtedly one of the great differentials of this study. By incorporating knowledge of clinical practice to the game, the result of the game will be more easily accepted by the clinical community and the patient will have an easier interaction with the game since he is used to perform certain movements during clinical evaluation sessions.

By incorporating a wearable technology to monitor and record motor signs of PD, the healthcare professional or researcher has at his disposal a tool to visualize events that are imperceptible to the unaided eye, complementing and expanding the usual clinical evaluation. Thus, using sensors that allow the professional to identify nuances about the disease may be important, and may even be the differential factor for identifying the beginning of a serious disease or a change of trajectory in the disease treatment. From this perspective, i.e., a real demand to accurately monitor some PD symptoms (bradykinesia, tremor or motor fluctuations), the RehaBEElitation emerges as an instrument that can be exactly useful in the clinical and even surgical environment, e.g., it can be used to evaluate the results of ablative surgeries and deep brain stimulation.

Several studies (Balci et al., 2021; Falla et al., 2021; Lubomski, Davis & Sue, 2020; Montanaro et al., 2022) point to a high incidence of depression, anxiety and other negative emotional aspects that accompany PD. This contributes to the patient not having the necessary engagement in rehabilitation processes or even clinical assessment. Thus, the use of a playful assessment scenario, such as the one provided by the game, can be the difference between a successful or an inaccurate assessment. It is known that the mere fact that a person with PD is being monitored can instantly affect their motor signs. Thus, if the person is immersed in the virtual reality provided by the game, the influence of external factors can be reduced, contributing to a more accurate assessment of the effectiveness of treatments and the clinical condition of the individual.

The requirements gathering stage for the development of the RehaBEElitation serious game considered the participation of patients and professionals with experience in PD (e.g., biomedical engineers, physiotherapists, physical educator, researchers, neurologist, neurosurgeon, and experts in computer science, ergonomics and human factors). Unlike several existing studies in the literature (Cikajlo et al., 2018; Fernández-González et al., 2019; Sánchez-Herrera-Baeza et al., 2020), the game was developed in such a way to place the end-user as the center of the development process. This may be fundamental for a good acceptability of the game. However, clinical evaluations should be conducted in the future to verify this hypothesis.

The game architecture was designed having in mind two actors: the individual with PD (e.g., the player) and the professional who will be performing the assessment (e.g., physical therapist, neurologist or a researcher). These actors can interact with the system at different times, and the game was modularized in order to customize interfaces for each type of actor. For instance, the therapist has exclusive access to glove calibration interfaces, user registration and experimental session customization. These modules are completely independent from other modules of the system, allowing future advances, such as the inclusion of modules for the generation of reports (e.g., for the presentation of the frequency of motor fluctuations, tremor and bradykinesia) and connection with other data management systems for people with PD (Folador et al., 2021b).

The system architecture considers the possibility of the user carrying out training sessions before experimental and clinical sessions. The independence of these steps is essential for the user to understand the game and execute the movements correctly, since it is through the execution of such movements that the clinical assessment will be performed. At the same time, the system requires that during the clinical assessment the user performs all the processes related to the four phases of the game to ensure the execution of a protocol that allows the assessment in different contexts, as recommended by the UPDRS clinical scale.

The system was modularized in order to implement parallel and concurrent programming at different levels. For instance, the microcontroller responsible for interfacing the glove with the game estimates all parameters independently from other running processes. This requires less computational effort, which contributes to the use of more accessible hardware, since in countries like Brazil the cost of computers is still high and modern computers are not always available in clinical environments. Furthermore, the independence between the processes contributes to the fluidity of the game and the control of the interface elements in real time.

Furthermore, the developed architecture allows the inclusion of new elements for monitoring the patient’s state. For instance, in the current version of the game, there is a module which is responsible for collecting and storing data from a commercial device (e.g., E4 wristband). This feature is desirable, since it extends the functionality of the developed tool without changing its fundamental characteristics. Thus, modularization also contributes to avoid obsolescence of the game, since other devices can be attached to it without compromising its basic operation. Finally, the inclusion of other sensors in the system enables the extension of the use of the game for the evaluation of new forms of motor and non-motor PD signs contemplated in its original design.

The developed system allows recording the biomedical and inertial signals of the participants. In addition, it records pre-defined events that can help the evaluation of the variability of biomedical signals during the period of the occurrence of the events. This functionality presents great clinical relevance, as it allows the assessment of the user in several contexts, for example, during the execution of different tasks and reception of different visual and sound stimuli.

The system design was developed in an incremental manner always considering the possibility of improvements due to unit test results (Fig. 6). Thus, even with the progress of the system development, new improvements were integrated into it according to the results of the usability evaluation performed at the design stage (Table 1). To evaluate the usability, the perception and experience of four evaluators were considered. In general, usability problems were identified with severity levels above 1 (that is, problems of medium, high or extremely high nature). By incorporating the necessary usability improvements to the game, we avoided that users are subjected to use a system still in an incipient stage (e.g., with basic usability problems), contributing to the good acceptance of the tool in the clinical scenario.

The heuristic evaluation performed contemplated all the 14 heuristics proposed by Nielsen-Shneiderman. It was possible to identify improvements to be implemented due to each of them, emphasizing the relevance of the use of heuristic evaluation still at the design stage of SG development. In a recent literature review, which contemplated the use of SG in PD (de Oliveira et al., 2021), it was not found any study that presented how the heuristics that aid the development of a game with good usability were contemplated. The application of this method in this study is also a contribution to the advancement of the design of SG for PD.

From the heuristic evaluation it was possible to identify the following usability issues classified as extremely serious (severity equal to 4): (i) the absence of a calibration step, enabling the adequacy of the game control to the motor impairment of the person with PD, could substantially affect the control of the game, causing serious gameplay problems; (ii) the meaning of a loss of data from an experimental session goes far beyond the loss of a file on a computer. This is a problem that can and should be seen as a usability problem. The loss of data may mean that the user is again exposed to a complex and long evaluation (since he may even be without the effect of medication) and that this previous experience may generate frustration and distrust in him, which totally interferes in the usability of the system. (iii) The lack of direct relationship between the evaluation performed by the game and that performed in a clinical scenario may generate problems related to the difficulty of understanding the tasks and performing movements, effectively jeopardizing the use of the game.

The game accessibility evaluation was performed to assess whether patients with PD were able to play the game in the same way as healthy patients (Shah et al., 2019). Statistical tests showed that the groups of healthy individuals and those with PD are homogeneous in terms of age. In general, as indicated by the MDS-UPDRS III and Hoehn and Yahr scores (Attachment S8), PD patients in the OFF medication state showed a higher degree of impairment compared to the ON state. However, despite all limitations, the assessment of the game accessibility revealed that RehaBEElitation can be played by individuals with PD in both medication states as well as by healthy individuals. Phase 3 was the only one that showed statistically significant differences between the scores of the healthy group with the PD group in the OFF state. As finger tapping is a movement that requires fine motor dexterity of the fingers, it is expected that a patient without the effect of a medication controlling the motor symptoms of the disease would indeed feel more difficulty in performing the movement than an individual without impairment.

Due to situations such as the one imposed by the COVID-19 pandemic, the need to develop technologies for remote use was observed. In this context, the RehaBEElitation serious game could be used for remote assessments; however, the way the game was implemented does not allow its use in this scenario at this time. In addition, the compliance with the heuristics for usability assessment also suggests that the game can be incorporated into clinical practice.

In the future, we intend to incorporate into the game components that also allow the assessment of lower limbs of individuals with PD, enabling, for example, gait assessment. In this way, it would be possible to perform a complete and objective evaluation of patients by using the developed system.

The system described in this article was registered with the Brazilian government (https://www.gov.br/inpi/en; INPI—registration number: BR 51 2021 001975 0) and some videos illustrating its operation are available as complementary material.

Conclusions

This study presented the architecture and the usability evaluation at the design stage of the RehaBEElitation game. The user-centered system was conceived from requirements of serious game projects, having in its conception the participation of a multidisciplinary team with experience in game development and PD. The proposed architecture was presented using the BPM in order to facilitate the reproduction of the system, and to extend its application to other scenarios of use considering the evaluation of other diseases. In the same way, the heuristic evaluation presented can serve as a guide for the development of new SG.

Some limitations of the game can be described, for instance, it cannot be applied remotely at home. This is due to the following reasons: (i) the game must be played in the presence of a professional, who can clarify doubts and observe the execution of the tasks; (ii) in several countries, such as Brazil, there is still no legal regulation that allows the use of systems of this nature at home. Thus, we chose not to incorporate this possibility of use, although the game architecture is flexible enough to allow future incorporation; (iii) the use of SG involves the registration of biomedical data, and the use of the system at home may compromise the data security; (iv) the use of games at home requires the availability of connection to stable network systems, often not available to users. Moreover, in the current version, the game cannot be played by hearing impaired patients, as they do not perceive important sound feedbacks emitted by the game. However, we are working to incorporate in the HMI device vibratory sensors to allow the delivery of tactile feedbacks to these individuals. Finally, the game cannot be played simultaneously by multiple players in the current version; however, the game architecture also enables the implementation of this feature.

In summary, the developed system allows the evaluation and monitoring of individuals with PD using an accessible serious game, objectively. The detailed description of the game architecture presented in this study, as well as the evaluation of the game usability at the design level can help professionals from different areas to develop more efficient systems and technologies.

Supplemental Information