Keywords

1 Introduction

Dementia is an umbrella term that includes a set of symptoms linked to disorders of the brain that progress over time and refers primarily to impairments of cognitive and motor functions [38]. A 2017 report suggested that 50 million people live with dementia worldwide [45] and this number is expected to rise to 132 million within the following thirty years [45]. This growth has made imperative the need for hospitalization and support of people with moderate to severe dementia with cognitive and motor function impairments. Even though hospitalization is necessary in some cases for People with Dementia (PwD), it is associated with negative effects on cognition and negative impact on the physical state of the person [1, 11, 19, 31, 50]. Therefore, hospitalization presents challenges that need to be addressed.

Exercise and physical activity in general have been associated with several benefits in mental and physical health for PwD. Many studies highlight that PwD who exercise on a regular basis have improved motor functions, including stronger upper and lower body muscle strength and better walking balance compared to those who do not engage in any type of physical activity [35, 39]. In addition, many studies report a negative relationship between cognitive decline and exercise, suggesting that cognitive decline could be delayed through systematic physical activity [22, 34, 35]. Thus, regular physical activity can improve functionalities related to the brain, preventing the further development of neurological conditions and cognitive decline [23].

Even though the benefits of physical activity and exercise are well documented, research also demonstrates that people living with dementia may experience several difficulties in taking up or continuing physical training. Indeed, barriers may arise from mental and physical impairments that limit or prevent patient engagement in exercise. In particular, as the disease progresses, PwD encounter difficulties with orientation, which can affect their understanding of the exercise task [40]. Attention, memory and language processing difficulties also interfere with their understanding. Therefore, it is especially challenging for a person with dementia to comprehend the exercise instructions given by the physician during the physical activity [48]. Another impediment related to dementia symptoms, is the increase of apathy, loss of motivation and interest in the self [5, 7, 20, 30] which contributes to patient reluctance to engage in any form of physical activity. As a result, most of the PwD find physical training to be boring and tedious [32, 42].

Interventions that aim to enhance the quality of life for people living with dementia can become difficult to achieve in more rigid settings including hospitals, where environmental and procedural restrictions are implemented depending on the risk that individuals may pose. As a result, further research is needed to develop novel interventions that can support and enhance care within confined environments. With this study, we aim to explore the use of Virtual Reality (VR) as a novel intervention to improve physical activity and therefore enhance the quality of life for patients with moderate to severe dementia who reside in a psychiatric hospital.

1.1 Related Work

Several studies that investigated the use of technology for physical training have shown promising results. One study [14] showed that the memory of PwD can be improved by playing a bowling game on Nintendo Wii on a regular basis. Another study [46] revealed that cognitive levels were improved by performing upper and lower limbs exergaming tasks, such as catching virtual coins or playing drums in front of a screen using customized controllers. Finally, improvements in cognition, gait and balance in PwD were also reported by a study which used the Wii Fit to compare virtual to normal walking tasks. This improvement was attributed to the high levels of presence experienced by PwD and was associated with exposure to the Virtual Environment [15]. It follows from these studies that VR can be an effective tool in improving the quality of life for PwD.

VR as a technology has been also used to improve physical training in healthy adults and athletes with promising results [27,28,29]. In general, most of the research related to VR and dementia focuses on the enhancement of specific skills which tend to decline within the course of the condition. Such skills include cognitiοn [4, 6, 22, 23, 34], memory [33], spatial navigation [8, 49], executive functions, such as planning activities [25] and attention enhancement [10, 24].

Within the context of Human-Computer Interaction (HCI), research mostly focuses on the specific needs of PwD interacting with technology. Past studies have examined the important elements that a virtual environment should incorporate in terms of design and development in both semi-immersive and fully-immersive modalities. An overall conclusion from past studies is that special focus should be placed on creating personalized experiences that take into account the preferences and skill level of each individual user [12, 13, 18, 43].

Our objective here is to discuss the applicability of VR for the physical training of people with moderate to severe dementia and to communicate our experiences in designing such a system. We present a set of guidelines, based on the issues encountered during the design, testing and execution of the experimental tasks, to enable the future development of VR applications for PwD in healthcare environments. To accomplish this, we present three interrelated studies that refer to: (1) System requirement analysis carried out through a workshop with experts in dementia care; (2) System interaction method assessment by testing two different types of interaction in VR, to identify the most suitable for PwD; and (3) A pilot study with patients performing three upper limb physiotherapy tasks in VR. To conclude, this paper presented an examination of issues encountered during the design, testing and execution of a VR system for physical training for people with moderate to severe dementia.

2 Apparatus

An Oculus Rift CV1 VRFootnote 1 head-mounted-display (HMD) was used to display the physical training tasks while interactions were effected using two Oculus Touch Controllers. Differences in Range of Motion (ROM) capabilities were measured with a goniometer (see Study 3, Materials) to avoid any frustration and enable PwD to successfully perform the exercise task based on the range of motion they were able to perform.

The VR experience was delivered using the Unity3DFootnote 2 game engine and the 3D models were created in MayaFootnote 3. The experience was also streamed to a laptop screen, mirroring the user’s real-time virtual interactions. This allowed the clinical staff and the researchers to observe silently the procedure.

3 Study 1. System Requirements Analysis

3.1 Study Design and Procedure

First, we conducted a thorough systematic literature review to inform the design of the VR application for a range of patients with neurodegenerative diseases, including those with dementia [41]. Then, a workshop was conducted to discuss potential VR environments and to identify the most suitable types of VR content with respect to the needs of people with moderate to severe dementia, informed by the knowledge gained from the systematic review.

3.2 Materials

Notes were taken by an HCI researcher during the workshop discussions. The aim of these observations was to identify potential VR environments and activities according to the needs of PwD.

3.3 Participants

Thirty-nine medical and paramedical personnel including clinical psychotherapists (n = 3), physiotherapists (n = 4), occupational (n = 2), speech (n = 2) and music/arts (n = 4) therapists, social workers (n = 1), gymnasts (n = 1) and caregivers (n = 22) attended the workshop.

3.4 Findings and Discussion

Study 1 included a 30-min consultative workshop to brainstorm about the suitable type of VR content for PwD. During the workshop, an HCI researcher provided an introduction of the potential uses of VR technology including information about what types of virtual environments were reported by previous literature as suitable for use with dementia patients. Environments included a cathedral, a forest, a beach, and a countryside [31, 50]. Then, all attendees rated the virtual environments independently (possible range between 0–2, where 0 = not suitable, 1 = neutral, and 2 = suitable). The highest-rated virtual environment was the forest. Therefore, it was agreed to use a forest as the backdrop to this study. A snapshot of the selected virtual environment is shown in Fig. 1.

Fig. 1.
figure 1

The three exercises performed during the training session.

Full size image

Then, the HCI researcher observed the regular physical training of five different PwD. After the training, four physiotherapists, two occupational therapists and a gymnast agreed on three essential tasks that a VR training session should include. The selected exercises were:

  1. 1.

    Overhead arm-stretching: Climbing up a vertical rope with 40 knots/targets. Each target represents each overhead arm-stretching move (Fig. 1A).

  2. 2.

    Overhead arm-raising: Climbing up a wall with 40 blocks/targets (Fig. 1B).

  3. 3.

    Seated cable row: Pulling a horizontal rope with 40 knots/targets towards their side (Fig. 1C).

All training activities involved upper body movements which can be performed actively from a seated position and are similar to the regular training routine. Seated position exercise training is mostly preferred in hospitalization training to avoid the risk of falling.

4 Study 2. System Interaction Method Assessment

4.1 Study Design and Procedure

The aim of this study was to assess the usability of different interaction methods in VR by PwD. Participants were asked to perform a boxing exercise via two conditions: (a) Touch Condition: Participants held a controller and touched a virtual ball by extending their hands, and (b) Touch and Grab Condition: Once participants touched the ball, as in the Touch Condition, they had to grab it using the grip button of the controller. Visual feedback was provided for each successful attempt by changing the colour of the ball from red to green (Fig. 2).

Fig. 2.
figure 2

Study 2 setting with a virtual ball turning green with each successful interaction (Color figure online)

Full size image

The study used a within-subject design and the order in which the two conditions were performed was counterbalanced to equate potential carry-over effects. The experiment lasted about 20 min. The findings prescribed the design factors for Study 3.

4.2 Materials

Accuracy and latency data from task execution were logged automatically by the computer. Additional data – independence, SQE and VAS – were collected by an HCI researcher located in the room where the VR physical training session took place. The metrics collected in each session were the following:

  1. 1.

    Task Performance. Assessed the ability to perform the boxing exercise correctly for 20 repetitions within a two-minute timeframe.

  2. 2.

    Task Independence. Evaluated by taking into account the number of times assistance was provided by the physiotherapist to the patient to perform the boxing exercise. More assistance yielded a lower score on this measure.

  3. 3.

    Time. Time was measured in seconds, from the first time that participants successfully performed the exercise until they completed it.

  4. 4.

    The Single Ease Question (SEQ) [44]. SEQ assessed the level of difficulty for each interaction on a 7-point Likert scale (1 = very difficult and 7 = very easy).

  5. 5.

    Visual Analog Scale (VAS) [6]. VAS was used as a psychometric response scale of depicted emoji to measure emotions (0 = happy and 5 = sad). The scale was used to allow patients to express their emotions towards each method of interaction. We asked patients to point to the specific emoji which matched their emotional state before and after each task execution.

4.3 Participants

Ten people with moderate to severe dementia (Male = 5; Female = 5), with ages between 75 to 85 years (M = 78.90, SD = 3.81), participated in this study. Their average number of years of living with dementia was 3.4 years (SD = 2.01) and ranged between 1–8 years. The mean Global Deterioration Scale rating (GDS; [36]) was 4.80 (SD = .92) “moderate to severe dementia” (range = 3–6: “mild to moderately severe”). Primary diagnoses included dementia in Alzheimer’s disease (n = 10) and secondary diagnoses included depressive episodes (n = 3), clinical anxiety (n = 3), and paranoid schizophrenia (n = 1). No participant had any prior experience using VR.

4.4 Findings and Discussion

Task Performance, Independence and Simplicity.

Only seven dementia patients were happy to use the VR system. The remaining reported experiencing claustrophobia (see Sect. 6) and did not complete the study.

An independent sample t-test revealed that patients scored significantly higher when performing the boxing exercise in the touch condition (M = .91, SD = .15) than in the touch and grab (M = .34, SD = .34); t (12) = 4.04, p = .002. In addition, an independent sample t-test revealed that PwD were depending more on the physiotherapist assistance in order to perform the exercise task correctly during the touch and grab condition than during the touch condition (M = 13.86, SD = 6.36) than during the touch and grab conditions (M = 1.86, SD = 2.91); t (12) = −4.54, p = .001. Finally, a significant difference was found in the scores of difficulty PwD encountered during the touch (M = 6.29, SD = 1.25) and the touch and grab (M = 2.57, SD = 2.44) conditions; t (12) = 3.58, p = .004.

The results suggest that PwD were able to complete the exercise tasks successfully, and with less assistance from the physiotherapist when simple interactions were assigned to the VR system. Our study validates the recommendation made by a previous study that suggested avoiding the use of buttons as control input when an interaction is required by people with mild dementia [13]. We also encourage researchers and developers to deliver simple interactions, avoiding the use of buttons as a control input.

In addition, we observed that, during the touch and grab condition, five out of seven PwD accidentally pressed the trigger button instead of the grip. Therefore, when buttons are considered necessary for the design, we propose to use the “trigger button” which is positioned to the front of the oculus controller instead of the “grip button” which is positioned to the side. We believe that patients naturally positioned their fingers to the front side of the controller because of the reduced finger mobility that is common with old age.

Learnability and Emotional Affect.

Task performance can also be assessed based on the time the PwD took to perform each task. Results showed that the time needed for the first completion in the touch condition (M = 2 s) was significantly lower than the touch and grab condition (M = 55 s); t (12) = −2.80, p = .016. Overall, most of the PwD were able to perform the task for the first time within 10 s of the experimental session in the touch condition, compared to about a minute for the touch and grab condition. Similarly, the analysis of the total time needed to complete the tasks revealed a significant difference between the touch condition (M = 38 s) and the touch and grab condition (M = 102 s); t (12) = −4.49, p = .001.

The response time affected, either positively or negatively, the intensity of the task performance. Therefore, and based on our findings, we suggest that it’s best to avoid complicated interactions with the use of buttons. To the best of our knowledge, this is the first study which examines the response time of the people with moderate to severe dementia for a given exercise task. Simple interactions where natural movements are used were found to provoke much more positive emotions to the PwD. This statement was validated by a 2 × 2 within-subjects ANOVA which compared the effect of the method of interaction on the patients’ emotions (VAS) at a pre- and post-exposure level. The results revealed a significant main effect of the method of interaction [F(1, 12) = 7.67, p = .017] with the touch condition evaluated as more enjoyable (M = 0.86, SD = 1.06) than the touch and grab (M = 3.28, SD = 2.06).

Based on the above results, we conclude that linking the buttons with the control input of the device is not a suitable solution for people with moderate to severe dementia. Therefore, we developed the VR system of Study 3 in accordance with these findings.

5 Study 3. A Pilot Study with Patients Confined at a Psychiatric Hospital Performing Physical Training in Virtual Reality

5.1 Study Design and Procedure

Patients were invited to use the VR system along with their own physiotherapist in a familiar room of the hospital. Their psychologist was also present to reduce the risk of behavioural disturbance and anxiety.

The experimental procedure followed six steps: First, patients were screened for inclusion. Then, a researcher observed each patient in care for 15 min prior to the session and recorded the person’s mood. PwD who were judged to be in a relatively good mood proceeded to a VR familiarization session. This was done to ensure that patients were able to use VR without any side effects (e.g., nausea).

Once the familiarization session was completed, patients took part in the VR physical training session (Overhead arm-stretching, Overhead arm-raising and Seated cable row as described in Sect. 3.4) (Fig. 3). During the session, quantitative data about task performance, reaction time, and independence of the patient were collected. Once the intervention was completed, patients returned to care where a familiar caregiver observed them for 15 min to record their mood. The maximum duration of the physical training session was set to 20 min to reduce the risk of adverse effects. Overall, each session lasted approximately 30 min.

Fig. 3.
figure 3

Dementia patients performing an overhead arm stretching exercise. Left: the physiotherapist assists the patient. Right: the patient performs the task on her own.

Full size image

5.2 Materials

  1. 1.

    Goniometer. A goniometer was used by the physiotherapist to assess ROM for each task. Each exercise task was personalized based on the pre-set ROM of each patient to ensure that training was carried out with the appropriate ROM and difficulty.

  2. 2.

    Task Performance. Measured by the number of times each task was performed correctly out of 40 repetitions.

  3. 3.

    Task Independence. As in Study 2, it was computed based on the number of times assistance was offered by the physiotherapist to the patient to perform the boxing exercise. ore assistance yielded a lower score on this measure.

  4. 4.

    Time. The time was measured in seconds and included: (1) the time participants needed to perform the exercise correctly for the first time (first repetition), (2) the overall time participants spent on each exercise, and (3) on the full VR session.

  5. 5.

    The Single Ease Question (SEQ). SEQ assessed the system usability (1 = very difficult and 7 = very easy).

  6. 6.

    Visual Analog Scale (VAS). VAS assessed the participants’ emotional state. The scale was administered before and after each session.

  7. 7.

    Observation Notes. Detailed observation notes were taken by an HCI researcher to classify the patient’s interactions and behavioural responses to the VR experience.

5.3 Participants

Ten people with moderate to severe dementia (Male = 5; Female = 5), aged between 65 to 90 years (M = 80.10, SD = 7.11) participated in Study 3. The average number of years of living with dementia was 3.5 years (SD = 1.35) and ranged between 1–5 years. The GDS mean was 4.71 (SD = .95) “moderate to severe dementia” (range = 3–6: “mild to moderately severe”). Primary diagnoses included: dementia in Alzheimer’s disease (n = 6) and unspecified dementia (n = 4). Secondary diagnoses included: depressive episodes (n = 4); clinical anxiety (n = 4) and paranoid schizophrenia (n = 1). Table 1 presents descriptive statistics about the goniometer data of the PwD ROM.

Table 1. Goniometer data (means and standard deviations).
Full size table

5.4 Findings and Discussion

Task Performance, Independence and Simplicity.

The overall mean success of patients in performing the overhead arm stretching exercise was .74 (SD = .22), for the seated cable row exercise .78, (SD = .21) and for the overhead arm raise exercise .84 (SD = .25). The high rates of VR task performance demonstrate the ability of people with moderate to severe dementia to use VR technology for their daily exercises.

In addition, and similarly to the usual care, assistance was requested in order to perform the tasks accurately. In particular, patients were in need of assistance approximately 14 times out of 40 during the overhead arm stretching exercise (M = 13.71, SD = 13.09), 12 times during the seated cable row exercise (M = 11.57, SD = 13.41) and 11 times during the overhead arm raise exercise (M = 11.00, SD = 15.69). However, it should be noted that even though assistance was required in order to perform the exercise tasks correctly, participants, in general, reported low difficulty in carrying out the exercises (M = 5.00, SD = 1.63).

These results suggest that VR can enhance and support the PwD physical training when assistance is provided by the physiotherapist. Coupled with the findings of Study 2, these results suggest that less complicated tasks can reduce the need for assistance and lead to improved levels of independence by the patients.

Learnability and Emotional Affect.

A paired sample t-test revealed a significant difference in the scores of the emotional state of PwD in pre- (M = 1.46, SD = .55) and post- (M = .76, SD = .29) VR exercising exposures, suggesting that VR exercise can significantly improve patient emotional states; t (6) = 3.26, p = .017. Previous research suggested that approximately half of the long-term hospitalized PwD are diagnosed with depression, which contributes negatively to the progression of the disease [20]. In the past, clinical research has shown that VR has been used successfully to enhance the treatment of mental health disorders such as depression, anxiety, and phobias [21, 26]. Here, we found that VR can have a positive effect on PwD. We consider this finding to be of vast importance since research has shown that depression leads to increased apathy and loss of motivation in dementia patients, which impacts negatively the engagement to physical activity [20, 30]. If VR can reduce the negative emotional responses in dementia, this could have a positive effect on the deceleration of the disease.

We also measured the time participants needed to successfully complete each exercise task for the first time. This was done to validate the simplicity of the requested task and to assess the understanding of task instructions. Descriptives statistics are presented in Table 2.

Table 2. Measured time means in seconds of the three exercise tasks by PwD.
Full size table

6 Implications and Recommendations Based on Content Analysis

A qualitative content analysis of the observational data was also conducted to discuss in depth the problems faced during the design, testing and execution of the experimental tasks and present recommendations for future deployment in healthcare services. Observed experiences regarding the more optimum design of the experimental setup were elaborated with quotes from our observational notes. The data were anonymized, and participants referred to as [PwD Number and Physiotherapist/Psychologist Number]. For example, a quote by the first patient and his/her physiotherapist is presented as: [Patient 1, Physiotherapist X].

Accept that VR is not for every PwD.

In contrast to other kinds of digital technologies, such as flat-screen interfaces, tablets and mobile devices, fully-immersive VR systems envelop the users view completely. Because of the material factors of the system, three out of ten PwD, all women with a mean age of 78.2 (SD = 3.78), refused to put on the HMD. These three patients reported suffering from severe clinical anxiety.

figure a

This is not the first time such responses were observed in PwD. Previous research on VR and patients with mild dementia reported that handheld systems (i.e., being able to hold the HMD with their hands) were preferred [18] since the system can link to a sense of claustrophobia [43]. However, and based on the nature of our study, such a solution was not an option, since patients had to perform upper body exercise tasks. We believe that building an exoskeleton to hold the VR display might increase participants tolerance and acceptance towards the device. It could also reduce the incidence of catastrophic behaviour, which yields a risk for the patient, the staff and the system.

Give Constant Feedback.

As dementia progresses, cognitive functions such as attention, memory and language comprehension, decline and interfere with the understanding of the exercise tasks [40, 48]. Therefore, to overcome such barriers, feedback for the upcoming movement should be provided to the PwD as a reminder. Our VR system used arrows to provide visual feedback for each target and to remind participants of their next move. We found that the visual feedback was confusing for one patient who often reached to grab the arrows instead of the targets. We observed that verbal instructions which were given by the physiotherapist to the PwD during the VR session, helped patients’ attention to the exercise task and improved performance.

figure b

These observations demonstrated that the verbal reward, via positive words such as “Bravo and Good”, might increase the PwD’s motivation to continue performing the given task. Previous research suggests that positive emotions activate brain areas related to reward, playing a critical role in motivation [2]. It is also documented that pleasant activities and rewards can relieve stress and improve motivation [47]. We, therefore, suggest incorporating in VR design, expressions that will present some form of motivation and approval for PwD.

Boost the Visual Targets.

Comparing the findings across Studies 2 and 3 we conclude that it is better to present the virtual targets as big shapes, with sharp colours and within the front field of view of PwD. Targets should also be stable as this is shown to improve performance. With respect to the observational data, during the seated cable row exercise, one patient grabbed the virtual rope and pulled it downward. The physiotherapist had to bring the patient back to the correct seated position to prevent hitting the ground.

figure c

In addition, it was observed that at some point during the exercise, all the patients tended to get confused and repeated the same steps for a couple of times. Patients repeatedly moved first in one direction and then in the opposite direction, instead of moving forward. In these cases, the physiotherapist assisted the patient. Based on this observation, we recommend avoiding the use of moving targets. We developed the VR system to simulate real-world interactions, and follow real-world physics rules. However, we suggest simplifying some interactions to avoid confusing PwD.

figure d

Include Content of Natural Environments.

Another impediment related to dementia symptoms are the experience of depressive episodes, the increased apathy and loss of motivation, along with the negative attitudes towards the engagement in physical activity [20, 30]. Therefore, to create an engaging and fun VR physical training we used a natural environment which was found to motivate PwD [16] while offering a pleasant experience with positive psychological effects, such as decreased tension, anger, and depression [3, 9, 17]. Our patients were found to enjoy the natural environment view.

figure e

Content of the Process of Reminiscence.

It was also observed that VR was able to trigger old memories for PwD. This is not the only study to suggest that VR is able to evoke positive memories on topics of family, geographical origins and travels [37]. In the past, research suggested that the recreation of experiences via VR could be very beneficial for PwD [18]. Therefore, we suggest that the design community aims to deliver personalised VR environments that might trigger memories of past experiences. 

figure f

Offer a Smooth Transition to Reality.

It is well documented by the literature that after VR exposure, PwD might experience a sense of disorientation [4, 43]. Once the VR HMD was removed from their head, our patients exhibited some disorientation.

figure g

We acknowledge the importance of smoothly transitioning PwD back to the physical space. Since this incident of disorientation happened at the very first patient’s exposure we then asked the physiotherapist and/or psychotherapist to inform the PwD when the experiment was over, and then to remove the HMD. No visual cues were presented to the PwD during the time the staff was informing them of the end of the session. We found that providing verbal prompts before removing the VR HMD improves the patient’s transition to the physical space. Having PwD for a couple of seconds without any visual cues might also improve smooth transition. We recommend for future VR development to incorporate features that will explain to the PwD that the experience is over. For example, a blue screen with a congratulatory message might appear. A soft voice might enhance this by informing participants that the experiment is over.

7 Conclusion

This paper presented an examination of issues encountered during the design, testing and execution of a VR system for physical training for people with moderate to severe dementia. The system was deployed into a confined mental health unit and thus the paper presents recommendations for designing VR systems for healthcare services. An iterative participatory design process was followed with input from health experts coming from diverse fields and from 20 people with moderate to severe dementia. Two methods of interaction and three different types of exercise were designed and tested in VR. Our analysis revealed the potential of VR physical training for PwD and presented a set of guidelines and recommendations for the future deployment in healthcare services: 1) continuous feedback should be provided to the PwD in order to perform the task correctly and without much supervision and assistance from the clinical staff; 2) visual targets should be stable, visible, and within the frontal field of view of the PwD so that the patients could interact easier and appropriately with the system; 3) personalised virtual environments relevant to the PwD’s interests and past memories improve the patient’s immersion and engagement, and 4) transitioning smoothly the PwD back to reality is of vast importance to reduce the risk of distress and disorientation.

The current study contributes to the emerging body of research in the use of VR technology with people with moderate to severe dementia. We trust that this paper lays the foundations for the future deployment of VR to enhance physical training for PwD in confined psychiatric hospitals.