Patient Acceptance of Self-Monitoring on a Smartwatch in a Routine Digital Therapy: A Mixed-Methods Study

2.1.1 Traditional Self-Monitoring.

The traditional way for patients to record their moods would be to keep a log on a paper diary. However, this approach presents several issues. Studies have shown that patients were likely to forget to report in the moment and tended to complete their logs retrospectively [95, 101]. In addition, retrospective logging is dependent on the person’s ability to accurately reflect on their past moods. It is also subject to recall biases [37, 99], including mood-related bias, or when mood recall is substantially influenced by the mood at the time of recall [12], events salience, which is the tendency to recall (infrequent) significant events [102, 104], and depressive symptoms, such as the tendency to recall negative information [12, 51, 98]. The task of self-report itself was also experienced as repetitive and binding because of the necessity to carry a journal throughout the day [111].

2.1.2 Mobile Self-Monitoring.

The advent of smartphones has facilitated Ecological Momentary Assessment (EMA) [103], and mood monitoring has become a frequent activity for a number of people [29]. EMA on a smartphone enables individuals to log their mood in the moment, avoiding retrospective reflection and therefore minimizing recall biases, which is particularly relevant in depression given its associated memory impairments [78]. EMA also permits the collection of contextual information (the time of the day, activity level, etc.) to situate the mood in one’s daily experiences [124]. For these reasons, EMA is becoming increasingly used for mood monitoring in individuals with mental health difficulties [24, 112]. Research also started to move away from fully manual EMA, which is said to be “limited by human effort” [57, p. 473], toward automated monitoring via sensors embedded in smart technologies [41]. This was particularly motivated by the potential for automatic data collection to improve self-report accuracy, lessen the risk of missing records, and reduce the burden associated with manual data entry [7, 35, 70]. Finally, research has explored semi-automated monitoring, combining manual self-report with automatic data collection to support patient awareness, long-term engagement, and sense of agency [21, 125]. Initial findings from this exploratory work indicate that semi-automated monitoring can be successfully used to monitor sleep and physical activity in mental health interventions [1, 6, 123], and leveraged in commercial apps [13, 79].

2.1.3 Self-Monitoring on a Smartwatch.

Wearable devices are increasingly used by the general population, particularly smartwatches for the monitoring of health-related behaviors (e.g., sleep and exercise [76]). Smartwatches have advantages over smartphones in this regard, affording the possibility of continuous monitoring of a wider range of physiological variables, the ability to provide biofeedback [116], and greater convenience [73]. In particular, most smartwatches are specifically designed for collecting and processing physiological and contextual data, giving insight into numerous facets of a person’s life. As a result, commercially available smartwatches have begun to be used in healthcare research [114] for applications such as the manual self-monitoring of symptoms (e.g., knee osteoarthritis [8]), automated self-monitoring of physiological irregularities (e.g., atrial fibrillation [75, 109, 113]), and detection of unhealthy behaviors (e.g., cigarette smoking [23, 97]). Studies have also explored the use of automated self-report on wrist-worn devices for mental health diagnosis (e.g., the detection of depression through sleep and heart rate monitoring [122]), interventions (e.g., understanding outcomes of therapy for social anxiety through heart rate and movements [11]), and symptom monitoring (e.g., physical activity monitoring to detect depressive symptoms [17, 72], stress [32], and schizophrenia relapse [85]). In addition, because mental health stigma has been a long-standing barrier to help-seeking and intervention compliance [2, 9], it is essential that self-report technologies maintain a high level of privacy. Smartwatches also enable EMA through microinteractions, potentially reducing the burden associated with self-report and supporting better user engagement [77]. Therefore, smartwatches’ proximity enabling immediate, discreet, and private interaction make them good candidates for supporting mood self-monitoring [26, 62].

To conclude, although smartwatches are widely used by the general population, their use in clinical contexts is still at a very early stage. Patients’ willingness to engage with self-report on a smartwatch will significantly be influenced by their acceptance of the device and its sensing capabilities [92]. Therefore, research aiming to use smartwatches with patients should investigate their acceptance of the technology and do so in real clinical settings [87].

2.3.1 Measurement Tools.

A common approach to measure acceptance involves the evaluation of each potential acceptance factor (e.g., perceived threat [52]) against self-reported usage behavior. The majority of studies employed questionnaires, relying on Likert scales [38, 47, 52, 117–121]. Each factor was evaluated through a number of measurement items, ranging from a minimum of 2 to a maximum of 11. Davis et al. [31] described the development of these measurement items as follows: (1) generating 14 candidate items for each construct, based on their definition; (2) pre-testing these items to refine the wording; (3) narrowing down the set to 10 items per construct; (4) assessing the reliability and validity of this subset; (5) narrowing down to 6 items per construct; and (6) repeating the validity assessment and narrowing down to 4 items per construct. Some studies piloted the questionnaire with focus groups [33, 117, 119], or a sample of users [33]. Most studies used Cronbach’s alpha coefficients to assess the internal consistency of the measurement questionnaire.

2.3.2 Measurement Timeline.

Most validation studies evaluated technology acceptance at different time points in the user journey. For instance, the TAM study [31] looked at the stages of pre-use, asking participants to fill in the first acceptance survey after watching a demo of the system, and post-use, giving the second questionnaire after 14 weeks of use. Drawing on this methodology but going a step further, the following studies assessed user acceptance at three time points in the user journey, including pre-use, after 1-month use, and after 3-month use [117–121]. This aligns with the body of work (later published) theorizing acceptance as a multi-stage process [34, 42, 56, 86, 100, 107]. Surprisingly, fewer recently published validation studies have taken such a longitudinal approach, instead assessing user acceptance at one single point, post-uptake of the technology [33, 38, 47, 52].

3.2.1 Supporting Consistent and Accurate Self-Report.

The core functionality of the Mood Monitor watch app is the mood self-report. Taking advantage of the ubiquity of the watch, this was designed in the form of an EMA, for the collection of mood data in daily life. The EMA was implemented by prompts on the watch screen, reminding the user to log their current mood at random times of the day (Figure 2(a)), to account for the variance of mood across the day [124]. Prompts are generally an effective way to get users to engage [10]. However, receiving prompts in this context (via a wearable device and as part of a clinical mental health intervention) might be perceived as intrusive and might be an obstacle to patient acceptance of the smartwatch app.

Fig. 2.

View Figure

After logging their mood, patients are presented with an evidence-based list of lifestyle elements likely to influence their mood, and are asked which one(s) they think might have affected it (see Figure 2(b)). Reflecting on these lifestyle factors is key in the identification of patterns in one’s behaviors and mood. Although interaction with this screen is optional (as in the desktop and mobile app), prompting patients with possible options is a step for encouraging introspection and action. The Mood Monitor app also contains a menu¹ enabling the independent logging of moods (Figure 3(a)).

Fig. 3.

View Figure

In addition to the mood logging, the Mood Monitor smartwatch app integrates the monitoring of daily data related to sleep (bedtime and number of hours slept) and physical activity (step count), giving context to the patient’s moods. Previously requiring manual logging, the monitoring of these lifestyle factors is automated on the smartwatch, therefore enabling a passive self-report, less subject to bias, human error (e.g., when typing in entries), and inconsistencies (e.g., gaps in the data). This automatically captured lifestyle data is available to both the patient and their therapist, and therefore has the potential to inform therapy sessions toward better personalization.

To complete its integration with the digital therapy platform, the mood and lifestyle entries recorded on the Mood Monitor smartwatch app are automatically uploaded to the online platform and accessible in the patient’s personal space.²

3.2.2 Encouraging Introspection and the Identification of Mood Patterns.

The Mood Monitor smartwatch app is also designed with the aim to encourage patients’ reflection and support them in identifying patterns. Although awareness might be gained from simply recording one’s mood and lifestyle habits, encouraging deeper reflection is essential to support behavior changes. After logging their mood on the Mood Monitor app, the user is directly brought to the application’s home screen, and is immediately presented with a visualization of their mood, bedtime, hours of sleep, and step count throughout the past week (see Figure 2(c)). Providing a detailed view of how the patient has been doing, this visualization acts as a prompt to reflection.

The app further encourages behavior change through cues materializing the user’s progress. First, three arrows under the current day’s report compare the lifestyle variables to those of the previous day—for instance, showing an improvement in the bedtime (up arrow for earlier bedtime), a stable time asleep (right arrow), or a decline in the step count (down arrow). Second, this progress is made visible in the icons, as their appearance evolves depending on how the patient has been doing: the moon representing the hours of sleep fills up, and the step count jogger either walks, runs slowly, or runs fast. Finally, to support frequent and consistent logging of mood, encouraging prompts (validated by a clinical psychologist) were displayed³ when users reached goals regarding usage of the mood logging, and the maintenance of a sleep routine, acting as positive reinforcement (Figure 4).

Fig. 4.

View Figure

Finally, we explored the addition of psycho-education snippets on the smartwatch (validated by a clinical psychologist) in the form the ‘Tips to stay well’ feature accessible from the application’s menu (see Figure 3(b)). Patients can go through this list of 31 brief educational pieces on lifestyle choices that may influence depression symptoms by tapping the ‘pulsing’ lotus icon.

3.2.3 Minimizing Patient Burden and Perceived Stigma.

The mood logging on the smartwatch was intended to be as effortless as possible for patients, to support their engagement with the self-report activity. This was implemented through the immediate interactions enabled by watchOS notifications. Due to the proximity of the watch, time is also saved in terms of the user not having to reach for their phone and logging into the program. In addition, with passive monitoring of sleep and physical activity, patients are not required to type in this information at the start and end of each day, thus eliminating a repetitive task that risked impeding patient engagement. The proximity of the watch, along with its small screen and discreet interactions (quick vibration on the wrist), enable the Mood Monitor app to deliver self-report more subtly and privately than via a smartphone or computer [63].

In line with the call for personalization in mobile health interventions [4, 50, 115], we chose to let users increase/reduce reminder frequency (from the default twice a day recommended by a clinical psychologist) by switching On/Off time ranges, at the first launch of the app (see Figure 3(c)).

Finally, creating consistency between the Mood Monitor watch app and the default iPhone and Apple Watch apps was important to support learnability and ease of use. The design of the weekly visualization of the moods and lifestyle habits required several iterations, to find the best way to display a large amount of information in a sufficiently concise format to fit the small smartwatch screen. After reviewing existing applications representing information over time on the watchOS, we eventually decided on a visual similar to that of the Apple Watch Weather app.

4.3.1 Measurement Timeline.

Our longitudinal approach to evaluate acceptance considered the three stages of the TAM (introduced by Nadal et al. [68]): pre-use, initial use, and sustained use. By conducting repeated measures of acceptance following this timeline, we aimed to identify the different facilitators and barriers to user acceptance at each stage, and used this data to inform the design of the technology.

4.3.2 Patient Demographics.

Multiple validated acceptance models examine user demographics [25, 120, 121], to enable detection of possible acceptability issues in specific user groups which would impede uptake of the technology. To examine possible associations between users’ demographics and pre-use acceptability of the technology, we asked participants, upon giving consent, to complete socio-demographic details online, including information on gender, age, ethnicity, employment status, marital status, and their experience with the smartwatch technology. For each question, a “Prefer not to answer” option was present.

4.3.3 Acceptance Mediators.

Grounded in the research validating acceptance models discussed previously, we examined user acceptance through the lens of acceptance mediators. This involved selecting among the existing validated acceptance models the best fit for our population and study context, and measuring its acceptance mediators. Because there presently exists no acceptance model specific to the mental health care context, we decided to draw on the HITAM developed by Kim and Park [52]. The HITAM includes the following acceptance mediators: perceived threat, perceived usefulness, perceived ease of use, attitude, behavioral intention, and usage behavior. As the study validating the HITAM did not provide the measurement questionnaire used, we reused question wordings from other validation studies measuring the same mediators. This resulted in the proposed AQ, which contains 15 measurement items, answerable through a 5-point Likert scale from strongly agree to strongly disagree. We assessed the key outcome usage behavior by collecting the amount of mood, sleep, and activity data recorded by each participant. The AQ was sent to participants at pre-use (day 0, T1), initial use (3 weeks, T2), and sustained use (8 weeks, T3). These three versions can be found in Appendix B. We adjusted the wording of the AQ to these measurement time points by referring to expected, present, or past experiences. Additionally, the number of times each participant accessed the ‘Tips to stay well’ feature was recorded.

4.3.4 Patient Satisfaction.

A review by Nadal et al. [68] showed that user satisfaction was often considered a factor of acceptance of digital health [15, 40, 59, 71, 89]. In the present study, we were interested in possible associations between long-term technology acceptance and patient satisfaction. At 8 weeks, along with the third AQ, we asked participants to complete the Satisfaction with Treatment measure [82], a 5-item questionnaire that had been used previously to evaluate patient satisfaction with the Space from Depression program [82].

4.4.1 Open-Ended Questions.

Each time participants received the AQ, they were given the opportunity to write about their experience with the Mood Monitor app. At T1, an open-ended question asked participants the reasons they had decided to use the watch app and enroll in the study. Similarly at T2, they were asked “How would you describe your use of the watch app?” and “What difficulties (if any) did you experience while installing or using the Mood Monitor app?” Finally at T3, a series of open-ended questions (see Appendix C) explored supplementary acceptance factors, which according to previous research might impact patient acceptance: satisfaction [15, 40, 59, 71, 89], engagement [84, 96], recommendation rate [16, 50, 89, 96], sharing [58], privacy protection [20], resistance to change [38], and match with expectations [89].

4.4.2 Post-Study Interviews.

Participants in the smartwatch group were offered to take part in a post-study interview to discuss their experience with the Mood Monitor. These semi-structured individual interviews were conducted by the first author; they informed each interviewee that they were a researcher and not a clinician, and explained the risk management procedure in place.⁷ Eight of the 35 participants took part in the interview, including five females and three males.

4.4.3 Thematic Analysis.

The data from both the open-ended questions and interviews was analyzed through a reflexive thematic analysis conducted by the first author (CN). In light of recent calls for more detailed reporting of thematic analysis practice in healthcare HCI [14], we have endeavored to provide a comprehensive methodological account of our analytic process. CN followed the approach of Braun and Clarke [22] and adopted a realist perspective, through which she analyzed participants’ words in a way that “treats language as though it is a direct conduit to the participants’ experience” [108]. CN took handwritten notes during the interviews. After talking to each participant, she would then reflect on her notes, complete them, and type them up in a Word document. CN then transcribed the interviews and checked the transcripts against the audio recordings. She familiarized herself with participants’ answers to the open-ended questions of the AQ by reading through them and formatting them into a unique document. She then conducted a first round of coding where she inductively coded patterns of meaning in the data. In a second round of coding, she selected those codes relevant to the research questions. The flexibility of this analysis method allowed for inductively coding the data and organizing the codes chronologically following the TAL. Finally, CN drew on the codes to generate themes and structured the analysis around the three stages of the TAL: pre-use acceptability, initial use acceptance, and sustained use acceptance.

4.4.4 Positionality Statement.

This statement reflects on CN’s positionality with regard to the collection and reflexive thematic analysis of participants’ qualitative data. Through her most recent works, CN has learned the importance of adopting a reflexive approach throughout the research process. Although such approach was not explicitly followed during this study, CN chose to engage in reflexivity a posteriori, and this statement reports this reflection. The study presented in this article is part of CN’s doctoral research on user acceptance of health and mental health care technologies. CN is a white, middle-class, cisgender woman in her late 20s. CN emigrated from France to Ireland, where she completed her doctoral degree and now works as a postdoctoral researcher in healthcare technology. Her background includes a master’s degree in HCI and a bachelor’s and technical degree in computer science. CN’s education reinforced her will to help vulnerable populations through her work; this led her to step into the field of mental health care. CN’s own mental health journey includes experiences of talking therapy and mindfulness sessions. As a feminist, gay woman, and supporter of human rights, CN has a special interest for design justice [27]. While she designed and implemented the Mood Monitor smartwatch app, she was conscious of the biases attached to her own identity and those related to being extensively involved in the development of the technology investigated. CN sought to maintain a critical position during data collection and analysis, and this being CN’s first experience developing an app for a smartwatch helped with conserving a critical view over her work. She also encouraged interview participants to be honest in their comments, explicitly distancing herself from the “development team” and never mentioning her role in the technology design.

4.5.1 Patient Clinical Outcomes Before and After Therapy.

Patients involved in the trial received the usual procedures for Improving Access to Psychological Therapies (IAPT, [93]). This included routine clinical assessments of patients with the Patient Health Questionnaire 9 (PHQ-9), Generalized Anxiety Disorder 7 (GAD-7), and Work and Social Adjustment Scale (WSAS) [39, 64]. To evaluate how the introduction of the smartwatch app impacted on patient outcomes, we compared the clinical scores of participants in both groups, obtained before and after the 8-week therapy.

4.5.2 Patient Usage of the Therapy Program.

We compared usage metrics between groups, including the total time spent on the platform, number of sessions, number of tools used, percentage of the program viewed, and number of reviews (metrics are detailed in the study protocol [66]). We also looked at usage of the self-report in both groups to ensure that the addition of the smartwatch was not detrimental to patient self-monitoring (e.g., patients might perceive the smartwatch prompts as a nuisance and choose not to wear the device).

4.5.3 Patient Acceptance of Self-Report.

We compared patient acceptance of self-report via the smartwatch app (experimental group) with that of self-report via the usual therapy platform.

By introducing the smartwatch app, we enable self-monitoring via an additional device and interface, separate from the usual therapy platform. Therefore, the survey questions related to interactions with the interface (“My interaction with the watch app is clear and understandable” (PEOU3)) or with the application as a whole (“Overall, I think that the watch app is useful in managing my mental wellbeing” (PU3)) in the experimental group do not find an exact equivalent for the treatment as usual group. Rather than transposing these questions into approximate equivalents, we chose to compute acceptancescores with a subset of 12 identical, equivalent questions.

6.3.1 Perceived Threat.

The perceived threat mediator reflects users’ concerns about their mental health and their willingness to take action to get better [52]. The repeated measures show that the perceived threat remained high throughout the intervention. Scores ranged from 3.0 to 5.0 (M = 4.600, SD = .4971) at pre-use, from 1.5 to 5.0 (M = 4.200, SD = .7611) at initial use, and from 2.5 to 5.0 (M = 4.370, SD = .6877) at sustained use. There was no evidence of a statistically significant difference between the scores across the three time points (F(2, 44) = 3.122, p = .054), which means that patients’ mental health concerns remained throughout therapy and so did their will to improve. Because a significant change in perceived threat (and therefore in overall technology acceptance) could be a result of improved/worsened clinical outcomes, examining patients’ clinical trajectories might help interpret this result if differences were observed.

6.3.2 Perceived Usefulness.

The perceived usefulness mediator reflects the degree to which users believe that the system will help improve their mental health [30]. Perceived usefulness of the Mood Monitor app was high overall throughout the measurement points. Scores ranged from 3.0 to 5.0 (M = 4.390, SD = .6022) at pre-use, from 3.0 to 5.0 (M = 4.222, SD = .7183) at initial use, and from 2.0 to 5.0 (M = 4.160, SD = .7919) at sustained use. There was no evidence of a statistically significant difference between the scores across the three time points (F(2, 44) = 1.303, p = .282). In the context of digital mental health, an interesting issue that emerged from discussions with clinical researchers is the potential for decreased symptoms to negatively impact perceived usefulness of the technology. As for the previous mediator, in cases where a significant change in these measures is noticed, researchers interpreting this data might find value in examining patients’ clinical trajectories.

6.3.3 Perceived Ease of Use.

The perceived ease of use mediator indicates the degree to which users believe that using a system will be free of physical and mental effort [30]. Findings reveal that participants rated self-report on the smartwatch as easy to use across the measurement points. Scores ranged from 3.0 to 5.0 (M = 4.293, SD = .6227) at pre-use, from 3.3 to 5.0 (M = 4.333, SD = .5545) at initial use, and from 3.0 to 5.0 (M = 4.407, SD = .6169) at sustained use. There was no evidence of a statistically significant difference between the scores across the three time points (F(2, 44) = .240, p = .787). Observing a significant improvement in perceived ease of use over time could be the marker of a learning curve in the use of technology; researchers in that case might find value in examining additional acceptance antecedents, such as computer self-efficacy [38, 52, 106, 117, 118].

6.3.4 Attitude.

The attitude mediator reflects the user’s overall affective reaction to using a technology [120]. The data show that users’ attitude toward use of the self-report on the smartwatch remained high over time. Scores ranged from 3.0 to 5.0 (M = 4.700, SD = .5402) at pre-use, from 3.3 to 5.0 (M = 4.708, SD = .4738) at initial use, and from 3.3 to 5.0 (M = 4.630, SD = .5606) at sustained use. There was no evidence of a statistically significant difference between the scores across the three time points (F(2, 44) = .195, p = .824). Although patients’ affective reaction to technology remained positive in our study, changes in attitude should be investigated because they can signify that users are not comfortable with aspects of the technology (e.g., sharing of personal information).

6.3.5 Behavioral Intention.

The behavioral intention mediator represents the degree to which individuals are willing to try to use a technology [3]. Despite a slight decrease in the participants’ intention to self-monitor on the smartwatch, the measure remained high throughout the intervention. The null spread at pre-use reveals patients’ strong willingness to take up the technology. Scores ranged from 4.0 to 5.0 (M = 4.829, SD = .3824) at pre-use, from 3.0 to 5.0 (M = 4.633, SD = .6149) at initial use, and from 1.5 to 5.0 (M = 4.352, SD = 1.0078) at sustained use. There was no evidence of a statistically significant difference between the scores across the three time points (F(2, 44) = .633, p = .536). Similarly as with perceived usefulness, a significant change in behavioral intention could be the result of an improvement/worsening in clinical symptoms. Moreover, such change might also be induced by users’ satisfaction (or lack of) with the system. Thus, examining clinical trajectories and user satisfaction might help with interpreting a change in individuals’ behavioral intention.

6.3.6 Usage Behavior.

The study of acceptance aims to determine facilitators and barriers to use of technology. Models of technology acceptance represent usage behavior as the final stage toward which the influence of the mediators is oriented. Examining usage behavior therefore provides direct insight into how user acceptance translates (or not) into usage. We first considered patients’ use of mood self-monitoring; this revealed that the large majority of patients (30 out of 35) engaged with mood recording on the smartwatch (Figure 10). The number of moods recorded on the smartwatch ranged from 0 to 167 (M = 27.03, SD = 38.311). In the group of 30 participants who used the mood monitoring, we observe differences in usage behavior, ranging from sporadic use of mood logging in half the group to more consistent use in the other half. In comparison, the number of moods logged on the desktop/mobile app (M = 3.14, SD = 10.244) ranged from 0 to 13 (Figure 11). The data was non-normally distributed. A Wilcoxon signed-rank test showed evidence of a statistically significant difference between the number of moods loggedon the smartwatch versus on the desktop/mobile app (Z = –3.529, p = .000).

Fig. 10.

View Figure

Fig. 11.

View Figure

We also observed significantly more consistent use of the mood self-report in the smartwatch group compared to the treatment as usual group, as illustrated in Figure 12, and confirmed this using a Mann-Whitney test⁹ (U = 336.0, p = .002). However, although smartwatch participants tended to record their moods with the smartwatch rather than with the desktop/mobile app, it is likely that the prompts sent by the smartwatch app further supported consistent self-report. Overall, these findings indicate that the high levels of acceptance captured by the AQ scores translated into actual use of mood logging on the smartwatch for most patients. Among the five participants who did not engage with the mood monitoring on the smartwatch, only one completed the three AQs and none participated in the post-study interview. Analysis of the non-use of the smartwatch intervention is therefore not straightforward,and the result is open to multiple interpretations, such as a difficulty to engage with digital therapy, technical difficulties, or non-receipt of the watch.

Fig. 12.

View Figure

Finally, we looked at patients' use of the “Tips to stay well”, a feature delivering psycho-educational material through the Mood Monitor smartwatch app. Findings reveal that two-thirds of participants in the smartwatch group never accessed the feature (Figure 13). Among the third of participants who did access this feature, the majority (n = 8) only used it once and only a few patients (n = 3) opened it several times (ranging from 3 to 10 times) throughout the 8-week period. Although the ‘Tips to stay well’ feature constitutes an add-on to the self-report intervention, the observed differences in usage raise the question of whether and how to examine user acceptance of auxiliary features, particularly where the feature is a small component of a larger intervention.

Fig. 13.

View Figure

7.1.1 I Want to Get Better.

Participants shared their concerns about their mental health status, which was a powerful motivation for the uptake of the technology: “I am open to all ways to help my mood improve” (P28). Patients’ willingness to “try anything to help myself feel better” (P19) spoke of the hopelessness experienced when seeking help for mental health difficulties. P7 commented,

“I would like to give it a try as I have tried everything with my best abilities and I am scared when it [the depression] is going to strike me again” (P7).

7.1.2 It Helps Me Check in with Myself.

The perceived efficiency of self-report on the smartwatch was also a significant determinant of patient acceptability, with participants mentioning how the smartwatch could enhance self-report activity. First, self-report on the watch was perceived as a way to support consistent monitoring of mood and lifestyle habits:

“It will enforce me to keep a record of my mood & exercise” (P1).

Second, patients mentioned that they felt this approach could help them “better gauge my mood” (P5) and “keep a regular check on my moods” (P9), therefore supporting an increased self-awareness. The duality of tracking mood and lifestyle habits was seen as particularly helpful to monitor both “my mental and physical well being” (P7). Particularly, patients were hoping to gain insight into sleep patterns as “this is something I am struggling with at the moment” (P26).

7.1.3 It Encourages Me to Adopt Healthier Habits.

Several patients mentioned that they would use self-report data to recognize mood patterns and, for example, “know how often [I] feel low and track what I do to stop feeling low” (P24). Specifically, they wrote about wanting to identify lifestyle habits influencing their mood to

“get a better understanding on situations that [are] contributing to depression with stuff like sleep deprivation and activity levels” (P27).

The use of the wearable device itself was seen as a good way to motivate behavioral change, as participant P4 comments, “I believe a smartwatch would encourage me to consistently exercise.”

Finally, a patient evoked that being able to share the data recorded on watch app might inform the supporter regarding their progress, for instance, “at times when I can’t see them through the app” (P6), and potentially give them more information to provide feedback on.

7.1.4 Self-Monitoring Is Easy and Convenient.

Being ‘better’ than paper-based self-report was a motivation to take on technology for some patients, for reasons including a greater ease of use and a higher convenience (“[the app] can be used anywhere at any time” (P6)). The EMA was also seen as a contributor to reducing the demands placed on patients, such as “recording things on paper or trying to remember [past events]” (P8).

7.1.5 I Am Familiar with Smartwatches.

Seeking a seamless integration of technology into their daily life, patients mentioned owning a smartwatch and health monitoring apps as incentives to use self-report on smartwatch:

“I use my Apple Watch daily . . . I don’t feel that this will be difficult to implement into my routine” (P23).

Familiarity with the smartwatch technology also triggered a certain enthusiasm in participants, as participant P16 notes, “I already own an Apple smart watch & enjoy using it for exercise.” This enthusiasm was sometimes mixed with curiosity:

“I have found it [my Apple Watch] useful before in tracking sleep and steps etc. However, I have never used it to track mood” (P34).

Therefore, technology uptake was supported not only by a smooth integration of the technology within the patient’s daily routine but also within their technological habits.

7.2.1 I Don’t Believe I Can Use the Smartwatch.

The smartwatch technology raised concerns in some patients, regarding their ability to use the device, “I don’t think that I am the best for something like that as I’m not good with technology” (Anonymous). The use of the device also raised concerns in one patient, worried about discomfort as they found wearing watches irritating to the skin.

7.3.1 It Helps Me Check in with Myself.

The most discussed facilitating element at initial use was patient perceived usefulness of self-report, as participant P9 comments, “it’s helpful to record the moods,” and participant P31, “I have found its helped me keep track of things better.” Several participants praised the reminders which supported consistent monitoring, explaining how they were able to “remember to log my mood data when it [the app] notifies me” (P23) and how they saw their engagement with self-report improved (P21). Another mentioned recurrent benefit was how wearing the smartwatch was “very useful for monitoring exercise” (P12) and motivated behavioral change, as participant P29 explains,

“Wearing [the] watch itself really makes me walk more and get fresh air” (P29).

By monitoring their mood and lifestyle habits with the watch app, patients reported an increased self-awareness, as the mood prompts were

“a good reminder to focus on myself & my feelings throughout the day” (P16).

Moreover, the automated tracking of sleep supported them in gaining insight into pre-existing issues, as participant P31 comments,

“[it’s] also interesting to me the data it gathers on my sleep as I know I don’t sleep much” (P31).

7.3.2 Self-Monitoring Is Easy and Convenient.

Participants’ remarks at the pre-use stage, highlighting the importance of a self-report approach that diminished their burden, found an echo at initial use. The perceived ease of use of the app was mentioned multiple times, and particularly the quick interactions enabled by the smartwatch technology and how they supported patient engagement:

“I do think that it is quicker and easier to log my mood on the watch” (P1).

Finally, self-report via a wearable device was once again evoked as a convenient support for EMA:

“It’s handy having the app on your person, so to speak, so you can log your mood easily” (P33).

7.4.1 I Don’t Believe I Can Use the Smartwatch.

Frustration could be felt in participant P7’s response to the second questionnaire. However, despite their struggles with smartphones, the patient was willing to give the smartwatch a try:

“I don’t have [a] laptop and only [use] this annoying small screen iphone . . . Don’t know how to make it [the smartwatch] work as I struggled with technology, but willing to learn” (P7).

This statement highlights that even strong barriers to acceptance might not stop one from wanting to use the technology, if stronger motivations exist.

7.4.2 The App Doesn’t Behave Reliably.

Usability issues and configuration issues (e.g., users not granting access to sensor data) in the Mood Monitor app causing the system to malfunction was the most frequently mentioned obstacle to use. Patients’ responses mentioned how the inconsistent behavior of the app did not match their expectations: “Some days it hasn’t asked me for the mood” (P21). This sometimes forced them to take actions to solve the issue: “I have to uninstall the watch app and reinstall it to get it working again” (P23).

Some participants were reassured seeing that, despite some inconsistencies in the behavior of the watch app, it was “still record[ing] my sleep, activity and mood etc.” (P36). However, for others, not knowing if the data was correctly recorded became a source of self-doubt, induced by the impression that one is not using the system as it was intended. Participant P4 writes,

“I am not sure if I am using the app correctly” (P4).

7.4.3 I Find It Difficult to Change My Habits.

By the time they received the study smartwatch in their post, some participants had already taken the habit to self-report their mood on the mobile app. Difficulty in changing one’s habits resulted in a delay in the uptake of the smartwatch app. For example, participant P28 writes,

“Didn’t use it straight away as I was recording the mood data on my smartphone” (P28).

Another patient’s comment reflected the impact of depression symptoms on one’s ability to change their habits:

“some days, I just don’t put it on as I am just being lazy etc.” (P30).

The language employed by P30 (“just being lazy”) is recurrent in the responses of people experiencing depression speaking of a misinterpretation of depression symptoms for laziness [61], or the judgment of their peers/relatives [88], often leading to an attitude of self-blame [55].

Finally, physical discomfort induced by the wearable impeded the continuous monitoring of sleep. Although wearing the watch at night was optional, some patients did so, hoping to get a more accurate reading of sleep patterns. However, keeping the watch on overnight was sometimes source of discomfort: “I find it quite uncomfortable always sleeping with the watch on” (P9).

7.4.4 The App Doesn’t Allow for Enough Personalization.

Several comments concerned the list of lifestyle elements presented after a user logged a mood, as participant P1 comments,

“the options it [the screen] gives me I don’t think are the reasons affecting my mood” (P1).

Participants mentioned ways designers could better tailor the app to their needs, for instance, through adding “more causes of moods . . . stress, finances etc.” (P9) or allowing the person to enter “your own reasons for why you feel that way sometimes other than what is on the app” (P20).

7.5.1 It Helps Me Check in with Myself.

At sustained use again, the most mentioned benefit of the smartwatch app was that it encouraged consistent mood monitoring, particularly through the mood reminders. Participant P5 comments, “every time I was prompted, I would log my mood,” and participant P36, “the reminders really helped otherwise I would have definitely forgot.”. Delivering self-report on the smartwatch also supported patient compliance: “I stopped logging my mood since I haven’t had the watch” (P33). In addition, participants highlighted how the reminders also helped “create a routine” (P28) and supported them “to stop and check in with how you feel” (P9). These new opportunities for reflection helped patients gain self-awareness and becoming “more aware of yourself, how you are physically and emotionally” (P5). Identifying one’s current mood can be challenging, as “we don’t always pay attention to our mood so closely” (P36). Participant P14 describes,

“When I got the reminder in the morning, I wasn’t sure how I was feeling. So when it asked me to record my mood, I actually took 2 min to understand how I’m feeling . . . I would carry one problem or another . . . But now, if I know that I’m in a bad mood, I know that I have to lay low, just let it pass and it’s going to be okay” (P14).

With a similar experience, other participants mentioned how receiving the mood reminders on the watch itself “broke the cycle . . . especially when I’m feeling anxious” (P5) and helped train their self-awareness, or, as participant P29 explains, “train myself to stop and think about my mood” (P29).

Finally, some patients pointed out the difficulty to “stop, take time, log the information and do it there and then” (P17) and the need for retrospective reflection:

“[the app] allowed me to make a note of effectively how I was feeling at that point in time, and then go back and retrospectively look at it” (P17).

7.5.2 It Encourages Me to Adopt Healthier Habits.

Regarding the pre-use motivation for the uptake of the app, participants commented on how the Mood Monitor supported behavioral change. First, the identification of patterns between mood and lifestyle habits was made easier (P26), helping patients understand “why I was feeling that way, what had changed for me to be like this” (P5).

Reflection was further encouraged by the app asking patients which elements might have affected their mood: “you have to give a reason [for your mood], it does make you more self-aware of the likely reason you are feeling good or bad” (P34). In particular, the impact of sleep and physical activity was rendered more explicit, as participant P36 explains,

“When I wasn’t getting enough sleep or if I was having too much sleep, I did actually notice that it was making me feel a bit grouchy or irritable the next day, and that’s something I never kind of linked together” (P36).

The increased self-awareness of patterns supported behavioral change in participants, enabling them to adopt healthier habits. Participant 36 comments,

“being more active or getting out and doing more things, then it actually made my mood a bit better sometimes I think. And before I’d be like ‘oh I don’t want to go do that, what’s the point’ but actually doing it did make a difference” (P36).

Finding the motivation to engage in physical activity is often challenging for individuals experiencing depression [18], and participants described how seeing their step count in the app gave them a ‘boost’ to get active, as participant P5 describes: “It motivated me to go for walks more . . . I don’t think I would have really been motivated to do that if I hadn’t had the smartwatch.” The Apple Watch daily prompts to stand up and encouragements to set and reach fitness goals acted as an additional motivation, as participant P11 notes: “[the watch] encouraged me to get out of bed and try and get active.” Similarly, the Apple Watch bedtime reminder and wake alarm (which participants were instructed to set up to enable sleep tracking) helped them maintain a sleep routine, particularly the bedtime reminders which acted as

“a cutoff point . . . otherwise you can blink and it will be 10:30. So yeah, I did find it helpful in keeping a routine” (P34).

7.5.3 Self-Monitoring Is Easy and Convenient.

The use of the smartwatch was praised by participants as a convenient delivery means for the mood self-report, making it “easier to record them [moods] there and then” (P1). Participant P5 comments doing “all my mood logging on the watch,” and participant P28 describes

“It’s more convenient than remembering to write it down or having to go online to do it” (P28).

The convenience of the mood self-report on the watch supported patient engagement with self-report, logging their mood “more frequently than I necessarily would have on the computer” (P33). Particularly, mood logging on the smartwatch was described as quick and effortless, compared to via the mobile and desktop app for the intervention which required an ‘extra effort’ (P34, P17). Participant P36 comments,

“it was a lot easier to do it on a smartwatch app because it’s not like everyday I’m going to want to log into SilverCloud [platform]” (P36).

The location and proximity of the smartwatch, allowing immediate interaction, further facilitated self-report, as participant P34 explains, “it’s there, and you’re not having to pick up your phone from somewhere else.” Finally, participants mentioned a lessened burden associated with the tracking of sleep—for instance, participant P26 explains that “trouble sleeping was much easier to track.”

7.5.4 I Don’t Fear the Judgment of Others.

Stigma associated with mental health difficulties was a source of worry for most participants, with some hiding their ongoing therapy: “it’s not something that I’d like to advertise to everybody” (P17). Engaging with the Mood Monitor watch app felt safe for participants, as it did not make their difficulties visible to others. Through their ‘subtle’ interface, the reminders enabled a discreet logging of the mood, as participant P17 explains, “if somebody saw it on my watch, they wouldn’t realize I was involved in some something like this program.” The smartwatch itself was described as a more private means to self-report:

“almost under the radar . . . sometimes if you’re on a big phone, you know, people can see more. If it’s just on your watch, nobody’s really interested” (P34).

Most patients also declared feeling comfortable when logging their mood in a social context, as participant P5 describes,

“We were out with friends this past weekend, and I got some reminders, and I felt very comfortable just kind of quietly logging it, and just taking a second to check in with myself” (P5).

For participants who were open to their relatives about undergoing therapy, the Apple Watch acted as a conversation starter (P33). The interest sparked by the smartwatch created opportunities to speak about the program, which felt empowering for some patients, such as participant P29 who was

“trying to make it quite casual, and I’d say ‘oh I’m just enjoying this program, it’s really good and then it actually allowed me to use Apple Watch for 6 weeks.’ So, I’m sort of telling people in a way that if you ever need help, there’s a way for you” (P29).

7.5.5 The Therapy Is Better Tailored to My Needs.

Participants reported feeling comfortable sharing the data collected through the Mood Monitor watch app with their supporter: “it was just the Mood Monitor, I did not mind sharing that” (P14). Trust in how their personal data was handled reinforced that feeling, as participant P29 comments,

“I am very confident because obviously I know they [the supporter] won’t be discussing any of my personal information, unless they think I have a life threatening moment . . . I do trust them” (P29).

This attitude was primarily motivated by the desire to get better. Participant P36 notes, “at the end of the day it’s in my best interest.”

First, participants believed that the more information the supporter has, the better they can help. Providing information about their daily mood and lifestyle habits appeared particularly helpful to “give them [the supporter] an idea of what I’ve been doing and how I’ve been feeling day to day” (P31), so that they could “understand how I exactly feel” (P29). With regard to lifestyle information, participants evoked how it could “give a lot deeper insight for the supporter . . . rather than just asking me ‘how have you been sleeping?”’ (P28). Furthermore, self-report data might provide complementary information to the clinical questionnaires, affected by recall bias. Participant P36 explains that filling in clinical surveys is

“very subjective, you might think you’ve been feeling a different way to how you actually have been feeling . . . it would be quite a good comparison for them to see actually how you felt every day for 2 weeks against the questionnaires” (P36).

Second, participants trusted the supporter’s expertise to identify pertinent elements in the data and tailor conversations (P29), detect warning signs (P23), and assess clinical outcomes (P10, P18).

7.5.6 It Is Part of My Routine.

Last, participants described how self-report “quickly became into a habit” (P29), or in the words of participant P26, “a part of normal life.” Wearing the watch both integrated into their routine, “I got into the habit of just putting it on every morning pretty much straight away” (P5), and with the technologies they used. Participant P33 describes,

“I’ve found it quite great that it sort of seamlessly worked, it integrated with everything that you already have” (P33).

7.6.1 The App Doesn’t Allow for Enough Personalization.

Most patients reported that the list of options from which they could select which factor(s) influenced their mood was not relevant to their situation, as participant P33 comments,

“I didn’t really find them [options] particularly relevant, so I got into a habit of just overlooking it . . . That definitely was something that kind of put me off using it” (P33).

Some participants suggested adding more reasons to explain the mood, such as family, friends, work, and other lifestyle factors which might also impact one’s mood (P34). Enabling customization of the list was also evoked, as participant P34 suggests, “hav[ing] the opportunity to constantly edit it and put in your own [factors] I think would be fantastic.”

Patients also missed being able to apply a valence to each factor, to record “if it affected in a positive or negative way” (P12). Participant P34 describes,

“I feel down because I haven’t done exercise, therefore, exercise is a reason. But equally, when I was logging me having a great mood after I’d gone for a nice run, then exercise was also a factor” (P34).

Participant P34 suggested adding an extra step to self-report, “where you’re like [selecting] ‘too much caffeine’ [or] ‘too little caffeine.”’ Participants also discussed how the smartwatch app could go a step further in terms of customization, to support their engagement with self-report. The use of generic encouraging prompts (e.g., ‘Well done’) was strongly contested: “I think the generic messages just wash over people because we get so many of them” (P17). Similarly, although the ‘Tips to stay well’ helped some patients adjust their habits (e.g., gradually reducing their caffeine intake (P34)), the recommendations they provided were perceived as too generic. Participant P36 notes,

“I felt like for me personally a lot of them were kind of like common sense or self-explanatory. It’s kind of like I know I need to do those things, sometimes it’s a little bit harder for me to do them but it was kind of like I was aware of those kind of things so throughout the time I think I looked at it once when I set it up and that was it” (P36).

This comment also reflects the difficulty of engaging in behavioral change despite being conscious of the importance of adopting healthy lifestyle habits (“I know I need to do those things”).

To further support behavioral change, participants suggested sending prompts “relevant to what you’ve done” (P17, P36). Such custom messages based on the self-monitoring data collected might also support introspection and action, as participant P33 explains,

“If there was poor sleep going on, asking the questions ‘Is everything OK? Is there something that you need to talk somebody about?’ . . . The more personalized, the better” (P33).

Such messages could be further enhanced through making explicit mention to self-report data:

“Something that was relevant to you personally, [e.g.] ‘So we’ve noticed that the last 3 days you’ve had less sleep and your mood is declining’ . . . you could then take an action” (P17).

However, through incorporating longer prompts, the smartwatch app would no longer rely on microinteractions which might impact user engagement with self-report [77]. Such design change therefore requires careful consideration and further investigation.

7.6.2 The App Doesn’t Behave Reliably.

Some participants managed to solve technical issues by reinstalling the app: “that was just an initial hiccup, but I got over that” (P5). However for others, the issues persisted—for instance, participant P17 comments, “I don’t think the reminders came through consistently.” This negatively impacted acceptance of the technology, as participant P33 notes, “I would have used it more regularly if the reminders worked.”

7.6.3 I Am Concerned about Sharing My Self-Report Data.

Sharing lifestyle data collected through the Mood Monitor with their supporter sometimes induced worries in participants: “it just puts a little more pressure on you” (P14). When asked how their supporter should use the data collected through self-report on the watch, participant P35 simply replied “With care,” reflecting the caution needed when dealing with sensitive data. In particular, patients expressed that they did not want to ‘feel trapped’ and under surveillance, as participant P5 explains,

“I’m not sure that I would necessarily want my clinician to be kind of Big Brother-ing on my sleeping trends” (P5).

Particularly, as much as they would like sharing data reflecting improvements, giving access to unsatisfactory data would be a cause of additional stress:

“It would have been nice if she [the supporter] had said ‘I see that you’ve been moving more, that’s really good!’ . . . but I wouldn’t want them to hold that against me if I haven’t been sleeping well or if I haven’t been exercising” (P5).

Participant P14 highlights that sharing ‘unsatisfactory’ data risks leading to feelings of self-blame: “What happens if I am not able to work out for 2 days? . . . they would think that I’m not working out, or I’m not doing good enough.” Pointing out the difficulty to maintain a sleep and exercise routine when experiencing depression symptoms, the participant further argued that self-report data shouldn’t be used to make them answerable for something they had little control over:

“I would want to work out some days, but my body has no energy. I can’t go and explain it to someone why I feel that way because that is how I feel . . . It’s okay if they have it [the data], right, but I don’t want any questions asked as to why” (P14).

Finally, lack of trust in the secure handling of data was an obstacle to self-monitor sleep on the watch. Participant P17, who owned a smart sleep mat, explains, “I chose not to share that information [sleep] . . . I was very concerned about having another source, another outlet which I wasn’t overly comfortable had been fully secured.”

7.6.4 It Disrupts My Routine.

For some participants, wearing the smartwatch disrupted their routine. Responses revealed that the smartwatch made switching off from technology difficult, as participant P36 describes,

“I couldn’t really switch off . . . having it on my arm and seeing it all the time, sometimes I felt a bit drained and like I wasn’t actually connected to the real world” (P36).

The automatic delivery of notifications and the frequent ‘Stand up’ and ‘Breathe’ prompts of the Apple Watch was described as “annoying” (P6), particularly when interrupting participants in the middle of work (P29, P34). Once again, the importance of personalization came up in the responses, with the suggestion of adjusting the sending of prompts (including the mood reminders (P36)) to one’s calendar: “[the app] would see that I’ve got a free spot in my calendar . . . It gives the watch a slot and it is more likely to get my attention when I’m not already busy” (P17).

9.3.1 Understanding Technology Uptake and the Evolution of Acceptance over Time.

Our measurement approach relied on previous literature arguing for the influence of user demographics [121], the different acceptance mediators (e.g., perceived threat) [52], and satisfaction with therapy [15, 40, 59, 71, 89] on user acceptance of digital health technologies. Findings revealed that looking through these three lenses provided valuable information on patient acceptance of the self-report on a smartwatch. First, analysis of patient demographics and pre-use acceptability scores permitted to highlight the influence of the relationship status element. Second, analyzing each mediator’s acceptance score obtained at pre-use, initial use, and sustained use revealed that these evolved in different manners. In particular, the lack of statistically significant difference in the overall acceptance and mediator scores at the three time points permitted to rule out potential risks, such as the emergence of obstacles to acceptance in the user journey. Finally, the strong correlation between sustained-use acceptance scores and patient satisfaction with therapy revealed that user acceptance of a small component of a mental health intervention (e.g., mood self-report) can be linked to satisfaction with the overall intervention. This suggests the potential value of investigating the relationship between these two elements.

9.3.2 Mixed-Measurement Methods to Get a Rich Understanding of Patient Acceptance.

The set of acceptance factors deemed important by patients differed (1) by stage of the user journey but also (2) by data collection method. For instance, although the perceived threat questionnaire score remained high throughout the user journey (indicating patients’ persistent concerns about their mental health), this factor was only brought up in conversations at the pre-use stage. This supports the importance of gathering users’ qualitative feedback at different stages of the user journey, to give them an opportunity to share what specific factors are important to them at that moment in time. In addition, although a body of work has equated satisfaction to acceptance [15, 19, 40, 44, 48, 49, 71, 72, 74, 89, 105], the findings of this research show that measuring satisfaction alone does not provide an understanding of the factors that determine patients’ usage behavior with the technology. On a higher level, the approach adopted to elicit patients’ opinion greatly impacts the outcome of an acceptance study in terms of understanding which factors play a role at which stage of the user journey. By adopting a methodology combining open questions with targeted questions, we respectively (1) gave patients the opportunity to explain what acceptance factor mattered to them most at specific time points, and (2) captured patient perspectives on aspects that they might have perceived as being of lesser importance, but which might still affect their technology acceptance. Therefore, we recommend measuring patient acceptance using a mixed-methods approach combining targeted and open measurement items to maximize insight and understanding into the evolution of patient acceptance of a technology.

Finally, capturing usage metrics together with a questionnaire addressing known mediators of acceptance enables between-group comparisons in user acceptance, which would help detect and explain problems if they were present.