1 Introduction

Smartphones have become an integral part of human daily life, and a focus of research conducted in our community [8, 9]. People find themselves using their smartphones under various challenging contexts [23]. Factors such as cold ambience [2, 18], encumbrance [10], walking [1], and ambient noise [16] have all been shown to hinder smartphone interaction [15] and cause situational impairments [19, 24]. While the effects of a number of situational impairments on mobile interaction have been studied and are established within the HCI research community, many situational impairments remain underexplored. In their overview of situational impairments, Sarsenbayeva et al. [15] identify a research gap concerning the effects of ambient light on mobile interaction, despite the fact that it is common to use one’s smartphone in varying light conditions (e.g., watching a movie with the lights off or interacting with the device while wearing sunglasses).

Therefore, in this paper we investigate the effect of ambient light conditions on mobile interaction. We quantify mobile interaction performance in terms of three everyday smartphone activities – target acquisition, visual search, and text entry – under three distinct ambient lighting conditions: (1) normal light condition (operationalised as recommended indoor light levels for easy to normal office work [12]), (2) dimmed light condition, and (3) normal light condition while wearing sunglasses. We limit our investigation to the effect of reduced illuminance conditions. We do not study the effect of bright light on smartphone interaction for a number of reasons. First, the existing literature has already established the adverse effect of bright light on performance in visual tasks on mobile device screens [7]. Second, we want to exclude the effect of confounding parameters, such as glare, that is caused by bright light and leads to a decrease in visual task performance on mobile phone screens [7].

Our study shows that dimmed light, as well as wearing sunglasses, negatively affects mobile interaction performance in terms of target acquisition time. We also show that tapping accuracy decreases under the dimmed light condition, while wearing sunglasses increases target memorising time. Our work contributes to the growing body of research in the HCI community on situational impairments.

2 Related Work

2.1 Situational Visual Impairments During Mobile Interaction

Tigwell et al. [22] identified ambient light as one of the leading causes of visual situational impairments. Ambient light has been shown to affect people’s perception as well as the clarity of a smartphone’s display. For example, Gong et al. [3] show that as ambient light intensity increases, mobile device screens become more challenging to use. This might be because increasing ambient brightness while decreasing monitor brightness reduces colour differentiation abilities, as found by Reinecke et al. [13]. Furthermore, it has been shown that for illuminance levels higher than 1000 lx, participants’ visual task performance declines at a faster rate compared to illuminance levels lower than 1000 lx, due to screen glare [7]. Dimmed ambient light has also been shown to visually impair users; however, only limited research has investigated its effect. For example, Lee et al. [6] investigated the effect of ambient illuminance on performance while reading e-papers. They found that search speed and illuminance level were directly proportional: with low search speed being associated with low levels of illuminance. Liu et al. [7] studied the effect of ambient light on handheld display image quality. The authors found that in darker environments, participants performed better in visual tasks as compared to bright environments. These findings are in line with findings from Kim et al. [5] which demonstrate that perceived image quality on screens decreases in bright environments. However, both of the aforementioned studies featured a limited number of participants (3 participants in [7], 10 participants in [5]). Furthermore, both of the studies focused on the perception of image quality on mobile device screens.

3 Method

In this study, we investigate the effect of an environmental factor – ambient light – on mobile interaction. In particular, we focus on mobile interaction under dimmed light conditions. We quantify interaction performance across three typical smartphone tasks: target acquisition, visual search, and text entry. We used the tasks developed and presented in a study by Sarsenbayeva et al. [16] in order to directly compare the effect of ambient light-induced situational impairments to the established effects of cold- [2, 18], noise- [16], and stress-induced [14] situational impairments.

3.1 Smartphone Tasks

In this study, we used a Samsung Galaxy S7 smartphone running Android 7.0 with \(1080\times 1920\) px screen size (similar to the one used in the studies by Sarsenbayeva et al. [16, 18]). To minimise sequence effects, participants completed the three tasks in a counterbalanced order. Furthermore, we minimised any potential learning effects by asking our participants to undergo extensive training in all three tasks prior to the start of the actual experiment. The participants completed the tasks in a standing position, interacting with the phone with the index finger of their dominant hand while holding the phone in their non-dominant hand.

Target Acquisition. In this task, participants tap circular targets (Radius \(=\) 135 px) with an indicated centre (Fig. 1A). The targets appear on a random position on the screen, one circle at a time. We asked our participants to tap the centre of the circles as precisely and quickly as possible. We measure participants’ performance in terms of their reaction speed (time required to tap targets) and accuracy (offset size).

Visual Search. In this task, participants are asked to find a target icon among 24 other icons, arranged according to a \(4\times 6\) grid [4]. The participants are first shown the icon, and given as much time as required to memorise it (Fig. 1B). Then, participants must find this icon in the subsequent screen (Fig. 1C). Target icons are selected randomly from the pool, and the icons are placed at random positions on the screen to minimise any potential learning effects. We quantify participants’ performance in terms of cognition (time to memorise an icon), reaction (time to find an icon), and accuracy (error rate).

Text Entry. In this task, participants are instructed to type a snippet of text shown in a text box. The texts are of two difficulties: (1) easy – consisting of only one sentence with commonly used words (Fig. 1D), and (2) difficult – consisting of several sentences with outdated words (Fig. 1E). For each round, participants are presented a randomly selected easy sentence (10 in total) and a randomly selected difficult sentence (10 in total). We measure how quick (character entry rate) and accurate (error rate) participants were in entering the text.

Fig. 1.
figure 1

Interface of the application with target acquisition task (A), visual search task (B–C), and text entry task with user’s input for easy and difficult texts (D–E).

Full size image

3.2 Participants

We recruited 28 participants through our university’s mailing lists. Participants are between 18 and 33 (\(M = 23, SD = 3.70\)) years old. In total, we recruited 19 female and 9 male participants. Our participants have a diverse range of educational background (e.g., Accounting, Actuarial Sciences, Biomedicine, Business, Chemistry, Food Science, Urban Planning). All participants have normal or corrected-to-normal vision (contact lenses) and are right-handed. All of our participants were used to wearing sunglasses.

3.3 Procedure

Our experiment contains three conditions: (1) normal light condition (recommended indoor light levels for easy to normal office work [12]), (2) dimmed light condition, and (3) normal light condition while wearing sunglasses. We followed the guidelines for illuminance standards in a working environment, and hence set the room’s illuminance to 335 lx for the normal light condition [12]. In the dimmed light condition, the illuminance of the room was set to 20 lx, a light level which we consider a dark environment to perform most activities. Finally, for the third condition, participants were required to wear non-polarised sunglasses with category 2 lenses under the same illuminance as the normal light condition. Our choice of sunglasses is justified by its popularity of use among the general population, as a category 2 lens provides a medium level of sun glare reduction and UV protection with a visible light transmission of 18–45% [21]. We ensured that the brightness level of the smartphone was kept constant at a medium level throughout the entire experiment, and disabled brightness auto-adjustment to ensure consistency in the study setup. Furthermore, we counterbalanced the presentation order of the conditions across the participants. At the end of the experiment we conducted semi-structured interviews with each participant to understand their perceived performance during the completion of the tasks. The Human Ethics committee of our university approved this experiment.

4 Results

To investigate the effect of ambient light on performance during smartphone interaction, we conducted a one-way repeated measures ANOVA on the aforementioned performance-measurement variables. We describe the results of our findings per each smartphone task. We removed extreme outliers from our data (3 individual data points in total from the whole dataset).

4.1 Target Acquisition

First, we investigated the effect of ambient light on target acquisition time (milliseconds). The result of a one-way repeated measures ANOVA revealed a statistically significant effect of ambient light on target acquisition time (\(F(2, 7607) = 8.20, p < 0.01\)). Post-hoc comparison using the Tukey HSD test (with Bonferroni corrections) showed that there is a significant difference (\(p = 0.02\)) between target acquisition time under the dimmed light condition (\(M = 495, SD = 110\)) and the normal light condition (\(M = 485, SD = 103\)). Moreover, our results show that the participants took a significantly longer time tapping a target (\(p < 0.01\)) while wearing sunglasses (\(M = 498, SD = 115\)) when compared to the normal light condition. However, there was no significant difference between the dimmed light condition and wearing sunglasses (\(p > 0.05\)). Mean values for target acquisition time are presented in Fig. 2(a).

We then examined the effect of ambient light on touch accuracy. A one-way repeated measures ANOVA showed a statistically significant effect of ambient light on the participants’ offset size (\(F(2, 7607) = 7.32, p < 0.01\)). Post-hoc comparison using the Tukey HSD test (with Bonferroni corrections) indicated that the offset size was significantly larger under the dimmed light condition (\(M = 49.50, SD = 26.70, p = 0.02\)) as compared to the normal light condition (\(M = 47.70, SD = 25.80\)). We also found a statistically significant difference in offset size between the dimmed light and sunglasses conditions (\(M = 46.7, SD = 26.40, p < 0.01\)). However, there was no significant difference between the offset size under normal light and sunglasses conditions. Mean values for offset size are presented in Fig. 2(b).

Fig. 2.
figure 2

Mean target acquisition time and offset size (95% CI)

Full size image

We also studied the effect of ambient light on effective throughput, calculated as proposed by Soukoreff and MacKenzie [20]. A one-way repeated measures ANOVA did not reveal a significant effect of ambient light on the effective throughput during target acquisition tasks.

4.2 Visual Search

We examined the effect of ambient light on the time taken to memorise (milliseconds) and subsequently find an icon (milliseconds). The result of a one-way repeated measures ANOVA revealed a statistically significant effect of ambient light on the time taken to memorise an icon (\(F(2, 2045) = 4.42, p = 0.01\)). Post-hoc comparisons using the Tukey HSD test (with Bonferroni corrections) indicated that participants took significantly longer time to memorise icons in the sunglasses condition (\(M = 744, SD = 271, p = 0.01\)) than the dimmed light condition (\(M = 703, SD = 206\)). The mean values to memorise an icon are presented in Fig. 3. However, we did not find a statistically significant difference between the normal light condition and the dimmed light condition (\(p > 0.05\)) for the time taken to memorise an icon. We found similar results when comparing the wearing sunglasses condition to the normal light condition (\(p > 0.05\)).

Fig. 3.
figure 3

Mean time to memorise an icon (95% CI)

Full size image

4.3 Text Entry

In the text entry task we measured participants’ performance in terms of time per character entry in milliseconds and total error rate [20]. We calculated character entry rate as time taken to input a character, while the total error rate was calculated as the ratio between the number of total errors and total entered characters. A one-way repeated measures ANOVA did not reveal a statistically significant effect of ambient light on either character entry rate or error rate (\(p > 0.05\)). We built two generalised linear mixed-effect models to describe the effect of ambient light on character entry rate and error rate. None of the predictors had a significant effect on text entry rate.

4.4 Qualitative Results

During the interview, our participants commented on their subjective perception of performance during the smartphone tasks. A number of participants (8 out of 28) mentioned that wearing sunglasses and dimmed light conditions affected their target acquisition time: “A little longer when I’m wearing the sunglasses.” (P02), “I was quicker when there was more light.” (P12), “Longer time was required under dimmed light” (P06). These findings are in line with our quantitative findings which show a negative effect of dimmed light and wearing sunglasses on target acquisition time.

Regarding the self-perceived accuracy during the target acquisition task, half of our participants (14 out of 28) indicated that they were more accurate under the normal light condition as compared to the dimmed light condition. “In the dimmed light it was harder to accurately tap the center of circles” (P04). These findings correspond to our quantitative data that shows that under the dimmed light participants were less accurate. Surprisingly, two of our participants claimed they were more accurate under the dimmed light condition, as the contrast of the screen was brighter and they could see the circle clearer: “In the dim light I felt I was more accurate because the circles were more visible” (P19).

When we asked the participants about their perceived performance on the time taken to memorising the icon during the visual search task, a large majority of our participants (17 out of 24) stated that the light did not affect their performance. However, these perceptive statements contradict our quantitative results, showing that participants took a significantly longer time when wearing sunglasses as compared to the dimmed light condition. Interestingly enough, one of our participants mentioned that it took them less time to memorise icons under the dimmed light condition: “When performing in dimmed light it takes less time to memorise icons” (P15).

Nevertheless, participants reported a negative effect of ambient light on their perceived performance in time taken to find an icon, even though our quantitative analysis does not support this observation. In total, 10 participants mentioned that they believe it took them a longer time to find an icon under the dimmed light: “It affects me so much. I took a bit longer to find the icons in a dim light condition” (P18); “When the light is on, I can find the icon easier compared to when the light is dimmed” (P13).

Regarding the text entry task, most of the participants (N = 19) claimed that the light did not have any effect on their performance. A total of 4 participants believed that they were slower to type under the dimmed light condition and when wearing sunglasses. “It took me more time to type text under dim light” (P20). Furthermore, 4 participants claimed to make more errors when the lights were dimmed. “It was more difficult for me to type the text accurately with dim lighting. I was more confident in typing under normal lighting” (P12). Nonetheless, our quantitative results did not show any significant support for these statements.

5 Discussion

5.1 Impact of Ambient Light on Mobile Interaction

Our findings show that participants took a significantly longer time to tap a target while wearing sunglasses and under the dimmed light as compared to normal light condition. However, only a minority of our participants (8 out of 28) reported the negative impact of ambient light on target acquisition time. This is an indication that dimmed light caused situational visual impairments in our participant without them noticing it. Previous work has shown that various environmental and internal factors have a different effect on target acquisition time. For example, previous research has shown that participants took a significantly longer time to tap circles under cold ambience due to stiff muscles [17, 18]. However, under music (fast and slow tempo) and urban noise (indoor and outdoor) conditions [16], and when exposed to stress [14], target acquisition time was significantly shorter due to the rhythm of the music, and the anxiety caused by urban noise and stress.

Furthermore, participants were significantly less accurate in target acquisition tasks under the dimmed light condition. This was confirmed in our qualitative data as the participants mentioned that they felt the negative effect of dimmed light on their interaction. In particular, our participants acknowledged that their perceived accuracy when tapping circles in dimmed environment is worse, compared to normal ambient illumination. This may be due to the fact that as the illuminance decreases, retinal dopaminergic activation from photoreceptors drops, and, hence causes a situational visual impairment [11]. In addition, our analysis did not reveal a significant effect of wearing sunglasses on participants’ tap accuracy as compared to normal light condition. This might be due to the fact that we used commonly available non-polarised sunglasses that are unlikely to cause strong visual impairments.

However, we observed a negative effect of wearing sunglasses on memorising time in visual search tasks. Moreover, we anticipate that the effect of wearing sunglasses under bright sun light might be exacerbated as the effect of glare contributes to the magnitude of the visual situational impairment. Although ambient light did not have a negative impact on our participants’ visual search time, the majority of our participants claimed that it took longer time to find an icon when the light was dimmed. However, this may be the case given the simple nature of the task, as prior research has shown that low illuminance levels are associated with slower search speed [6].

Finally, our analysis did not reveal a negative effect of dimmed light or wearing sunglasses on performance during text entry tasks. This may be due to the fact that we used a limited number of text entry tasks that are not sufficient to observe the effect of ambient light on text entry performance. However, previous work has shown a significant effect on participants exposed to meaningful speech on a similar typing task (i.e. listening to someone speak in a language they understand while typing on their smartphone) [16]. This confirms that different situational factors have a different effect on typical smartphone tasks.

5.2 Accommodating Ambient Light-Induced Situational Impairments

In summary, we demonstrate the negative impact of dimmed ambient light on fundamental smartphone interaction tasks. We argue that accounting for situational visual impairments in mobile interaction is important, as the effect might accumulate as task complexity rises. Moreover, Tigwell et al. [22] argue that the value of reducing the effects of situational visual impairments grows as the importance of the task increases. Previous work has proposed different methods to accommodate for situational visual impairments during mobile interaction. For example, in the study by Tigwell et al. [23] participants suggested increasing the contrast of the screen to reduce the effect of situational visual impairments. Moreover, Reinecke et al. [13] suggest increasing button sizes and adjusting the background colour to reduce the adverse effect of situational visual impairments on mobile interaction. As smartphones already come equipped with an ambient light sensor, these methods can be applied once a detrimental ambient light condition is detected and the user is performing a particular task (e.g., target acquisition task under the dimmed light). Given that different people have differences in their perception of contrast colours, the ambient light sensor together with adaptive techniques (e.g., screen contrast, background colour) could be used to build personalised interfaces to improve the smartphone interaction experience, beyond simply adjusting the screen brightness as is the case with current devices.

5.3 Limitations

We acknowledge several limitations in this study. First, the study settings were strictly controlled. In particular, we examined only two levels of ambient illumination – normal and the dimmed light, and do not investigate the effect of bright ambient light (outdoor illuminance) on smartphone interaction performance. The reason for this exclusion is to eliminate the effect of additional external factors, such as glare and ambient noise, on smartphone interaction performance. Finally, our experiment is limited to three types of smartphone tasks. We argue that these tasks are representative of the vast majority of activities that typical users undertake while using their smartphone.

6 Conclusion

In this study, we investigate the effect of three ambient light conditions on smartphone interaction performance in target acquisition, visual search, and text entry tasks. We found that dimmed ambient light significantly impairs target acquisition. Participants took a significantly longer time to hit targets while wearing sunglasses or are under dimmed light, as compared to the normal light condition. Furthermore, participants were less accurate when tapping targets under the dimmed light condition. We also show that participants took longer to memorise icons while wearing sunglasses when completing visual search tasks. Our findings enhance the understanding of situational visual impairments impact on mobile interaction and contribute to the growing body of research in the HCI community on situational impairments.