Keywords

1 Introduction

In special education centers for people with visual impairments, raised-line maps are particularly important. An advantage of these maps is that they can be explored with both hands, allowing efficient two-handed exploration strategies [4]. However, making raised-line maps is a tedious process, which requires tactile document specialists to adapt and simplify visual maps. Moreover, maps have to be printed out on a special paper that generates raised lines when heated. As a result, raised-line maps cannot be changed or updated easily when the content is out-of-date.

Digital Interactive Maps (DIMs) are a good alternative to raised line maps [13]. They can be generated on demand and are easily modifiable. DIMs can be displayed on a touch sensitive surface (e.g. a tablet) and explored with the fingers. DIMs are not combined with any physical objects, i.e. they are entirely digital. During exploration, they provide auditory and/or vibrotactile feedback according to touch positions on the map [13]. Previous work has proposed to explore DIMs on multitouch surfaces using one [39] or several fingers (usually one finger of each hand) [5, 19]. However, single-finger exploration raises perceptual and cognitive issues [29]. Two-handed interaction allows for preserving exploration strategies [19] but raises the challenge of providing the appropriate feedback for each hand without ambiguity. As bimanual interactions support cognitive processes related to spatial perceptions, it is then necessary to design and evaluate them to explore DIMs in ubiquitous context (e.g. with the use of an Xperia Touch). To the best of our knowledge, no previous work has: (1) explored the design of appropriate feedback for two-handed exploration of digital maps (2) compared the impact of feedback on the performance and strategies used during one- vs. two-handed map exploration.

In this work, we investigated these two research questions. First, we studied the design of unilateral vs. bilateral feedback for two-handed exploration of digital maps without vision. To this end, we proposed a design space for two-handed exploration feedback (see Fig. 1), which combines two axes: location of the feedback (unilateral vs. bilateral), and feedback similarity (same vs. different feedback). To compare the usability of the different techniques, we conducted a first study with 15 blindfolded subjects. Results revealed that bilateral feedback outperforms unilateral feedback, and that the feedback similarity has little influence on exploration performance.

Fig. 1.
figure 1

The design space combines two feedback dimensions: spatial location and similarity.

Full size image

Then, we explored the impact of unilateral vs. bilateral feedback on performance and exploration strategies in a study with 12 people with visual impairments. In that study, the one-handed technique with unilateral feedback is similar to existing map exploration techniques on a tablet with accessibility feature activated. The results show that the two-handed technique with bilateral feedback is between 30% and 43% faster than the other techniques, and is preferred. We also identified, for each study and task, the two-handed exploration strategies used by the participants, revealing the impact of the feedback on such strategies.

To summarize, we explored the design and the evaluation of two-handed exploration techniques with different feedback on digital maps, and compared it with one-handed exploration. Our contributions are: (1) the exploration of all techniques resulting from the combination of our two axes (spatial location of the feedback and feedback similarity); (2) the results of an evaluation of this design space with 15 blindfolded people; (3) the results of an evaluation with 12 users with visual impairments comparing one-handed exploration vs. two-handed exploration techniques (with unilateral and bilateral feedback) for map exploration.

2 Related Work

2.1 Exploring DIMs on Touch Sensitive Surfaces

Digital interactive maps (DIMs) can be explored by directly moving the finger over the display surface. Important elements of the map (buildings, streets, regions, etc.) are rendered by auditory and/or vibratory feedback. For instance, TouchOverMap was designed for a smartphone [33] and provided a basic overview of a map layout. Other prototypes were designed on tablets and provided access to maritime, choropleth or city maps [8, 25, 37]. Some projects have also been developed on large tabletops [26]. One persistent issue with DIMs based on multitouch is that feedback can be ambiguous: when multiple fingers are moving on the surface, the user does not know which finger triggers the feedback. To release ambiguity, the user must raise up all the fingers and touch back, one finger after the other, to locate the rendered elements. Another possibility is to use only one finger to explore the map [31]. For instance, Bardot et al. [3] designed a one-handed technique to explore DIMs using a smartwatch. Vocal and vibratory feedback was provided by the smartwatch according to the finger location on the DIM. In this technique, the feedback was collocated with the moving hand.

Although users can use gestures with several fingers (two, three or four fingers) to access menus, two-handed exploration is not possible with VoiceOver or TalkBack. Hence map applications depending on them (e.g. Plans, Google Maps, Ariadne GPS, etc.) do not rely on two-handed exploration. For instance, it is not possible to use one finger as a reference point (anchor) while the other is moving around, which can provide the relative location of different elements in the map.

2.2 Multimodal Feedback

Giudice et al. [16] designed multimodal (audio and tactile) feedback that was triggered when graphics were explored by touch on a tablet. They showed that vibro-audio interface is a good multimodal solution for providing access to dynamic graphical information. They also showed that it supports the development of mental representations of the explored graphics. Goncu et al. [17] designed a more complex device that allowed the user to use the index finger of both hands, and that provided vibrations with small vibrators attached to the fingers and controlled by the tablet. The results showed that blind participants were able to understand different graphics including tables, line graphs and floorplans. Although multimodal feedback distributed on both hands may intuitively seems more efficient that single multimodal feedback, it has never been evaluated systematically.

2.3 Parallel Feedback

The “cocktail party effect” is a known research problem [1, 22]. It is the ability for a person to focus on a single speaker among several parallel conversation or background noise. Results showed that the ability of a listener depend on different factors (such as the type, the number or the location). How many information can be gather from several conversations while focus on one is unclear. Guerreiro et al. [18] proposed a study using several concurrent speech channels. They aimed to understand how people with VI can glean relevant information over two, three or four channels. The results showed that users where able to easily get the relevant information over two or three concurrent channels.

2.4 Haptic Exploration Strategies

Single hand exploration of an unknown digital map relies on a few strategies [38, 43]. The perimeter strategy consists in exploring an area for the identification of shape, size and contour characteristics. The gridline strategy involves searching for elements of a configuration with a systematic exploration of the display. The cyclic strategy consists in moving from one element to another in the same order, and then returning to the starting point. The reference point strategy is a back and forth movement between two elements. Previous studies [38] showed that grid and perimeter strategies are specially used for the first step of the exploration called “discovery phase”, whereas cyclic and reference point strategies better serve the memorization of the map configuration.

In a recent study [19], Guerreiro et al. investigated strategies used during one-handed and two-handed target finding on a touchscreen. Complex behaviors involving both hands have been observed and six two-handed strategies have been identified: path scan consists in a structured scanning path; focused consists in searching a small sub-section; to-the-point corresponds to a path starting close to the target, which is already known by the user; freeform represents an unstructured strategy, while freeform symmetry describes the same strategy with two-hands; finally, trailing finger is a variant of freeform with an offset between hands. Although authors did not specifically compare one-handed vs. two-handed performance, they pointed out that they found no difference in exploration time. These strategies are consistent with previous work on how people with VI explore unknown spaces by foot [15, 23] or using tactile input [28].

In our study, we designed different types of bilateral feedback for digital map exploration. We then assessed the effects of unilateral vs. bilateral feedback on exploration strategies used with different one- vs. two-handed interaction techniques.

3 Bilateral Feedback for Two-Handed Exploration

While unilateral feedback can be easily produced using a multitouch surface, bilateral feedback requires locating both hands and providing unambiguous cues. The use of two wearable devices such as smartwatches can comply with this requirement since they have sound and vibration capabilities. A smartwatch or a vibrating bracelet can be worn on each hand, providing bilateral feedback that is collocated with each hand. To render the elements of a map (i.e. region, cities and borders), we used TTS to render the name of cities or regions, and vibratory feedback to locate map elements, such as borders between regions.

Interaction for one-handed exploration of digital maps (e.g. on tablets and smartphones) is based on audio and vibratory feedback [25, 39]. Audio feedback is often used to render text and descriptions (e.g. name of regions) while vibratory feedback is preferred to render elements that must be located accurately (e.g. points or lines). We leveraged this approach for two-handed exploration. The challenge was to provide feedback corresponding to both hands actions, while still being cognitively manageable. We know that using different feedback for each hand, for instance using different voices (e.g. male or female) and different vibration patterns (e.g. single or dual), can help to identify which hands triggers it [32]. Then we proposed a design space based on the combination of two design factors: spatial location of the feedback (unilateral or bilateral) and similarity of the feedback (same or different).

The combination of these two design factors resulted in four feedback techniques (see Fig. 1): (1) unilateral with the same feedback (SF) for both hands (U-SF); (2) unilateral with different feedback (DF) for each hand (U-DF); (3) bilateral with the same feedback for both hands (B-SF); and (4) bilateral with different feedbacks for each hand (B-DF). It is interesting to note that the condition U-SF is similar to exploring a DIM on a touch screen with two fingers. We also designed a Control condition including one hand only, which is similar to exploring a DIM with only one finger on a tablet with the accessibility feature activated.

For the conditions with different feedback (U-DF and B-DF), we used a female voice and one vibration for the left hand, and a male voice and two vibrations for the right hand. For the unilateral conditions (U-SF and U-DF), if the user moves both hands at the same time, the feedback corresponds to the latest detected movement.

4 Materials and Methods

In the current paper, we present two studies on two-handed exploration of maps without vision. In this section, we detail the rationale for such an approach, as well as the experimental task and apparatus, which was the same for both studies.

4.1 Rationale

Including people with visual impairments (VI) in controlled studies is not a simple task due to limited availability [35]. Three different approaches might be adopted to evaluate interaction techniques for people with VI: (1) Gathering sighted people and people with VI in the same study, which is appropriate when the study is not too long because we cannot expose users with VI to long studies; (2) When the study is too long, it is possible to conduct separate evaluations: a first study with blindfolded people to reduce the design space, and a second study with people with VI to assess performance; (3) Conducting a study on a reduced group of users with VI, which fits well with evaluating qualitative user experience, but does not with evaluating performance.

We chose the best approach (2) for analyzing performance during a long study. We adopted a two-step approach with two distinct populations: blindfolded users are non-experts of tactile exploration. They were exposed to all the conditions (Study 1). On the other hand, people with VI are, in general, expert in tactile exploration and were exposed to a subset of conditions (Study 2).

4.2 Maps

We asked participants to explore digital maps similar to country maps, i.e. composed of regions. To create the digital maps, we used the Voronoi algorithm [14], configured to randomly generate 30 regions of different areas fitting an A3 format (29.7 × 42 cm). We included constraints in our Voronoi algorithm to define the targets regions: each map has the same complexity but with different target positions and size. The maps were different for each participant and each task.

4.3 Task and Instructions

Each session included two tasks corresponding to the discovery and memorization phases observed during haptic map exploration [38]. The first task was to explore a digital map and find 4 regions as fast as possible. To simplify the task, the names of the 4 regions were the same for all trials but their location was pseudo randomly changed. After 2 min, if the participant did not find the 4 regions, we considered the trial as a failure. Before switching to the second task, we indicated the eventual missing regions to the participant.

The second task was to compare the distance between 3 regions among the 4 regions found in task 1. For each trial, the three target regions were pseudo randomly chosen (the regions were chosen in order to remove comparisons that were too easy; and the trials order was random). After 2 min, if the participant did not answer the question, we considered the trial as a failure.

During the training session, we explained the techniques and the tasks. Participants started with a training map for each technique. Participants notified the experimenter when they found a specific region, and when they identified the two regions with the shortest distance.

4.4 Design and Procedure

The experiment followed a within-participant design with Interaction Technique as main factor. A block included three trials with the same technique. We counterbalanced the blocks order across participants using a Latin Square. We informed participants that they could take a break between blocks. Before using each technique, participants completed a familiarization session during which the technique was explained, and they completed a few trials of each task until they felt comfortable with the technique (around 5 min).

4.5 Apparatus

For finger tracking, we used TopCodes tags [24] attached to the index fingers of each hand (Fig. 2). We tracked them with a Logitech C270 webcam (1280 × 720 px, 50 Hz) located above the exploration surface. We used two SimValley AW-414 Android smartwatches (91 g, 45 × 44 × 14 mm), connected to the computer running the study with TCP sockets over a local Wi-Fi.

Fig. 2.
figure 2

Tags on index fingers (left) and setup (right).

Full size image

4.6 Collected Data

We logged all tracking data, completion times and success rates for the two tasks. At the end of each block, participants had to fill a NASA-TLX questionnaire [21] about the technique they just used. At the end of the study, users had to classify the techniques by preference order and subjective effectiveness.

4.7 Identification of Exploration Strategies

We coded the exploration strategies according to Simonnet and Vieilledent [38] and Guerreiro et al. [19] (see Related Work section). We first unified equivalent strategies (i.e. gridline and path scan; back-and-forth and to-the-point). As a result, we had a set of eight strategies corresponding to the discovery (path scan, perimeter, freeform, freeform symmetry) and memorization (focused, back-and-forth, cyclic and point-of-reference) phases. During data analysis, we also included an additional discovery strategy named asymmetry (i.e. a different strategy with each hand).

The “Path scan” strategy corresponds to hand movements that clearly show a systematic and well organized exploration of the drawing. The “Perimeter” strategy is another discovery strategy that consists in corresponds to displacements along the physical limits of the interactive surface. The “Freeform” strategy represents an unstructured strategy, which does not explicitly show any form of organization. The “Freeform symmetry” is another unstructured strategy, but with the two-hands moving in similar directions (in mirror or in parallel). The “Focused” strategy consists in searching a small sub-section of the drawing in order to relocate a previously detected element of the drawing. The “Back-and-forth” strategy corresponds to back and forth movements between two elements of the drawing that have been previously located. The “Cyclic” strategy is a movement between three or more elements of the drawing, which are relocated one after the other before coming back to the starting element (e.g. A – B – C – A). Finally, the “Point-of-reference” strategy consists in star-like movements between three or more targets (e.g. A – B – A – C – A). The memorization strategies can help to locate the elements relative to each other.

In order to detect the discovery strategies in each trial, we relied on three independent judges inspecting the exploration paths. We generated one image showing the complete exploration path, and a set of images corresponding to successive periods (30 s duration) extracted from the whole exploration path. The visual coding by the judges was divided in two steps: first, they coded 5% of the trials set, and then they compared their respective coding in order to agree on the coding. Finally, they encoded the entire set of trials, and they collectively decided (i.e. reach a consensus if needed) which discovery strategy was the most observed in each trial.

Since three memorization strategies (back-and-forth, cyclic and point-of-reference) were difficult to identify visually, we developed three algorithms to perform the identification process. We validated the three algorithms with unit tests and with thorough visual exploration of many subparts of the exploration traces. In the following sections of the paper, when the memorization strategies were observed in every trials, we report the average number of occurrences per trial. When the memorization strategies were observed in some trials only, we report the percentage of trials in which these strategies were observed.

4.8 Analysis

Recently, the null-hypothesis significance testing (NHST) has come under criticism within the statistics [2, 9] and HCI communities [11, 12]. We thus report our results using estimation techniques with confidence intervals (0.95), instead of p-value statistics, consistent with APA recommendations [40].

5 Study 1: Exploration of the Design Space

The goal of this first study was to compare the impact of the design factors during map exploration.

5.1 Participants

We recruited 15 sighted participants (3 females) aged between 19 and 29 (M = 23, SD = 3). Participants were blindfolded for the study. They were recruited at the university: 12 of them were undergraduate students, 2 were PhD students, and one was research assistant. Fourteen participants were right-handed and one left-handed. Six participants owned a smartwatch. None of them had hearing problems.

5.2 Interaction Techniques

We compared the four two-handed techniques from our design space (U-SF, U-DF, B-SF and B-DF) and the control technique. As mentioned earlier, the control technique is similar to exploring a DIM with one finger on a tablet with the accessibility feature activated; and U-SF condition is similar to exploring a DIM with two fingers.

5.3 Results

In total, we collected 5 techniques × 2 tasks × 3 repetitions × 15 participants = 450 trials.

Task 1: Completion Time and Success Rate.

On average, participants took 64.4 s (CI [55.8, 72.4]) using one hand only (Control), 68.9 s (CI [61.4, 76.5]) using two hands with unilateral feedback (average completion time for U-SF and U-DF because they are similar), and 45.5 s (CI [40.2, 51.6]) using two hands with bilateral feedback (average completion time for B-SF and B-DF because they are similar). Hence, results show a difference between techniques concerning completion time (Fig. 3). The Control and unilateral techniques were slower than bilateral techniques. Results did not show any difference regarding the feedback similarity factor (i.e. between U-SF and U-DF, or between B-SF and B-DF). Among all the trials, only nine trials were considered as failure. For each condition, it represents: 2 for Control (4.44%), 6 for U-SF (13.3%) and 1 for U-DF (2.22%).

Fig. 3.
figure 3

Mean time in s to complete Task 1.

Full size image

Task 2: Completion Time and Success Rate.

Concerning the comparison of map elements location (task 2), we observed that, very often, participants remembered the position of the regions to be compared from task 1, and hence answered directly without re-exploring the map (62.5% of the trials). For the remaining trials with map exploration, we observed no difference in completion time between techniques: on average it took participants 24.8 s to complete the task. Regarding success rates, only 2 U-SF trials were incorrect (4.44%).

Strategies Used.

During task 1, across the different conditions, the most used strategies were “Path scan” (71% of the trials) and “Freeform” (15.5%). The prevalence of the “Path scan” strategy was true for all the techniques (Control: 73.3%; U-SF: 61.1%; U-DF: 66.6%; B-SF: 76.7% and B-DF: 77.2%).

During task 2, when users re-explored the map, the most used discovery strategies were “Freeform” (75.7% of the trials) and “Path scan” (13.7%). Participants also used memorization strategies: “Back-and-forth” movements with the same hand between two regions were observed in 28.8% of the trials for Control, 20% for U-SF, 26.6% for U-DF, 22.2% for U-SF, 31.1% for B-SF, and 28.8% for B-DF. “Cyclic” strategy over three regions was observed in 4% of the trials for Control, and not observed with the other techniques. The participants did not used the “Point-of-reference” strategy.

Simultaneous Hand Movements.

To further investigate two-handed exploration, we identified simultaneous movements for the different techniques. Simultaneous movements correspond to exploratory movements with both hands at the same time. On average, participants used simultaneous movements in 15.7% of the trials using U-SF, in 17.4% using U-DF, in 72.1% using B-SF, in 75.8% using B-DF. This clearly illustrates how the feedback impacts behavior. With bilateral feedback, participants did use their two hand at the same time without any difficulty to manage parallel audio and vibratory feedback.

Subjective Report.

User Preference. Overall, participants preferred using the bilateral technique with different feedback (B-DF). Among the 15 participants, 10 chose this technique as their favorite, 3 chose the bilateral with same feedback (B-SF), and 2 the unilateral with different feedback (U-DF).

User Feedback.

All the participants appreciated moving both hands at the same time, and most of them added positive comments about the bilateral feedback. P3 appreciated not “wasting time on determining which hand I’m using”. P5 found that “feedback is stronger” with bilateral feedback. Regarding the B-DF technique, users enjoyed “listening to different voices because it allows me to easily identify which hand triggered the feedback” (P11). Concerning unilateral feedback, P8 subjectively reported exploring slowly, and explained that he had to “move his hands one after the other in order not to miss any regions”. P12 felt “less confident using U-SF and U-DF techniques because the exploration was less intuitive”.

5.4 Discussion

In this first experiment, we found out that bilateral feedback techniques allow participants to perform task 1 faster than the other techniques (control and both unilateral techniques). This result confirms that bilateral feedback makes it easier for participants to locate elements on the digital map. The type of feedback (same or different) did not affect performance, as we found no differences between B-SF and B-DF in terms of accuracy or completion time. However, a large majority of participants (10/15) preferred having a different feedback (B-DF) compared to the same feedback (B-SF). In addition, most users were able to build an accurate mental representation of the map during exploration (task 1), and active comparison of elements location during task 2 was not necessary in a majority of trials.

6 Study 2: Impact of Feedback on Performance and Exploration Strategies

We ran a second experiment with 12 users with VI, who have an expertise in two-handed exploration of raised-line diagrams compared to the participants of the previous study. Since feedback similarity did not raise differences in the first study, we focused on the location of the feedback (unilateral vs. bilateral) for two-handed exploration.

6.1 Interaction Techniques

In order to shorten the experiment time in this study, we focused on three conditions only: Control, U-SF and B-DF. We made that choice according to the results of the first study. Indeed, there was no difference between U-SF and U-DF on one side, and B-SF and B-DF on the other side (see Fig. 3). Secondly, as mentioned earlier, the Control and U-SF conditions are similar to exploring a digital map on a tablet, with one or two fingers respectively. We selected the Control condition because there is no feedback ambiguity, and because people with visual impairment already use a similar technique on regular phones and tablets. Hence, as stated in its name, it represents a good control condition. B-DF was selected because it is the best and preferred technique identified in the first study.

6.2 Participants

We recruited 12 people with VI (5 females) aged between 21 and 73 years old (M = 52, SD = 15). The level of visual impairment varied: 9 of them were totally blind, and 3 had residual light perceptions. All participants use VoiceOver daily, and three of them use a map application (i.e. Ariadne GPS) on their smartphone. Seven participants had already used a smartwatch before, and two participants own a smartwatch. None of them had hearing problems. All of them were used to exploring raised-line diagrams (Table 1).

Table 1. Description of the visually impaired participants
Full size table

6.3 Maps

To design the maps, we used the Voronoi algorithm to randomly generate 30 regions on each map. We generated 216 maps in order to use a different map in each trial (for each participant, condition, and task). In addition, in order to verify whether bilateral feedback improves performance on the comparison task too, we systematically changed the map between task 1 and task 2, which means that participants had to locate four new regions in task 2 before comparing them. Then, both discovery and memorization strategies should appear in task 2.

6.4 Results

In total, we collected 3 techniques × 2 tasks × 3 repetitions × 12 participants = 216 trials.

Task 1: Completion Time and Success Rate.

Results show a clear difference between techniques concerning completion time to find the four regions (task 1). Participants completed the task faster with bilateral feedback (50 s, CI [42.9, 58.8]) than with Control (76 s (CI [66.7, 88.3]) or unilateral (93 s, CI [80.7, 107.2]), as illustrated in Fig. 4. This difference is quite important, since B-DF is 31.2% faster than Control and 42.9% than U-SF. Eleven trials that lasted longer than 2 min were considered as failure: 7 Control (19.4%) and 4 U-SF (11.1%).

Fig. 4.
figure 4

Mean time in s to complete Task 1.

Full size image

Task 2: Completion Time and Success Rate.

On average, participants completed the comparison task, i.e. finding the four regions and answering the question, in 92.8 s (CI [69.7, 118.5]) with B-DF, in 110 s (CI [92.8, 141.3]) with Control and in 137 s (CI [113.1, 171]) with U-SF. The ratio analysis confirmed the difference between B-DF and U-SF: the ratio and the corresponding CIs was above 1 (meaning that the B-DF is faster than U-SF) (Fig. 5).

Fig. 5.
figure 5

Mean completion time in seconds (left), and ratio (right) for Task 2.

Full size image

The time needed to find the four regions for the first time was, on average, 83 s with B-DF, 94 s with Control and 122 s with U-SF. These times are higher than those observed in task 1 because participants mixed up discovery and memorization strategies from the beginning of task 2.

The percentage of correct answers was 77% (CI [54.5, 99.5]) with the Control condition, 50% (CI [27.5, 72.50]) with U-SF, and 77% (CI [57.25, 96.75]) with B-DF. Because the task had to be completed within 4 min, 2 trials were considered as failure, corresponding to 1 Control (0.9%) and 1 U-SF (0.9%).

Strategies Used.

During task 1, when exploring with the control technique, the most used strategy was the “Path scan” (83.3% of the trials), and then “Perimeter” (8.3%). When exploring with two hands and unilateral feedback (U-SF), the most used strategy was the “Path scan” (66.6% of the trials) then “Freeform” (16.6%; see Fig. 6 right). When exploring with two hands and bilateral feedback (B-DF), the most used strategy was the “Path scan” (74.9%) and then “Freeform” (11.1%). We also identified when hands explore in symmetry (“Path scan”; Fig. 6 left). This strategy appeared 41.6% with unilateral feedback, and 72.2% with bilateral feedback.

Fig. 6.
figure 6

Example of strategies used (the underlying digital map is virtual. It is displayed for illustration purpose only). Left: “Path scan” strategy (P4 with B-DF, task 1, simultaneous hand movements). Right: “Freeform” (P2 with U-SF, task 2, sequential hand movements). Brown and blue line correspond to left and right hands movements respectively. (Color figure online)

Full size image

Because users were exposed to a new map during task 2, discovery strategies were also observed. The “Path scan” was used in 88.9% of the trials with the Control technique, 91.6% with U-SF, and 77.7% with B-DF.

In task 2, we more specifically focused on the memorization strategies (“Back-and-forth”, “Cyclic”, and “Point-of-reference”). “Back-and-forth” movements were observed in every trials, and the number of occurrences in each trial varied according to the technique being used: 5.3 (CI [4.1, 6.3]) for Control, 3.6 (CI [2.8, 5.4]) for U-SF, and 2.4 (CI [1.9, 3.8]) for B-DF (see Fig. 7).

Fig. 7.
figure 7

Example of “Back-and-forth” strategies used (the underlying digital map is virtual. It is displayed for illustration purpose only). Left: P2 with U-SF, task 2. Right: P7 with B-DF, task 2. The blue line corresponds to right hand movements. (Color figure online)

Full size image

Contrary to “Back-and-forth” strategy, “Cyclic” and “Point-of-reference” strategies were observed in some trials only. The “Cyclic” strategy with the same hand (i.e. going from region A, to B, to C, and back to A) over 3 regions was observed in 33.3% of the Control trials, 38.8% of the U-SF trials, and 22.2% of the B-DF trials. “Cyclic” strategy over 4 regions was observed in 30.5% of the Control trials, and only 5.5% of the U-SF and B-DF trials (see Fig. 8).

Fig. 8.
figure 8

Example of “Cyclic” strategies used (the underlying digital map is virtual. It is displayed for illustration purpose only). Left: P12 with U-SF, task 2. Right: P9 with B-DF, task 2. The blue line corresponds to right hand movements. (Color figure online)

Full size image

The “Point-of-reference” strategy with the same hand (going from A to B, back to A, and then to C) was observed in 58.3% of the Control trials, 41.6% of the U-SF trials, and 33.3% of the B-DF trials (see Fig. 9).

Fig. 9.
figure 9

Example of “Point-of-reference” strategies used (the underlying digital map is virtual. It is displayed for illustration purpose only). Left: P5 with U-SF, task 2. Right: P5 with B-DF, task 2. Brown and blue lines correspond to left and right hands movements respectively. (Color figure online)

Full size image

Finally, we observed the number of occurrences of memorization strategies realized with two hands (i.e. placing one hand on one target region, as an anchor, and using the other hand to re-locate other target regions): 2.5 (CI [2.27, 3.06]) occurrences per trial for U-SF, and 2.8 (CI [2.27, 3.37) occurrences for B-DF. These results show that participants combined the use of both hands during two-handed exploration to compare the relative location of two regions.

In the Annex of the paper, we have added many figures corresponding to the different strategies used during exploration.

Simultaneous Hand Movements.

With unilateral feedback (U-SF), 37% of the exploration time was based on simultaneous hands movements. This percentage increased to 70% with bilateral feedback (B-DF), which highlights the impact of feedback on the use of the two hands. Even though both hands were involved with U-SF, users mainly did sequential movements. On the contrary, they effortlessly do simultaneous hand movements with B-DF.

Subjective Report.

User Preference. 9 out of 12 participants ranked the bilateral technique as their favorite, 2 preferred the control condition (none of these two people use any map application on his/her smartphone or tablet), and 1 preferred unilateral feedback. Participants preferring bilateral feedback mentioned that using both hands in parallel, and hence hearing two concurrent voices, “does not raise any cognitive issue” (P1, P3, P4, P7, P8, P9, P10, P11 and P12). Two of them reported that they are regularly “confronted to different audio stimuli at the same time in their daily lives” (P3, P4) (i.e. listening to street noises while talking to a person). However, two of them reported that bilateral feedback requires “more concentration and cognitive load” (P2, P5), but that using it for such a short task was not disturbing. All participants reported feeling the vibration. Eleven out of twelve participants reported that wearing two smartwatches for the bilateral technique was convenient as “it is easy to understand where the feedback comes from” (P1, P10). One participant said that “wearing two smartwatches is useless” (P6 - same participant that scored unilateral technique first).

NASA-TLX.

We did not observe any major difference on the different dimensions of the NASA TLX questionnaire (mental demand, physical demand, temporal demand, effort and frustration) between the three techniques. Participants evaluated the performance of B-DF and U-SF (M = 75 and M = 74 respectively) better than control (M = 63).

7 Discussion

7.1 Main Findings

These two studies showed that bilateral feedback improves map exploration and more specifically the localization and comparison of specific elements on the map. We can consider [4] that there is a difference of raised-line exploration expertise between users with VI (study 2) and blindfolded users (study 1). Because the results show the same trends in these two studies, it seems that expertise does not have any effect on the usability of the bilateral techniques. A more surprising result is that exploring the map with two hands but triggering unilateral feedback was not faster than one-handed exploration. This is an important result showing than an efficient two-handed exploration technique for digital maps must provide bilateral feedback.

7.2 The Effect of Bilateral Feedback on Exploration Strategies

We observed different exploration strategies depending on the technique being used. With the two-handed technique and bilateral feedback, users explored the map with their two hands, and each hand mainly explored one half of the map only. In task 2 (comparison of map elements), users also took advantage of using two hands: quite often, one hand was used as an anchor while the other hand was moving around. Consequently, they did less back and forth movements between elements. With these bimanual strategies, based on adapted bilateral feedback, users were more efficient when exploring a map and when comparing the location of different regions. When the bilateral feedback was not provided (U-SF), users were less efficient and preferred using one hand only.

7.3 Recommendation for Touch Devices

It is important to note that two of our experimental conditions (Control and U-SF) may correspond to a situation where users with VI explore maps on a tablet with TTS activated (e.g. VoiceOver or TalkBack). In such a situation, they are free to explore the map with one or two fingers (corresponding to Control and U-SF respectively), but with ambiguous feedbacks. When it is not possible to use bilateral unambiguous feedback, our results show that it is more efficient to use one finger only.

Our results also show that bilateral feedback may improve tactile exploration, provided that both hands are identified during exploration. In order to implement bilateral feedback on touchscreens, it is mandatory to identify and track each finger unambiguously (or at least right and left hands if only one finger is used on each hand). Recent studies show that finger identification can be based on finger orientation [41], or on external hardware such as a wearable device, or a camera [20].

7.4 Other Applications for Bilateral Feedback

A recent study [4] has shown that people with VI intuitively use two hands to explore different types of raised line graphics, such as common drawings, mathematical graphs or neighborhood maps. Bilateral feedback could be of interest to render these different graphics accessible on touch surfaces. For instance, a two-handed exploration of digital graphs could allow the user to explore the horizontal or vertical axes with one hand, and the graph itself with the other [34]. It could also be used on tagged drawings or photos (e.g. on Facebook) to explore tags with both hands, which is similar to our experimental task. Previous work [4] showed that the average of simultaneous exploration time for people with VI on raised-line diagrams is 74.6%. In the current study, we observed that simultaneous exploration using bilateral technique was 70%. These two results are comparable and show that people with VI can perform similar two-handed exploration on raised-line diagrams and digital maps.

8 Conclusion and Future Work

In this paper, we designed and evaluated four feedback techniques for two-handed exploration of digital interactive maps by users with visual impairments. These techniques resulted from a design space combining feedback location and similarity. The results, which were comparable for both blindfolded and users with visual impairments, show the advantages of two-handed vs. one-handed exploration, providing that bilateral feedback is available and unambiguous. Although we observed these results on graphics that represent maps, we are convinced that they can apply to other interactive accessible graphics (figurative drawings, graphs, etc.). Future work will focus on how to design affordable hand and finger tracking techniques that could provide multimodal bilateral feedback. Moreover, future work will investigate bimanual interactions on complex digital maps (i.e. with different data sets).