Detecting individuals' spatial familiarity with urban environments using eye movement data

Highlights

• It is the first study using real-world eye movement data to infer spatial familiarity.

• Performance of different time window sizes and feature types were compared.

• We achieved a best accuracy of 81% in a 10-fold classification and 70% in the leave-one-task-out classification.

• We found important eye movement features for detecting users' spatial familiarity.

• Only a few seconds (e.g., 5 s) of eye movement data is sufficient for spatial familiarity detection.

Abstract

The spatial familiarity of environments is an important high-level user context for location-based services (LBS). Knowing users' familiarity level of environments is helpful for enabling context-aware LBS that can automatically adapt information services according to users' familiarity with the environment. Unlike state-of-the-art studies that used questionnaires, sketch maps, mobile phone positioning (GPS) data, and social media data to measure spatial familiarity, this study explored the potential of a new type of sensory data - eye movement data - to infer users' spatial familiarity of environments using a machine learning approach. We collected 38 participants' eye movement data when they were performing map-based navigation tasks in familiar and unfamiliar urban environments. We trained and cross-validated a random forest classifier to infer whether the users were familiar or unfamiliar with the environments (i.e., binary classification). By combining basic statistical features and fixation semantic features, we achieved a best accuracy of 81% in a 10-fold classification and 70% in the leave-one-task-out (LOTO) classification. We found that the pupil diameter, fixation dispersion, saccade duration, fixation count and duration on the map were the most important features for detecting users' spatial familiarity. Our results indicate that detecting users' spatial familiarity from eye tracking data is feasible in map-based navigation and only a few seconds (e.g., 5 s) of eye movement data is sufficient for such detection. These results could be used to develop context-aware LBS that adapt their services to users' familiarity with the environments.

Introduction

Currently, location-based services (LBS) often provide adaptive services based on various factors of users and user contexts, such as the user location, walking speed, road conditions, weather, and user history visits. Understanding users and their needs is key to LBS (Huang, Gartner, Krisp, Raubal, & Weghe, N.V.d., 2018; Jiang & Yao, 2006). In addition to providing well-established services (e.g., planning the shortest route and recommending points of interest based on the user location), recent LBS integrate solutions that are more tailored to high-level user context and user information needs. Examples include suggesting beautiful, pleasant and quiet routes rather than the shortest route (Huang et al., 2014; Quercia, Schifanella, & Aiello, 2014) and even adapting the map content according to users' visual behavior (Anagnostopoulos et al., 2017; Giannopoulos, Kiefer, & Raubal, 2015).

This study focused on an important high-level user context that has received little attention: users' familiarity with the environment. Knowing users' familiarity level of environments is helpful for enabling context-aware LBS that can automatically adapt information services according to users' familiarity with the environment (Savage, Chun, Chavez, & Mexico, 2011; Schmid & Richter, 2006; van Haeren & Mackaness, 2015; Zhou, Weibel, & Huang, 2021). First, displaying adaptive content on maps according to users' familiarity level can optimize the communication of map information, and thus increase the effectiveness and/or efficiency of map services. This will avoid overwhelming familiar users with unnecessary detailed information, or providing insufficient information for unfamiliar users. This can also help to decrease the user's cognitive load (Bunch & Lloyd, 2006) in processing geoinformation. Second, users' spatial familiarity level affects their route choice in wayfinding (Lovelace, Hegarty, & Montello, 1999; van Haeren & Mackaness, 2015). Users who are familiar with an environment may be more willing to actively explore new changes and new things, whereas unfamiliar users may want to find their destinations more quickly and accurately. Therefore, a navigation service can provide familiar users with more interesting routes and places and offer unfamiliar users with the shortest routes. Third, LBS adapted to users' familiarity with the environment can potentially help them acquire new spatial knowledge much more easily, because they can relate new places to their previously known places (Merriman, Ondřej, Roudaia, O'Sullivan, & Newell, 2016).

Unlike state-of-the-art studies that used questionnaires, mobile phone positioning (GPS) data, and social media data to measure spatial familiarity (as reviewed in Section 2.1), this study focused on the use of a new type of sensory data - eye movement data - to infer users' familiarity with urban environments. People perceive and understand urban environments largely through the visual world. Human visual behavior is considered associated with users' high-level cognitive states and intentions (Henderson, Shinkareva, Wang, Luke, & Olejarczyk, 2013). Thus, tracking people's eye movements to infer high-level user context is a more intuitive method than mining their GPS and social media data. Furthermore, advances in eye-tracking technology have made eye trackers lighter, less expensive, and more accurate. Eye trackers may become ubiquitous sensors (similar to GPS) that are embedded in portable devices such as smartphones, tablets, and glasses in the near future. Therefore, eye-gaze based human-computer interactions (HCIs) (e.g., detecting users' familiarity by tracking eye movements in real time and providing tailored information according to their familiarity level) may be possible in future LBS (Anagnostopoulos et al., 2017).

In this study, we explored the potential of eye movement data to infer users' familiarity with environments using a machine learning approach. We collected users' eye movement data when they were navigating familiar and unfamiliar urban environments. We extracted two sets of eye movement features (namely, basic statistical features and fixation semantic features) to characterize users' visual behavior. We then trained and cross-validated a random forest classifier to infer whether the users were familiar or unfamiliar with the environments (i.e., binary classification). While the results of this work can be used to enable familiarity-aware LBS, such an investigation using mobile eye tracking may also be helpful for research on activity/intention recognition, tourist behavior, environment perception, and urban planning.

The manuscript has five additional sections. Related work on measuring spatial familiarity and eye tracking is reviewed in Section 2. Methods are described in Section 3. Results are presented in Section 4 and discussed in Section 5. Finally, we draw conclusions and propose future work in Section 6.