Using eye movement activity as a correlate of cognitive workload
Introduction
Terminal air traffic control facilities perform radar operations to direct aircraft during the departure, descent, and approach phases of flight. In general, controllers at these facilities direct aircraft that are transitioning from the en route phase through to the approach phase into a destination airport located within the terminal airspace. In addition, terminal controllers also direct aircraft for departures. During all controller operations, aircraft entering or departing the airspace must be separated at safe distances. Because of the high density of traffic and dynamic aircraft maneuvers in the terminal airspace, controllers face additional difficulties and increased workload during adverse weather conditions (Ahlstrom, 2003). Adverse weather creates safety hazards for pilots, constrains the usable airspace for air traffic controllers, and reduces the overall capacity of the National Airspace System (NAS).
Controllers need weather information to optimize the use of routes and runways, improve planning, and provide weather advisories to pilots. Today, terminal controllers maintain their weather situation awareness by receiving weather briefings from supervisors and by viewing six levels of precipitation on their workstation. In addition, controllers receive reports of hazardous weather conditions that pilots encounter during flight. However, the controllers lack detailed display information about storm movements that would enable them to effortlessly correlate weather with relevant sector traffic. Therefore, providing controllers with advanced weather displays could be one way to improve the efficiency and safety of terminal operations during adverse weather conditions. Nonetheless, although accurate and timely weather information is important for the mitigation of both delays and safety risks, it is not clear what types of weather displays would be most useful for controllers (Ahlstrom, 2003). Too much weather information on controller workstation displays could contribute to display clutter and therefore interfere with the controllers’ ability to extract critical traffic data. Optimal weather displays, on the other hand, could increase controller efficiency via increased traffic throughput, improved weather advisories to pilots, and reduced workload associated with controlling traffic during adverse weather conditions. However, implementing weather displays in the air traffic domain carries with it some degree of risk. Specifically, as we add more information displays to an already complex system, we could potentially increase controller workload and reduce efficiency rather than enhancing control operations. Therefore, researchers could benefit from a way to measure controller workload during weather tool use. They could then evaluate the impact on controller performance.
Because workload can negatively affect controller performance and increase the probability of operating hazards, researchers have spent a great deal of effort developing measures and probes of controller workload (Averty et al., 2002, Averty et al., 2004; Athénes et al., 2002; Collet et al., 2003). Air traffic control researchers have commonly used a workload probe called the Air Traffic Workload Input Technique (ATWIT: Stein, 1985, Stein, 1991, Stein, 1998). The ATWIT is a one-dimensional workload rating method that allows controllers to provide subjective estimates of workload while they perform control operations. When employed during high-fidelity human-in-the-loop simulations, this method allows controllers to indicate their instantaneous workload by pressing one of ten keyboard buttons labelled from 1 (low workload—all tasks completed) to 10 (high workload—some tasks left uncompleted). The method has very broad applicability and has been used in a variety of studies including workload assessments during evaluations of situation awareness (Endsley et al., 2000), display concepts (Endsley et al., 1999), Air-Ground Traffic Management (DAG-TM) concepts (Lee et al., 2005), and air traffic control communications (Rantanen et al., 2005).
Aside from using self-reported workload ratings as a gauge of operator workload, researchers have used eye movement parameters that are found to correlate with cognitive demands. For example, researchers have used parameters like blink rate and duration, pupil diameter (PD), saccadic extent, fixation frequency, and dwell time, to estimate the cognitive requirements of different tasks (Brookings et al., 1996; Van Orden et al., 2000, Van Orden et al., 2001; Veltman and Gaillard, 1998; Wilson, 2001; Wilson and Caldwell, 2002). In general, research has found blink rate and blink duration (BD) to decline as a function of increased workload (Van Orden et al., 2000; Veltman and Gaillard, 1998; Zeghal et al., 2002). Research has also found PD to increase as a function of cognitive processing demands (Iqbal et al., 2004, Iqbal et al., 2005; Lin et al., 2003; Van Orden et al., 2000; Zeghal et al., 2002). Furthermore, increases in workload have also been found to increase the number of saccades, decrease saccade duration (Rognin et al., 2004; Zeghal et al., 2002) and increase the frequency of long fixations (>500 ms: see Van Orden, 2000; Van Orden et al., 2000).
A potential benefit of using eye movement activity as a correlate of cognitive workload is that it provides for the possibility of capturing fluctuations in workload that occur over short time intervals. For example, system interactions or weather display interactions are difficult to capture with subjective rating methods without interrupting the operator's task. Furthermore, the frequency by which researchers collect workload ratings over time has a practical limit. Traditionally, researchers have collected ratings every 5 min, but shorter 2- (Kuk et al., 1999) and 1-min (Stein, 1985) intervals have also been used for certain applications. We hypothesize that eye movement activity measures, because of their potential for use as workload correlates, could provide more continuous, moment-to-moment measures of workload. With this, we could possibly detect differences in workload not reflected in subjective workload ratings.
Section snippets
Study purpose
Our purpose with the present study was to investigate the effect on controller operations and workload from the use of weather displays. Specifically, we wanted to assess the impact from weather displays on severe weather avoidance, controller efficiency, controller-pilot communications, and the safety of airspace operations. Additionally, we also wanted to examine whether eye movement activity recordings could be used as reliable correlates of cognitive workload while evaluating whether
Participants
Six terminal controllers participated as subjects in the simulation (M age=36.8 years [SD=6.3] and M job experience=13 years [SD=6.7]). We solicited the controllers from nationwide facilities, and their participation was voluntary.
Simulation setup
We used a high-fidelity simulator that emulates the modern workstations used in select terminal facilities. We presented weather displays directly on the controller workstation, or on an auxiliary weather display (Ahlstrom et al., 2004) located on top of the
Predictions
Our main hypotheses for the present study are that providing weather displays to controllers should increase their weather situation awareness and therefore:
- 1.
increase operational efficiency by means of an increased traffic throughput,
- 2.
enhance severe weather avoidance,
- 3.
reduce the workload associated with controlling traffic during adverse weather conditions.
For the remaining dependent measures we had no a priori knowledge leading us to predict the direction of the effect. For instance, we might
Data analysis
The system components of our simulator like the workstation interface, the traffic generation facility, the simulation scenario control, the weather display system, the ATWIT workload collection system, and the oculometer recording system, are all synchronized and driven by a single simulator clock. Furthermore, the simulator system continuously time stamps all dependent measures and controller system interactions during data collection (e.g., aircraft data entries and weather display
Severe weather avoidance
Our hypothesis that weather displays would enhance controllers’ ability to reduce aircraft weather cell penetrations was not confirmed by the penetration data. Fig. 4 shows the mean number of weather cell penetrations for the three simulation conditions by weather scenario. There were no significant differences in the mean number of penetrations for levels 4–6 between weather display conditions (auxiliary and workstation) and the control condition. Overall, there were very few levels 4–6
Subjective workload ratings
We evaluated the mean workload ratings as a function of simulation condition and weather scenario. A contrast showed that our hypothesis that the use of weather displays should reduce controllers’ workload (−0.5[auxiliary]−0.5[workstation]+1[control]) was not significant. Workload ratings were uniformly low, ranging from 2.3 to 3 on the 10-point scale, and there were no significant differences in mean workload ratings between weather display conditions and the control condition for Storm 1 or 2.
Eye movement activity analysis
We first performed an analysis of blink frequency and BD. We did not find a decrease in blink frequency with an increasing number of aircraft in the sector, and the slope of the regression line was not statistically different from zero. However, our analysis of BD did show an effect of the number of aircraft in the sector. Fig. 10 shows the mean BD as a function of the number of aircraft, with the mean BD decreasing with an increasing number of aircraft, R2=0.595, SE=0.008, F (1, 6)=8.81, p
Discussion
In the current study, we found a significant impact of weather displays on controller efficiency with increases in sector throughput (i.e., completed flights) up to 10%. By providing controllers with weather displays it appears that we enhance the controllers’ ability to detect approaching weather, monitor its movement, and understand its effect on future operations. In operational settings, this should increase controllers’ efficiency for timing of arrivals, for vectoring and adjustment of
References (41)
- et al.
Psychophysiological responses to changes in workload during simulated air traffic control
Biological Psychology
(1996) - et al.
Using eye movement parameters for evaluating human–machine interface frameworks under normal control operation and fault detection situations
International Journal of Human Computer Studies
(2003) - et al.
Combined eye activity measures accurately estimate changes in sustained visual task performance
Biological Psychology
(2000) Current trends in the display of aviation weather
Journal of Air Traffic Control
(2003)Advanced weather displays for TRACON controllers
Journal of Air Traffic Control
(2005)- Ahlstrom, U., Friedman-Berg, F., 2006. TRACON controller weather information needs: III. Human-in-the-loop simulation...
- et al.
Weather information display system (WIDS)
Journal of Air Traffic Control
(2004) Series 5000 Model 501 Eye Tracking System with Head Mounted Optics [Computer software]
(2005)- et al.
ATC complexity and controller workload: trying to bridge the gap
- et al.
Evaluating a new index of mental workload in real control situation using psychophysiological measures
Mental workload in air traffic control: an index constructed from field tests
Aviation, Space, and Environmental Medicine
Subjective aspects of mental workload in air traffic control
Confidence intervals in within-subject designs: a simpler solution to Loftus and Masson's method
Tutorial in Quantitative Methods for Psychology
The use of predictive displays for aiding controller situation awareness
Situation awareness: a comparison of measures
The Integrated Terminal Weather System (ITWS)
The Lincoln Laboratory Journal
Repeated-measures contrasts for “multiple-pattern” hypotheses
Psychological Methods
En Route Generic Airspace Evaluation (DOT/FAA/CT-TN97/7)
The Influence of Generic Airspace on Air Traffic Controller Performance (DOT/FAA/CT-TN95/38)
Task-evoked pupillary response to mental workload in human–computer interaction
Cited by (226)
Physiological workload assessment for highly flexible fine-motory assembly tasks using machine learning
2024, Computers and Industrial EngineeringInvestigating cognitive workload in irrelevant speech-based information communication with visual distractions: Pleasant or distracted?
2024, International Journal of Industrial ErgonomicsMethodology for augmented reality-based adaptive assistance in industry
2024, Computers in IndustryA systematic review on the influence factors, measurement, and effect of driver workload
2023, Accident Analysis and PreventionEnhanced Interactivity in VR-based Telerobotics: An Eye-tracking Investigation of Human Performance and Workload
2023, International Journal of Human Computer StudiesUsing eye tracking to examine expert-novice differences during simulated surgical training: A case study
2023, Computers in Human Behavior