Neural dynamics of spreading attentional labels in mental contour tracing

Highlights

• A recurrent neural network is designed to explain properties of mental contour tracing.

• Tracing is achieved by propagation of rate enhancement along the target contour.

• Tracing is modulated by the multi-scale contour and L-junction detection units.

• Model’s tracing performance is consistent with behavioral data.

Abstract

Behavioral and neural data suggest that visual attention spreads along contour segments to bind them into a unified object representation. Such attentional labeling segregates the target contour from distractors in a process known as mental contour tracing. A recurrent competitive map is developed to simulate the dynamics of mental contour tracing. In the model, local excitation opposes global inhibition and enables enhanced activity to propagate on the path offered by the contour. The extent of local excitatory interactions is modulated by the output of the multi-scale contour detection network, which constrains the speed of activity spreading in a scale-dependent manner. Furthermore, an L-junction detection network enables tracing to switch direction at the L-junctions, but not at the X- or T-junctions, thereby preventing spillover to a distractor contour. Computer simulations reveal that the model exhibits a monotonic increase in tracing time as a function of the distance to be traced. Also, the speed of tracing increases with decreasing proximity to the distractor contour and with the reduced curvature of the contours. The proposed model demonstrated how an elaborated version of the winner-takes-all network can implement a complex cognitive operation such as contour tracing.

Introduction

Vision starts with the parallel registration of features such as color, shape, or motion in dedicated processing streams (Lennie, 1998). The output of feature detectors is further elaborated by a set of Gestalt grouping rules to form a spatial representation of perceptual belongingness among objects and of figure-ground relationships (Wagemanas, 2014). Although the perceptual organization of a scene according to Gestalt rules involves sophisticated computations, it is not sufficient to extract all information that is of interest to the observer (Tsotsos, Kotseruba, Rasouli, & Solbach, 2018). For instance, the perception of spatial relations between distal image parts, such as whether they are connected or whether they lie inside or outside of the same bounding surface, cannot be computed by any type of spatially limited feature detectors (Minsky & Papert, 1988). Fig. 1 illustrates this fact. Whether the patterns presented in Fig. 1A and B contain one or two black spirals is not immediately apparent.

Ullman, 1984, Ullman, 1996 suggested that human observers comprehend spatial relations by applying visual routines on the representation offered by early vision. Visual routines refer to a set of cognitive operations that engage attention to bind together parts of the scene that remained ungrouped by the early visual representation. Attention labels distant features to render their spatial relations explicit. In a similar vein, Roelfsema and Houtkamp (2011) distinguished between base grouping, which depends on a fast extraction of image features and their conjunctions, and incremental grouping, which involves the engagement of slow, serial labeling of image elements that belong to the same perceptual group. Incremental grouping relies on object-based attention to highlight the representation of one perceptual group in an input image composed of many competing groups. Incremental grouping requires the establishment of dynamic links between neurons encoding features of the attended object and, at the same time, disabling connections to neurons encoding an unattended object.

Mental contour tracing is an example of a visual routine or incremental grouping process that has been extensively studied at both psychological and neural levels. It is engaged when we attempt to determine whether two image regions are connected, as illustrated in Fig. 1. The detection of connectedness is important because connected image parts are likely to belong to the same objects, whereas disconnected parts usually belong to different objects (Roelfsema & Singer, 1998). In the laboratory, contour tracing is studied by a task where observers are required to determine whether two dots lie on the same contour in a pattern consisting of two (or more) intermingled contours. A typical finding is that the time it takes to provide an answer increases monotonically, but not linearly, with the distance between dots on the contour. The key factor determining the speed of tracing is the distance on the contour, and not the Euclidean distance between the dots, which is kept constant (Jolicoeur et al., 1986, Pringle and Egeth, 1988). Furthermore, tracing exhibits scale invariance, since the absolute size of the contour does not influence the speed of tracing. This suggests the involvement of multiple spatial scales (Jolicoeur & Ingleton, 1991).

To isolate relevant factors contributing to the dynamics of tracing, Jolicoeur, Ullman, and Mackay (1991) devised simple stimuli consisting of a set of parallel straight lines or parallel curved lines. An example of the type of stimuli they used is depicted in Fig. 2. Jolicoeur et al. (1991) systematically varied the distance between the target and distractor contours (Fig. 2A–D) and the amount of curvature in the curved contours (Fig. 2E–H) to measure their impact on the dynamics of tracing. Their results revealed that (a) the tracing time increases monotonically and roughly linearly with the length of the contour, (b) the tracing speed decreases with decreased spacing between the target and distractor contours, and (c) the tracing speed decreases with increased contour curvature. These findings help to explain why tracing times were not a linear function of the distance in studies employing two contours that wiggle around each other. In such stimuli, the distance between the target and distractor contours, as well as their curvature, vary considerably along the path that needs to be traced. Therefore, we will focus on the results of Jolicoeur et al.’s (1991) study in our modeling efforts.

Several studies examined the question of whether attention moves or spreads along the contour. According to the zoom lens model, attention makes discrete jumps from one part of the contour to the next. The size of jumps is flexibly adjusted to avoid making mistakes (McCormick and Jolicoeur, 1991, McCormick and Jolicoeur, 1994). Therefore, the size of the jump is smaller when the target contour is near the distractor, and it increases when the distractor contour is further away. Crundall, Dewhurst and Underwood (2008) provided further support for this model by demonstrating that participants do not notice changes that occur near the beginning of the contour, when the tracing operator has sufficient time to move away from the starting point.

On the other hand, three studies (Houtkamp et al., 2003, Roelfsema et al., 2010; Scholte, Spekreijse, & Roelfsema, 2001) found that tracing involves highlighting all elements of the same contour. These results imply that tracing operates similarly to object-based attention: it selects all spatial locations occupied by the same object. In other words, tracing creates a visual representation where a grouped array of locations is selected (Cosman and Vecera, 2012, Hollingworth et al., 2012, Vatterott and Vecera, 2015).

Neural recordings in the monkey primary visual cortex (V1) suggest that contour tracing is associated with elevated firing rates in neurons whose receptive fields fall on the target contour, relative to neurons whose receptive fields fall on the distractor contour (Roelfsema, Lamme, & Spekreijse, 1998). Next, it was found that firing rate modulation occurs earlier for neurons located near the start of the tracing process (fixation point) and later for neurons located further away along the contour. Importantly, the response enhancement for neurons encoding early segments of the contour remained approximately constant during the whole trial, thus providing direct support for the idea that attention spreads rather than moves along the contour (Roelfsema, Khayat, & Spekreijse, 2003). Moreover, the timing of the response enhancement on neurons encoding a distal contour segment depends on how close the target and distractor contour are placed (Pooresmaeili & Roelfsema, 2014). If there is a small gap between proximal segments of the target and distractor contours, then the response enhancement on the distal segment of the target contour is delayed relative to the stimulus with a wider gap. This is consistent with the behavioral findings on the effect of contour spacing on the speed of tracing (Jolicoeur et al., 1991).

Wannig, Stanisor, and Roelfsema (2011) found that enhanced activity is initiated by the external cue and automatically spreads from the cued to the neighboring neurons if they share the same feature selectivity, namely color or orientation. Interestingly, attentional modulation in the contour tracing task was not observed in all tested neurons. About half of the neurons were not affected by the attention at all, and they were labeled as N-neurons — as opposed to A-neurons, which exhibited response enhancement (Pooresmaeili, Poort, Thiele, & Roelfsema, 2010). Finally, it should be noted that the reviewed studies were not able to discern whether the source of the tracing signal arises from the horizontal connections within the V1 or via feedback connections from the extrastriate cortex. Roelfsema and Houtkamp (2011) proposed that contour tracing is a consequence of the interactions within and between cortical areas V1, V2, and V4 organized in a dynamic processing hierarchy termed a growth cone. The size of the cone is dynamically adjusted depending on the stimulus conditions. When the target and distractor contours are far apart, a larger cone is activated that encompasses higher hierarchical levels containing neurons with larger receptive sizes. Contour tracing consequently advances faster along the contour (see also Jeurissen et al., 2016, Pooresmaeili and Roelfsema, 2014).

The aim of the present study is to develop a neurocomputational account of the dynamics of mental contour tracing consistent with the reviewed behavioral and neural data. The model rests upon a feature-based winner-takes-all (F-WTA) network, recently proposed by Marić and Domijan (2018). The F-WTA network is a recurrent competitive map with local interactions between excitatory units and global inhibition mediated by a single inhibitory unit. It is capable of simultaneous selection of many winners based on top-down guidance. We have demonstrated how to embed the F-WTA network into a larger neural architecture incorporating multi-scale contour and L-junction detection networks. The output of the contour and L-junction detection is used to guide the lateral excitatory interactions within the F-WTA network. In this way, enhanced activity in the F-WTA network spreads along the target contour as fast as possible without making mistakes – that is, without activity spillover to the distractor contour.