Neural dynamics of spreading attentional labels in mental contour tracing
Introduction
Vision starts with the parallel registration of features such as color, shape, or motion in dedicated processing streams (Lennie, 1998). The output of feature detectors is further elaborated by a set of Gestalt grouping rules to form a spatial representation of perceptual belongingness among objects and of figure-ground relationships (Wagemanas, 2014). Although the perceptual organization of a scene according to Gestalt rules involves sophisticated computations, it is not sufficient to extract all information that is of interest to the observer (Tsotsos, Kotseruba, Rasouli, & Solbach, 2018). For instance, the perception of spatial relations between distal image parts, such as whether they are connected or whether they lie inside or outside of the same bounding surface, cannot be computed by any type of spatially limited feature detectors (Minsky & Papert, 1988). Fig. 1 illustrates this fact. Whether the patterns presented in Fig. 1A and B contain one or two black spirals is not immediately apparent.
Ullman, 1984, Ullman, 1996 suggested that human observers comprehend spatial relations by applying visual routines on the representation offered by early vision. Visual routines refer to a set of cognitive operations that engage attention to bind together parts of the scene that remained ungrouped by the early visual representation. Attention labels distant features to render their spatial relations explicit. In a similar vein, Roelfsema and Houtkamp (2011) distinguished between base grouping, which depends on a fast extraction of image features and their conjunctions, and incremental grouping, which involves the engagement of slow, serial labeling of image elements that belong to the same perceptual group. Incremental grouping relies on object-based attention to highlight the representation of one perceptual group in an input image composed of many competing groups. Incremental grouping requires the establishment of dynamic links between neurons encoding features of the attended object and, at the same time, disabling connections to neurons encoding an unattended object.
Mental contour tracing is an example of a visual routine or incremental grouping process that has been extensively studied at both psychological and neural levels. It is engaged when we attempt to determine whether two image regions are connected, as illustrated in Fig. 1. The detection of connectedness is important because connected image parts are likely to belong to the same objects, whereas disconnected parts usually belong to different objects (Roelfsema & Singer, 1998). In the laboratory, contour tracing is studied by a task where observers are required to determine whether two dots lie on the same contour in a pattern consisting of two (or more) intermingled contours. A typical finding is that the time it takes to provide an answer increases monotonically, but not linearly, with the distance between dots on the contour. The key factor determining the speed of tracing is the distance on the contour, and not the Euclidean distance between the dots, which is kept constant (Jolicoeur et al., 1986, Pringle and Egeth, 1988). Furthermore, tracing exhibits scale invariance, since the absolute size of the contour does not influence the speed of tracing. This suggests the involvement of multiple spatial scales (Jolicoeur & Ingleton, 1991).
To isolate relevant factors contributing to the dynamics of tracing, Jolicoeur, Ullman, and Mackay (1991) devised simple stimuli consisting of a set of parallel straight lines or parallel curved lines. An example of the type of stimuli they used is depicted in Fig. 2. Jolicoeur et al. (1991) systematically varied the distance between the target and distractor contours (Fig. 2A–D) and the amount of curvature in the curved contours (Fig. 2E–H) to measure their impact on the dynamics of tracing. Their results revealed that (a) the tracing time increases monotonically and roughly linearly with the length of the contour, (b) the tracing speed decreases with decreased spacing between the target and distractor contours, and (c) the tracing speed decreases with increased contour curvature. These findings help to explain why tracing times were not a linear function of the distance in studies employing two contours that wiggle around each other. In such stimuli, the distance between the target and distractor contours, as well as their curvature, vary considerably along the path that needs to be traced. Therefore, we will focus on the results of Jolicoeur et al.’s (1991) study in our modeling efforts.
Several studies examined the question of whether attention moves or spreads along the contour. According to the zoom lens model, attention makes discrete jumps from one part of the contour to the next. The size of jumps is flexibly adjusted to avoid making mistakes (McCormick and Jolicoeur, 1991, McCormick and Jolicoeur, 1994). Therefore, the size of the jump is smaller when the target contour is near the distractor, and it increases when the distractor contour is further away. Crundall, Dewhurst and Underwood (2008) provided further support for this model by demonstrating that participants do not notice changes that occur near the beginning of the contour, when the tracing operator has sufficient time to move away from the starting point.
On the other hand, three studies (Houtkamp et al., 2003, Roelfsema et al., 2010; Scholte, Spekreijse, & Roelfsema, 2001) found that tracing involves highlighting all elements of the same contour. These results imply that tracing operates similarly to object-based attention: it selects all spatial locations occupied by the same object. In other words, tracing creates a visual representation where a grouped array of locations is selected (Cosman and Vecera, 2012, Hollingworth et al., 2012, Vatterott and Vecera, 2015).
Neural recordings in the monkey primary visual cortex (V1) suggest that contour tracing is associated with elevated firing rates in neurons whose receptive fields fall on the target contour, relative to neurons whose receptive fields fall on the distractor contour (Roelfsema, Lamme, & Spekreijse, 1998). Next, it was found that firing rate modulation occurs earlier for neurons located near the start of the tracing process (fixation point) and later for neurons located further away along the contour. Importantly, the response enhancement for neurons encoding early segments of the contour remained approximately constant during the whole trial, thus providing direct support for the idea that attention spreads rather than moves along the contour (Roelfsema, Khayat, & Spekreijse, 2003). Moreover, the timing of the response enhancement on neurons encoding a distal contour segment depends on how close the target and distractor contour are placed (Pooresmaeili & Roelfsema, 2014). If there is a small gap between proximal segments of the target and distractor contours, then the response enhancement on the distal segment of the target contour is delayed relative to the stimulus with a wider gap. This is consistent with the behavioral findings on the effect of contour spacing on the speed of tracing (Jolicoeur et al., 1991).
Wannig, Stanisor, and Roelfsema (2011) found that enhanced activity is initiated by the external cue and automatically spreads from the cued to the neighboring neurons if they share the same feature selectivity, namely color or orientation. Interestingly, attentional modulation in the contour tracing task was not observed in all tested neurons. About half of the neurons were not affected by the attention at all, and they were labeled as N-neurons — as opposed to A-neurons, which exhibited response enhancement (Pooresmaeili, Poort, Thiele, & Roelfsema, 2010). Finally, it should be noted that the reviewed studies were not able to discern whether the source of the tracing signal arises from the horizontal connections within the V1 or via feedback connections from the extrastriate cortex. Roelfsema and Houtkamp (2011) proposed that contour tracing is a consequence of the interactions within and between cortical areas V1, V2, and V4 organized in a dynamic processing hierarchy termed a growth cone. The size of the cone is dynamically adjusted depending on the stimulus conditions. When the target and distractor contours are far apart, a larger cone is activated that encompasses higher hierarchical levels containing neurons with larger receptive sizes. Contour tracing consequently advances faster along the contour (see also Jeurissen et al., 2016, Pooresmaeili and Roelfsema, 2014).
The aim of the present study is to develop a neurocomputational account of the dynamics of mental contour tracing consistent with the reviewed behavioral and neural data. The model rests upon a feature-based winner-takes-all (F-WTA) network, recently proposed by Marić and Domijan (2018). The F-WTA network is a recurrent competitive map with local interactions between excitatory units and global inhibition mediated by a single inhibitory unit. It is capable of simultaneous selection of many winners based on top-down guidance. We have demonstrated how to embed the F-WTA network into a larger neural architecture incorporating multi-scale contour and L-junction detection networks. The output of the contour and L-junction detection is used to guide the lateral excitatory interactions within the F-WTA network. In this way, enhanced activity in the F-WTA network spreads along the target contour as fast as possible without making mistakes – that is, without activity spillover to the distractor contour.
Section snippets
Model overview
The neural model of contour tracing consists of three components: The F-WTA network, the contour detection network (CDN), and the L-junction detection network (LDN). In this chapter, the components are informally described first to provide an understanding of how they contribute to the tracing. In the second part, the formal specification of each model component is provided.
The effect of distractor proximity
The basic property of contour tracing is that the time to connect starting and ending points on the contour increases monotonically with the distance between these points. Moreover, when tracing occurs along straight lines, its speed is modulated by the proximity of the target and distractor contours (Jolicoeur et al., 1991). As the proximity is increased, the tracing becomes increasingly slower, even though the distance to be traced is kept constant. To demonstrate that the proposed model can
Discussion
To account for behavioral findings on contour tracing, McCormick and Jolicoeur, 1991, McCormick and Jolicoeur, 1994 developed a zoom lens model that captures many of its features. They have identified five component processes that a contour tracing operator should have: (1) a process that can determine whether there is only one contour within the receptive field, (2) a zoom process that can shrink or expand the size of the receptive field until an optimal size is reached such that only one
Acknowledgment
This work was supported by the University of Rijeka under Grant uniri-drustv-18-177.
References (112)
- et al.
Decoding successive computational stages of saliency processing
Current Biology
(2011) Two-dimensional spectral analysis of cortical receptive field profiles
Vision Research
(1980)- et al.
Using afterimages to test neural mechanisms for perceptual filling-in
Neural Networks
(2004) - et al.
Cortical dynamics of feature binding and reset: Control of visual persistence
Vision Research
(1994) A mechanism for cognitive dynamics: Neuronal communication through neuronal coherence
Trends in Cognitive Sciences
(2005)- et al.
Visual brain and visual perception: How does the cortex do perceptual grouping?
Trends in Neuroscience
(1997) - et al.
Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex
Vision Research
(2000) - et al.
A neural network architecture for figure-ground separation of connected scenic figures
Neural Networks
(1991) - et al.
A simple cell model with dominating opponent inhibition for robust image processing
Neural Networks
(2004) - et al.
Dendrites: Bug or feature?
Current Opinion in Neurobiology
(2003)
Winner-take-all networks for physiological models of competitive learning
Neural Networks
Sustained and transient mechanisms in human vision: temporal and spatial properties
Vision Research
Cortical gamma oscillations: The functional key is activation, not cognition
Neuroscience & Biobehavioral Reviews
A growth-cone model for the spread of object-based attention during contour grouping
Current Biology
A neural model of the temporal dynamics of figure-ground segregation in motion perception
Neural Networks
Do gamma oscillations play a role in cerebral cortex?
Trends in Cognitive Sciences
Activity-dependent regulation of synapses by retrograde messengers
Neuron
The spatial profile of visual attention in mental curve tracing
Vision Research
Synchrony unbound: A critical evaluation of the temporal binding hypothesis
Neuron
Neuronal synchrony: A versatile code for the definition of relations?
Neuron
Modeling visual attention via selective tuning
Artificial Intelligence
Visual routines
Cognition
A model of contextual interactions and contour detection in primary visual cortex
Neural Networks
Synaptic computation
Nature
Neurons in monkey visual area V2 encode combinations of orientations
Nature Neuroscience
Attention and mental primer
Mind & Language
Traditional and new principles of perceptual grouping
Reinforcement learning of linking and tracing contours in recurrent neural networks
PLoS Computational Biology
A backwards progression of attentional effects in the ventral stream
Proceedings of the National Academy of Sciences of the United States of America
Perceiving geometric patterns: From spirals to inside-outside relations
IEEE Transactions on Neural Networks
Object-based attention overrides perceptual load to modulate visual distraction
Journal of Experimental Psychology: Human Perception and Performance
Attractor dynamics of network UP states in the neocortex
Nature
A neural model of figure-ground organization
Journal of Neurophysiology
Attentional and automatic processes in line tracing: Is tracing obligatory?
Perception & Psychophysics
Does attention move or spread during mental curve tracing?
Perception & Psychophysics
Spatial attention is necessary for object-based attention: Evidence from temporal-order judgments
Attention, Perception, & Psychophysics
Object-based attention: Shifting or uncertainty?
Attention Perception, & Psychophysics
Neural mechanisms of visual feature binding investigated with microelectrodes and models
Visual Cognition
The logic of inhibitory connectivity in the neocortex
Neuroscientist
Attention gates visual coding in the human pulvinar
Nature Communications
Interactions of afterimages for orientation and color: Experimental data and model simulations
Perception & Psychophysics
A simple neural network exhibiting selective activation of neuronal ensembles: From winner-take-all to winners-share-all
Neural Computation
Brightness perception, illusory contours, and corticogeniculate feedback
Visual Neuroscience
Contour enhancement, short term memory, and constancies in reverberating neural networks
Studies in Applied Mathematics
3-D vision and figure-ground separation by visual cortex
Perception & Psychophysics
Neural dynamics of 1-D and 2-D brightness perception: A unified model of classical and recent phenomena
Perception & Psychophysics
Figure-ground separation of connected scenic figures: Boundaries, filling-in, and opponent processing
Maintenance of semantic information in capacity-limited item short-term memory
Psychonomic Bulletin & Review
The spatial distribution of attention within and across objects
Journal of Experimental Psychology: Human Perception and Performance
Parallel and serial grouping of image elements in visual perception
Journal of Experimental Psychology: Human Perception and Performance
Cited by (4)
An interactive cortical architecture for perceptual organization by accentuation
2024, Neural NetworksA multi-scale neurodynamic implementation of incremental grouping
2022, Vision ResearchCitation Excerpt :Computer simulations showed that the model’s behavior captures temporal dynamics of neuronal activity reported in the studies reviewed above. In this way, the model goes beyond previous attempts to simulate properties of incremental grouping and object-based attention (Brosch et al., 2015; Grossberg & Raizada, 2000; Marić & Domijan, 2019; Tsotsos & Kruijne, 2014). Fig. 1 depicts a multi-scale model of incremental grouping consisting of two networks: a contour detection network (CDN) and an L-junction detection network (LDN).
Picking up the Pieces: Sex Differences in Mechanisms of Curve Tracing
2021, Canadian Journal of Experimental Psychology