Surface versus edge-based determinants of visual recognition

doi:10.1016/0010-0285(88)90024-2

Cognitive Psychology

Volume 20, Issue 1, January 1988, Pages 38-64

https://doi.org/10.1016/0010-0285(88)90024-2 Get rights and content

Abstract

Two roles hypothesized for surface characteristics, such as color, brightness, and texture, in object recognition are that such information can (a) define the gradients needed for a $212$ -D sketch so that a 3-D representation can be derived (e.g., Marr & Nishihara, 1978) and (b) provide additional distinctive features for accessing memory. In a series of five experiments, subjects either named or verified (against a target name) brief (50–100 ms) presentations of slides of common objects. Each object was shown in two versions: professionally photographed in full color or as a simplified line drawing showing only the object's major components (which typically corresponded to its parts). Although one or the other type of picture would be slightly favored in a particular condition of exposure (duration or masking), overall mean reaction times and error rates were virtually identical for the two types of stimuli. These results support a view that edge-based representations mediate real-time object recognition in contrast to surface gradient or multiple cue representations. A previously unexplored distinction of color diagnosticity allowed us to determine whether color (and brightness) was employed as an additional feature in accessing memory for those objects or conditions where there might have been an advantage for the color slides. For some objects, e.g., banana, fork, fish, and camera, color is diagnostic as to the object's classification. For other objects, e.g., chair, pen, mitten, and bicycle pump, color is not diagnostic, as such objects can be of any color. If color was employed in accessing memory, color-diagnostic objects should have shown a relative advantage when presented as color slides compared to the line drawing versions of the same objects. Also, this advantage would be magnified when subjects could anticipate the color of an object in the verification task, particularly on NO trials when the foil was of a different color. Neither an overall advantage for color-diagnostic objects when presented in color nor a magnification of a relative advantage on the NO trials in the verification task was obtained. Although differences in surface characteristics such as color, brightness, and texture can be instrumental in defining edges and can provide cues for visual search, they play only a secondary role in the real-time recognition of an intact object when its edges can be readily extracted.

References (24)

T.O. Binford
Inferring surfaces from images
Artificial Intelligence
(1981)
P. Jolicoeur et al.
Picture and names: Making the connection
Cognitive Psychology
(1984)
E. Rosch et al.
Basic objects in natural categories
Cognitive Psychology
(1976)
A. Treisman et al.
A feature integration theory of attention
Cognitive Psychology
(1980)
S. Ullman
Visual routines
Cognition
(1984)
I. Biederman
Recognition-by-components: A theory of human image understanding
Psychological Review
(1987)
Biederman, I., Blickle, T. W., Teitelbaum, R. C., & Klatsky, G. J. Object search in nonscene displays. Journal of...
I. Biederman et al.
Processing redundant information
Journal of Experimental Psychology
(1970)
I. Biederman et al.
Recognition of objects that require texture specification
(1987)
J.S. Bruner
Going beyond the information given

J.S. Bruner et al.

On the perception of incongruity: A paradigm

Journal of Personality

(1949)

E.J. Gibson

Cited by (512)

Butterfly effects in perceptual development: A review of the ‘adaptive initial degradation’ hypothesis
2024, Developmental Review
Human perceptual development evolves in a stereotyped fashion, with initially limited perceptual capabilities maturing over the months or years following the commencement of sensory experience into robust proficiencies. This review focuses on the functional significance of these developmental progressions. Specifically, we review findings from studies of children who have experienced alterations of early development, as well as results from corresponding computational models, which have recently provided compelling evidence that specific attributes of early sensory experience are likely to be important prerequisites for later developing skills in several perceptual domains such as vision and audition. Notably, the limitations of early sensory experience have therein emerged as scaffolds, rather than hurdles, being causally responsible for the acquisition of later perceptual proficiencies, while dispensing with these limitations has the perhaps counter-intuitive consequence of compromising later development. These results have implications for understanding why normal trajectories of perceptual development are sequenced in the way that they are, help account for the perceptual deficits observed in individuals with atypical histories of sensory development, and serve as guidelines for the creation of more robust and effective training procedures for computational learning systems.
The roles of edge-based and surface-based information in the dynamic neural representation of objects
2023, NeuroImage
We combined multivariate pattern analysis (MVPA) and electroencephalogram (EEG) to investigate the role of edge, color, and other surface information in the neural representation of visual objects. Participants completed a one-back task in which they were presented with color photographs, grayscale images, and line drawings of animals, tools, and fruits. Our results provide the first neural evidence that line drawings elicit similar neural activities as color photographs and grayscale images during the 175–305 ms window after the stimulus onset. Furthermore, we found that other surface information, rather than color information, facilitates decoding accuracy in the early stages of object representations and affects the speed of this. These results provide new insights into the role of edge-based and surface-based information in the dynamic process of neural representations of visual objects.
On the importance of severely testing deep learning models of cognition
2023, Cognitive Systems Research
Researchers studying the correspondences between Deep Neural Networks (DNNs) and humans often give little consideration to severe testing when drawing conclusions from empirical findings, and this is impeding progress in building better models of minds. We first detail what we mean by severe testing and highlight how this is especially important when working with opaque models with many free parameters that may solve a given task in multiple different ways. Second, we provide multiple examples of researchers making strong claims regarding DNN-human similarities without engaging in severe testing of their hypotheses. Third, we consider why severe testing is undervalued. We provide evidence that part of the fault lies with the review process. There is now a widespread appreciation in many areas of science that a bias for publishing positive results (among other practices) is leading to a credibility crisis, but there seems less awareness of the problem here.
A novel feature-scrambling approach reveals the capacity of convolutional neural networks to learn spatial relations
2023, Neural Networks
Convolutional neural networks (CNNs) are one of the most successful computer vision systems to solve object recognition. Furthermore, CNNs have major applications in understanding the nature of visual representations in the human brain. Yet it remains poorly understood how CNNs actually make their decisions, what the nature of their internal representations is, and how their recognition strategies differ from humans. Specifically, there is a major debate about the question of whether CNNs primarily rely on surface regularities of objects, or whether they are capable of exploiting the spatial arrangement of features, similar to humans. Here, we develop a novel feature-scrambling approach to explicitly test whether CNNs use the spatial arrangement of features (i.e. object parts) to classify objects. We combine this approach with a systematic manipulation of effective receptive field sizes of CNNs as well as minimal recognizable configurations (MIRCs) analysis. In contrast to much previous literature, we provide evidence that CNNs are in fact capable of using relatively long-range spatial relationships for object classification. Moreover, the extent to which CNNs use spatial relationships depends heavily on the dataset, e.g. texture vs. sketch. In fact, CNNs even use different strategies for different classes within heterogeneous datasets (ImageNet), suggesting CNNs have a continuous spectrum of classification strategies. Finally, we show that CNNs learn the spatial arrangement of features only up to an intermediate level of granularity, which suggests that intermediate rather than global shape features provide the optimal trade-off between sensitivity and specificity in object classification. These results provide novel insights into the nature of CNN representations and the extent to which they rely on the spatial arrangement of features for object classification.
Stereopsis provides a constant feed to visual shape representation
2023, Vision Research
Citation Excerpt :
For instance, we can recognize 3D shapes only defined by shading (Ramachandran, 1988; Aubin & Arguin, 2014). Furthermore, line drawn objects and photographs of objects can be recognised easily (Biederman & Ju, 1988). It seems to be even possible to achieve shape constancy with shapes defined only by their edges (Biederman, 1987).
The contribution of stereopsis in human visual shape perception was examined using stimuli with either null, normal, or reversed binocular disparity in an old/new object recognition task. The highest levels of recognition performance were observed with null and normal binocular disparity displays, which did not differ. However, reversed disparity led to significantly worse performance than either of the other display conditions. This indicates that stereopsis provides a continuous input to the mechanisms involved in shape perception.
Does the brain's ventral visual pathway compute object shape?
2022, Trends in Cognitive Sciences
A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway – the primary substrate underlying object recognition – may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

View all citing articles on Scopus

^☆: This research was supported by research Grants F492083C0086 and 86-0106 from the Air Force Office of Scientific Research.

View full text

Surface versus edge-based determinants of visual recognition☆

Abstract

Artificial Intelligence

Cognitive Psychology

Cognitive Psychology

Cognitive Psychology

Cognition

Recognition-by-components: A theory of human image understanding

Psychological Review

Processing redundant information

Journal of Experimental Psychology

Recognition of objects that require texture specification

Going beyond the information given

On the perception of incongruity: A paradigm

Journal of Personality