Introduction

Ensemble perception is, essentially, the perception of groups of things. It has been referred to by various names over the years, including set perception, summary statistical perception, and sometimes even texture perception, among other terms. Ensemble perception is ubiquitous. The norm in natural scenes is to encounter groups or crowds of things, features, textures, or objects. Ensemble perception is therefore relevant throughout perception science. It has been reported in many arenas, not just within vision but across modalities. In particular, there has been rapid growth in the subfield of ensemble visual perception in the last few years. This special issue of AP&P reflects that growth, focusing on vision in particular.

At the August, 2019 European Conference on Visual Perception (ECVP), Solomon and Hochstein organized a symposium on “Ensemble Perception: Theory and Experiment.” Talks included: Five Dichotomies in the Psychophysics of Ensemble Perception (Joshua A. Solomon), Ensemble Perception and the Inference of the Invisible (David Whitney, Zhimin Chen, and Allie Yamanashi), Perceptual Averaging: How to Separate Inattention From Representational Noise? (Aire Raidvee and Jüri Allik), The Properties of Large Receptive Fields Can Explain Ensemble-Related Phenomena (Igor S. Utochkin), and The How and Why of Ensemble Perception (Shaul Hochstein). (See abstracts at Perception, 48(2S), 1–236, 2019). Following the symposium we approached the editors of Attention, Perception & Psychophysics to edit a special issue on Ensemble Processing.

Goals of the special issue

There have been several recent reviews on ensemble perception (Alvarez, 2011; Bauer, 2015; Cohen et al., 2016; Haberman & Whitney, 2012; Hochstein et al., 2015; Whitney & Yamanashi Leib, 2018). This introductory chapter and the special issue as a whole are therefore not intended as reviews, though there are theoretically oriented papers in the special issue that include synthesis across fields (e.g., Solomon, 2020). Instead, our goal in this special issue is to reflect on the exponential growth in the field, the directions that are being taken, and the connections that are made to other areas of vision science (e.g., Cui et al., 2021).

Organization of the special issue

The special issue is organized into seven broad groups of articles. The organization is inevitably fuzzy in virtue of the diversity and growth in the field, but it is also intentional: it is not meant to be rigid. Instead, it is just a rough guide to help the reader navigate between different topics and make connections between them.

  • Section 1: Low-level ensemble perception

The first section encompasses what can arguably be coined lower-level ensemble perception, including ensemble properties such as size (Haberman & Suresh, 2020; Allik et al., 2014; Ariely, 2001; Corbett & Oriet, 2011; Khayat & Hochstein, 2019; Morgan et al., 2008; Solomon, 2010), position (Alvarez & Oliva, 2008; Sun et al., 2021), color and contrast (Bauer, 2009; Khayat & Hochstein, 2019; Rajendran et al., 2020), etc. Of course, the term “low-level” is loaded, and this grouping is not intended to suggest (or exclude) any particular model of visual processing; it does not preclude feedback or necessitate strict feedforward hierarchical processing. For example, ensemble numerosity (Katzin et al., 2020; Lee et al., 2021; Melcher et al., 2020; Rosenbaum et al., 2021) might be an early representation or a late one (or both). Effects of size constancy (Haberman & Suresh, 2020; Tiurina & Utochkin, 2019) might well not be strictly low-level.

  • Section 2: Mid and high-level ensemble perception

The second section encompasses what might be considered mid- and high-level ensemble perception. For our purposes, this operational division could simply reflect visual information beyond features like color and position. This section encompasses the majority of the papers in this special issue, which is, perhaps, unsurprising because the long history of research on ensemble perception stretches back over 100 years and that literature largely focused on the perception of ensemble features (Bauer, 2015).

Much of the new work reported in this special issue explores higher level ensemble representations, the interactions among levels, and the underlying mechanisms. For example, ensemble perception of spatial properties may not require magnocellular or coarse spatial processing (Lee & Chong, 2020), but it does involve information drawn from multiple spatial scales (Sweeny et al., 2020).

A canonical example of high-level ensemble perception is the perception of face crowds, and there has been a great deal of work on ensemble face perception since Haberman’s initial demonstrations (Haberman & Whitney, 2007). In this special issue, several groups explored ensemble face perception. Han et al. (2020) found ensemble coding of holistic representations of faces and Ji and Hayward (2020) found that observers have metacognitive awareness of their explicit ensemble face percepts. Echoing some of the work on faces (e.g., Han et al., 2020), ensemble perception of biological motion speed (Nguyen et al., 2020; Sweeny et al., 2013) also suggests that configural or holistic information can be represented as an ensemble.

  • Section 3: Summary statistics beyond the average

Most of the work on summary statistical perception has been about the average of a group of things. However, ensemble perception occurs for statistical moments beyond just the “average.” Several papers in the special issue explore this issue, such as the effect of variance and outliers on ensemble representations (Avcı & Boduroglu, 2021; Semizer & Boduroglu, 2021; Allard, 2021; Sama et al., 2021) and the representation of variance and statistical distributions, per se (Cha et al., 2020; Im et al., 2020).

More abstract perceptual dimensions may also be represented as ensembles (Leib et al., 2016; Leib et al., 2020). For example, ensemble perception of numerosity or numerical values (Brezis et al., 2015; Katzin et al., 2020; Lee et al., 2021; Melcher et al., 2020; Rosenbaum et al., 2021; Solomon & Morgan, 2018), explicit ensemble judgments of abstract aesthetic impressions (Brielmann & Pelli, 2020), and implicit ensemble perception of category prototypes (Khayat & Hochstein, 2019) even with novel image sets that have never been seen before (Khayat et al., 2021). There is also research suggesting that high level cognition and cultural factors can impinge on ensemble perception (Peng et al., 2020).

  • Section 4: Temporal properties of ensemble perception and ensemble perception of temporal properties

The role of time in ensemble representations has been little explored in the past. This is one of the potentially fast-growing areas of ensemble perception work in the future. The papers in this section represent the forefront of the field. Melcher et al. (2020) report qualitatively different relationships between exposure duration and the accuracies of small and large numerosity estimates. Zhu et al. (2020) explicitly probe ensemble representations of duration and provide a model for how the quality of those representations depends on the statistics of the ensemble. Corbett et al. (2020) found negative aftereffects of adaptation to the average duration of sequentially presented sets of visual events, suggesting independent neural mechanisms that are selectively sensitive over a limited range of average durations.

  • Section 5: Attention-related effects

The role of attention in ensemble perception has been studied but remains hotly debated. The papers in this special issue explore a range of related issues. Several examine the role of saliency (Iakovlev & Utochkin, 2020), participant expertise (Sun & Gauthier, 2020) and practice-induced learning (Cha et al., 2020; Hochstein & Pavlovskaya, 2020) on ensemble perception. Another paper proposes a method for distinguishing noise from effects of inattention (Raidvee et al., 2021) when considering the efficiency of ensemble integration (Solomon, 2010; Solomon et al., 2011; Solomon et al., 2016). Additional papers explore ensemble representations within and outside the focus of attention (Chen, Zhuang, et al., 2020), during competing tasks like multiple object tracking (Alzahabi & Cain, 2021), and even with active ignoring (Rafiei et al., 2020).

  • Section 6: Theoretical contributions and reviews

A few papers in the special issue take a broader view. Solomon (2020) outlines a rubric for studying ensemble perception and guidelines for psychophysical explorations of it. Cui and Liu (2021) explore some promising connections between data visualization, ensemble perception, and education. Khayat et al. (2021) suggest that perceptual representations are initially coded in terms of an average computed from a set along with the differences of individual items from that mean (Benna & Fusi, 2019), such that under limited viewing (or memory) conditions only the mean (prototype or “ancestor”) survives.

  • Section 7: The relation between ensemble perception and memory

Although the field of visual memory is vast, there are increasingly apparent connections to ensemble representations because of their efficiency of representation and currency in perceptual experience. Papers by Papenmeier and Timm (2020) and Chen, Kocsis, et al. (2020) explore how visual working memory may be impacted by summary statistics (see Brady & Alvarez, 2011). A Paper by Zepp et al. (2021) tests ensembles in an iconic memory task, relating ensemble mean to central tendency recall (Dubé, 2019; Hollingworth, 1910).

What has the special issue not captured?

The most obvious limit of this special issue is that it only captures vision. However, ensemble and summary statistical perception occur in other modalities, including audition (e.g., Piazza et al., 2013) and probably including smell, taste, and somatosensation. Hopefully the next special issue will reflect this growth.

There are a range of specific debates that are not directly addressed in this special issue. For example, there has been a heated philosophical debate about ensembles as the currency of conscious experience, whether ensembles provide evidence for “consciousness overflow,” and other significant questions directly related to ensemble perception in the philosophy literature (e.g., Bayne & McClelland, 2019; Cohen et al., 2016; McClelland & Bayne, 2016; Openshaw & Weksler, 2020). These have been running essentially in parallel to the empirical work, and we have little representation for that debate in this special issue. This is unfortunate but also an unsurprising consequence of the empirical focus of both the research and the journal. A future special issue that more actively bridges the philosophy with the empirical work would be fruitful.

The diversity and growth in the field of ensemble perception in the last twenty years has been remarkable. We hope that this special issue reflects some of the promise it holds in helping to address questions in areas ranging from cross modal perception to consciousness, and from data visualization to memory.