Introduction

Given the limitation of human brain resources, attention mechanisms are essential to prioritise relevant information for ongoing cognitive processes. In addition to their function in filtering sensory data during perception, these mechanisms have also been shown to covertly shift attention within the contents of working memory (Baddeley, 2003; van Ede et al., 2017). Neuroimaging (for a review, see Gazzaley & Nobre, 2012) and behavioural studies (for a review, see Souza & Oberauer, 2016) showed that covert attention mechanisms contribute to strengthening working memory representations while granting us the capacity to dynamically prioritize any of them (Chun et al., 2011).

The pupillary response to light is a low-level reflex regulated by the parasympathetic neural pathway (Beatty & Lucero-Wagoner, 2000) and dedicated to the optimization of visual acuity under varying levels of luminance (Campbell & Gregory, 1960). A recent stream of studies demonstrated that variations in the pupil diameter to luminance can inform us about the covert attention mechanisms supporting visual cognition (for a review, see Mathôt & Van der Stigchel, 2015). They showed that pupil size is not only modulated by the actual luminance but also by the perceived luminance, hence the presence of a sun in the image background increases pupil constriction compared to an equiluminant image without a sun (Binda et al., 2013; Naber & Nakayama, 2013). Further studies showed that variations in pupil diameter can actually reveal covert attention shifts towards bright or dark areas during visual detection tasks although participants were fixating a portion of the screen where luminance was kept constant: covertly attending a bright area provoked a constriction compared to attending to a dark area (Mathôt et al., 2013, 2014; Unsworth & Robison, 2017). Other studies showed that the pupil may also react to the difference in luminance between objects no longer present on the screen but whose spatial location is maintained in working memory (Hustá et al., 2019; Zokaei et al., 2019). The theoretical implication is that the pupil light response anticipates the need to adapt to the luminance of the area where an object is likely to (re-)appear (Binda et al., 2014; Mathôt et al., 2014; Mathôt et al., 2017; Wang & Munoz, 2018) or to the luminance of the object itself (Zokaei et al., 2019), hypothetically to optimize the upcoming perception. However, it was suggested by Ebitz and Moore (2019) that the modulation of the pupil size is so small that it likely yields negligible improvement in visual acuity. It would thus be surprising that such a complex mechanism with so little impact on vision would have been selected and preserved by evolution. One possibility is that attentional modulation of the pupil light response is a vestigial competency that was selected as it conferred an important advantage on more ancestral nervous systems and maintained through evolution because it did not hamper vision or attentional mechanisms. While previous results highlight an intriguing relationship between pupil size and visual cognition, they all imply attentional shifts towards objects, or their memorized position, which were presented in the physical environment. Indeed, all these studies show that pupil size responds to the storing, manipulation or recalling of concrete items associated with a physical object or luminance area. But it remains unknown whether the interaction between pupil size response and attention extends to the cognitive processing of concepts, devoid of brightness information, with no physical counterpart in the environment. Such a finding would indicate that the pupil light response is modulated by attention even when it is not at the service of perception, challenging the functionalist hypothesis proposed to account for previous findings.

Number comparison provides a unique example of a cognitive process that is embodied in a spatial frame of reference, opening the door to the exploration of attention shifts on fully imaginary contents. Spatial-numerical associations (SNAs) suggest that numbers are mentally mapped in serial order on a spatial medium where small numbers are represented on the left side and large numbers on the right side (Dehaene, 1992; Hawes & Ansari, 2020; Hubbard et al., 2005). The classical method to measure covert attentional shifts elicited by number processing relies on the participants’ performance in a secondary task (e.g., detecting a lateralized target; Fischer et al., 2003). This method has low reliability as attested by several failed replications (e.g., Colling et al., 2020; Fattorini et al., 2015; van Dijck et al., 2014) and is subject to response biases not implying spatial attention (Andres et al., 2020; Galarraga et al., 2021; Gevers et al., 2006; Proctor & Cho, 2006). A more direct approach to SNAs has consisted of using an eye-tracker to record overt attention shifts during various numerical tasks (Hartmann et al., 2015; 2016; Loetscher et al., 2010; Masson et al., 2018; Myachykov et al., 2016; Salvaggio et al., 2019, 2022). The results converged to show that the serial order processing of numbers in working memory elicits eye movements along a left-to-right oriented axis. However, differences in eye position during the discrimination of small and large numbers hardly reached 1° of visual angle, which may be interpreted as a weak association between number comparison and spatial attention. The eye drift actually fails to reveal the full extent of covert attention shifts, as they only reflect the visible part of internal attention mechanisms. Popular theoretical proposals suggested that covert attention shifts are mandatorily the result of programming eye movements that are finally inhibited (Craighero et al., 2004; Rizzolatti et al., 1987). Accordingly, it was thought that the lateralized eye movements recorded during number processing were informative about the covert attentional shifts operating the mental numerical continuum. However, recent neuropsychological (Masson et al., 2020) and psychophysical (Hanning & Deubel, 2020; Hanning et al., 2019) studies showed that covert shifts of attention can be elicited without the ability to perform an eye movement. Because it is now proven that covert attention shifts rely on mechanisms that do not depend entirely on eye movements, it is essential for theoretical elaboration to get a direct measure of the covert attention shifts induced by abstract numerical concepts, which is not subsumed to gaze behaviour like in previous studies (Myachykov et al., 2016; Salvaggio et al., 2019).

The present study thus investigates whether covert shifts of attention induced by abstract concepts (i.e., numbers) can be traced, independently from overt attention shifts, at the level of pupil size adjustments to incoming light. The method consists of measuring the pupil diameter of participants performing an auditory number comparison task in front of a screen split into a dark and a bright area. Under strict control of central fixation on an area with constant luminance, variations of pupil diameter should reflect the side covertly attended by the participants as they classify numbers as smaller or larger than 45. According to the assumption that number comparison relies on a left-to-right oriented mental continuum, pupil dilation should be accentuated when a number smaller than the reference is processed in front of a left dark–right bright image, or when a number larger than the reference is processed in front of right dark–left bright image.

Method

Participants

Twenty French-speaking University students took part in this experiment (12 males and eight females; three left-handed; mean age and standard deviation, M = 22, SD = 1.8 years). They all had normal or corrected-by-lenses vision and did not report any antecedent of mathematical learning disability when asked by the experimenter. They were not aware of the hypotheses being tested. The experiment was conducted in accordance with the ethical standards established by the Declaration of Helsinki. Based on the results of a previous study with non-numerical stimuli, we estimated that a sample size of 20 participants is sufficient to achieve a 95% power to detect a true difference between the experimental conditions “Congruent to Dark” and “Congruent to Bright” (Cohen’s d = 0.8) with α = 0.05 using a one-tailed paired t-test (Mathôt et al., 2013).

Apparatus

The experiment was run on a PC equipped with a 22-in. LCD screen (1,920 × 1,080 pixels; refresh rate: 60 Hz). The participant wore Sennheiser PC8 USB headphones equipped with a microphone (Sennheiser, Wedemark, Germany). The software OpenSesame (v.3.2.8) controlled the stimulus presentation and the recording of the verbal response (Mathôt et al., 2012). An Eyelink 1000 desktop-mounted camera was used to track eye movements (SR Resarch, Mississauga, Canada; sampling rate: 1,000 Hz; average accuracy range: 0.25° angle to 0.5° angle; gaze tracking range of 32° angle horizontally and 25° angle vertically). Participants were placed 60 cm away from the screen. Prior to each experimental block, the eye tracker was calibrated to the screen using a built-in 9-point protocol. The analyses were made with MATLAB 2018a (MathWorks Inc., 2018).

Stimulus materials

The auditory stimuli consisted of French number words whose magnitude ranged from 21 to 69, excluding decade numbers (i.e., 30, 40, 50, 60) and the number ‘45’, which was used as an internalized fixed reference for the comparison task. The number words were recorded in a stereo audio file whose duration was adjusted to 750 ms. The screen background was divided in three parts of different colours (see Mathôt et al., 2013). The left and right parts of the screen were either white bright (105.3 cd/m2, 20° wide) or dark black (0.2 cd/m2, 20° wide), while the middle part was grey (26.5 cd/m2, 6.19° wide) and contained a central white fixation dot (diameter: 0.5°) (see Fig. 1). Each number word was presented twice, once with each screen configuration. The experiment contained 48 trials for each combination of magnitude and location of bright/dark parts on the screen, giving 192 trials presented in a random order within two 8-min blocks. Thus, the screen configuration was not fixed within a block and could change randomly from trial to trial.

Fig. 1
figure 1

Visual depiction of the display and the main conditions

Full size image

Task and procedure

Each trial started with a 1-pt eye-tracker recalibration, used for drift correction, consisting of a central white fixation dot on a grey background. Then, the split-screen display appeared, with the location of the bright and dark parts counterbalanced across the experiment. A number word was played for 750 ms in the headphones 2,000 ms after the display onset. Participants were instructed to look constantly at the central white fixation cross and to say aloud whether the number was “smaller” (“plus petit” in French) or “larger” (“plus grand”) in comparison to the fixed reference (‘45’). Pupil size response was recorded during 3,000 ms post number onset.

Data analysis

Erroneous trials were excluded from further analyses (0.3% of dataset). The spatial and temporal parameters of the eye movements and the pupil diameter were extracted using Eyelink® Data Viewer (SR Research Ltd., Mississauga, Ontario, Canada). Missing data resulting from eye blinks were linearly interpolated if they were shorter than 500 ms. We interpolated data in 1,872 out of 3,840 trials for a mean duration of 172 ms. The trials in which participants’ eye gaze deviated more than 2° from the fixation dot were removed (7.9%). The data trimming left an equivalent amount of trials per configuration of screen and magnitude (Congruent to Bright Small 8.1%; Congruent to Bright Large 9.9%; Congruent to Dark Small 6.1%; Congruent to Dark Large 7.8%; all comparisons p > . 127), ruling out the possibility that the difficulty of some trials might be accidentally associated more frequently with one of the two configurations. In the remaining data (92%), we downsampled the pupil signal (10 Hz) to create bins of 100 ms and used a high-pass filter (digital Chebyshev filter, 1/6 Hz cutoff frequency) to highlight trial-related changes in the diameter. The difference between Congruent to Bright (small numbers and bright left, large numbers and bright right) and Congruent to Dark (small numbers and dark left, large numbers and dark right) was tested in every bin using a paired t-test and clusters were constituted by grouping all bins that showed a significant difference (p < 0.05). The multiple comparison issue was addressed by means of a nonparametric random permutation procedure in which pupil size was permuted randomly between the conditions across all the bins of a given trial and compared statistically by paired t-tests (for a similar procedure, see Sahan et al., 2021; Salvaggio et al., 2019, 2022; Maris & Oostenveld, 2007). Significant clusters were isolated using the same criteria as those described for the analysis of the non-permuted data. Following a conservative criterion (Maris & Oostenveld, 2007), the largest detected cluster was selected, and all its t-values were summed. The same procedure was repeated 1,000 times and the sum of t-values obtained after each permutation was stored. The decision was taken by comparing the sum of the t-values obtained for each cluster revealed by the analysis of the non-permuted data to the distribution of the sums of the t-values derived from the random permutations. The probability of observing a significant difference between conditions in a cluster was inferred from the position of the sum in the normal distribution created by random permutations. The p-value was equal to 1 minus the percentile of this position. The test was significant if the p-value was inferior to α = 0.05.

Results

Response latencies were calculated from number onset. The average response latency was 1,167 ± 171 ms for small numbers and 1,144 ± 157 ms for large numbers, t(19) = 1.46, p = 0.161.

The comparison of the curves of pupil size between the condition “Congruent to Dark” and the condition “Congruent to Bright” showed a significant difference between 1,700 and 2,700 ms after number onset (p < 0.001), indicating that the pupil size was larger when number magnitude cued attention to the dark side (see Fig. 2). The effect was very consistent as 80% of the participants showed the effect in the predicted direction. This effect was only noticeable in a period of time situated after the participants had given their response. This might suggest that covert attentional shifts are in fact an epiphenomenon that is not related to the computation of the numbers. However, pupil size modulation is a very slow process that can be related to cognitive events occurring several hundred milliseconds before the absolute pupil size was modified (e.g., Mathôt, 2018; Mathôt et al., 2013). It is thus possible that covert attentional shifts were made before the response was given and are only noticeable in absolute pupil size after the response was given.

Fig. 2
figure 2

Filtered pupil size across time. On the vertical axis, an arbitrary unit depicts change of pupil aperture. The horizontal axis represents the time course of the trial, in milliseconds, from number onset. Pupil was significantly larger in trials congruent to the dark side from 1,700 to 2,700 ms post-onset (i.e., small numbers if dark was left or large numbers if dark was right). The framed area on the curves indicates the only significant cluster. The framed graph represents the mean effect separately for each participant within the period where the significant difference was observed. The data are ordered by the magnitude of the difference in change in pupil size, which was calculated by computing pupil size for numbers Congruent to Dark minus pupil size for numbers Congruent to Bright. Average response onset (± 1 SEM) is represented as a vertical bar

Full size image

To detect earlier trace of the covert attentional shifts associated with the number comparison process, we made an exploratory analysis on the pupil derivative (i.e., velocity). Larger pupil velocity reflects the moment at which the dilation of the pupil starts to accelerate in the Congruent to Bright condition in comparison to the Congruent to Dark condition. This period should precede the moment at which the absolute size of the pupil differed between the two conditions, offering a suitable method to detect the onset of the top-down modulation of pupil size. Accordingly, this complementary analysis revealed that the pupil velocity increased more steeply when the number is Congruent to Dark than when it is Congruent to Bright, between 1,000 ms and 1,200 ms after number onset (p = 0.003). This period begins between the audio presentation of the number, whose duration was 750 ms, and the average response time (Fig. 3).

Fig. 3
figure 3

Pupil derivative across time. On the vertical axis, a positive value indicates dilatation of the pupil size while a negative value indicates a constriction of the pupil size. The horizontal axis represents the time course of the trial, in milliseconds, from number onset. Pupil dilated significantly more in trials congruent to the dark side from 1,000 to 1,200 ms post-onset (i.e., small numbers if dark was left or large numbers if dark was right). The framed area on the curves indicates the only significant cluster. The framed graph represents the mean effect separately for each participant within the period where the significant difference was observed. The data are ordered by the magnitude of the difference in change in pupil size, which was calculated by computing the pupil derivative for numbers Congruent to Dark minus the pupil derivative for numbers Congruent to Bright. Average response onset (± 1 SEM) is represented as a vertical bar

Full size image

Finally, in order to exclude that our findings could be related to subtle differences in eye position, we performed a control analysis on the horizontal eye movements (100 Hz downsampled) induced by small and large numbers. Results did not reveal any difference in horizontal eye movements between the small and the large number condition (maximal average position from centre of screen: 0.19°, see Fig. 4).

Fig. 4
figure 4

Difference in horizontal eye movements between large and small number trials, across time. The figure is vertically scaled to represent only the middle grey part of the display, and degrees of visual angles depict gaze position. The horizontal axis represents the time course of the trial, in milliseconds, from number onset. No significant differences were observed between small and large numbers. Average response onset (± 1 SEM) is represented as a vertical bar

Full size image

Discussion

The present study investigated how low-level visual mechanisms, i.e. the pupil light response, react to the coding of number magnitude, a question that has mainly focused on the perception of non-symbolic numbers so far (i.e., dots on a screen; Castaldi et al., 2021). Here, we recorded pupil size to study the internal attention shifts emerging from the manipulation of number words presented auditorily. Participants had to compare numbers to a fixed reference while fixating the screen centre, and covert shifts of attention were quantified from the pupillary response to luminance differences between the left and right sides of the screen.

The innovative finding of this study comes from the pupillometry analysis that revealed that pupil dilation was accentuated when the dark half of the screen coincided with the assumed position of the number on a left-to-right spatial medium. Screen configuration was paired equally often with large and small numbers, or far and close numbers, ruling out the possibility that the pupil size increase revealed by the comparison between Congruent to Dark and Congruent to Bright trials might result from imbalanced difficulty due to an over-representation of large numbers in the former condition (Kahneman, 1973). Moreover, as the participants’ gaze and the luminance of the visual scene were maintained constant throughout the trials, the differences in pupil diameter can only be explained by covert shifts of attention elicited by the numerical comparison task. While the effect on the absolute pupil size was noticeable during a period situated after the participants gave their answer, complementary analysis on pupil velocity revealed that pupil dilation in the Congruent to Dark condition started to surpass pupil dilation in the Congruent to Bright condition already during number processing, before the response onset. This early contribution suggests that attention shifts help the number comparison process, and are not just epiphenomenal to response production. It also sheds light on the difficulties encountered by left or right neglect patients when they have to discriminate small or large numbers (e.g., Masson et al., 2016, 2017; Salillas et al., 2009; Vuilleumier et al., 2004; Zorzi et al., 2002, 2012).

Models of numerical cognition have long assumed that magnitude processing relied on covert attention shifts along a mental numerical continuum, but these models mostly developed on theoretical grounds (e.g., Chen & Verguts, 2010, 2012). Evidence from hemineglect studies is not decisive because these studies cannot fully exclude an association of non-functionally related symptoms (e.g., Doricchi et al., 2005; Storer & Demeyere, 2014; van Dijck et al., 2011). Obviously, it is not possible to explain why neglect patients would have difficulties to judge small and not large numbers, unless one assumes a congruent relationship with their inability to orient attention to the left side of space. Nevertheless, it is difficult to formally ascribe the numerical difficulties of neglect patients to covert attention as no distinction is made in these studies between covert and overt attention deficits, such as the gaze deviation opposite to the neglected side. Mental chronometry studies in healthy participants have often been presented as complementary and reinforcing evidence that number magnitude processing involves covert attention. However, these studies were subsequently questioned within alternative interpretative frameworks (Andres et al., 2020; Galarraga et al., 2021). First, the ability of classical target detection paradigms (e.g., Fischer et al., 2003; Galfano et al., 2006; Ristic et al., 2006; Schuller et al., 2015) to inform the issue of covert attention shifts during number processing has been debated since the publication of failure to replicate studies (e.g., Colling et al., 2020; Pellegrino et al., 2021). Second, these paradigms only provide indirect evidence for covert shifts, as the interference of number magnitude with spatial attention is inferred from the performance of participants in a non-numerical task (e.g., latency of responses to lateralized targets), opening the door to alternative explanations resting on stimulus–response associations that do not imply attention orientation. A valuable alternative to the attentional account is that congruency effects emerged from similarity in the coding of number magnitude and target location (Gevers et al., 2006). In the case of numerical processing, the polarity correspondence principle states that numerical stimuli (e.g., small vs. large numbers) and response side (e.g., left vs. right) get associated with negative or positive polarities that induce compatibility effects improving performance when they overlap (i.e., a small number and a left response would receive a negative polarity, a large number and a right response would receive a positive polarity; Proctor & Cho, 2006). While the polarity correspondence account and the attentional account may co-exist, there is good evidence that the latter is hindered in tasks involving explicit lateralized response (Andres et al., 2020; Galarraga et al., 2021). The same holds for the interaction between number position and target detection in simple dot-detection tasks: in principle, a congruency effect is likely to occur as soon as the structure of the task allows dimensional overlap between perceptual, verbal, or conceptual codes (Kornblum et al., 1990). Eye tracking allows the measurement of spontaneous eye movements, as a proxy for attention shifts during number magnitude processing, without necessarily requiring binary classification tasks or lateralized detection tasks. However, these attention shifts are assumed to operate on a mental numerical continuum, internally, which implies that they are only partially reflected in lateralized eye movement. Despite its importance for theoretical elaboration, evidence that number processing involves covert attention shifts has thus remained very debated. We succeeded in bringing the proof of the existence of covert attentional shifts for number comparison that cannot be explained by alternative polarity correspondence mechanisms, using a pupillometry method that is not subsumed to eye movements, thus complying strictly with the criteria of covert attention. We showed that the attention shifts indexed by the pupil light response were not related to eye movements, even subtle ones like microsaccades, tremors or drifts, because only the attended side of the screen – not the eye position – changed with number magnitude (see Fig. 4).

Our study also suggests that the attentional biases elicited by numbers are larger than what the aforementioned studies, measuring overt shifts of attention, revealed. Indeed, the gaze drift observed in previous studies never exceeded 1° amplitude, but this indirect eye-tracking method likely underestimated the amplitude of the shifts. Our pupillometry method outperforms previous eye-tracking methods by showing that attention can be shifted further away from the gaze position, covertly, by at least 3° of visual angle, though further experiments are needed to ensure that this estimate is not biased by the cortical magnification factor, which would predict pupil changes even for attention shifts that are smaller than 3°. Moreover, the effect size was larger in the present study (d = 0.866) than in our previous study where attention shifts were estimated from the eye drift (d = 0.69; Salvaggio et al., 2019). This adds up to previous evidence showing the superiority of the method to measure the wideness of the minimal size of the attentional window (Tkacz-Domb & Yeshurun, 2018; Yeshurun, 2019).

The finding of pupil size variations in response to fully imaginary contents is theoretically relevant for the understanding of the interaction between internal attention and vision. We showed a real-time crosstalk between internal attention and pupil size in response to abstract numerical concepts, devoid of brightness information, and with no physical counterpart in the external world. This finding contrasts with previous results because it shows that the pupil light response is modulated by attention even when it is not in the service of perception, challenging the functionalist view that pupil size adjusts to prepare to the upcoming perception (Mathôt, 2018; Wang & Munoz, 2018; Zokaei et al., 2019). Our results favor a mechanistic view highlighting a mandatory and automatic interaction between the two systems so that any stimulus that activates the attention system will inevitably influence pupil size – without implying an instrumental relation. At the brain level, this implies that the modulation of cortical activity by attention will be accompanied by a modulation of the cortical mechanism underlying the pupil light response whether it is instrumental or not for perception.

In conclusion, our results provide new evidence that the system responsible for variations in the pupil diameter operates in close interaction with the system responsible for attention orientation (Mathôt & Van der Stigchel, 2015; Mathôt et al., 2013). This evidence shows that the pupil size response is modulated by covert attention shifts triggered by the mental manipulation of internal numerical representations on a visuospatial medium. Because the mental manipulation of numbers did not aim to guide visual perception (i.e., there was no target to detect or remember), the top-down modulation of pupil size cannot be assigned to the optimization of visual acuity in preparation for an upcoming stimulus, as previous results suggested (Mathôt, 2018; Zokaei et al., 2019). Our results support a mechanistic process whereby any attention bias, whether it is instrumental or not for perception, leads to a congruent modulation of pupil size. This makes it possible to use the pupil diameter as an index of the covert shifts of attention elicited by numbers, opening new avenues for the study of cognitive abilities as abstract as mathematical thinking.