2.1 Human perception fluctuates between epochs of enhanced and reduced sensitivity to external information

We began by selecting 66 studies from the Confidence database [20] that investigated how human participants (N = 4,317) perform binary perceptual decisions (Fig 1A; see Methods for details on inclusion criteria). As a metric for perceptual performance (i.e., the sensitivity to external sensory information), we asked whether the participant’s response and the presented stimulus matched (stimulus-congruent choices) or differed from each other (stimulus-incongruent choices; Fig 1B and 1C) in a total of 21.05 million trials.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Concept.

(A) In binary perceptual decision-making, a participant is presented with stimuli from 2 categories (A vs. B; solid line) and reports consecutive perceptual choices via button presses (solid line). All panels below refer to these stimulated example data. (B) When the response matches the external stimulus information (i.e., overlap between the dotted and solid lines in panel (A)), perceptual choices are stimulus-congruent (red line). When the response matches the response at the preceding trial, perceptual choices are history-congruent (blue line). (C) The dynamic probabilities of stimulus- and history-congruence (i.e., computed in sliding windows of ±5 trials) fluctuate over time. (D) The mode of perceptual processing is derived by computing the difference between the dynamic probabilities of stimulus- and history-congruence. Values above 0% indicate a bias toward external information, whereas values below 0% indicate a bias toward internal information. (E) In computational modeling, internal mode is caused by an enhanced impact of perceptual history. This causes the posterior (black line) to be close to the prior (blue line). Conversely, during external mode, the posterior is close to the sensory information (log likelihood ratio, red line). (F) The bimodal inference model (M1) explains fluctuations between externally and internally biased modes (left panel) by 2 interacting factors: a normative accumulation of evidence according to parameter H (middle panel), and antiphase oscillations in the precision terms ω_LLR and ω_ψ (right panel). (G) The control models M2-M5 were constructed by successively removing the antiphase oscillations and the integration of information from the bimodal inference model. Please note that the normative-evidence-accumulation model (M4) corresponds to the model proposed by Glaze and colleagues [51]. In the no-evidence-accumulation model (M5), perceptual decisions depend only on likelihood information (flat priors).

https://doi.org/10.1371/journal.pbio.3002410.g001

In a first step, we asked whether the ability to accurately perceive sensory stimuli is constant over time or, alternatively, fluctuates in periods of enhanced and reduced sensitivity to external information. We found perception to be stimulus-congruent in 73.46% ± 0.15% of trials (mean ± standard error of the mean; Fig 2A), which was highly consistent across the selected studies (S1A Fig). In line with previous work [8], we found that the probability of stimulus-congruence was not independent across successive trials: At the group level, stimulus-congruent perceptual choices were significantly autocorrelated for up to 15 trials (Fig 2B), controlling for task difficulty and the sequence of presented stimuli (S2 Fig).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Internal and external modes in human perceptual decision-making.

(A) In humans, perception was stimulus-congruent in 73.46% ± 0.15% (in red) and history-congruent in 52.7% ± 0.12% of trials (in blue; upper panel). History-congruent perceptual choices were more frequent when perception was stimulus-incongruent (i.e., on error trials; lower panel), indicating that history effects impair performance in randomized psychophysical designs. (B) Relative to randomly permuted data, we found highly significant autocorrelations of stimulus-congruence and history-congruence (dots indicate intercepts ≠ 0 in trial-wise linear mixed effects modeling at p < 0.05). Across trials, the autocorrelation coefficients were best fit by an exponential function (adjusted R² for stimulus-congruence: 0.53; history-congruence: 0.72) as compared to a linear function (adjusted R² for stimulus-congruence: 0.53; history-congruence: 0.51), decaying at a rate of γ = −1.92×10⁻³ ± 4.5×10⁻⁴ (T(6.88×10⁴) = −4.27, p = 1.98×10⁻⁵) for stimulus-congruence and at a rate of γ = −6.11×10⁻³ ± 5.69×10⁻⁴ (T(6.75×10⁴) = −10.74, p = 7.18×10⁻²⁷) for history-congruence. (C) Here, we depict the number of consecutive trials at which autocorrelation coefficients exceeded the respective autocorrelation of randomly permuted data within individual participants. For stimulus-congruence (upper panel), the lag of positive autocorrelation amounted to 3.24 ± 2.39×10⁻³ on average, showing a peak at trial t+1 after the index trial. For history-congruence (lower panel), the lag of positive autocorrelation amounted to 4.87 ± 3.36×10⁻³ on average, peaking at trial t+2 after the index trial. (D) The smoothed probabilities of stimulus- and history-congruence (sliding windows of ±5 trials) fluctuated as a scale-invariant process with a 1/f power law, i.e., at power densities that were inversely proportional to the frequency. (E) The distribution of phase shift between fluctuations in stimulus- and history-congruence peaked at half a cycle (π denoted by dotted line). (F) The average squared coherence between fluctuations in stimulus- and history-congruence (dotted line) amounted to 6.49 ± 2.07×10⁻³%. (G) We observed faster RTs for both stimulus-congruence (as opposed to stimulus-incongruence, β = −0.14 ± 1.6×10⁻³, T(1.99×10⁶) = −85.84, p < 2.2×10⁻³⁰⁸) and history-congruence (β = −9.56×10⁻³ ± 1.37×10⁻³, T(1.98×10⁶) = −6.97, p = 3.15×10⁻¹²). (H) The mode of perceptual processing (i.e., the difference between the smoothed probability of stimulus- vs. history-congruence) showed a quadratic relationship to RTs, with faster RTs for stronger biases toward both external sensory information and internal predictions provided by perceptual history (β₂ = −19.86 ± 0.52, T(1.98×10⁶) = −38.43, p = 5×10⁻³²³). The horizontal and vertical dotted lines indicate maximum RT and the associated mode, respectively. (I) Confidence was enhanced for both stimulus-congruence (as opposed to stimulus-incongruence, β = 0.48 ± 1.38×10⁻³, T(2.06×10⁶) = 351.54, p < 2.2×10⁻³⁰⁸) and history-congruence (β = 0.04 ± 1.18×10⁻³, T(2.06×10⁶) = 36.85, p = 3.25×10⁻²⁹⁷). (J) In analogy to RTs, we found a quadratic relationship between the mode of perceptual processing and confidence, which increased when both externally and internally biased modes grew stronger (β₂ = 39.3 ± 0.94, T(2.06×10⁶) = 41.95, p < 2.2×10⁻³⁰⁸). The horizontal and vertical dotted lines indicate minimum confidence and the associated mode, respectively.

https://doi.org/10.1371/journal.pbio.3002410.g002

At the level of individual participants, the autocorrelation of stimulus-congruence exceeded the respective autocorrelation of randomly permuted data within an interval of 3.24 ± 2.39×10⁻³ trials (Fig 2C). In other words, if a participant’s experience was congruent (or incongruent) with the external stimulus information at a given trial, her perception was more likely to remain stimulus-congruent (or stimulus-incongruent) for approximately 3 trials into the future. The autocorrelation of stimulus-congruence was corroborated by logistic regression models that successfully predicted the stimulus-congruence of perception at the index trial t = 0 from the stimulus-congruence at the preceding trials within a lag of 16 trials (S3 Fig).

These results confirm that the ability to process sensory signals is not constant over time but unfolds in multitrial epochs of enhanced and reduced sensitivity to external information [8]. As a consequence of this autocorrelation, the dynamic probability of stimulus-congruent perception (i.e., computed in sliding windows of ±5 trials; Fig 1C) fluctuated considerably within participants (average minimum: 35.46% ± 0.22%, maximum: 98.27% ± 0.07%). In line with previous findings [9], such fluctuations in the sensitivity to external information had a power density that was inversely proportional to the frequency in the slow spectrum [11] (power ∼ 1/f^β, β = −1.32 ± 3.14×10⁻³, T(1.84×10⁵) = −419.48, p < 2.2×10⁻³⁰⁸; Fig 2D). This feature, which is also known as a 1/f power law [36,37], represents a characteristic of scale-free fluctuations in complex dynamic systems such as the brain [38] and the cognitive processes it entertains [9,10,13,39,40].

2.2 Humans fluctuate between external and internal modes of sensory processing

In a second step, we sought to explain why perception cycles through periods of enhanced and reduced sensitivity to external information [4]. We reasoned that observers may intermittently rely more strongly on internal information, i.e., on predictions about the environment that are constructed from previous experiences [19,31].

In perception, serial dependencies represent one of the most basic internal predictions that cause perceptual decisions to be systematically biased toward preceding choices [22–33]. Such effects of perceptual history mirror the continuity of the external world, in which the recent past often predicts the near future [30,31,34,35,41]. Therefore, as a metric for the perceptual impact of internal information, we computed whether the participant’s response at a given trial matched or differed from her response at the preceding trial (history-congruent and history-incongruent perception, respectively; Fig 1B and 1C).

First, we confirmed that perceptual history played a significant role in perception despite the ongoing stream of external information. With a global average of 52.7% ± 0.12% history-congruent trials, we found a small but highly significant perceptual bias towards preceding experiences (β = 16.18 ± 1.07, T(1.09×10³) = 15.07, p = 10⁻⁴⁶; Fig 2A) that was largely consistent across studies (S1B Fig) and more pronounced in participants who were less sensitive to external sensory information (S1C Fig). Importantly, history-congruence was not a corollary of the sequence of presented stimuli: History-congruent perceptual choices were more frequent at trials when perception was stimulus-incongruent (56.03% ± 0.2%) as opposed to stimulus-congruent (51.77% ± 0.11%, β = −4.26 ± 0.21, T(8.57×10³) = −20.36, p = 5.28×10⁻⁹⁰; Fig 2A, lower panel). Despite being adaptive in autocorrelated real-world environments [19,34,35,42], perceptual history thus represented a source of bias in the randomized experimental designs studied here [24,28,30,31,43]. These serial biases were effects of choice history, i.e., driven by the experiences reported at the preceding trial, and could not be attributed to stimulus history, i.e., to effects of the stimuli presented at the preceding trial (S1 Text).

Second, we asked whether perception cycles through multitrial epochs during which perception is characterized by stronger or weaker biases toward preceding experiences. In close analogy to stimulus-congruence, we found history-congruence to be significantly autocorrelated for up to 21 trials (Fig 2B), while controlling for task difficulty and the sequence of presented stimuli (S2 Fig). In individual participants, the autocorrelation of history-congruence was elevated above randomly permuted data for a lag of 4.87 ± 3.36×10⁻³ trials (Fig 2C), confirming that the autocorrelation of history-congruence was not only a group-level phenomenon. The autocorrelation of history-congruence was corroborated by logistic regression models that successfully predicted the history-congruence of perception at an index trial t = 0 from the history-congruence at the preceding trials within a lag of 17 trials (S3 Fig).

Third, we asked whether the impact of internal information fluctuates as a scale-invariant process with a 1/f power law (i.e., the feature typically associated with fluctuations in the sensitivity to external information [9,10,13,39,40]). The dynamic probability of history-congruent perception (i.e., computed in sliding windows of ±5 trials; Fig 1C) varied considerably over time, ranging between a minimum of 12.77% ± 0.14% and a maximum 92.23% ± 0.14%. In analogy to stimulus-congruence, we found that history-congruence fluctuated at power densities that were inversely proportional to the frequency in the slow spectrum [11] (power ∼ 1/f^β, β = −1.34 ± 3.16×10⁻³, T(1.84×10⁵) = −423.91, p < 2.2×10⁻³⁰⁸; Fig 2D).

Finally, we ensured that fluctuations in stimulus- and history-congruence are linked to each other. When perceptual choices were less biased toward external information, participants relied more strongly on internal information acquired from perceptual history (and vice versa, β = −0.05 ± 5.63×10⁻⁴, T(2.1×10⁶) = −84.21, p < 2.2×10⁻³⁰⁸, controlling for fluctuations in general response biases; S1 Text). Thus, while sharing the 1/f power law characteristic, fluctuations in stimulus- and history-congruence were shifted against each other by approximately half a cycle and showed a squared coherence of 6.49 ± 2.07×10⁻³% (Fig 2E and 2F; we report the average phase and coherence for frequencies below 0.1 1/N_trials; see Methods for details).

In sum, our analyses indicate that perceptual decisions result from a competition of external sensory signals with internal predictions provided by perceptual history. We show that the impact of these external and internal sources of information is not stable over time but fluctuates systematically, emitting overlapping autocorrelation curves and antiphase 1/f profiles.

These links between stimulus- and history-congruence suggest that the fluctuations in the impact of external and internal information are generated by a unifying mechanism that causes perception to alternate between 2 opposing modes [18] (Fig 1D): During external mode, perception is more strongly driven by the available external stimulus information. Conversely, during internal mode, participants rely more heavily on internal predictions that are implicitly provided by preceding perceptual experiences. The fluctuations in the degree of bias toward external versus internal information created by such bimodal inference may thus provide a novel explanation for ongoing fluctuations in the sensitivity to external information [4,5,18].

2.3 Internal and external modes of processing facilitate response behavior and enhance confidence in human perceptual decision-making

The above results point to systematic fluctuations in the decision variable [44] that determines perceptual choices, causing enhanced sensitivity to external stimulus information during external mode and increased biases toward preceding choices during internal mode. As such, fluctuations in mode should influence downstream aspects of behavior and cognition that operate on the perceptual decision variable [44]. To test this hypothesis with respect to motor behavior and metacognition, we asked how bimodal inference relates to response times (RTs) and confidence reports.

With respect to RTs, we observed faster responses for stimulus-congruent as opposed to stimulus-incongruent choices (β = −0.14 ± 1.6×10⁻³, T(1.99×10⁶) = −85.84, p < 2.2×10⁻³⁰⁸; Fig 2G). Intriguingly, while controlling for the effect of stimulus-congruence, we found that history-congruent (as opposed to history-incongruent) choices were also characterized by faster responses (β = −9.56×10⁻³ ± 1.37×10⁻³, T(1.98×10⁶) = −6.97, p = 3.15×10⁻¹²; Fig 2G).

When analyzing the speed of response against the mode of sensory processing (Fig 2H), we found that RTs were shorter during externally oriented perception (β₁ = −11.07 ± 0.55, T(1.98×10⁶) = −20.14, p = 3.17×10⁻⁹⁰). Crucially, as indicated by a quadratic relationship between the mode of sensory processing and RTs (β₂ = −19.86 ± 0.52, T(1.98×10⁶) = −38.43, p = 5×10⁻³²³), participants became faster at indicating their perceptual decision when biases toward both internal and external mode grew stronger.

In analogy to the speed of response, confidence was higher for stimulus-congruent as opposed to stimulus-incongruent choices (β = 0.04 ± 1.18×10⁻³, T(2.06×10⁶) = 36.85, p = 3.25×10⁻²⁹⁷; Fig 2I). Yet, while controlling for the effect of stimulus-congruence, we found that history-congruence also increased confidence (β = 0.48 ± 1.38×10⁻³, T(2.06×10⁶) = 351.54, p < 2.2×10⁻³⁰⁸; Fig 2I).

When depicted against the mode of sensory processing (Fig 2J), subjective confidence was enhanced when perception was more externally oriented (β₁ = 92.63 ± 1, T(2.06×10⁶) = 92.89, p < 2.2×10⁻³⁰⁸). Importantly, however, participants were more confident in their perceptual decision for stronger biases toward both internal and external mode (β₂ = 39.3 ± 0.94, T(2.06×10⁶) = 41.95, p < 2.2×10⁻³⁰⁸). In analogy to RTs, subjective confidence thus showed a quadratic relationship to the mode of sensory processing (Fig 2J).

Consequently, our findings predict that human participants lack full metacognitive insight into how strongly external signals and internal predictions contribute to perceptual decision-making. Stronger biases toward perceptual history thus lead to 2 seemingly contradictory effects, more frequent errors (S1C Fig) and increasing subjective confidence (Fig 2I and 2J). This observation generates an intriguing prediction regarding the association of between-mode fluctuations and perceptual metacognition: Metacognitive efficiency should be lower in individuals who spend more time in internal mode, since their confidence reports are less predictive of whether the corresponding perceptual decision is correct. We computed each participant’s M-ratio [45] (meta-d′/d′ = 0.85 ± 0.02) to probe this hypothesis independently of interindividual differences in perceptual performance. Indeed, we found that biases toward internal information (as defined by the average probability of history-congruence) were stronger in participants with lower metacognitive efficiency (β = −2.98×10⁻³ ± 9.82×10⁻⁴, T(4.14×10³) = −3.03, p = 2.43×10⁻³).

In sum, the above results indicate that reporting behavior and metacognition do not map linearly onto the mode of sensory processing. Rather, they suggest that slow fluctuations in the respective impact of external and internal information are most likely to affect perception at an early level of sensory analysis [46,47]. Such low-level processing may thus integrate perceptual history with external inputs into a decision variable [44] that influences not only perceptual choices but also the speed and confidence at which they are made.

In what follows, we probe alternative explanations for between-mode fluctuations, test for the existence of modes in mice, and propose a predictive processing model that explains fluctuations in mode by ongoing shifts in the precision afforded to external sensory information relative to internal predictions driven by perceptual history.

2.4 Fluctuations between internal and external mode cannot be reduced to general response biases or random choices

The core assumption of bimodal inference—that ongoing changes in the sensitivity to external information are driven by internal predictions induced via perceptual history—needs to be contrasted against 2 alternative hypotheses: When making errors, observers may not engage with the task and respond stereotypically, i.e., exhibit stronger general biases toward one of the 2 potential outcomes or simply choose randomly.

Logistic regression confirmed that perceptual history made a significant contribution to perception (β = 0.11 ± 5.79×10⁻³, z = 18.53, p = 1.1×10⁻⁷⁶) over and above the ongoing stream of external sensory information (β = 2.2 ± 5.87×10⁻³, z = 375.11, p < 2.2×10⁻³⁰⁸) and general response biases toward one of the two possible outcomes (β = 15.19 ± 0.08, z = 184.98, p < 2.2×10⁻³⁰⁸).

When eliminating perceptual history as a predictor of individual choices, Akaike information criterion (AIC; [48]) increased by δ_AIC = 1.64×10³ (see S4 Fig for parameter- and model-level inference at the level of individual observers). Likewise, when eliminating slow fluctuations in history-congruence as a predictor of slow fluctuations in stimulus-congruence across trials, we observed an increase in AIC by δ_AIC = 7.06×10³. These results provided model-level evidence against the null hypotheses that fluctuations in stimulus-congruence are driven exclusively by choice randomness or general response bias (see S1 Text and S5 Fig for an in-depth assessment of general response bias).

To confirm that changes in the sensitivity to external information are indicative of internal mode processing, we estimated full and history-dependent psychometric curves during internal, external, and across modes [21]. If, as we hypothesized, internal mode processing reflects an enhanced impact of perceptual history, one would expect a history-dependent increase in biases and lapses as well as a history-independent increase in threshold. Conversely, if internal mode processing were driven by random choices, one would expect a history-independent increase in lapses and threshold and no change in bias. In line with our prediction, we found that internal mode processing was associated with a history-dependent increase in bias and lapse as well as a history-independent increase in threshold (S1 Text and S6 Fig). This confirmed that internal mode processing is indeed driven by an enhanced impact of perceptual history.

In line with this, the quadratic relationship between mode and confidence (Fig 2J) suggested that biases toward internal information do not reflect a postperceptual strategy of repeating preceding choices when the subjective confidence in the perceptual decision is low. Moreover, while responses became faster with longer exposure to the experiments of the Confidence database, the frequency of history-congruent choices increased over time, speaking against the proposition that participants stereotypically repeat preceding choices when not yet familiar with the experimental task (S1 Text).

Taken together, our results argue against recurring intervals of low task engagement, which may be signaled by stereotypical or random responses, as an alternative explanation for the phenomenon that we identify as bimodal inference.

2.5 Mice fluctuate between external and internal modes of sensory processing

In a prominent functional explanation for serial dependencies [22–28,32,33,46], perceptual history is cast as an internal prediction that leverages the temporal autocorrelation of natural environments for efficient decision-making [30,31,34,35,41]. Since this autocorrelation is one of the most basic features of our sensory world, fluctuating biases toward preceding perceptual choices should not be a uniquely human phenomenon.

To test whether externally and internally oriented modes of processing exist beyond the human mind, we analyzed data on perceptual decision-making in mice that were extracted from the International Brain Laboratory (IBL) dataset [21]. We restricted our analyses to the basic task [21], in which mice responded to gratings of varying contrast that appeared either in the left or right hemifield with equal probability. We excluded sessions in which mice did not respond correctly to stimuli presented at a contrast above 50% in more than 80% of trials (see Methods for details), which yielded a final sample of N = 165 adequately trained mice that went through 1.46 million trials.

We found perception to be stimulus-congruent in 81.37% ± 0.3% of trials (Fig 3A, upper panel). In line with humans, mice were biased toward perceptual history in 54.03% ± 0.17% of trials (T(164) = 23.65, p = 9.98×10⁻⁵⁵; Figs 3A and S1D). Since the basic task of the IBL dataset presented stimuli at random in either the left or the right hemifield, we expected stronger biases toward perceptual history to decrease perceptual performance. Indeed, history-congruent choices were more frequent when perception was stimulus-incongruent (61.59% ± 0.07%) as opposed to stimulus-congruent (51.81% ± 0.02%, T(164) = 31.37, p = 3.36×10⁻⁷¹; T(164) = 31.37, p = 3.36×10⁻⁷¹; Fig 3A, lower panel), confirming that perceptual history was a source of bias [24,28,30,31,43] as opposed to a feature of the experimental paradigm.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Internal and external modes in mouse perceptual decision-making.

(A) In mice, 81.37% ± 0.3% of trials were stimulus-congruent (in red) and 54.03% ± 0.17% of trials were history-congruent (in blue; upper panel). History-congruent perceptual choices were not a consequence of the experimental design, but a source of error, as they were more frequent on stimulus-incongruent trials (lower panel). (B) Relative to randomly permuted data, we found highly significant autocorrelations of stimulus-congruence and history-congruence (dots indicate intercepts ≠ 0 in trial-wise linear mixed effects modeling at p < 0.05). Please note that the negative autocorrelation of stimulus-congruence at trial 2 was a consequence of the experimental design (S2 Fig). As in humans, autocorrelation coefficients were best fit by an exponential function (adjusted R² for stimulus-congruence: 0.44; history-congruence: 0.52) as compared to a linear function (adjusted R² for stimulus-congruence: 3.16×10⁻³; history-congruence: 0.26), decaying at a rate of γ = −6.2×10⁻⁴ ± 5.93×10⁻⁴ (T(3.55×10⁴) = −1.05, p = 0.3) for stimulus-congruence and at a rate of γ = −6.7×10⁻³ ± 5.94×10⁻⁴ (T(3.69×10⁴) = −11.27, p = 2.07×10⁻²⁹) for history-congruence. (C) For stimulus-congruence (upper panel), the lag of positive autocorrelation was longer in comparison to humans (4.59 ± 0.06 on average). For history-congruence (lower panel), the lag of positive autocorrelation was slightly shorter relative to humans (2.58 ± 0.01 on average, peaking at trial t+2 after the index trial). (D) In mice, the dynamic probabilities of stimulus- and history-congruence (sliding windows of ±5 trials) fluctuated as a scale-invariant process with a 1/f power law. (E) The distribution of phase shift between fluctuations in stimulus- and history-congruence peaked at half a cycle (π denoted by dotted line). (F) The average squared coherence between fluctuations in stimulus- and history-congruence (dotted line) amounted to 3.45 ± 0.01%. (G) We observed shorter trial durations (TDs) for stimulus-congruence (as opposed to stimulus-incongruence, β = −1.12 ± 8.53×10⁻³, T(1.34×10⁶) = −131.78, p < 2.2×10⁻³⁰⁸), but longer TDs for history-congruence (β = 0.06 ± 6.76×10⁻³, T(1.34×10⁶) = 8.52, p = 1.58×10⁻¹⁷). (H) TDs decreased monotonically for stronger biases toward external mode (β₁ = −4.16×10⁴ ± 1.29×10³, T(1.35×10⁶) = −32.31, p = 6.03×10⁻²²⁹). The horizontal and vertical dotted lines indicate maximum TD and the associated mode, respectively. (I) For TDs that differed from the median TD by no more than 1.5 × MAD (median absolute distance; [49]), mice exhibited a quadratic component in the relationship between the mode of sensory processing and TDs (β₂ = −1.97×10³ ± 843.74, T(1.19×10⁶) = −2.34, p = 0.02). This explorative post hoc analysis focuses on trials at which mice engage more swiftly with the experimental task. The horizontal and vertical dotted lines indicate maximum TD and the associated mode, respectively.

https://doi.org/10.1371/journal.pbio.3002410.g003

At the group level, we found significant autocorrelations in both stimulus-congruence (42 consecutive trials) and history-congruence (8 consecutive trials; Fig 3B), while controlling for the respective autocorrelation of task difficulty and external stimulation (S2 Fig). In contrast to humans, mice showed a negative autocorrelation coefficient of stimulus-congruence at trial 2, which was due to a feature of the experimental design: Errors at a contrast above 50% were followed by a high-contrast stimulus at the same location. Thus, stimulus-incongruent choices on easy trials were more likely to be followed by stimulus-congruent perceptual choices that were facilitated by high-contrast visual stimuli [21].

At the level of individual mice, autocorrelation coefficients were elevated above randomly permuted data within a lag of 4.59 ± 0.06 trials for stimulus-congruence and 2.58 ± 0.01 trials for history-congruence (Fig 3C). We corroborated these autocorrelations in logistic regression models that successfully predicted the stimulus-/history-congruence of perception at the index trial t = 0 from the stimulus-/history-congruence at the 33 preceding trials for stimulus-congruence and 8 preceding trials for history-congruence (S3 Fig). In analogy to humans, mice showed antiphase 1/f fluctuations in the sensitivity to internal and external information (Fig 3D–3F).

The above results confirm that fluctuations between internally and externally biased modes generalize to perceptual decision-making in mice. Following our hypothesis that bimodal inference operates at the level of perception, we predicted that between-mode fluctuations modulate a decision variable [44] that determines not only perceptual choices but also downstream aspects of mouse behavior [44]. We therefore asked how external and internal modes relate to the trial duration (TD, a coarse measure of RT in mice that spans the interval from stimulus onset to feedback; [21]). Stimulus-congruent (as opposed to stimulus-incongruent) choices were associated with shorter TDs (δ = −262.48 ± 17.1, T(164) = -15.35, p = 1.55×10⁻³³), while history-congruent choices were characterized by longer TDs (δ = 30.47 ± 5.57, T(164) = 5.47, p = 1.66×10⁻⁷; Fig 3G).

Across the full spectrum of the available data, TDs showed a linear relationship with the mode of sensory processing, with shorter TDs during external mode (β₁ = −4.16×10⁴ ± 1.29×10³, T(1.35×10⁶) = −32.31, p = 6.03×10⁻²²⁹, Fig 3H). However, an explorative post hoc analysis limited to TDs that differed from the median TD by no more than 1.5 × MAD (median absolute distance; [49]) indicated that, when mice engaged with the task more swiftly, TDs did indeed show a quadratic relationship with the mode of sensory processing (β₂ = −1.97×10³ ± 843.74, T(1.19×10⁶) = −2.34, p = 0.02, Fig 3I).

As in humans, it is important to ensure that ongoing changes in the sensitivity to external information are driven by perceptual history and cannot be reduced to general choice biases or random behavior. Logistic regression confirmed a significant effect of perceptual history on perceptual choices (β = 0.51 ± 4.49×10⁻³, z = 112.84, p < 2.2×10⁻³⁰⁸), while controlling for external sensory information (β = 2.96 ± 4.58×10⁻³, z = 646.1, p < 2.2×10⁻³⁰⁸) and general response biases toward one of the 2 outcomes (β = −1.78 ± 0.02, z = −80.64, p < 2.2×10⁻³⁰⁸). When eliminating perceptual history as a predictor of individual choices, AIC increased by δ_AIC = 1.48×10⁴, arguing against the notion that choice randomness and general response bias are the only determinants of perceptual performance in mice (see S4 Fig for parameter- and model-level inference in individual subjects).

In mice, fluctuations in the strength of history-congruent biases had a significant effect on stimulus-congruence (β₁ = −0.12 ± 7.17×10⁻⁴, T(1.34×10⁶) = −168.39, p < 2.2×10⁻³⁰⁸) beyond the effect of ongoing changes in general response biases (β₂ = −0.03 ± 6.94×10⁻⁴, T(1.34×10⁶) = −48.14, p < 2.2×10⁻³⁰⁸). Eliminating the dynamic fluctuations in history-congruence as a predictor of fluctuations in stimulus-congruence resulted in an increase in AIC by δ_AIC = 2.8×10⁴ (see S1 Text and S5 Fig for an in-depth assessment of general response bias).

When fitting full and history-conditioned psychometric curves to the IBL data [21], we observed that internal mode processing was associated with a history-dependent increase in bias and lapse as well as a history-independent increase in threshold (S1 Text and S7 Fig). Over time, the frequency of history-congruent choices increased alongside stimulus-congruence and speed of response as mice were exposed to the experiment, arguing against the proposition that biases toward perceptual history reflected an unspecific response strategy in mice who were not sufficiently trained on the IBL task (S1 Text and S8 Fig).

In sum, these analyses confirmed that the observed fluctuations in sensitivity to external sensory information are driven by dynamic changes in the impact of perceptual history and cannot be reduced to general response bias and random choice behavior.

2.6 Fluctuations in mode result from coordinated changes in the impact of external and internal information on perception

The empirical data presented above indicate that, for both humans and mice, perception fluctuates between external and modes, i.e., multitrial epochs that are characterized by enhanced sensitivity toward either external sensory information or internal predictions generated by perceptual history. Since natural environments typically show high temporal redundancy [34], previous experiences are often good predictors of new stimuli [30,31,35,41]. Serial dependencies may therefore induce autocorrelations in perception by serving as internal predictions (or memory processes; [9,13]) that actively integrate noisy sensory information over time [50].

Previous work has shown that such internal predictions can be built by dynamically updating the estimated probability of being in a particular perceptual state from the sequence of preceding experiences [35,46,51]. The integration of sequential inputs may lead to accumulating effects of perceptual history that progressively override incoming sensory information, enabling internal mode processing [19]. However, since such a process would lead to internal biases that may eventually become impossible to overcome [52], changes in mode may require ongoing wave-like fluctuations [9,13] in the perceptual impact of external and internal information that occur irrespective of the sequence of previous experiences and temporarily decouple the decision variable from implicit internal representations of the environment [19].

Following Bayes’ theorem, binary perceptual decisions depend on the log posterior ratio L of the 2 alternative states of the environment that participants learn about via noisy sensory information [51]. We computed the posterior by combining the sensory evidence available at time point t (i.e., the log likelihood ratio LLR) with the prior probability ψ, weighted by the respective precision terms ω_LLR and ω_ψ: (1)

We derived the prior probability ψ at time point t from the posterior probability of perceptual outcomes at time point L_t−1. Since a switch between the 2 states can occur at any time, the effect of perceptual history varies according to both the sequence of preceding experiences and the estimated stability of the external environment (i.e., the hazard rate H; [51]): (2)

The LLR was computed from inputs s_t by applying a sigmoid function defined by parameter α that controls the sensitivity of perception to the available sensory information (see Methods for details on s_t in humans and mice): (3) (4)

To allow for bimodal inference, i.e., alternating periods of internally and externally biased modes of perceptual processing that occur irrespective of the sequence of preceding experiences, we assumed that likelihood and prior vary in their influence on the perceptual decision according to fluctuations governed by ω_LLR and ω_ψ. These antiphase sine functions (defined by amplitudes a_LLR/ψ, frequency f, and phase p) determine the precision afforded to the likelihood and prior [53]. The implicit antiphase fluctuations are mandated by Bayes-optimal formulations in which inference depends only on the relative values of prior and likelihood precision (i.e., the Kalman gain; [54]). As such, ω_LLR and ω_ψ implement a hyperprior [55] in which the likelihood and prior precisions are shifted against each other at a dominant timescale defined by f: (5) (6)

Finally, a sigmoid transform of the posterior L_t yields the probability of observing the perceptual decision y_t at a temperature determined by ζ⁻¹: (7)

We used a maximum likelihood procedure to fit the bimodal inference model (M1; Fig 1F) to the behavioral data from the Confidence database [20] and the IBL database [21], optimizing the parameters α, H, amp_LLR, amp_ψ, f, p, and ζ (see Methods for details and S2 Table for a summary of the parameters of the bimodal inference model). We validated our model in 3 steps:

First, to show that bimodal inference does not emerge spontaneously in normative Bayesian models of evidence accumulation but requires the ad hoc addition of antiphase oscillations in prior and likelihood precision, we compared the bimodal inference model to 4 control models (M2 to M5; Fig 1G). In these models, we successively removed the antiphase oscillations (M2 to M4) and the integration of information across trials (M5) from the bimodal inference model and performed a model comparison based on AIC.

Model M2 (AIC₂ = 9.76×10⁴ in humans and 4.91×10⁴ in mice) and Model M3 (AIC₃ = 1.19×10⁵ in humans and 5.95×10⁴ in mice) incorporated only oscillations of either likelihood or prior precision. Model M4 (AIC₄ = 1.69×10⁵ in humans and 9.12×10⁴ in mice) lacked any oscillations of likelihood and prior precision and corresponded to the normative model proposed by Glaze and colleagues [51]. In model M5 (AIC₄ = 2.01×10⁵ in humans and 1.13×10⁵ in mice), we furthermore removed the integration of information across trials, such that perception depended only in incoming sensory information (Fig 1G).

The bimodal inference model achieved the lowest AIC across the full model space (AIC₁ = 8.16×10⁴ in humans and 4.24×10⁴ in mice) and was clearly superior to the normative Bayesian model of evidence accumulation (δ_AIC = −8.79×10⁴ in humans and −4.87×10⁴ in mice; S9 Fig).

As a second validation of the bimodal inference model, we tested whether the posterior model predicted within-training and out-of-training variables. The bimodal inference model characterizes each subject by a sensitivity parameter α (humans: α = 0.5 ± 1.12×10⁻⁴; mice: α = 1.06 ± 2.88×10⁻³) that captures how strongly perception is driven by the available sensory information, and a hazard rate parameter H (humans: H = 0.45 ± 4.8×10⁻⁵; mice: H = 0.46 ± 2.97×10⁻⁴) that controls how heavily perception is biased by perceptual history. The parameter f captures the dominant timescale at which likelihood (amplitude humans: a_LLR = 0.5 ± 2.02×10⁻⁴; mice: a_LLR = 0.39 ± 1.08×10⁻³) and prior precision (amplitude humans: a_ψ = 1.44 ± 5.27×10⁻⁴; mice: a_ψ = 1.71 ± 7.15×10⁻³) fluctuated and was estimated at 0.11 ± 1.68×10⁻⁵ 1/N_trials and 0.11 ± 1.63×10⁻⁴ 1/N_trials in mice.

As a sanity check for model fit, we tested whether the frequency of stimulus- and history-congruent trials in the Confidence database [20] and IBL database [21] correlated with the estimated parameters α and H, respectively. As expected, the estimated sensitivity toward stimulus information α was positively correlated with the frequency of stimulus-congruent perceptual choices (humans: β = 0.84 ± 0.26, T(4.31×10³) = 32.87, p = 1.3×10⁻²¹¹; mice: β = 1.93 ± 0.12, T(2.07×10³) = 16.21, p = 9.37×10⁻⁵⁶). Likewise, H was negatively correlated with the frequency of history-congruent perceptual choices (humans: β = −11.84 ± 0.5, T(4.29×10³) = −23.5, p = 5.16×10⁻¹¹⁵; mice: β = −6.18 ± 0.66, T(2.08×10³) = −9.37, p = 1.85×10⁻²⁰).

Our behavioral analyses reveal that humans and mice show significant effects of perceptual history that impaired performance in randomized psychophysical experiments [24,28,30,31,43] (Figs 2A and 3A). We therefore expected that humans and mice underestimated the true hazard rate of the experimental environments (Confidence database [20]: ± 1.58×10⁻⁵); IBL database [21]: ± 6.48×10⁻⁵). Indeed, when fitting the bimodal inference model to the trial-wise perceptual choices, we found that the estimated (i.e., subjective) hazard rate H was lower than for both humans (β = −6.87 ± 0.94, T(61.87) = −7.33, p = 5.76×10⁻¹⁰) and mice (β = −2.91 ± 0.34, T(112.57) = −8.51, p = 8.65×10⁻¹⁴).

To further probe the validity of the bimodal inference model, we asked whether posterior model quantities could explain aspects of the behavioral data that the model was not fitted to. We predicted that the posterior decision variable L_t not only encodes perceptual choices (i.e., the variable used for model estimation) but also predicts the speed of response and subjective confidence [30,44]. Indeed, the estimated trial-wise posterior decision certainty |L_t| correlated negatively with RTs in humans (β = −4.36×10⁻³ ± 4.64×10⁻⁴, T(1.98×10⁶) = −9.41, p = 5.19×10⁻²¹) and TDs mice (β = −35.45 ± 0.86, T(1.28×10⁶) = −41.13, p < 2.2×10⁻³⁰⁸). Likewise, subjective confidence reports were positively correlated with the estimated posterior decision certainty in humans (β = 7.63×10⁻³ ± 8.32×10⁻⁴, T(2.06×10⁶) = 9.18, p = 4.48×10⁻²⁰).

The dynamic accumulation of information inherent to our model entails that biases toward perceptual history are stronger when the posterior decision certainty at the preceding trial is high [30,31,51]. Due to the link between posterior decision certainty and confidence, confident perceptual choices should be more likely to induce history-congruent perception at the subsequent trial [30,31]. In line with our prediction, logistic regression indicated that history-congruence was predicted by the posterior decision certainty |L_t-1| extracted from the model (humans: β = 8.22×10⁻³ ± 1.94×10⁻³, z = 4.25, p = 2.17×10⁻⁵; mice: β = −3.72×10⁻³ ± 1.83×10⁻³, z = −2.03, p = 0.04) and the subjective confidence reported by the participants (humans: β = 0.04 ± 1.62×10⁻³, z = 27.21, p = 4.56×10⁻¹⁶³) at the preceding trial.

As a third validation of the bimodal inference model, we used the posterior model parameters to simulate synthetic perceptual choices and repeated the behavioral analyses conducted for the empirical data. Simulations from the bimodal inference model closely replicated our empirical results: Simulated perceptual decisions resulted from a competition of perceptual history with incoming sensory signals (Fig 4A). Stimulus- and history-congruence were significantly autocorrelated (Fig 4B and 4C), fluctuating in antiphase as a scale-invariant process with a 1/f power law (Fig 4D–4F). Simulated posterior certainty [28,30,44] (i.e., the absolute of the log posterior ratio |L_t|) showed a quadratic relationship to the mode of sensory processing (Fig 4H), mirroring the relation of RTs and confidence reports to external and internal biases in perception (Figs 2G, 2H, 3G and 3H,). Crucially, the overlap between empirical and simulated data broke down when we removed the antiphase oscillations or the accumulation of evidence over time from the bimodal inference model (S10–S13 Figs).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Internal and external modes in simulated perceptual decision-making.

(A) Simulated perceptual choices were stimulus-congruent in 71.36% ± 0.17% (in red) and history-congruent in 51.99% ± 0.11% of trials (in blue; T(4.32×10³) = 17.42, p = 9.89×10⁻⁶⁶; upper panel). Due to the competition between stimulus- and history-congruence, history-congruent perceptual choices were more frequent when perception was stimulus-incongruent (i.e., on error trials; T(4.32×10³) = 11.19, p = 1.17×10⁻²⁸; lower panel) and thus impaired performance in the randomized psychophysical design simulated here. (B) At the simulated group level, we found significant autocorrelations in both stimulus-congruence (13 consecutive trials) and history-congruence (30 consecutive trials). (C) On the level of individual simulated participants, autocorrelation coefficients exceeded the autocorrelation coefficients of randomly permuted data within a lag of 2.46 ± 1.17×10⁻³ trials for stimulus-congruence and 4.24 ± 1.85×10⁻³ trials for history-congruence. (D) The smoothed probabilities of stimulus- and history-congruence (sliding windows of ±5 trials) fluctuated as a scale-invariant process with a 1/f power law, i.e., at power densities that were inversely proportional to the frequency (power ∼ 1/f^β; stimulus-congruence: β = −0.81 ± 1.18×10⁻³, T(1.92×10⁵) = −687.58, p < 2.2×10⁻³⁰⁸; history-congruence: β = −0.83 ± 1.27×10⁻³, T(1.92×10⁵) = −652.11, p < 2.2×10⁻³⁰⁸). (E) The distribution of phase shift between fluctuations in simulated stimulus- and history-congruence peaked at half a cycle (π denoted by dotted line). The dynamic probabilities of simulated stimulus- and history-congruence were therefore were strongly anticorrelated (β = −0.03 ± 8.22×10⁻⁴, T(2.12×10⁶) = −40.52, p < 2.2×10⁻³⁰⁸). (F) The average squared coherence between fluctuations in simulated stimulus- and history-congruence (black dotted line) amounted to 6.49 ± 2.07×10⁻³%. (G) Simulated confidence was enhanced for stimulus-congruence (β = 0.03 ± 1.71×10⁻⁴, T(2.03×10⁶) = 178.39, p < 2.2×10⁻³⁰⁸) and history-congruence (β = 0.01 ± 1.5×10⁻⁴, T(2.03×10⁶) = 74.18, p < 2.2×10⁻³⁰⁸). (H) In analogy to humans, the simulated data showed a quadratic relationship between the mode of perceptual processing and posterior certainty, which increased for stronger external and internal biases (β₂ = 31.03 ± 0.15, T(2.04×10⁶) = 205.95, p < 2.2×10⁻³⁰⁸). The horizontal and vertical dotted lines indicate minimum posterior certainty and the associated mode, respectively.

https://doi.org/10.1371/journal.pbio.3002410.g004

In sum, computational modeling suggested that between-mode fluctuations are best explained by 2 interlinked processes (Fig 1E and 1F): (i) the dynamic accumulation of information across successive trials mandated by normative Bayesian models of evidence accumulation and (ii) ongoing antiphase oscillations in the impact of external and internal information.

3.1 Bimodal inference represents a pervasive aspect of perceptual decision-making in humans and mice

A growing body of literature has highlighted that perception is modulated by preceding choices [22–28,30,32,33]. Our work provides converging cross-species evidence supporting the notion that such serial dependencies are a pervasive and general phenomenon of perceptual decision-making (Figs 2 and 3). While introducing errors in randomized psychophysical designs [24,28,30,31,43] (Figs 2A and 3A), we found that perceptual history facilitates postperceptual processes such as speed of response [42] (Figs 2G and 3G) and subjective confidence in humans (Fig 2I).

At the level of individual traits, increased biases toward preceding choices were associated with reduced sensitivity to external information (S1 Fig) and lower metacognitive efficiency. When investigating how serial dependencies evolve over time, we observed dynamic changes in the strength of perceptual history (Figs 2 and 3B) that created wavering biases toward internally and externally biased modes of sensory processing. Between-mode fluctuations may thus provide a new explanation for ongoing changes in perceptual performance [6–11].

In computational terms, serial dependencies may leverage the temporal autocorrelation of natural environments [31,46] to increase the efficiency of decision-making [35,43]. Such temporal smoothing [46] of sensory inputs may be achieved by updating dynamic predictions about the world based on the sequence of noisy perceptual experiences [22,31], using algorithms based on sequential Bayes [25,42,51] such as Kalman [35] or Hierarchical Gaussian filtering [54]. At the level of neural mechanisms, the integration of internal with external information may be realized by combining feedback from higher levels in the cortical hierarchy with incoming sensory signals that are fed forward from lower levels [56].

Yet, relying too strongly on serial dependencies may come at a cost: When accumulating over time, internal predictions may eventually override external information, leading to circular and false inferences about the state of the environment [57]. Akin to the wake–sleep algorithm in machine learning [58], bimodal inference may help to determine whether errors result from external input or from internally stored predictions: During internal mode, sensory processing is more strongly constrained by predictive processes that auto-encode the agent’s environment. Conversely, during external mode, the network is driven predominantly by sensory inputs [18]. Between-mode fluctuations may generate an unambiguous error signal that aligns internal predictions with the current state of the environment in iterative test-update cycles [58]. On a broader scale, between-mode fluctuations may regulate the balance between feedforward versus feedback contributions to perception and thereby play an adaptive role in metacognition and reality monitoring [59].

We hypothesized that observers have certain hyperpriors that are apt for accommodating fluctuations in the predictability of their environment, i.e., people believe that their world is inherently volatile. To be Bayes optimal, it is therefore necessary to periodically reevaluate posterior beliefs about the parameters that define an internal generative model of the external sensory environment. One way to do this is to periodically suspend the precision of prior beliefs and increase the precision afforded to sensory evidence, thus updating Bayesian beliefs about model parameters.

The empirical evidence above suggests that the timescale of this periodic scheduling of evidence accumulation may be scale-invariant. This means that there may exist a timescale of periodic fluctuations in precision over every window or length of perceptual decision-making. Bimodal inference predicts perceptual decisions under a generative model (based upon a hazard function to model serial dependencies between subsequent trials) with periodic fluctuations in the precision of sensory evidence relative to prior beliefs at a particular timescale. Remarkably, a systematic model comparison based on AIC indicated that a model with fluctuating precisions has much greater evidence, relative to a model in the absence of fluctuating precisions. This ad hoc addition of oscillations to a normative Bayesian model of evidence accumulation [51] allowed us to quantify the dominant timescale of periodic fluctuations mode at approximately 0.11 1/N_trials in humans and mice that is appropriate for these kinds of paradigms.

3.2 Bimodal inference versus normative Bayesian evidence accumulation

Could bimodal inference emerge spontaneously in normative models of perceptual decision-making? In predictive processing, the relative precision of prior and likelihood determines their integration into the posterior that determines the content of perception. At the level of individual trials, the perceptual impact of internal predictions generated from perceptual history (prior precision) and external sensory information (likelihood precision) are thus necessarily anticorrelated. The same holds for mechanistic models of drift diffusion, which understand choice history biases as driven by changes in the starting point [51] or the drift rate of evidence accumulation [32]. Under the former formulation, perceptual history is bound to have a stronger influence on perception when less weight is given to incoming sensory evidence, assuming that the last choice is represented as a starting point bias. The effects of choice history in normative Bayesian and mechanistic drift diffusion models can be mapped onto one another via the Bayesian formulation of drift diffusion [60], where the inverse of likelihood precision determines the amount of noise in the accumulation of new evidence, and prior precision determines the absolute shift in its starting point [60].

While it is thus clear that the impact of perceptual history and sensory evidence are anticorrelated at each individual trial, we here introduce antiphase oscillations as an ad hoc modification to model slow fluctuations in prior and likelihood precision that evolve over many consecutive trials and are not mandated by normative Bayesian or mechanistic drift diffusion models. The bimodal inference model provides a reasonable explanation of the linked autocorrelations in stimulus- and history-congruence, as evidenced by formal model comparison, successful prediction of RTs and confidence as out-of-training variables, and a qualitative reproduction of our empirical data from posterior model parameter as evidence against over- or underfitting.

Of note, similar non-stationarities have been observed in descriptive models that assume continuous [61] or discrete [12] changes in the latent states that modulate perceptual decision-making at slow timescales. A recent computational study [12] has used a Hidden Markov model to investigate perceptual decision-making in the IBL database [21]. In analogy to our findings, the authors observed that mice switch between temporally extended strategies that last for more than 100 trials: During engaged states, perception was highly sensitive to external sensory information. During disengaged states, in turn, choice behavior was prone to errors due to enhanced biases toward one of the 2 perceptual outcomes [12]. Despite the conceptual differences to our approach (discrete states in a Hidden Markov model that correspond to switches between distinct decision-making strategies [12] versus gradual changes in mode that emerge from sequential Bayesian inference and ongoing oscillations in the impact of external relative to internal information), it is tempting to speculate that engaged/disengaged states and between-mode fluctuations might tap into the same underlying phenomenon.

3.3 Task engagement and residual motor activation as alternative explanations for bimodal inference

As a functional explanation for bimodal inference, we propose that perception temporarily disengages from internal predictions to form stable inferences about the statistical properties of the sensory environment. Between-mode fluctuations may thus elude circular inferences that occur when both the causes and the encoding of sensory stimuli are volatile [19,57]. By the same token, we suggest that fluctuations in mode occur at the level of perceptual processing [26,30,46,47] and are not a passive phenomenon that is primarily driven by factors situated up- or downstream of sensory analysis.

How does attention relate to phenomenon of between-mode fluctuations? According to predictive processing, attention corresponds to the precision afforded to the probability distributions that underlie perceptual inference [53]. From this perspective, fluctuations between external and internal mode can be understood as ongoing shifts in the attention afforded to either external sensory information (regulated via likelihood precision) or internal predictions (regulated via prior precision). When the precision of either likelihood or prior increases, posterior precision increases, which leads to faster RTs and higher confidence. Therefore, when defined from the perspective of predictive processing as the precision afforded to likelihood and prior [53], fluctuations in attention may provide a plausible explanation for the quadratic relationship of mode to RTs and confidence (Figs 2H–2J, 3I, and 4I).

Outside of the predictive processing field, attention is often understood in the context of task engagement [62], which varies according to the availability of cognitive resources that are modulated by factors such as tonic arousal, familiarity with the task, or fatigue [62]. Our results suggest that internal mode processing cannot be completely reduced to intervals of low task engagement: In addition to shorter RTs and elevated confidence, choices during internal mode were not random or globally biased but driven by perceptual history (S1 Text). Moreover, our computational model identified the dominant timescale of between-mode fluctuations at 0.11 1/N_trials, which may be compatible with fluctuations in arousal [63] but is faster than to be expected for the development of task familiarity or fatigue.

However, in interpreting the impact of between-mode fluctuations on perceptual accuracy, speed of response, and confidence, it is important to consider that global modulators such as tonic arousal are known to have nonlinear effects on task performance [64]: In perceptual tasks, performance seems so be highest during midlevel arousal, whereas low- and high-level arousal lead to reduced accuracy and slower responses [64]. This contrasts with the effects of bimodal inference, where accuracy increases linearly as one moves from internal to external mode, and responses become faster at both ends of the mode spectrum.

Of note, high phasic arousal has been shown to suppress biases in decision-making in humans and mice across domains [65–67], including biases toward perceptual history [28] that we implicate in internal mode processing. While the increase in response speed and history congruence over time (S1 Text) may argue against insufficient training as an alternative explanation for internal mode processing, it may also be indicative of waning arousal. The multiple mechanistic mappings to RTs and confidence warrant more direct measures of arousal (such as pupil size [28,63–65,67,68], motor behavior [63,68], or neural data [69]) to better delineate bimodal inference from fluctuations in global modulators of task performance.

Residual activation of the motor system may provide another contribution to serial biases in perceptual choices [70]. Such motor-driven priming may lead to errors in randomized psychophysical designs, resembling the phenomenon that we identify as internally biased processing [71]. Moreover, residual activation of the motor system may lead to faster responses and thus constitutes an alternative explanation for the quadratic relationship of mode with RTs [70]. The observation of elevated confidence for stronger biases toward internal mode speaks against the proposition that residual activation of the motor system is the primary driver of serial choice biases, since strong motor-driven priming should lead to frequent lapses that are typically associated with reduced confidence [72]. Likewise, perceptual history effects have repeatedly been replicated in experiments with counterbalanced stimulus–response mappings [30].

No-response paradigms, in which perceptual decisions are inferred from eye movements alone, could help to better differentiate perceptual from motor-related effects. Likewise, video tracking of response behavior and neural recordings from motor- and premotor cortex, which has recently been released for the IBL database [21], may provide further insight into the relation of motor behavior to the perceptual phenomenon of between-mode fluctuations.

3.4 Limitations and open questions

Our results suggest bimodal inference as a pervasive aspect of perceptual decision-making in humans and mice. However, a number of limitations and open questions have to be considered:

First, this work sought to understand whether fluctuations between internal and external mode, which we initially observed in an experiment on bistable perception in humans [19], represent a general phenomenon that occurs across a diverse set of perceptual decision-making tasks. Our analysis of the Confidence database [20] therefore collapsed across all available experiments on binary perceptual decision-making. Individual experiments differed with respect to the stimuli, the manipulation of difficulty, the timing of trials, and the way responses were collected but were highly comparable with respect to the central variables of stimulus- and history-congruence (S1 Fig).

The variability across experiments, which we considered as random effects in all statistical analyses, enabled us to assess whether bimodal inference represents a general phenomenon in perceptual decision-making but limited the precision at which we were able to investigate the relation of mode to behavioral variables such as timing, task difficulty, RT, or confidence. This issue is partially resolved by our analyses of the IBL database, which replicated our findings in an experiment that was highly standardized with respect to timing, task difficulty, and behavioral readout [21]. It will be an important task for future research to validate our results on bimodal inference in a standardized dataset of comparable volume in humans, which is, to our knowledge, not yet available.

Second, our results point to an attraction of perception toward preceding choices. Previous work has shown that perceptual decision-making is concurrently affected by both attractive and repulsive serial biases that operate on distinct timescales and serve complementary functions for sensory processing [27,73,74]: Short-term attraction may serve the decoding of noisy sensory inputs and increase the stability of perception, whereas long-term repulsion may enable efficient encoding and sensitivity to change [27]. In the data analyzed here, history biases tended to be repetitive (Figs 2A, 3A, S6, and S7), and only 2 of the 66 experiments of the Confidence database [20] showed significant alternating biases (S1 Fig). However, as we show in S14 Fig, fluctuations in both alternating and repeating history biases generate overlapping autocorrelation curves. Our analysis of between-mode fluctuations is therefore not tied exclusively to repeating biases but accommodates alternating biases as well, such that both may lead to internally biased processing and reduced sensitivity to external sensory information. Future work could apply our approach to paradigms that boost alternating as opposed to repeating biases, as this would help to better understand how repetition and alternation are linked in terms of their computational function and neural implementation [27].

A third open question concerns the computational underpinnings of bimodal inference. The addition of slow antiphase oscillations to the integration of prior and likelihood represents an ad hoc modification of a normative Bayesian model of evidence accumulation [51]. While the bimodal inference model is supported by formal model comparison, the successful prediction of out-of-training variables, and the qualitative reproduction of our empirical data in simulations from posterior model parameters, it is an important task for future research to test (i) whether between-mode fluctuations can emerge spontaneously in hierarchical models of Bayesian inference, (ii) whether modes are continuous [19] or discrete [12], and (iii) whether bimodal inference can be causally manipulated by experimental variables. We speculate that between-mode fluctuations may separate the perceptual contribution of internal predictions and external sensory data in time, creating unambiguous learning signals that benefit inference about the precision of prior and likelihood, respectively. This proposition should be tested empirically by relating the phenomenon of bimodal inference to performance in, e.g., reversal learning, probabilistic reasoning, or metacognition.

A final important avenue for further research on bimodal inference is to elucidate its neurobiological underpinnings. Since between-mode fluctuations were found in humans and mice, future studies can apply noninvasive and invasive neuroimaging and electrophysiology to better understand the neural mechanisms that generate ongoing changes in mode in terms of their neuroanatomy, neurochemistry, and neurocircuitry.

Establishing the neural correlates of externally and internally biased modes will enable exiting opportunities to investigate their role for adaptive perception and decision-making: Causal interventions via pharmacological challenges, optogenetic manipulations, or (non)invasive brain stimulation will help to understand whether between-mode fluctuations are implicated in resolving credit-assignment problems [18,75] or in calibrating metacognition and reality monitoring [59]. Answers to these questions may provide new insights into the pathophysiology of hallucinations and delusions, which have been characterized by an imbalance in the impact of external versus internal information [56,76,77] and are typically associated with metacognitive failures and a departure from consensual reality [77].

4.1 Resource availability

4.2 Experimental model and subject details

4.3 Method details

4.4 Quantification and statistical procedures

All aggregate data are reported and displayed with errorbars as mean ± standard error of the mean.