Introduction

Mental chronometry relies on the analysis of response time (RT) across different experimental conditions to make inferences on underlying cognitive processes (Donders, 1868). Beyond comparing mean RT in each experimental condition, one can study RT distributions (Luce, 1986). De Jong et al., (1994) introduced a particular method, called delta plot, to describe the link between response speed and the size of an experimental effect. This method consists first in computing RT quantiles in each condition, and then plotting their difference against their mean. Using this method, one can see for instance whether the magnitude of an experimental factor is larger for slower than for faster responses.

Delta plots have proven to be especially fruitful in the study of tasks involving a stimulus-response compatibility (SRC) manipulation, such as the Simon task (Simon, 1969). In this task, subjects have to choose between a left- and a right-hand key press according to the color of a visual stimulus presented a few degrees either to the left or the right of a fixation point. Mean RTs are typically slower for contralateral stimulus-response associations (incompatible trials) than for ipsilateral ones (compatible trials). De Jong et al., (1994) found that the spatial correspondence effect (difference between incompatible and compatible trials) was larger for fast responses than for slow ones, resulting in decreasing delta plots.

This result has been consistently replicated (for reviews, see Pratte et al., (2010), Proctor et al., (2011), and Schwarz and Miller (2012)). Delta plots that level-off for slower RTs, or even become decreasing, are not restricted to the Simon task and have also been observed in other tasks involving some kind of conflict, such as the Eriksen flanker task (e.g., Burle et al., 2014, Mattler 2003, Ridderinkhof et al., 2005, Wylie et al., 2007, 2009), masked priming tasks (Eimer, 1999; Finkbeiner & Caramazza, 2008; Atas & Cleeremans, 2015), or tasks involving lying and truth-telling (Debey et al., 2015). Such experimental results stand in sharp contrast to the positive linear relation between mean and standard deviation of RTs usually observed (Luce, 1986; Wagenmakers & Brown, 2007).

These observations suggest that the shape of delta plots might be the signature of some specific cognitive processes. For instance, some argued that it reflects the spontaneous decay of an automatic response (Hommel 1993, 1994), while others proposed that it results from the deployment of an active inhibitory process (De Jong et al., 1994, Ridderinkhof 2002a, b). Recently, various cognitive architectures compatible with the particular relation between RT and the size of the interference effect observed in conflict tasks have been proposed (see, e.g., Schwarz and Miller (2012), Ulrich et al., (2015), and Hübner and Töbel (2019)).

All of these theories have in common proposing a cognitive interpretation of distribution of the interference effect. While delta plots provide a compelling visualization of the relation between RT and the size of the interference effect, it is not obvious how they could be translated into numerical measures suitable for statistical purposes (as required, for instance, to test one of the above-mentioned theories). The aim of this article is precisely to solve this issue, by providing a quantitative measure of the distribution of the interference effect. It is important to keep in mind that, by themselves, delta plots say nothing about the cognitive processes at play (Zhang & Kornblum, 1997). The same is true about the index we propose: it is a measure of the effects of the cognitive processes, not of their nature and causes. After introducing this new index, we will illustrate its properties and compare it to other methods through simulations of a computational model for conflict tasks (Ulrich et al., 2015) and reanalyses of an actual dataset.

The Interference Distribution Index

Definition

While the following analysis might be applied to data from any type of task, we will consider a task involving an SRC manipulation, such as the Simon task. Formally, let F be the cumulative distribution function of RTs for correct trials for which stimulus and response are incompatible, and G be the cumulative distribution function of RTs for correct trials for which stimulus and response are compatible. Given a cumulative distribution function F, its associated quantile function is defined as : \(F^{-1}(p)=\inf \left \{x\in \mathbb R\left |F(x)\geq p \right .\right \}\). Let D(F,G)(p) be the size of the interference effect, as measured by the difference of the RTs between conditions at the pth quantile (i.e., D(F,G)(p) = F− 1(p) − G− 1(p)). The delta plot corresponding to F and G is obtained by plotting D(F,G)(p) against the mean of the RTs in each condition at the pth quantile (i.e., \(M(F,G)(p) =\frac {1}{2} \left [F^{-1}(p)+ G^{-1}(p)\right ]\)). When there is no ambiguity about the underlying distribution functions, we will slightly abuse notations, and simply note M and D instead of D(F,G) and M(F,G), respectively. The total interference effect is given by \(\mu (D)={{\int \limits }_{0}^{1}} D(p)dp\). We will assume, throughout the remainder of this article, that μ(D) > 0.

Our aim is to quantify the relation between RTs and the size of the interference effect. To do so, we will describe how the total interference effect is spread across RT quantiles. Specifically, the proportion of the total interference effect contained in the p × 100% fastest (pairs of) trials is given by:

$$ L(p) = \frac{1}{\mu(D)}{{\int}_{0}^{p}}D(x)dx. $$
(1)

This function is very similar to the Lorenz curve (Lorenz, 1905). It is noteworthy, however, that a Lorenz curve would require that D be a quantile function, which is not the case here (since it is the difference between two quantile functions). L(p) represents the share of the total interference effect that is contained in the p × 100% fastest (pairs of) trials. Note that L(0) = 0 and L(1) = 1. Because of its resemblance to the Lorenz curve, we will call the “Lorenz-interference plot” the graphical representation of L. If the interference effect is constant across RT trials, then L(p) = p for all p, and the Lorenz-interference plot will coincide with the diagonal. The more the interference effect is concentrated among the fastest trials, the higher will be the Lorenz-interference plot. The area under the Lorenz-interference plot is thus a natural measure of the distribution of the interference effect, very much like the Gini index (Gini, 1912) is related to the area under the Lorenz plot. Formally, for any interference effect D, we propose the following interference distribution index:

$$ I(D) = {{\int}_{0}^{1}} L(p)dp. $$
(2)

We call this index the Interference Distribution Index (IDI). Observe that if D is constant, \(I(D)=\frac {1}{2}\). On the other hand, if D is decreasing, \(I(D)>\frac {1}{2}\). In particular, if all the interference effect is concentrated in the fastest pair of trials, I(D) = 1. Finally, if D is increasing, \(I(D)<\frac {1}{2}\). If all the interference effect is concentrated in the slowest pair of trials, I(D) = 0. It has sometimes been observed that the interference effect becomes negative at high percentiles (translating into negative delta plots). In such a case, the Lorenz-interference plot will decrease for these highest percentiles. Therefore, these percentiles will contribute negatively to the IDI (because the interference effect will then be less concentrated among slowest trials).

We made available an R package called RTconflict, which allows to easily implement the IDI.Footnote 1 In practice, the quantile functions must be approximated from data. We set the number of quantiles to the number of trials in the condition that contains the smallest number of observations. We then crucially rely on the quantile function in R. This function allows choosing the method used to compute quantiles among the nine methods presented in Hyndman and Fan (1996). We implemented the default method, but allow the user to choose other methods.

Properties of the IDI

We will now turn to some properties of the IDI. First, the IDI only depends on the interference effect, i.e., on the difference between F− 1 and G− 1. Thus, it is invariant to any change that does not impact F− 1G− 1. We call this property “location invariance”. Assume, for instance, that for some reason the motor execution time increases in the same way in both conditions for some quantiles. Then the IDI will not be modified.

Next, one can easily see that, if both F− 1 and G− 1 are scaled by the same positive constant, the Lorenz-interference plot will not change (because both the numerator and the denominator of L will then be multiplied by the same constant), and so will therefore be the case for the IDI. In other words, the IDI is invariant with respect to a uniform increase of the interference effect across quantiles. We call this property the “scale invariance property”. It is an important property, as it allows measuring the distribution of the interference effect independently from its overall magnitude. This does not imply by any means that the overall interference effect level is not meaningful, but that it should be disentangled from its distributional properties.

The IDI has another invariance property that allows an easy measure of the distribution of aggregated individual interference effects. Let us consider two interference effects, D1 and D2, with the same total magnitude (i.e., μ(D1) = μ(D2)). Then, it is easy to check that the IDI corresponding to the weighted average of D1 and D2 is simply the weighted average of each IDI. Now, what if μ(D1)≠μ(D2)? Recall that we are interested in the distribution of the interference effect, and not in its level. We can thus normalize D1 and D2 by μ(D1) and μ(D2), respectively, and aggregate them linearly. We then obtain:

$$ D = \frac{1}{2} \frac{D_{1}}{\mu(D_{1})} + \frac{1}{2} \frac{D_{2}}{\mu(D_{2})}. $$

By the above property, we get:

$$ I(D) = \frac{1}{2} I\left( \frac{D_{1}}{\mu(D_{1})}\right) + \frac{1}{2} I\left( \frac{D_{2}}{\mu(D_{2})}\right). $$

Finally, by the scale invariance property, we obtain:

$$ I(D) = \frac{1}{2} I(D_{1}) + \frac{1}{2} I(D_{2}). $$

In other words, the aggregated IDI is nothing but the average of individual IDIs. We call this property the “aggregation property”.

Yet, all this tells little about how the IDI reflects the relation between RT and the size of the interference effect. In that respect, the IDI has a fundamental property that we call the “transfer property”. Assume that the level of interference effect increases for some quantile p, and decreases of the exact same amount for some higher quantile q. Such a transformation leaves the overall interference effect unchanged, but more concentrated among the fastest trials. It is easy to see that such a transformation indeed results in an increase of the IDI. Moreover, the larger the distance between p and q, the larger is this increase. On the other hand, the variation of the IDI is the same wherever this change occurs in the interference effect distribution (given the distance between p and q).

Finally, the IDI is a continuous function. This means that small changes in the RT distributions will result in small changes in the IDI. This property, while a bit technical, has its importance: it means that the IDI will not overreact to small measurement errors.

Importantly, it turns out that not only does the IDI have all the properties described above, but it is also the only index (up to an increasing transformation) that satisfies them all. In other words, if another index were to satisfy all these properties, it would be an increasing transformation of the IDI. This fact is proven in Appendix A.

Alternative measures of the interference effect distribution

To the best of our knowledge, mainly two methods are currently used to measure the distribution of the interference effect, and both are directly based on delta plots. A very common method consists of quantifying the decay of the interference effect by the opposite of the slope of the last segment of the delta plot (Delta Slope Index, DSI) (Ridderinkhof, 2002a; Forstmann et al., 2008; van den Wildenberg et al., 2010). This method has been used to determine the effects of various psychiatric or neurodegenerative disorders on inhibitory control mechanisms, such as attention deficit/hyperactivity disorder (e.g., Ridderinkhof et al., (2005) and Suarez et al., (2015)), Tourette’s syndrome (e.g., Wylie et al., (2013)), alien hand syndrome (McBride et al., 2013), Parkinson’s disease (e.g., van Wouwe et al., (2016) and Wylie et al., (2010)), mild cognitive impairment (Wylie et al., 2007) and their modulation by psychotropic drugs (e.g., Wylie et al., (2012) and van Wouwe et al., (2016)). It has also been used to study the development of inhibitory control across the lifespan (e.g., Bub et al., (2006), Ambrosi et al., (2019), de Bruin and Della Sala (2017), and Joyce et al., (2014)), the effect of hypoxia (Davranche et al., 2016), acute exercise (e.g. Davranche et al., (2009)), working memory capacity (Meier and Kane, 2015), and social context (Sharma et al., 2010; Belletier et al., 2015).

The DSI satisfies scale invariance, but not location invariance. In particular, a similar (not necessarily uniform) increase of both F− 1 and G− 1 (which lets the interference effect unchanged) would actually yield a decrease of the DSI. This index also does not satisfy the aggregation property, which means that the average of the DSI across subjects is difficult to interpret. These difficulties can be circumvented by considering the quantile version of DSI, i.e., by considering the slope of the last segment of a discretized version of D. However, neither the DSI, nor its quantile version satisfy the principle of transfer, as they are totally insensitive to any change in the early quantiles of the RT distribution. Finally, it should be stressed that the DSI requires a discretization of the RTs, which entails a substantial loss of information. Moreover, the number of bins considered is quite arbitrary, and might affect the results, as illustrated in “Mental fatigue and control”.

Another, albeit less frequent method, consists of measuring the distribution of the interference effect by the opposite of the trend of the delta plot (Delta Trend Index, DTI), as estimated by a linear regression (De Jong et al., 1994; Pratte et al., 2010). As the DSI, and for the same reasons, the DTI satisfies the scale invariance property, but neither location invariance nor the aggregation property. It does, however, satisfy the transfer property. One might thus prefer to replace it by the trend of D. Formally, let T(D) be the opposite of the trend of D, as estimated by an OLS regression. It is obvious that T does not satisfy the invariance property. This can however be easily fixed by normalizing it by μ(D). We then obtain the following Interference Trend Index (ITI):

$$ ITI = \frac{T(D)}{\mu(D)}. $$

This index satisfies all the properties of the IDI, and is actually equivalent to it: as shown in Appendix A, it is an affine transformation of the IDI. Note that the ITI is slightly distinct from the usual measure of the trend of the delta plot in three respects: first, it is the trend of the interference effect; second, it is computed on all quantiles (instead of a subset of them, e.g., deciles, thereby avoiding the issue of discretization); and third, it is normalized by the mean interference effect. Among other things, this means that the average of individual ITIs can easily be interpreted, contrary to that of delta plot trends.

Illustrations

In order to better understand the differences between the IDI and the DSI, we will compare the results of these methods on different sets of data. First, we will turn to data generated by a computational model for conflict tasks. This will allow us to see how various cognitive processes, as defined within this model, impact these measures. Next, we will reanalyze experimental data from a study on the impact of mental fatigue on cognitive control (Joyce et al., 2014).Footnote 2

Diffusion model for conflict tasks

The drift diffusion model (DDM, Ratcliff (1978)) is one of the most popular computational model of RTs in two-alternative forced choice. It essentially postulates that the evidence favoring an option is accumulated until it reaches a threshold, at which stage a decision is made. While the DDM proved to be very successful in modeling RTs in many paradigms, it has been shown that it cannot account for decreasing delta slopes (Pratte et al., 2010; Schwarz and Miller, 2012).

The diffusion model for conflict tasks (DMC, Ulrich et al., (2015)) is a versatile diffusion model that can account for delta plots that level off, or even decrease, as observed in experiments involving SRC manipulations, such as the Simon task. The DMC is an extension of the DDM, where the amount of evidence in favor of one option results from the sum of two diffusion processes: a controlled process, with a constant drift rate (μc), and an automatic process, which generates an expected accumulated evidence that increases in favor of one option (the response compatible to the stimulus) before progressively fading. This dynamics is described by a rescaled Gamma density function, with a peak amplitude A, a shape parameter α and a characteristic time τ.Footnote 3 The peak amplitude determines the strength of the automatic activation, while the shape parameter and characteristic time are related to the speed of the fading of automatic activation, be it due to an active suppression or a passive decay. The shape parameter is generally fixed at 2 (Ulrich et al., 2015). With this parameterization, the characteristic time corresponds to the peak latency of the automatic activation. More precisely, the smaller the parameter τ, the faster the decline of the automatic activation.

We simulated the DMC for various values of τ, and computed the IDI and DSI for each simulated data set. The DSI was calculated from deciles of the delta plots. Observe that the IDI and the DSI are supposed to vary in the same direction, i.e., to be larger whenever the interference effect is more concentrated among the fastest responses. We thus expected these indices to mainly be impacted by the characteristic time, and, specifically, to decrease when τ increases.

We kept the parameters related to the controlled processes and the speed-accuracy strategy fixed, with values chosen in a plausible range (Ulrich et al., 2015; Servant et al., 2016; White et al., 2017), while letting the peak amplitude and the characteristic time varying (see legend of Fig. 1 for more details).

Fig. 1
figure 1

Heat maps representing the values of the Delta Slope Index (left) and the Interference Distribution Index (right) for simulated data from DMC as a function of the peak amplitude A and the characteristic time τ. A varies between 0.015 and 0.04 (by steps of 0.001), and τ varies between 0.035 and 0.160 (by steps of 0.005). Other parameters are kept fixed with the following values: μc = 0.5, b = 0.05, Ter = .03, α = 2. The model is simulated in seconds, with a diffusion coefficient equal to 0.1. For each set of parameter values, 160,000 trials were simulated

Figure 1 clearly shows that the DSI is not a very good measure of the interference effect distribution. It is strongly affected by the peak of the automatic activation A, and is not monotone with respect to the characteristic time. This implies that there is a range of values of τ such that, for a fixed A, the DSI increases with τ, which is exactly the opposite to what is expected. The IDI seems, in that respect, more satisfactory, as it globally decreases when τ increases, and is much less sensitive to variations of A.

In order to further illustrate the differences between the IDI and the DSI, we plot the cumulative distributions for compatible and incompatible trials, delta plots, and Lorenz-interference plots for two sets of parameters (see the legend of Fig. 2 for details). As τ increases, the interference effect is more concentrated among slowest trials. This is clearly apparent when inspecting the cumulative distributions for compatible and incompatible trials in Fig. 2: the distance between the cumulative distributions increases more in the bottom panel (corresponding to τ = 0.06) than in the top panel (corresponding to τ = 0.03). The Lorenz-interference plots confirm this fact. For instance, the 50% fastest trials represent 52% of the interference effect when τ = 0.03, while they represent only 40% of the interference effect when τ = 0.06. As a consequence, the IDI is larger for τ = 0.03 (IDI = 0.52) than for τ = 0.06 (IDI = 0.44). Yet, the DSI index is smaller for τ = 0.03 (DSI = 0.29) than for τ = 0.06 (DSI = 0.37). The reason for this result appears clearly when one inspects the delta plots. While the slope of the last segment is indeed steeper for τ = 0.06, it becomes decreasing later (for the ninth decile, instead of the sixth decile for τ = 0.03). The DSI fails to take into account this difference.

Fig. 2
figure 2

Cumulative distributions for compatible and incompatible trials (left), delta plots (middle), and Lorenz-interference plots (right) for two characteristic times: τ = 0.03 (top) and τ = 0.06 (bottom), and A = 0.03. 100,000 trials were simulated for each value of τ. Other parameters were kept identical to those in Fig. 1. DSI and IDI are displayed with the cumulative distribution functions

Mental fatigue and control

We now turn to an actual data set in order to investigate to what extent the choice of an index of the interference effect distribution might lead to different conclusions. Joyce et al., (2014) investigated the impact of mental fatigue (measured by time on task) on control, defined as the suppression of an automatic activation. We reanalyze here their results. During two experimental sessions, 24 subjects completed four sets of five blocks of 96 trials (i.e., 1920 trials of the Simon task per session). We group trials of both sessions in two conditions: the first two sets (condition “early”) and the last two sets (condition “late”). We expect to find that fatigue induces a depletion of cognitive control. According to Ridderinkhof (2002a) activation-suppression theory, the interference effect should be more concentrated among fast trials in the “early” than in the “late” condition. We compare the DSI and the IDI across conditions using a Bayesian parameter estimation (Kruschke, 2013), implemented with the R package BEST (Kruschke and Meredith, 2018). We use BEST default priors with 10,000 burn-in steps, and report 95% highest-density intervals (HDI).

We start with the DSI. The comparison of slopes requires deciding the number of quantiles used to compute the slopes. We first compare slopes between the last two deciles (see Fig. 3, left panel), and observe that there is no significant difference between the two conditions (early condition: DSI = 0.11, late condition: DSI = 0.061, HDI of the difference: [− 0.03,0.12]). On the other hand, comparing the slopes between the last quintiles reveals a better control for the early condition (early condition: DSI = 0.13, late condition: DSI = 0.051, HDI of the difference: [0.020,0.13]). These results show that the DSI might lead to a different conclusion depending on the number of quantiles considered (which is an arbitrary choice).

Fig. 3
figure 3

Delta plots in the early condition (red) and late conditions (blue) averaged across subjects, for five (left panel) and ten (right panel) bins

We now turn to the IDI. Interference effects (D) and Lorenz-interference plots are shown in Fig. 4, where group interference plots are obtained by computing the average of individual ones. The IDI index reveals a better control in the early condition compared to the late condition (early condition: IDI = 0.61, late condition: IDI = 0.51, HDI of difference : [0.001,0.096]), suggesting that time-on-task might impair cognitive control, as expected.

Fig. 4
figure 4

Interference effect (representing the interference effect at each quantile, left) and Lorenz-interference plots (representing the share of the mean interference effect that is contained in the p × 100% fastest trials, right) in the early (red) and late (green) conditions. The Lorenz-interference plot corresponding to early condition (red) is below the Lorenz-interference plot corresponding to late condition (green), which corresponds to a larger IDI in the early condition as compared to the late condition

Discussion

Delta plots are a natural way to represent the link between response speed and the size of an experimental effect, by plotting the difference between RT quantiles in each experimental factor against their means. It has been shown that, while they are generally linearly increasing, they can sometimes deviate from this pattern in tasks involving SRC effects, such as the Simon task (De Jong et al., 1994). This observation leads to interpreting the distribution of the interference effect in SRC tasks as the signature of some specific cognitive processes (De Jong et al., 1994, Ridderinkhof 2002a, b, Schwarz & Miller 2012, Ulrich et al., 2015, Pratte et al., 2010, Proctor et al., 2011). In that context, it is important to be able to measure the distribution of the interference effect, independently of any cognitive theory.

In this article, we aimed at providing such a measure. While extremely useful as a graphical device, delta plots are not a pure representation of the distribution of the interference effect, as they also account for the average response time at each quantile. We thus step back from the delta plots to the distribution of the interference effect, and wonder how it can be quantified. We proposed a simple measure (the Interference Distribution Index, IDI), that is essentially a concentration index, measuring to what extent the interference effect is concentrated among the fastest trials. It is associated to a graphical representation (the Lorenz-interference effect), that represents the proportion of the total interference effect concentrated within the p% fastest trials. We show that this measure has several convenient properties, in terms of invariance with respect to scale transformation, sensitivity to increase of concentration among fastest trials, and aggregation of individual data. Moreover, we show that this measure is the only one (up to an arbitrary increasing transformation) that possesses all these properties. We also show how this measure can be applied to specific computational models and actual experimental data.

It should be noted that we have restricted attention, throughout this article, to the interference effect among correct trials. The rationale for this choice is that the amount and timing of errors are likely to reflect different cognitive mechanisms than the distribution of the interference effect. In particular, it has been argued that the timing of errors could be the signature of the strength of response capture (see van den Wildenberg et al., (2010), for a review), and an index dedicated to measure this effect has been proposed (Servant et al., 2018). It must also be stressed that, by construction, the IDI does not depend on the size of the interference effect. Again, this does not mean that this information is not important, but simply that we tried to design an index that specifically measures the distribution of the interference effect. The specificity of chronometric indices might help disentangle different important aspects of cognitive processes.

Finally, it might be worth noting that, while new, the IDI is closely related to the intuition of the authors who first proposed to use delta plots to quantify the distribution of the interference effect (De Jong et al., 1994). Indeed, these authors suggested using the trend of the delta plot, as approximated by a linear regression, as a measure of the distribution of the interference effect. It turns out that this measure does not allow a simple aggregation across individuals (because it relies on delta plots), and does not satisfy a simple invariance property with respect to changes that do not impact the interference effect (location invariance). However, these issues can easily be solved by considering the trend of the interference effect instead of that of the delta plot. Yet, this measure would depend on the overall interference effect. To solve this problem, one can simply normalize it by the total interference effect. It turns out that the resulting measure (the ITI) is equivalent to the IDI. In other words, the present work can be seen as a rationalization of the use of the trend of the interference effect as a measure of its distribution. As a side result, it comes with an intuitive graphical representation (the Lorenz-interference plot).