Stochastic thermodynamics of bipartite systems: transfer entropy inequalities and a Maxwell's demon interpretation

Thermodynamics of information processing started a long time ago with a thought experiment about 'violations' of the second law achieved by Maxwell's demon [1]. Among the seminal contributions to this field (see [2] for a collection of papers) are Szilard's engine [3], Landauer's principle [4] and Bennett's work [5].

More broadly, access to small systems where fluctuations are not negligible is now possible and an understanding of the relation between thermodynamics and information has become a problem of practical interest. For example, experimental verifications of the conversion of information into work [6] and of Landauer's principle [7] have been realized. Moreover, considerable theoretical progress has been made recently with the derivation of second law inequalities [8]–[11] and fluctuation relations [12]–[18] for feedback driven systems. The study of simple models has also played an important role [19]–[33]. In particular, Mandal and Jarzynski [34] (see also [35]–[38]) have introduced a model that clearly demonstrates an idea expressed by Bennett [5]: a tape can be used to do work as it randomizes itself.

In related work [39, 40], we have studied the relation between the rate of mutual information and the thermodynamic entropy production in bipartite systems. By a bipartite system we mean a Markov process with states that are determined by two variables such that in a transition between states only one of the variables can change. We have obtained an analytical upper bound on the rate of mutual information and developed a numerical method to estimate the Shannon entropy rates of continuous time series [40].

In this paper we take the view that a bipartite system provides a simple and convenient description of a Maxwell's demon. More precisely, considering a subsystem y as a Maxwell's demon, we show that work extraction by the other subsystem x leads to an entropy decrease of the external medium, with this entropy decrease being bounded by the entropy reduction of x due to its coupling with y. As an advantage of this Maxwell's demon realization, the full thermodynamic cost is easily accessible, being given by the standard entropy production of the bipartite system.

Moreover, we also study transfer entropy within our setup. Transfer entropy is an informational theoretical measure of how the dynamics of a process depends on another process [41], which is an important concept in the analysis of time series [42]. Ito and Sagawa [43] have recently obtained a fluctuation relation for very general dynamics involving the entropy variation of the external medium due to a subsystem and transfer entropy. Here, we obtain a similar fluctuation relation for a bipartite system. This fluctuation relation implies that the entropy decrease of the external medium due to subsystem x is bounded by the transfer entropy from x to y, where y can be interpreted as a Maxwell's demon.

Hence, we find three different bounds for the entropy decrease of the external medium due to x: the entropy reduction of x, the transfer entropy from x to y and the entropy increase of the external medium due to y. We show that the entropy reduction of x is always the best bound. Furthermore, by studying a particular four state model we observe that the transfer entropy can be larger than the entropy increase of the medium due to y.

The paper is organized as follows. In the next section, we define bipartite systems and the thermodynamic entropy production. Moreover, we explain in which sense a subsystem can be interpreted as a Maxwell's demon. We define the transfer entropy and obtain an analytical upper bound for it in section 3. In section 4 we prove an integral fluctuation relation involving the transfer entropy and show that the transfer entropy from x to y is larger than the entropy reduction of x due to y. Our results are illustrated with a simple four state system in section 5, where we summarize and compare the different inequalities obtained in this paper. We conclude in section 6.

2.1. Basic definitions and inequalities

We restrict to a class of Markov processes which we call bipartite [39, 40]. The states are labeled by the pair of variables (α, i), where α ∈{1, ..., Ω_x} and i ∈{1, ..., Ω_y}. The transition rates from (α, i) to $(\beta ,j)$, are given by

$$\begin{equation}w_{ij}^{\alpha \beta }\equiv \left \{ \begin{array}{@{}ll} w^{\alpha \beta }_i & \quad\qquad {\rm if}\; i=j\;{\rm and}\;\alpha \neq \beta , \\ w^{\alpha }_{ij} & \quad\qquad {\rm if}\; i\neq j\;{\rm and}\;\alpha =\beta ,\\ 0 & \quad\qquad {\rm if}\; i\neq j\;{\rm and}\; \alpha \neq \beta . \end{array}\right .\label {defrates2} \end{equation} \tag{ 1 }$$

The central feature of the network of states is that when a jump occurs only one of the variables changes. Examples of a bipartite systems, inter alia, are stochastic models for cellular sensing [39, 44, 45], where one variable could represent the activity of a receptor and the other the concentration of some phosphorylated internal protein. We also denote a state of the system at time t by z(t) = (x(t), y(t)), where x(t) ∈{1, ..., Ω_x} and y(t) ∈{1, ..., Ω_y}. Hence, the subsystem x is related to the variable denoted by Greek letters and the subsystem y is related to the Roman letters.

The rate of entropy increase of the external medium [46] is then divided into two parts, one caused by jumps in the x variable and the other by jumps in the y variable. More precisely, we define

$$\begin{equation} \sigma _x\equiv \sum _{i,\alpha }P_i^\alpha \sum _{\beta \neq \alpha } w^{\alpha \beta }_i\ln \frac {w^{\alpha \beta }_i}{w^{\beta \alpha }_i} \label {sdef2} \end{equation} \tag{ 2 }$$

and

$$\begin{equation} \sigma _y\equiv \sum _{i,\alpha }P_i^\alpha \sum _{j\neq i} w^\alpha _{ij}\ln \frac {w^\alpha _{ij}}{w^\alpha _{ji}}, \label {sdef3} \end{equation} \tag{ 3 }$$

where $P_i^{\alpha }$ is the stationary probability distribution. The total entropy production, which fulfills the second law of thermodynamics, is then given by [46]

$$\begin{equation} \sigma \equiv \sigma _x+\sigma _y\ge 0. \label {sdef1} \end{equation} \tag{ 4 }$$

The rate σ_x (σ_y) can be interpreted as the rate of increase of the entropy of the external medium due to the dynamics of the x (y) subsystem. It is important to notice that σ_x is not a coarse grained entropy rate [47]–[49]: knowing only the x time series is not sufficient to calculate σ_x.

The rate of change of the Shannon entropy of the system is known to be zero in the stationary state; this can be written as [46]

$$\begin{equation} \sum _{i,\alpha }P_i^\alpha \sum _{\beta \neq \alpha } w^{\alpha \beta }_i\ln \frac {P^{\alpha }_i}{P^{\beta }_i}+ \sum _{i,\alpha }P_i^\alpha \sum _{j\neq i} w^{\alpha }_{ij}\ln \frac {P^{\alpha }_i}{P^{\alpha }_j}=0. \label {sys01} \end{equation} \tag{ 5 }$$

Considering the term originating due to the x jumps we define

$$\begin{equation}h_x\equiv \sum _{i,\alpha }P_i^\alpha \sum _{\beta \neq \alpha } w^{\alpha \beta }_i\ln \frac {P^{\alpha }_i}{P^{\beta }_i}. \label {eq:hx_def} \end{equation} \tag{ 6 }$$

We interpret this quantity as the rate at which the entropy of the subsystem x is reduced due to its coupling with y. This can be understood in the following way. If the state of the subsystem y is i, then the stationary probability of state α given i is $P(\alpha |i)= P_i^\alpha /\sum _\beta P_i^\beta$. Therefore, the rate of change of the Shannon entropy of the subsystem x for y = i is just $\sum _{\alpha }P_i^\alpha \sum _{\beta \neq \alpha } w^{\alpha \beta }_i\ln ({P(\alpha |i)}/{P(\beta |i)})$. Summing over all possible y we obtain (6). In this view it is as if the subsystem x's transition rates $w_i^{\alpha \beta }$ and probabilities P(α|i) depend on time due to the y jumps. In appendix A we consider a functional of the stochastic trajectory that when averaged gives h_x. With this functional, this interpretation of h_x becomes even more clear. Similarly, for the subsystem y we have

$$\begin{equation}h_y\equiv \sum _{i,\alpha }P_i^\alpha \sum _{j\neq i} w^{\alpha }_{ij}\ln \frac {P^{\alpha }_i}{P^{\alpha }_j}. \label {eq:hy_def} \end{equation} \tag{ 7 }$$

From (5) it follows that

$$\begin{equation}h_x=-h_y, \label {sys0} \end{equation} \tag{ 8 }$$

i.e., the entropy reduction of subsystem x equals the entropy increase of subsystem y.

Besides the second law inequality for the full system (4), we also have for the total entropy production caused by transitions in the subsystem x

$$\begin{equation} \sigma _x+h_x= \sum _{i,\alpha }P_i^\alpha \sum _{\beta \neq \alpha } w^{\alpha \beta }_i\ln \frac {w^{\alpha \beta }_iP_i^\alpha } {w^{\beta \alpha }_iP_i^\beta }\ge 0. \label {totx} \end{equation} \tag{ 9 }$$

This inequality has been considered explicitly in [50] and is a direct consequence of the log sum inequality. In appendix A we prove a more general integral fluctuation relation which implies (9). This fluctuation relation is similar to the fluctuation relation for the house-keeping entropy obtained by considering the dual dynamics [46, 51]. The same inequality is valid for the subsystem y,

$$\begin{equation} \sigma _y+h_y= \sum _{i,\alpha }P_i^\alpha \sum _{j\neq i} w^\alpha _{ij}\ln \frac {w^\alpha _{ij}P_i^\alpha } {w^\alpha _{ji}P_j^\alpha }\ge 0.\label {toty} \end{equation} \tag{ 10 }$$

2.2. Subsystem y as a Maxwell's demon

Let us now consider a case where σ_x is negative, i.e., the entropy of the external medium decreases at rate − σ_x due to the subsystem x dynamics. From relations (8)–(10) we obtain the following inequalities:

$$\begin{equation} \sigma _y\ge h_x \ge -\sigma _x. \label {eq:sx_hy_sy} \end{equation} \tag{ 11 }$$

The second inequality can be interpreted in the following way. If we consider the subsystem y as a Maxwell's demon, then the rate of entropy reduction of the external medium − σ_x is bounded by the rate h_x at which the entropy of the subsystem x is reduced due to its coupling to the Maxwell's demon. Furthermore, the first inequality contains an integrated description with the rate at which entropy increases in the Maxwell's demon − h_y = h_x being bounded by the rate of entropy increase in the external medium due to the y dynamics σ_y.

For a more specific interpretation involving the first law we assume that the transition rates take the following local detailed balance form:

$$\begin{equation} \ln \frac {w_i^{\alpha \beta }}{w_i^{\beta \alpha }}= (E_i^\alpha -E_i^\beta )-\omega _i^{\alpha \beta }, \end{equation} \tag{ 12 }$$

where $E_i^\alpha$ is the internal energy of the state (α, i) and $\omega _i^{\alpha \beta }$ is the work extracted from the system in the jump $(\alpha ,i)\to (\beta ,j)$. We set k_BT = 1 throughout the paper. In the stationary state the rate of internal energy change due to x jumps is given by

$$\begin{equation} \epsilon _x= \sum _{i,\alpha ,\beta }P_i^{\alpha } w_i^{\alpha \beta }(E_i^\beta -E_i^\alpha ). \end{equation} \tag{ 13 }$$

The rate of extracted work is

$$\begin{equation} \omega ^{{\rm out}}= \sum _{i,\alpha ,\beta }P_i^{\alpha } w_i^{\alpha \beta }\omega _i^{\alpha \beta }. \end{equation} \tag{ 14 }$$

From the relation σ_x = −_x − ω^out and the first law we identify σ_x as the dissipated heat due to the x jumps. Likewise σ_y is the dissipated heat due to y jumps. For the special case _x = 0, we have − σ_x = ω^out: the second inequality in (11) implies that the rate of extracted work is bounded by h_x. The first inequality in (11) means that the rate of entropy decrease h_x is bounded by the heat σ_y that is dissipated by the demon.

Let us make a comparison with the model introduced by Mandal and Jarzynski [34], where a tape composed of bits interacts with a system connected to a heat bath. By increasing the Shannon entropy of the tape the system can deliver work to a work reservoir. In the above interpretation, this delivered work corresponds to − σ_x and the tape is analogous to the subsystem y: the subsystem x can deliver work by increasing the entropy of the subsystem y. In this sense we can see the subsystem y as an information or entropy reservoir (see [37, 38] for definitions of an information reservoir).

Summarizing the above discussion, bipartite systems provide a particularly transparent description of Maxwell's demon, with the full thermodynamic cost being easily accessible through the standard second law inequality (4). We proceed by defining transfer entropy, which, as we will show in section 4, also provides a bound for − σ_x.

We first consider a discrete time Markov chain with time spacing τ and transition probabilities corresponding to the transition rates (1), i.e.,

$$\begin{equation}W_{ij}^{\alpha \beta }\equiv \left \{ \begin{array}{@{}ll} w^{\alpha \beta }_i\tau &\quad\qquad {\rm if}\; i=j\;{\rm and}\; \alpha \neq \beta , \\ w^{\alpha }_{ij}\tau &\quad\qquad {\rm if}\; i\neq j\;{\rm and}\;\alpha =\beta ,\\ 0 &\quad\qquad {\rm if}\; i\neq j\;{\rm and}\;\alpha \neq \beta ,\\ 1-\sum _{k\neq i} w^{\alpha }_{ik}\tau -\sum _{\gamma \neq \alpha } w^{\alpha \gamma }_{i}\tau & \quad\qquad {\rm if}\; i=j\;{\rm and}\;\alpha =\beta . \end{array}\right .\label {defrates} \end{equation} \tag{ 15 }$$

We denote the full state of the system at time nτ by z_n = (x_n, y_n), where x_n ∈{1, ..., Ω_x} and y_n ∈{1, ..., Ω_y}. For the case where x_n−1 = α, $x_{n}=\beta$, y_n−1 = i and y_n = j, we represent the transition probability $W_{ij}^{\alpha \beta }$ by W(x_n, y_n|x_n−1, y_n−1). Furthermore, the stochastic trajectory of the full process is written as $z_0^n= (z_0,z_1,\ldots ,z_n)$. Whereas $z_0^n$ is Markovian, the stochastic trajectories of the two coarse grained processes $x_0^n$ and $y_0^n$ are in general non-Markovian.

The Shannon entropy rate is a measure of how much the Shannon entropy of a stochastic trajectory increases as we increase the length of the trajectory n. For a generic process $a_0^n$ (where a = x, y, z) it is defined as

$$\begin{equation}H_{a}\equiv -\lim _{n\to \infty }\frac {1}{n\tau }\sum _{a_0^n} P[a_0^n]\ln P[a_0^n],\label {entrate} \end{equation} \tag{ 16 }$$

where $P[a_0^n]$ is the probability of the trajectory $a_0^n$. In particular, since the full process is Markovian, its Shannon entropy rate is given by [52]

$$\begin{eqnarray} &&H_{z} = -\frac {1}{\tau }\sum _{i,j,\alpha ,\beta }P^\alpha _i W_{ij}^{\alpha \beta }\ln W_{ij}^{\alpha \beta } \nonumber \\ &&\quad \,\,\, = -\sum _{i,\alpha ,\beta \atop \alpha \neq \beta }P^\alpha _i w_{i}^{\alpha \beta }(\ln \tau +\ln w_{i}^{\alpha \beta }-1) -\sum _{i,j\alpha \atop i\neq j}P^\alpha _iw_{ij}^{\alpha } (\ln \tau +\ln w_{ij}^\alpha -1)+\textrm {O}(\tau ).\label {entZ2} \end{eqnarray} \tag{ 17 }$$

The equality in the second line is convenient for the subsequent discussion where we will take the limit τ → 0. In general, a similar formula for the rates H_x and H_y in terms of the stationary distribution is not known and these Shannon entropy rates have to be calculated numerically [39, 40], [53]–[55].

A closely related quantity is the conditional Shannon entropy, which is defined as

$$\begin{equation}H(a_n|a_0^{n-1})\equiv \frac {1}{\tau }\sum _{a_0^n} P[a_0^n]\ln P[a_n|a_0^{n-1}]. \label {conddef} \end{equation} \tag{ 18 }$$

In the limit of n →∞ we have $H(a_n|a_0^{n-1})\to H_a$. Moreover, the conditional Shannon entropy decreases for increasing n: knowledge of a longer past decreases randomness [52]. Therefore, the conditional Shannon entropies $H(x_n|x_0^{n-1})$ and $H(y_n|y_0^{n-1})$, which can be calculated in terms of the stationary probability distribution, provide upper bounds on the Shannon entropy rates H_x and H_y, respectively. More precisely, it can be shown that for any finite n and up to order τ, the conditional Shannon entropies are given by [40]

$$\begin{equation}H(x_n|x_0^{n-1})= -\sum _{i,\alpha }P^\alpha _i \sum _{\beta \neq \alpha } w_{i}^{\alpha \beta }(\ln \tau + \ln \overline {w}^{\alpha \beta }-1)+\textrm {O}(\tau ) \label {condux} \end{equation} \tag{ 19 }$$

and

$$\begin{equation}H(y_n|y_0^{n-1}) = -\sum _{i,\alpha }P^\alpha _i \sum _{j\neq i} w_{ij}^{\alpha }(\ln \tau + \ln \overline {w}_{ij}-1)+ \textrm {O}(\tau ), \label {conduy} \end{equation} \tag{ 20 }$$

where

$$\begin{equation} \overline {w}^{\alpha \beta }\equiv \sum _{i=1}^{\Omega _y}P(i|\alpha ) w_{i}^{\alpha \beta }= \frac {1}{P^\alpha }\sum _{i=1}^{\Omega _y} P_i^\alpha w_{i}^{\alpha \beta } \label {avgalpha} \end{equation} \tag{ 21 }$$

and

$$\begin{equation} \overline {w}_{ij}\equiv \sum _{\alpha =1}^{\Omega _x} P(\alpha |i) w_{ij}^{\alpha }= \frac {1}{P_i}\sum _{\alpha =1}^{\Omega _x} P_i^\alpha w_{ij}^{\alpha },\label {avgi} \end{equation} \tag{ 22 }$$

with $P_i= \sum _{\beta }P_i^\beta$ and $P^\alpha = \sum _{j}P_j^\alpha$.

The transfer entropy from x to y is defined as [41]

$$\begin{equation}T^n_{x\to y }\equiv H(y_n|y_0^{n-1})-H(y_n|x_0^{n-1},y_0^{n-1})\ge 0, \label {transferxydef} \end{equation} \tag{ 23 }$$

where $H(y_n|x_0^{n-1},y_0^{n-1})\equiv -{\tau }^{-1}\sum _{x_0^{n-1},y_0^n} P[x_0^{n-1},y_0^{n}]\ln P[y_n|x_0^{n-1},y_0^{n-1}]$. It is the reduction on the conditional Shannon entropy of the y process generated by knowing the x process. In other words, it measures the dependence of y on x or the flow of information from x to y. In the same way, the transfer entropy from y to x is written as

$$\begin{equation}T^n_{y\to x }\equiv H(x_{n}|x_0^{n-1})-H(x_n|x_0^{n-1},y_0^{n-1})\ge 0. \label {transferyxdef} \end{equation} \tag{ 24 }$$

The transfer entropy is in general not symmetric, i.e., $T^n_{x\to y }\neq T^n_{y\to x}$.

The conditional Shannon entropy $H(x_n|x_0^{n-1},y_0^{n-1})$ can be written as

$$\begin{eqnarray}H(x_n|x_0^{n-1},y_0^{n-1}) & =& H(x_n|x_{n-1},y_{n-1})\nonumber \\ & =& -\sum _{i,\alpha }P^\alpha _i\sum _{\beta \neq \alpha } w_{i}^{\alpha \beta }(\ln \tau + \ln w_{i}^{\alpha \beta }-1)+\textrm {O}(\tau ), \label {condx} \end{eqnarray} \tag{ 25 }$$

where the first equality comes from the fact that the full process is Markovian and in the second equality we have performed the substitutions x_n−1 → α, y_n−1 → i and $x_n\to \beta$. Analogously, we obtain

$$\begin{eqnarray}H(y_n|x_0^{n-1},y_0^{n-1}) &=& H(y_n|x_{n-1},y_{n-1})\nonumber \\ & =& -\sum _{i,\alpha }P^\alpha _i\sum _{j\neq i} w_{ij}^{\alpha }(\ln \tau + \ln w_{ij}^{\alpha }-1)+\textrm {O}(\tau ). \label {condy} \end{eqnarray} \tag{ 26 }$$

In this paper we are interested in the transfer entropy in the continuous time limit τ → 0, which is defined as

$$\begin{eqnarray} &&\mathcal {T}_{x\to y}\equiv \lim _{\tau \to 0}\lim _{n\to \infty } T^n_{x\to y}= \lim _{\tau \to 0}\left (H_y+\sum _{i,\alpha }P^\alpha _i\sum _{j\neq i} w_{ij}^{\alpha }(\ln \tau + \ln w_{ij}^{\alpha }-1)\right ), \label {contransferxy} \end{eqnarray} \tag{ 27 }$$

where we used relation (26) in the second equality. The conditional Shannon entropies diverge as lnτ for τ → 0 but the transfer entropy is well behaved in this limit. More clearly, from formula (20), the Shannon entropy rate $H_y= \lim _{n\to \infty } H(y_n|y_0^{n-1})$ diverges as $-\sum _{i,\alpha }P^\alpha _i\sum _{i\neq j} w_{ij}^{\alpha }\ln \tau$ in the limit τ → 0, canceling the lnτ term in (27).

Moreover, as the conditional Shannon entropy (20) for finite n bounds the Shannon entropy rate H_y from above, from equations (20) and (26) we obtain an analytical upper bound on $\mathcal {T}_{x\to y}$, given by

$$\begin{equation} \overline {\mathcal {T}}_{x\to y}= \sum _{i,\alpha }P_i^\alpha \sum _{j\neq i}w^\alpha _{ij}\ln \frac {w^\alpha _{ij}}{\overline {w}_{ij}}. \label {upperxy} \end{equation} \tag{ 28 }$$

This result is similar to the analytical upper bound on the rate of mutual information (see appendix B) we obtained in [39, 40]. Similarly, for the transfer entropy from y to x, which is defined as $T^n_{y\to x }\equiv H(x_n|x_0^{n-1})-H(x_n|x_0^{n-1},y_0^{n-1})$, we obtain

$$\begin{equation} \overline {\mathcal {T}}_{y\to x}= \sum _{i,\alpha } P_i^\alpha \sum _{\beta \neq \alpha }w^{\alpha \beta }_{i} \ln \frac {w^{\alpha \beta }_{i}}{\overline {w}^{\alpha \beta }}. \end{equation} \tag{ 29 }$$

Furthermore, if there is a clear timescale separation, i.e., if the x process is much faster than y, $w^{\alpha \beta }_i\gg w^{\alpha }_{ij}$, then $\mathcal {T}_{x\to y}\to \overline {\mathcal {T}}_{x\to y}$ [40]. Similarly, if the y process is much faster $\mathcal {T}_{y\to x}\to \overline {\mathcal {T}}_{y\to x}$.

While considering the discrete time case and taking the limit τ → 0 is convenient to calculate the upper bound (28), it is more efficient to consider continuous time trajectories with their waiting times to obtain the transfer entropy numerically. This issue and the relation between the transfer entropy and the rate of mutual information are discussed in appendix B.

4.1. Integral fluctuation relation

We consider a generic functional of the random variables $x^n_{n-1}$ and $y_0^{n}$, which is written as $\hat {F}[x_n,y_n;x_{n-1},y_0^{n-1}]$. The average of the functional is denoted by angular brackets, i.e.,

$$\begin{eqnarray} &&\langle \hat {F}\rangle \equiv \sum _{x^n_{n-1},y_0^{n}} \hat {F}[x_n,y_n;x_{n-1},y_0^{n-1}] W[x_n,y_n|x_{n-1}, y_{n-1}]P[x_{n-1},y_0^{n-1}], \end{eqnarray} \tag{ 30 }$$

where we used the Markov property $P[x_{n-1}^n,y_0^{n}]= W[x_n,y_n|x_{n-1},y_{n-1}]P[x_{n-1},y_0^{n-1}]$. Note that W denotes a transition probability where the time index n is irrelevant. In particular, we define the functionals

$$\begin{equation} \hat {\sigma }_x[x_n,y_n;x_{n-1},y_0^{n-1}]\equiv \ln \frac {W[x_n,y_n|x_{n-1},y_{n-1}]}{W[x_{n-1},y_{n}|x_{n},y_{n-1}]} \end{equation} \tag{ 31 }$$

and

$$\begin{equation} \hat {T}_{x\to y}[x_n,y_n;x_{n-1},y_0^{n-1}]\equiv \ln \frac {P[x_{n-1}|y_0^{n-1}]}{P[x_{n}|y_0^{n-1}]}= \ln \frac {P[x_{n-1},y_0^{n-1}]}{P[x_{n},y_0^{n-1}]}. \label {defMxyfr} \end{equation} \tag{ 32 }$$

We can prove the following integral fluctuation relation:

$$\begin{eqnarray} \langle \exp (-\hat {\sigma }_x-\hat {T}_{x\to y}) \rangle & =& \sum _{x^n_{n-1},y_0^{n}}\left (\frac {W[x_{n-1}, y_n|x_{n},y_{n-1}]}{W[x_{n},y_{n}|x_{n-1},y_{n-1}]} \frac {P[x_{n},y_0^{n-1}]}{P[x_{n-1},y_0^{n-1}]}\right )\nonumber \\ &&\times \,W[x_n,y_n|x_{n-1},y_{n-1}]P[x_{n-1},y_0^{n-1}]\nonumber \\ &=& \sum _{x^n_{n-1},y_0^{n}} W[x_{n-1},y_n|x_{n},y_{n-1}] P[x_{n},y_0^{n-1}]\nonumber \\ &=& \sum _{x_{n-1},y_n,x_n,y_0^{n-1}} W[x_{n-1},y_n|x_{n},y_{n-1}] P[x_{n},y_0^{n-1}]\nonumber \\ &=& \sum _{x_n,y_0^{n-1}} P[x_{n},y_0^{n-1}]= 1. \label {fttransf} \end{eqnarray} \tag{ 33 }$$

This relation does not depend on the transition probabilities having the form (15), therefore, it is valid also for systems that are not bipartite. Using Jensen's inequality we then obtain

$$\begin{equation} \langle \hat {\sigma }_x\rangle +\langle \hat {T}_{x\to y}\rangle \ge 0. \label {jensentr} \end{equation} \tag{ 34 }$$

Finally, it is straightforward to show that

$$\begin{equation} \sigma _x= \frac {1}{\tau }\langle \hat {\sigma }_x\rangle . \label {sigmaS} \end{equation} \tag{ 35 }$$

Furthermore, as we show in appendix C,

$$\begin{equation} \mathcal {T}_{x\to y}= \lim _{\tau \to 0}\lim _{n\to \infty } \frac {1}{\tau }\langle \hat {T}_{x\to y}\rangle . \label {transferT} \end{equation} \tag{ 36 }$$

Therefore, inequality (34) implies

$$\begin{equation} \sigma _x+\mathcal {T}_{x\to y}\ge 0. \label {inetransf} \end{equation} \tag{ 37 }$$

A closely related fluctuation relation for causal networks has recently been obtained by Ito and Sagawa [43]. To obtain a fluctuation relation similar to (33) using the framework from [43], the stochastic trajectory of a bipartite system should be viewed as a causal network with two connected rows, corresponding to the x and y processes.

Comparing our autonomous system, where there are no explicit measurements or feedback, with standard feedback driven systems, the inequality (37) is analogous to the second law inequality for feedback driven systems as derived in [10]. As pointed out in [17], the quantity that bounds the extracted work in feedback driven systems is precisely the transfer entropy from the system to the controller performing the measurements, which is equivalent to $\mathcal {T}_{x\to y}$.

4.2. Comparison between h_x and $\mathcal {T}_{x\to y}$

We would like to compare the bounds − σ_x ≤ h_x and $-\sigma _x\le \mathcal {T}_{x\to y}$ in order to assess which one is stronger. By considering the average $\langle \hat {\sigma }_x+ \hat {T}_{x\to y}\rangle$, we obtain the following inequality:

$$\begin{eqnarray} \left \langle \hat {\sigma }_x+\hat {T}_{x\to y} \right \rangle &=& \sum _{x^n_{n-1},y_{n-1}^{n}}\sum _{y_{0}^{n-2}} W[x_n,y_n|x_{n-1},y_{n-1}]P[x_{n-1},y_0^{n-1}]\nonumber \\ &&\times \, \ln \left (\frac {W[x_{n},y_{n}|x_{n-1},y_{n-1}]}{W[x_{n-1}, y_n|x_{n},y_{n-1}]}\frac {P[x_{n-1},y_0^{n-1}]} {P[x_{n},y_0^{n-1}]}\right )\nonumber \\ &\ge & \sum _{x^n_{n-1},y_{n-1}^{n}} W[x_n,y_n|x_{n-1},y_{n-1}]P[x_{n-1},y_{n-1}]\nonumber \\ &&\times \, \ln \left (\frac {W[x_{n},y_{n}|x_{n-1},y_{n-1}]}{W[x_{n-1}, y_n|x_{n},y_{n-1}]}\frac {P[x_{n-1},y_{n-1}]}{P[x_{n},y_{n-1}]}\right ) = \langle \hat {\sigma }_x\rangle +\langle \hat {h}_{x}\rangle , \label {ineq} \end{eqnarray} \tag{ 38 }$$

where we used the log sum inequality for the sum over $y_0^{n-2}$ and the definition

$$\begin{equation} \hat {h}_x[x_n,y_n;x_{n-1},y_0^{n-1}]\equiv \ln \frac {P[x_{n-1},y_{n-1}]}{P[x_{n},y_{n-1}]}. \end{equation} \tag{ 39 }$$

By noting that $\langle \hat {h}_{x}\rangle /\tau = h_x+\textrm {O}(\tau )$, we obtain that the inequality (38) in the limit τ → 0 implies

$$\begin{equation}h_x\le \mathcal {T}_{x\to y}. \label {hxmxy} \end{equation} \tag{ 40 }$$

Therefore, h_x provides a better bound on − σ_x than the transfer entropy $\mathcal {T}_{x\to y}$.

The above inequality can also be seen in a different way. In general, we do not know an analytical expression for the transfer entropy in terms of the stationary distribution. With the upper bound (28) and the inequality (40) we find

$$\begin{equation}h_x\le \mathcal {T}_{x\to y}\le \overline {\mathcal {T}}_{x\to y}. \label {upperlower} \end{equation} \tag{ 41 }$$

Summarizing inequalities (11) and (41), we obtain

$$\begin{equation} -\sigma _x\le h_x\le \left \{ \begin{array}{l} \sigma _y\\ \mathcal {T}_{ x\to y}\le \overline {\mathcal {T}}_{ x\to y}. \end{array}\right . \label {summine} \end{equation} \tag{ 42 }$$

Knowing now that h_x is the best bound on − σ_x we would also like to investigate whether there is a unique relation between σ_y and the transfer entropy $\mathcal {T}_{x\to y}$. The following example will demonstrate that there is no such inequality.

To illustrate our results we consider the simplest bipartite system which is a four state model. The transition rates are defined in figure 1. The parameters γ_x and γ_y set the timescales of the x and y transitions, respectively. We consider the case where q_y ≤ q_x ≤ 1/2 so that the probability current runs in the clockwise direction. In this case, σ_y and − σ_x are both positive.

Zoom In Zoom Out Reset image size

We can interpret the model of figure 1 as follows. We consider two coupled proteins x and y that each can be in an inactive or active state, represented by 1 and 2, respectively. A chemical reaction, with chemical potential difference Δμ_x ≥ 0, drives the transitions of the x protein favoring the states (1,2) and (2,1), where the proteins are in different configurations. Local detailed balance is then written as

$$\begin{equation} \ln \frac {1-q_x}{q_x}= \Delta \mu _x. \end{equation} \tag{ 43 }$$

Another chemical reaction drives the y transitions, also favoring anti-alignment of the proteins, implying in the local detailed balance relation

$$\begin{equation} \ln \frac {1-q_y}{q_y}= \Delta \mu _y. \end{equation} \tag{ 44 }$$

The condition q_y ≤ q_x ≤ 1/2 reads Δμ_y ≥ Δμ_x ≥ 0. In this case the chemical reaction driving the y transitions feeds work into the system at a rate σ_y and the system does work against the chemical reaction driving the x transitions at a rate − σ_x.

Explicitly, the stationary current is given by

$$\begin{equation}J= P_1^2\gamma _xq_x-P_1^1\gamma _x(1-q_x), \end{equation} \tag{ 45 }$$

with the stationary probabilities $2P_1^1=2P^2_2=(\gamma _xq_x+\gamma _yq_y)/(\gamma _x+\gamma _y)$ and $2P_1^2=2P^1_2=1-2P_1^1$. Moreover, the rate of extracted work is given by

$$\begin{equation} -\sigma _x=2J\ln \left (\frac {1-q_x}{q_x}\right )\ge 0, \end{equation} \tag{ 46 }$$

and the rate of energy input is

$$\begin{equation} \sigma _y=2J\ln \left (\frac {1-q_y}{q_y}\right )\ge 0. \end{equation} \tag{ 47 }$$

The rate of entropy reduction of subsystem x due to its coupling to y is

$$\begin{equation}h_x=2J\ln \left (\frac {\gamma _x(1-q_x)+\gamma _y(1-q_y)} {\gamma _xq_x+\gamma _yq_y}\right )\ge 0. \end{equation} \tag{ 48 }$$

The upper bound of the transfer entropy reads

$$\begin{equation} \overline {\mathcal {T}}_{x\to y}=2P_1^2\gamma _yq_y \ln \frac {q_y}{r_y}+2P_1^1\gamma _y(1-q_y)\ln \frac {1-q_y}{r_y}, \end{equation} \tag{ 49 }$$

where $r_y\equiv 2P_1^2q_y+2P_1^1(1-q_y)$. An analytical expression for the transfer entropy $\mathcal {T}_{x\to y}$ is not known but we can determine it numerically.

Zoom In Zoom Out Reset image size

In figure 2 we compare the three different bounds on the extracted work − σ_x. Besides the illustration of inequalities (42), we can also see that $\mathcal {T}_{x\to y}$ approaches $\overline {\mathcal {T}}_{x\to y}$ as the y process becomes slower, as discussed in section 3. The main result we obtain from these plots is the crossing of the transfer entropy $\mathcal {T}_{x\to y}$ and σ_y, with the input σ_y being smaller near equilibrium, q_x = q_y, and larger in the far from equilibrium limit, q_y → 0.

We have studied a series of second law like inequalities valid for bipartite systems. Besides the standard entropy production (4), the entropy production of a subsystem (9) and the inequalities involving transfer entropy (37) and (40) have been analyzed. Moreover, inspired by the fluctuation relation recently obtained by Ito and Sagawa [43] we have obtained the fluctuation relation (33), which in the continuous time limit leads to the inequality involving transfer entropy. From the summary of the inequalities (42) we have obtained that h_x, the rate of entropy reduction of x due to the coupling with y, provides the best bound on − σ_x. As a particularly interesting interpretation, we have shown that a bipartite system provides a transparent realization of Maxwell's demon, with an integrated description of the subsystem and demon being easily accessible through the standard entropy production.

By analyzing a simple four state model we have shown that the transfer entropy $\mathcal {T}_{x\to y}$ can be larger than the entropy rate proportional to the heat dissipated by the demon σ_y. While the crossing between $\mathcal {T}_{x\to y}$ and σ_y has been obtained for a specific model we conjecture it to be more general because it depends on two general properties: $\mathcal {T}_{x\to y}$ not being zero in equilibrium, where σ_y = 0, and $\mathcal {T}_{x\to y}$ being finite when the y transition rate goes to zero, where σ_y diverges. Furthermore, as the transfer entropy is generally not zero in equilibrium it should be useless as a bound on − σ_x near equilibrium, e.g., in the linear response regime.

It is interesting to compare this work with [37], where a simplified version of the Mandal and Jarzynski model [35] for a tape interacting with a thermodynamic system was analyzed. In [37] the entropy (or information) reservoir is a tape composed of a sequence of bits, while here it is the y subsystem. Moreover, a series of inequalities similar to (42) have been obtained in [37], with the Shannon entropy difference of the tape providing the best bound on the extracted work, as is the case of h_x here. Likewise, the mutual information between the tape and the system crosses the full input of work to reset the tape (see also [28]), corresponding to the crossing of $\mathcal {T}_{x\to y}$ and σ_y here.

Summarizing, the series of inequalities studied here, and the methods to calculate quantities like the rate of mutual information and transfer entropy that we have developed in [40], form a solid theoretical framework for bipartite systems, which constitute an important class of Markov processes. Among possible applications of our results, the investigation of bipartite models for cellular sensing is an interesting direction for future work.

Support by the ESF through the network EPSD is gratefully acknowledged.

The full Markovian stochastic trajectory from time 0 to T is denoted by $z(t)_0^T=(z_0,\tau _0;z_1,\tau _1;\ldots ;z_N,\tau _N)$, where the waiting times fulfill τ₀ + τ₁ + ⋯ + τ_N = T and N is the number of jumps in the trajectory. The probability density of a trajectory is written as

$$\begin{equation}P[z(t)_0^T]= P(z_0)\prod _{n=0}^{N-1}w_{z_nz_{n+1}} \prod _{n=0}^{N}\exp (-\lambda _{z_n}\tau _n), \label {pathprob} \end{equation} \tag{ A.1 }$$

where P(z₀) is the initial distribution, w_znzn+1 denotes the transition rate from z_n to z_n+1 defined in (1) and λ_zn ≡∑ _zw_znz is the escape rate.

In order to obtain the fluctuation relation leading to (6) we consider the modified transition rates

$$\begin{equation}u_{ij}^{\alpha \beta }\equiv \left \{ \begin{array}{ll} w^{\beta \alpha }_iP_i^\beta /P_i^\alpha & \quad\qquad {\rm if}\; i=j\quad {\rm and}\quad \alpha \neq \beta , \\ w^{\alpha }_{ij} & \quad\qquad {\rm if}\; i\neq j\quad {\rm and}\quad \alpha =\beta ,\\ 0 & \quad\qquad {\rm if}\; i\neq j\quad {\rm and}\quad \alpha \neq \beta , \end{array}\right . \end{equation} \tag{ A.2 }$$

and denote the path probability (A.1) obtained with these modified transition rates by $P^{\dagger }[z(t)_0^T]$, where the initial probability is also P(z₀). The escape rates for u are written as ψ_zn ≡∑ _zu_znz. Furthermore, we define the functionals

$$\begin{eqnarray} &&\Delta H_x[z(t)_0^T]\equiv \sum _{n=0}^{N-1} \delta _{y_n,y_{n+1}}\ln \frac {P_{y_{n}}^{x_{n}}} {P_{y_{n+1}}^{x_{n+1}}}=\sum _{n=0}^{N-1} \ln \frac {P_{y_{n}}^{x_{n}}}{P_{y_{n}}^{x_{n+1}}},\label {hxfunc} \end{eqnarray} \tag{ A.3 }$$

$$\begin{eqnarray} &&\Delta S_x[z(t)_0^T]\equiv \sum _{n=0}^{N-1} \delta _{y_n,y_{n+1}} \ln \frac {w_{z_nz_{n+1}}}{w_{z_{n+1}z_{n}}}, \end{eqnarray} \tag{ A.4 }$$

where δ_{yn, yn+1} is the Kronecker delta function. In the limit T →∞ we have 〈ΔH_x〉/T → h_x and 〈ΔS_x〉/T → σ_x, where the angular brackets here denote an integral over all stochastic paths (note that this is different from section 4).

The usual ratio of path probabilities is then given by

$$\begin{equation} \frac {P^{\dagger }[z(t)_0^T]}{P[z(t)_0^T]}= \exp (-\Delta H_x[z(t)_0^T]-\Delta S_x[z(t)_0^T]+\Lambda _x[z(t)_0^T]), \label {ratioprob} \end{equation} \tag{ A.5 }$$

where the functional $\Lambda _x[z(t)_0^T]\equiv \sum _{n=0}^{N}(\lambda _{z_n}-\psi _{z_n})\tau _n$ comes from the fact that the escape rates of w and u are different. Using standard methods [46], relation (A.5) implies

$$\begin{equation} \langle \exp \left (-\Delta H_x[z(t)_0^T]-\Delta S_x[z(t)_0^T]+\Lambda _x[z(t)_0^T]\right ) \rangle =1. \end{equation} \tag{ A.6 }$$

From Jensen's inequality and

$$\begin{equation} \langle \Lambda _x\rangle /T\to \sum _{i,\alpha }P_i^{\alpha } \sum _{\beta \neq \alpha }(w^{\alpha \beta }_i-w^{\beta \alpha }_i P_i^{\beta }/P_i^{\alpha })=0, \end{equation} \tag{ A.7 }$$

we obtain the second law for the subsystem x (9).

Let us make the following remarks. One could consider an x transition from α to $\beta$ as dependent on time due to changes in the variable i in the transition rates $w_{i}^{\alpha \beta }$. Within this view, the rate u corresponds to a sort of 'adjoint' dynamics and relation (A.5) is similar to the ratio of probabilities involving the forward adjoint trajectory in the fluctuation relation for the house-keeping entropy derived in [51]. Furthermore, denoting by N_y the number of jumps for which the variable y changes and considering the interval between two y jumps [n_y, n_y + 1] we write

$$\begin{equation} \delta H_x(n_y)= \ln \frac {P^{x_i}_j}{P^{x_f}_j}, \end{equation} \tag{ A.8 }$$

where x_i is the x state at the time of the jump n_y, x_f is the x state at the time of the jump n_y + 1 and j is the y state in the time interval between the jumps n_y and n_y + 1. The functional (A.3) can then be written as

$$\begin{equation} \Delta H_x[z(t)_0^T]\equiv \sum _{n_y=0}^{N_y-1} \delta H_x(n_y). \end{equation} \tag{ A.9 }$$

In this form it becomes clear that the rate h_x = 〈ΔH_x〉/T in the large T limit is the rate of entropy reduction of subsystem x due to the subsystem y dynamics.

Using the notation of appendix A, the continuous time Shannon entropy rate is given by [56]

$$\begin{eqnarray} \mathcal {H}_{z}&\equiv & -\lim _{T\to \infty }\frac {1}{T}\sum _{z(t)_0^T} P[z(t)_0^T]\ln P[z(t)_0^T]\nonumber \\ &=&\sum _{i,\alpha }P^\alpha _i\sum _{\beta \neq \alpha } w_{i}^{\alpha \beta }(\ln w_{i}^{\alpha \beta }-1) +\sum _{i,\alpha }P^\alpha _i\sum _{j\neq i} w_{ij}^{\alpha }(\ln w_{ij}^\alpha -1).\label {entratez} \end{eqnarray} \tag{ B.1 }$$

If we compare this formula with (17) we see that the continuous time entropy rate does not show the lnτ divergence.

The coarse grained trajectories are written as $x(t)_0^T=(x_0,\tau _0^x;x_1,\tau _1^x;\ldots ;x_{N_x},\tau _{N_x}^x)$ and $y(t)_0^T=(y_0,\tau _0^y;y_1,\tau _1^y;\ldots ;y_{N_y},\tau _{N_y}^y)$, where $\tau _0^x+\tau _1^x+\cdots +\tau _{N_x}^x=\tau _0^y+\tau _1^y+ \cdots +\tau _{N_y}^y=T$ and N_x (N_y) is the number of jumps where the x (y) variable changes. Due to the bipartite nature of the transition rates N = N_x + N_y. The continuous time Shannon entropy rate of the y process is defined as

$$\begin{equation} \mathcal {H}_{y}\equiv -\lim _{T\to \infty }\frac {1}{T} \sum _{y(t)_0^T}P[y(t)_0^T]\ln P[y(t)_0^T]. \end{equation} \tag{ B.2 }$$

In contrast to lim_τ→0H_y, this continuous time Shannon entropy rate does not have the divergent term proportional to lnτ. Hence, the transfer entropy (27) can be written as

$$\begin{equation} \mathcal {T}_{x\to y}= \mathcal {H}_y-\sum _{i,\alpha } P^\alpha _i\sum _{j\neq i} w_{ij}^{\alpha }(\ln w_{ij}^{\alpha }-1). \end{equation} \tag{ B.3 }$$

Therefore, to calculate the transfer entropy we just have to obtain the non-Markovian entropy rate $\mathcal {H}_y$; this can be achieved by using a numerical method to estimate the Shannon entropy for non-Markovian continuous time processes developed in [40].

We can apply the same procedure to the transfer entropy from y to x, which is written as

$$\begin{equation} \mathcal {T}_{y\to x}= \mathcal {H}_x-\sum _{i,\alpha } P^\alpha _i\sum _{\beta \neq \alpha } w_{i}^{\alpha \beta } (\ln w_{i}^{\alpha \beta }-1).\label {eq:Myx_cont} \end{equation} \tag{ B.4 }$$

Finally, we would like to point out the relation

$$\begin{equation} \mathcal {I}\equiv \mathcal {H}_x+\mathcal {H}_y-\mathcal {H}_z= \mathcal {T}_{x\to y}+\mathcal {T}_{y\to x} \end{equation} \tag{ B.5 }$$

between the rate of mutual information $\mathcal {I}$ and the transfer entropy. The rate of mutual information measures how correlated the x and y processes are, without any specific direction. Obviously, the lower bound on the transfer entropy (40) can be extended to a lower bound on the rate of mutual information, i.e., $\mathcal {I}\ge |h_x|$.

We start by rewriting (32) in the from

$$\begin{equation} \frac {1}{\tau }\langle \hat {T}_{x\to y}\rangle = H(x_n|y_0^{n-1})-H(x_{n-1}|y_0^{n-1}). \end{equation} \tag{ C.1 }$$

We are interested in the limit n →∞ and for n large enough we can substitute $H(x_{n-1}|y_0^{n-1})$ by $H(x_{n}|y_0^{n})$, leading to

$$\begin{equation} \frac {1}{\tau }\langle \hat {T}_{x\to y}\rangle = H(x_n|y_0^{n-1})-H(x_{n}|y_0^{n})= H(y_n|y_0^{n-1})-H(y_{n}|x_n,y_0^{n-1}). \end{equation} \tag{ C.2 }$$

Hence, from the definition (27), in order to prove (36) we have to show that

$$\begin{equation}H(y_{n}|x_n,y_0^{n-1})= -\sum _{i,j,\alpha \atop i\neq j} P^\alpha _i w_{ij}^{\alpha }(\ln \tau + \ln w_{ij}^{\alpha }-1)+\textrm {O}(\tau ). \label {eq:A_Hvar1} \end{equation} \tag{ C.3 }$$

Comparing this with (25), which reads

$$\begin{equation}H(y_{n}|x_{n-1},y_0^{n-1})= -\sum _{{i,j,\alpha \atop i\neq j}} P^\alpha _i w_{ij}^{\alpha }(\ln \tau + \ln w_{ij}^{\alpha }-1)+\textrm {O}(\tau ), \label {eq:A_Hvar2} \end{equation} \tag{ C.4 }$$

it is then analogous to demonstrate that

$$\begin{equation} \left \langle \ln \left \{\frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1}, y_0^{n-1}]}\right \}\right \rangle =\textrm {O}(\tau ^2). \label {eq:Otau2} \end{equation} \tag{ C.5 }$$

The term inside the logarithm can be written as

$$\begin{eqnarray} &&\frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1},y_0^{n-1}]}= \frac {P[y_n,x_n,y_0^{n-1}]}{P[x_n,y_0^{n-1}]P[y_n|x_{n-1}, y_{n-1}]}\nonumber \\ &&\quad = \frac { \sum _{\tilde x_{n-1}} W[x_{n},y_{n}|\tilde {x}_{n-1},y_{n-1}] P[\tilde {x}_{n-1},y_0^{n-1}] }{ \sum _{\tilde x_{n-1},\tilde y_{n}} W[x_{n},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}]P[\tilde {x}_{n-1},y_0^{n-1}] \sum _{\tilde {x}_n}W[\tilde {x}_n,y_n|x_{n-1},y_{n-1}] }.\label {eq:BracketDots} \end{eqnarray} \tag{ C.6 }$$

In the following we consider three cases.

First, we consider y_n≠y_n−1, which implies x_n = x_n−1. Equation (C.6) takes the form

$$\begin{eqnarray} &&\label {firsteq} \frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1},y_0^{n-1}]}= \frac { W[x_{n-1},y_{n}|x_{n-1},y_{n-1}] P[x_{n-1},y_0^{n-1}]}{ \sum _{\tilde x_{n-1},\tilde y_{n}} W[x_{n-1},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}]P[\tilde {x}_{n-1},y_0^{n-1}] W[x_{n-1},y_n|x_{n-1},y_{n-1}]}\nonumber \\ &&\quad = \frac {1}{ \sum _{\tilde x_{n-1},\tilde y_{n}} W[x_{n-1},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}] ({P[\tilde {x}_{n-1},y_0^{n-1}]}/{P[x_{n-1},y_0^{n-1}]}) }=1+\textrm {O}(\tau ), \end{eqnarray} \tag{ C.7 }$$

where in the last equality we used $W[x_{n-1},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}]=1+\textrm {O}(\tau )$ for $\tilde {x}_{n-1}= x_{n-1}$ and $\tilde {y}_{n}= y_{n-1}$, and $W[x_{n-1},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}]=\textrm {O}(\tau )$ otherwise.

Second, we take x_n≠x_n−1, implying y_n = y_n−1. The term (C.6) is now written as

$$\begin{eqnarray} \frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1},y_0^{n-1}]} &{=}& \frac { \sum _{\tilde x_{n-1}} W[x_{n},y_{n-1}|\tilde {x}_{n-1},y_{n-1}] P[\tilde {x}_{n-1},y_0^{n-1}] }{ \sum _{\tilde x_{n-1},\tilde y_{n}} W[x_{n},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}]P[\tilde {x}_{n-1},y_0^{n-1}] \sum _{\tilde {x}_n} W[\tilde {x}_n,y_{n-1}|x_{n-1},y_{n-1}]}\nonumber \\ &{=}&1+\textrm {O}(\tau ). \label {secondeq} \end{eqnarray} \tag{ C.8 }$$

where we used $\sum _{\tilde {x}_n}W[\tilde {x}_n,y_{n-1}|x_{n-1},y_{n-1}]=1+\textrm {O}(\tau )$.

Third, we consider x_n = x_n−1 and y_n = y_n−1. It is convenient to define L through the equality W[x, y|x^', y^'] = δ_{x, x^'}δ_{y, y^'} + τL[x, y|x^', y^'], where L is the stochastic matrix corresponding to the transition rates (1). The term (C.6) now becomes

$$\begin{eqnarray} \frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1},y_0^{n-1}]} &=& \frac { 1+\sum _{\tilde x_{n-1}} \tau L[x_{n-1},y_{n-1}|\tilde {x}_{n-1},y_{n-1}] ({P[\tilde {x}_{n-1},y_0^{n-1}]}/{P[x_{n-1},y_0^{n-1}]}) }{ 1+ \sum _{\tilde {x}_{n-1},\tilde y_{n}} \tau L[x_{n-1},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}] ({P[\tilde {x}_{n-1},y_0^{n-1}]}/{P[x_{n-1},y_0^{n-1}]}) }\nonumber \\ &&\times \, \left (1+\sum _{\tilde {x}_n}\tau L[\tilde {x}_n,y_{n-1} |x_{n-1},y_{n-1}]\right )^{-1}. \label {eq:Average3rdCase} \end{eqnarray} \tag{ C.9 }$$

Finally, the quantity (C.5) can be separated into three contributions: one for y_n≠y_n−1, the second for x_n≠x_n−1 and the third for x_n = x_n−1 and y_n = y_n−1. Considering equations (C.7) and (C.8), we see that the first two contributions give O(τ²), leading to

$$\begin{eqnarray} \left \langle \ln \frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1}, y_0^{n-1}]}\right \rangle &=&\sum _{x_{_n-1},y_0^{n-1}}\left (1+\tau L[x_{_n-1},y_{n-1} |x_{_n-1},y_{n-1}]\right )P[x_{n-1},y_0^{n-1}]\nonumber \\ &&\times \,\ln \frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1},y_0^{n-1}]}+ \textrm {O}(\tau ^2). \end{eqnarray} \tag{ C.10 }$$

Using equation (C.9) we obtain

$$\begin{eqnarray} &&\left \langle \ln \frac {P[y_n|x_n,y_0^{n-1}]}{P[y_n|x_{n-1}, y_0^{n-1}]}\right \rangle \nonumber \\ &&= \sum _{\tilde x_{n-1},x_{n-1},y_0^{n-1}} \tau L[x_{n-1},y_{n-1}|\tilde {x}_{n-1},y_{n-1}] P[\tilde {x}_{n-1},y_0^{n-1}]\nonumber \\ &&-\, \sum _{\tilde {x}_{n-1},x_{n-1},\atop \tilde y_{n},y_0^{n-1}} \tau L[x_{n-1},\tilde {y}_{n}|\tilde {x}_{n-1},y_{n-1}] P[\tilde {x}_{n-1},y_0^{n-1}] \nonumber \\ &&-\, \sum _{\tilde x_{n},x_{n-1},y_0^{n-1}}\tau L[\tilde {x}_n,y_{n-1}|x_{n-1},y_{n-1}]P[x_{n-1},y_0^{n-1}] +\textrm {O}(\tau ^2)=\textrm {O}(\tau ^2), \end{eqnarray} \tag{ C.11 }$$

where, from ∑ _{x, y}L[x, y|x^', y^'] = 0 for all x^', y^', the term in the third line is zero and the terms in the second and fourth lines cancel. This concludes the proof of (C.5), which implies (36).