Correlation and unmediated cheap talk in repeated games with imperfect monitoring

Abstract

This paper studies n-player $(n\ge 3)$ undiscounted repeated games with imperfect monitoring. We prove that all uniform communication equilibrium payoffs of a repeated game can be obtained as Nash equilibrium payoffs of the game extended by unmediated cheap talk. We also show that all uniform communication equilibrium payoffs of a repeated game can be reached as Nash equilibrium payoffs of the game extended by a pre-play correlation device and a cheap-talk procedure that only involves public messages; furthermore, in the case of imperfect public and deterministic signals, no cheap talk is conducted on the equilibrium path.

Appendix A: Proofs

We first present the proofs of the results in the public monitoring case. In particular, to prove Proposition 5.2, we construct correlated equilibria in which no cheap talk is used on the equilibrium path. Next we explain how to generalize this proof to establish Proposition 4.1 in the private monitoring case. We then show that Theorem 4.2 follows from Proposition 4.1 and an application of the MPC protocols introduced in the Online Appendix. Finally, we outline a detailed sketch of proof of Theorem 6.1 (the stochastic-signal case).

1.1 Proof of Proposition 5.1

Fix a canonical communication equilibrium $({\bar{c}},\tau ^*)$ with payoff $w\in C$, where ${\bar{c}}$ is a canonical communication device and $\tau ^*$ is truthful and obedient. Under public monitoring, all players observe the same public signal at each stage. It is therefore without loss to assume that the mediator observes the realized public signals. Thus, for each $t, {\bar{c}}_t$ can be viewed as a mapping from the set of past recommendations and public signals $(a_1,s_1,\ldots ,a_{t-1},s_{t-1})$ to the set of distributions of actions $\Delta (A)$.

Now we define an autonomous device ${\hat{c}}=({\hat{c}}_t)$, which does not observe the public signals. First set ${\hat{c}}_1={\bar{c}}_1$, draw $a_1$ from ${\hat{c}}_1$, and recommend each player i to play $a^i_1$. Next for each $s_1\in S$, let ${\hat{c}}_2(a_1)(s_1)={\bar{c}}_2(a_1,s_1)$, that is, ${\hat{c}}_2$ maps A into $\Delta (A)^S$. Then draw $(a_2(s_1))_{s_1\in S}$ from ${\hat{c}}_2(a_1)$ and inform each player i the vector $(a^i_2(s_1))_{s_1\in S}$. Inductively, for each $t\ge 2$ and $(s_1,\ldots ,s_{t-1})$, let ${\hat{c}}_t(a_1,\ldots ,a_{t-1})(s_1,\ldots ,s_{t-1})={\bar{c}}_t(a_1,s_1,a_2(s_1),s_2,\ldots , a_{t-1}(s_1,\ldots ,s_{t-2}),s_{t-1})$, so that ${\hat{c}}_t$ maps $A^{t-1}$ into $\Delta (A)^{S^{t-1}}$. Similarly, draw $(a_t(s_1,\ldots ,s_{t-1}))_{(s_1,\ldots ,s_{t-1})\in S^{t-1}}$ and inform each player the corresponding vector of recommendations.

Given the constructed autonomous device ${\hat{c}}$, the obedient strategy of each player i is to play the recommended action $a^i_t(s_1,\ldots ,s_{t-1})$ if the realized public history is $(s_1,\ldots ,s_{t-1})$ at stage t. Since ${\hat{c}}$ is defined without referring to the actual play, no player learns more information under ${\hat{c}}$ than under ${\bar{c}}$ after any realized history at any stage. It follows that the obedient strategy profile is an equilibrium of this extended game, with the corresponding equilibrium payoff w. This completes the proof.

1.2 Proof of Proposition 5.2

Fix any $w\in JR$ with $w=u(p)$ for some $p\in {\mathcal {P}}$. We construct a correlated equilibrium with public cheap talk such that the equilibrium payoff vector is w and cheap talks are only conducted off the equilibrium path. By Proposition 4.5 in Mertens et al. (2014), it suffices to show that for any $\varepsilon _0>0$, there exist $T\in \mathbb {N}$ and an $\varepsilon _0$-correlated equilibrium with public cheap talk of the T-period finitely repeated game with payoff within $\varepsilon _0$ of w.³⁷ Recall that $M=\max _{i,a}|u^i(a)|$. Let $\varepsilon =\varepsilon _0/(1+5M)$. Let $T_1$ be such that $T_1\ge \max \{T_0, (M/\varepsilon )^6\}$, where $T_0$ is defined in the “approachability strategy” $c^a$ at the end of Sect. 3 [see inequality (6)]. Let $T_2$ be such that $T_2\ge T_1/\varepsilon$ and for any $T\ge T_2, T\exp (-\varepsilon \sqrt{T}/n|A|)\le \varepsilon$. Fix $T\ge T_2$.

Define a correlation device ${\bar{c}}=(\Omega , P)$ as follows:

• $\Omega =\Omega _{I}\times \Omega _{II}\times \Omega _{III}$, where $\Omega _{I}=\prod _{i\in N}\Omega ^i_{I}, \Omega ^i_{I}=\prod _{t=1}^T\Omega ^i_{I,t}, \Omega ^i_{I,t}=A^i_t\cup (\prod _{j\in N} A^j_t), \Omega _{II}=\prod _{i\in N}(\prod _{t\ge 1} \mathbb {N}^{S^{t-1}})$, and $\Omega _{III}=\prod _{i\in N} A^{-i}$.

• $P=P_{I}\otimes P_{II}\otimes P_{III}$, where $P_{I}$ is a probability on $\Omega _{I}, P_{II}$ is a probability on $\Omega _{II}$, and $P_{III}$ is a probability on $\Omega _{III}$. $(\Omega _{I}, P_{I})$ generates the recommended actions for players to follow on the equilibrium path. $(\Omega _{II}, P_{II})$ is used by players to coordinate punishment in case a deviation is detected and there are multiple suspects. $(\Omega _{III}, P_{III})$ is used by players to punish any single player to his correlated minmax.

• $P_{III}=\otimes _{i\in N} P^i_{III}$, where for each $i, P^i_{III}\in \Delta (A^{-i})$ is any minimizer of the problem

  $$\begin{aligned} v^i=\min _{x^{-i} \in \Delta \left( A^{-i} \right) } \max _{a^i\in A^i} \sum _{a^{-i}} x^{-i}(a^{-i}) u^i(a^i,a^{-i}). \end{aligned}$$

  Each player i is informed of the action used to punish any player $j\ne i$, so that player j will effectively be punished to his correlated minmax level by others. Note that here we only need to draw $a^{-i}$ once because this punishment never occurs on the equilibrium path.³⁸ Since the correlated minmax action profile against player i is drawn from $P^{i}_{III}\in \Delta (A^{-i})$ independently from the actions that player i receive from $P^j_{III}$ for all $j\ne i$, on the equilibrium path player i does not learn any information about the actions recommended by $P^{i}_{III}$ to other players when he is punished this way. We also note that $(\Omega _{III}, P_{III})$ can be discarded since we allow players to use cheap talk in punishment phases and correlated minmax a single player can be achieved through cheap talk among all other players.

• $P_{II}$ is constructed from the “approachability strategy” $c^a$ together with an authentication scheme as follows: Recall that the aim of $P_{II}$ is to generate correlation to implement the punishment strategy in the corresponding communication equilibrium. Since the correlation device is only present before the game starts, we need it to generate a collection of recommended action profiles for each history $(a_1,s_1,\ldots ,a_{t-1},s_{t-1})$ of the mediator and then translate them to recommendations that only depend on the public history $(s_1,\ldots , s_t)$. Moreover, equally important is the design of an encoding-decoding scheme so that no player is able to learn the realized recommendations without the help from others.

  Specifically, given $c^a$, the mediator first draws $a_1\in A$ according to $c^a_1$; then for each $s_1$, she draws $a_2(s_1)\in A$ from $c^a_2(a_1,s_1)$; then for each $(s_1,s_2)$, she draws $a_3(s_1,s_2)=a_3(a_1,a_2(s_1),s_1,s_2)\in A$ from $c^a_3(a_1,s_1,a_2,s_2)$, and so on. That is, the mediator draws all recommendations for all possible public histories at the pre-play stage. Then the recommendations are encrypted and sent to each player as follows. For each i, let $K^i=|A^i|$, write $A^i=\{a^{i}[1],\ldots ,a^i[K^i]\}$, and pick an integer $L^i$ large enough such that $K^i<\varepsilon (L^i-1)/M$. Suppose that after history $(s_1,\ldots ,s_{t-1})$, the recommended action for player i is $a^i[k]$. The mediator randomly chooses $K^i$ different integers $\{l^i[1],\ldots ,l^i[K^i]\}$ from the set $\{1,2,\ldots ,L^i\}$, where $l^i[\kappa ]$ corresponds to the $\kappa$-th action $a^i[\kappa ]$ for player i. Then she sends the ordered sequence $(l^i[1],\ldots ,l^i[K^i])$ to player i and the integer $l^i[k]$ to player $i+1$ (modulo n). Finally, she takes a random permutation of $(l^i[1],\ldots ,l^i[K^i])$ and sends the resulting ordered sequence to all other players in $N\setminus \{i,i+1\}$.

  By construction, player i’s recommendations from $(P_{II},\Omega _{II})$ are encoded so that he does not know which actions are recommended unless player $i+1$ broadcasts the decoding keys for i truthfully for all public histories; moreover, no player in $N\setminus \{i,i+1\}$ learns any information about player i’s recommendations other than the fact that player $i+1$ has announced valid keys. Finally, whenever player $i+1$ broadcasts a number different from the truth encoding key, all other players will find his announcement invalid with high probability.³⁹

• $P_I$, which generates the main path of play, is defined by the following procedure: Let ${\hat{p}}\in \Delta (A)$ be the convex combination of the distribution $p\in \Delta (A)$ and the uniform distribution ${\mathcal {U}}\in \Delta (A)$, i.e., ${\hat{p}}=(1-\eta )p+\eta {\mathcal {U}}$, where $\eta <\min \{\varepsilon /M, T^{-\frac{1}{6}}\}$.⁴⁰ The mediator draws a sequence of action profiles $(a_1,\ldots ,a_T)\in A^T$ independently and identically according to the distribution ${\hat{p}}$. Each player i is informed of the sequence of recommendations $(a^i_1,\ldots ,a^i_T)$. Moreover, independently at each stage $t\le T$, (1) with probability $\eta$, all players are informed of the action profile $a_t$ instead of their own recommendations, and (2) with probability $\eta ^2/n$, only $n-1$ players are informed of $a_t$; so that, for each player i, (1) conditioning on observing $a_t$, he believes that with probability at least $1-\eta$, all other players also observe $a_t$; and (2) conditioning on observing $a^i_t$, he believes that with probability at least $\eta ^2/n$, all other players observe $a_t$.

• Each player i is supposed to follow the recommended actions $(a^i_1,\ldots , a^i_T)$ on the equilibrium path and switch to the recommended actions from $(P_{II},\Omega _{II})$ once he detects a deviation from the main path, i.e., he observes $a_t$ but $s_t\ne f(a_t)$. In the latter case, a public cheap-talk phase is added at each stage: at the first punishment stage each player $i+1$ truthfully broadcasts the encoding key $l^i[k]$ for player i and follows the recommended action $a^{i+1}[k']$ after learning the encoding key $l^{i+1}[k']$ from player $i+2$; at each later stage each player again truthfully broadcasts the encoding keys for every public history and follow the recommended actions. If any player fails to announce any encoding key or announces an invalid key, then all other players know the identity of that player. Since $(\Omega _{II}, P_{II})$ is constructed from the approachability strategy $c^a$, if there are at least $T_0$ stages in the punishment phase, then no player i can get an expected average payoff higher than $w^i+\varepsilon$ from playing non-obediently.

  Finally, in cases where a single deviant is identified, that is, either a deviation from the recommended action is attributed to a single player or a deviation from the public cheap talk is detected, players switch to playing the corresponding recommended action from $(P_{III},\Omega _{III})$ without cheap talk.

Note that by construction:

• (1) At each stage t where player i only observes $a^i_t$ and other players play according to $P_I$, if player i deviates in a way that could be detected, i.e., choosing an ${\hat{a}}^i_t$ such that $f({\hat{a}}^i_t,a^{-i}_t)\ne f(a^i_t,a^{-i}_t)$ for some $a^{-i}_t$, then the probability that all other players detect such a deviation is at least $\eta ^3/(n|A|)$.

• (2) At each stage t where player i observes $a_t$ and other players play according to $P_I$, it is possible that player i can profitably deviate without being detected. However, since $\eta$ is small and there are less than $\eta T$ stages like this, so that the improvement in average payoff is at most $\eta M <\varepsilon$.

• (3) At each stage t where player i observes $a_t$ and other players play according to $P_I$, if player i deviates in a way that could be detected, then the probability that all other players detect such a deviation is at least $1-\eta$. At each t of these stages with $t<T-T_0$, the average continuation payoff of player i is at most $(1-\eta )(w^i+\varepsilon )+\eta M <x^i+2\varepsilon$ with probability at least $1-\varepsilon$.

• (4) The play in the last $T_0$ stages has little impact on the T-stage average payoff: $T_0\le \varepsilon T$.

  Finally, we need to show that along the main path of play, at those stages where each player i only observes $a^i_t$, profitable deviations (measured by the T-stage average payoff) lead to punishment with high probability. Fix $i\in N$. Define the random variable $Z\in \{0,\ldots , T\}$ as the number of stages where player i is deviating in a way that could have been detected by all other players yet not. Let z denote the realized value of Z. The idea in the following steps is that a small z does not give player i high payoffs and a large z occurs with small probability:

  • (1) $z\le \varepsilon T$. For each such z, there are at most $z+1$ stages where player i deviates yet other players do not switch to the punishment phase, and we also drop the last $T_0-1$ stages. That is, there are at most $T_0+z$ stages where player i’s stage game payoff is at most M. For any of the $T-T_0-z$ stages left, either player i deviates to indistinguishable actions, which is not profitable by assumption, or all other players are at the punishment phase, which will be effective with probability at least $1-\varepsilon$. In the former case, player i’s expected stage game payoff is bounded above by

    $$\begin{aligned} (1-\eta )x^i+\eta M \le w^i+\varepsilon . \end{aligned}$$

    In the latter case, player i’s expected average payoff is at most $x^i+\varepsilon$ with probability at least $1-\varepsilon$.

    Therefore, for each $z\le \varepsilon T$, player i’s average payoff is bounded above by

    $$\begin{aligned} \frac{1}{T}\left[ (T_0+z)M + (T-T_0-z) \left( w^i+\varepsilon \right) \right] \le w^i+(1+2M)\varepsilon , \end{aligned}$$

    with probability at least $1-\varepsilon$.

  • (2) $z > \varepsilon T$. For each such z, the probability that $Z=z$ is at most

    $$\begin{aligned} \left( 1-\frac{\eta ^3}{n|A|}\right) ^z \le \left( 1-\frac{1}{n|A|\sqrt{T}}\right) ^z \le \exp \left( -\frac{z}{n|A|\sqrt{T}}\right) \le \exp \left( -\frac{\varepsilon \sqrt{T}}{n|A|}\right) . \end{aligned}$$

    Therefore, the probability that $z\ge \varepsilon T$ is at most

    $$\begin{aligned} T\exp \left( -\frac{\varepsilon \sqrt{T}}{n|A|}\right) \le \varepsilon . \end{aligned}$$

    Since M is an upper bound on the stage game payoffs, the unconditional average payoff is at most $\varepsilon M$.⁴¹

  • (3) Summing up the two cases, player i’s average payoff is bounded above by

    $$\begin{aligned} (1-\varepsilon )(w^i+(1+2M)\varepsilon )+\varepsilon M + \varepsilon M \le w^i + (1+4M)\varepsilon \le w^i+\varepsilon _0. \end{aligned}$$

This completes the proof.

1.3 Proof of Proposition 4.1

The proof is similar to and generalizes the construction in the proof of Proposition 5.2. Here we give a detailed sketch. Specifically, fix any $w\in JR$ with $w=u(p)$ for some $p\in {\mathcal {P}}$, and consider the correlation device ${\bar{c}}$ defined in Sect. A.2. Because now each player observes private signals along the play, a public cheap-talk phase is added at the beginning of each stage, where players broadcast their observed private signals at the previous stage.

The timing of this extended game is as follows. Before the game starts, the correlation device sends to each player sequences of recommended actions (or action profiles occasionally) and encoded instructions for punishment as in Sect. A.2. At each stage t, before choosing their actions $a_t$, players simultaneously and publicly announce the private signals observed at stage $t-1$, i.e., $m_t=s_{t-1}$. In equilibrium, players always announce the true signals and follow the recommended actions at every stage. When a deviation is detected, i.e., one of the reported signals is not compatible with the recommended action profile, players switch to the punishment phase. The approachability strategy profile is implemented in the punishment phase as in Sect. A.2. At each stage of the punishment phase, players first simultaneously broadcast the observed private signals from the previous stage and the encoding keys for the punishment actions at this stage; then they choose actions based on these encoding keys as well as the instructions sent by the correlation device.

Now we verify that the above strategy profile is a Nash equilibrium of the game extended by the correlation device and public cheap talk. First, since $p\in {\mathcal {P}}$, no undetectable deviation is profitable by the definition of ${\mathcal {P}}$. Second, following the argument in Sect. A.2, any $\varepsilon$-profitable deviation of player i, which either leads to an inconsistent report by this player or makes other players observe an inconsistent signal, will be detected by all other players with high probability if the time horizon T is large enough. Finally, consider the punishment phase, which again lasts for at least $T_0$ stages. Since $x\in JR$, if all players follow the punishment instructions, then player i’s expected payoff is not more than $w^i+\varepsilon$. On the other hand, player i may deviate during the punishment phase, i.e., he could (1) report a signal different from his observation, (2) choose an action different from the suggestion by the correlation device, or (3) announce an invalid encoding key. By the definition of the approachability strategy, the first two types of deviations cannot yield player i more than $w^i+\varepsilon$; by the construction of the encoding scheme, the last type of deviation will be detected by all other players with high probability, which triggers a correlated minmax punishment of player i. Therefore, player i’s expected payoff is bounded above by $w^i+\varepsilon$ during the punishment phase.

1.4 Proof of Theorem 4.2

Fix any $w\in JR$ with $w=u(p)$ for some $p\in {\mathcal {P}}$. We adapt the construction in the proof of Proposition 4.1 (see Sect. A.3) to obtain a Nash equilibrium of the extended game with almost public cheap talk. Specifically, we replace the correlation device ${\bar{c}}$ in Sect. A.1 with a long pre-play cheap-talk phase, which generates the same set of recommendations and instructions as ${\bar{c}}$. Moreover, as in Sect. A.3, players announce their private signals from the previous stage at each public cheap-talk phase along the play.

During the pre-play cheap-talk phase, all players execute the secure MPC (see the Online Appendix) infinitely many times to generate (1) a sequence of recommended action profiles as $(\Omega _I,P_I)$, (2) a sequence of encoded punishment actions as $(\Omega _{II},P_{II})$, and (3) a sequence of correlated minmax punishment actions as $(\Omega _{III},P_{III})$. In other words, players have to talk infinitely long before they start playing the repeated game, in order to generate the required sequences of actions for the entire subsequent play.⁴² Note that for each probability distribution in $(\Omega _{I},P_{I}), (\Omega _{II},P_{II})$ and $(\Omega _{III},P_{III})$, the secure MPC runs independently. In particular, deviations at the computation for drawing action profiles with one distribution do not affect the computation for action profiles with another probability distribution. Therefore, we can have many computation phases to compute the sequences of actions as required. If a deviation is detected during the pre-play cheap-talk phase, all the non-deviating players know the identity of the deviant; they then minmax the deviant permanently.

At the end of the pre-play cheap-talk phase, each player is only informed of his own sequences of recommendations as the correlation device ${\bar{c}}$ achieves. The players then follow the recommended actions on the equilibrium path. The rest of the proof is the same as that in Sect. A.3.

1.5 Proof of Theorem 4.3

Fix any $w\in JR$ with $w=u(p)$ for some $p\in {\mathcal {P}}$. We construct a cheap-talk phase at the beginning of each stage. For each $t\ge 1$, the cheap-talk phase at stage t consists of three sub-phases:

• (1) Signal sub-phase: Each player i first broadcasts the private signals $s^i_{t-1}$ observed from the previous stage of play. In the public monitoring case, the signal sub-phase is not needed.

• (2) Message sub-phase: Each player i then publicly announce the messages $m^i_{t-1}$ that he sent and received during the computation sub-phase at stage $t-1$. This and the previous sub-phases are used to check whether deviation occurred in period $t-1$. That is, players can recover the realized recommended action profile in period $t-1$ from the messages $(m^i_{t-1})_i$ during the MPC in period $t-1$. Combined with the realized signals $(s^i_{t-1})_i$, they can check whether deviation occurred in period $t-1$.

• (3) Computation sub-phase: If no deviation is detected, then all players conduct the secure MPC to generate an action profile $a_t=(a^i_t)_{i\in N}$ from the full-support distribution ${\hat{p}}\in \Delta (A)$ defined in Sect. A.2, so that each player i only learns his own component $a^i_t$ of the profile. If a deviation is detected, then players switch to the punishment phase and conduct the secure MPC to generate the corresponding action profile from the approachability strategy; each player again only learns his own component of the profile.

Each player is supposed to report the true signals and messages and follow the computation protocol at each cheap-talk phase, and follow the computed action at each stage of play. As shown in the Online Appendix, deviations at the message or computation sub-phases are always detected with high probability; therefore, players are willing to play the prescribed strategies in the message and computation sub-phases. Moreover, by definition a player’s profitable deviations either lead to a private signal of some other player that is different from what that player expects to observe, or result in a less informative signal, which does not match the signal that the player should have observed if he were to follow the recommendations. In either case, after detecting a deviation players switch to the joint punishment using approachability strategies, in which case players’ payoffs are no more than their equilibrium payoffs. Since a profitable deviation involves deviating at many stages, it follows from the same arguments in Sect. A.4 that profitable deviations during the play are detected with high probability. Therefore, players are willing to follow the prescribed strategies during the signal sub-phase and follow the recommended actions at every stage. Hence, the above strategy profile is a uniform equilibrium of the cheap-talk extension.

1.6 Proof of Theorem 6.1

Fix any $w\in C$ with $u(p)=w$ for some $p\in \Delta (A)$. We will construct a cheap talk extension of the T-period finitely repeated game and an $\varepsilon _0$-equilibrium with payoff within $\varepsilon _0$ of w.⁴³ There are again two phases: main path and punishment. The play starts from the main path. The idea is to conduct statistical tests along the main path of play. Since signals are stochastic, along the main path players’ strategies are defined on blocks of stages. That is, within a block players choose the same mixed actions independently for a large number of stages, and then decide whether some player has deviated at the end of the block.

Specifically, let ${\tilde{p}}=(1-\eta )p+\eta {\mathcal {U}}$, for some $\eta >0$. For any $m\in \mathbb {Z}_+$, Each block $B_m$ on the main path consists of $2^{q_{m}}$ stages, where $(q_m)_{m\ge 1}$ is a increasing sequence of integers. Independently at each stage t of the block $B_m$, each player i first draws a number $l^i_t$ uniformly from $\{1,2,\ldots ,|S^i|\}$. Since each private signal $s^i_t\in S^i$ also corresponds to a unique number between 1 and $|S^i|$ via a one-to-one map $k^i:S^i\rightarrow \{1,2,\ldots ,|S^i|\}$, player i will then encode the signal $s^i$ with $l^i_t$ and get the encoded signal $k^i_t=k^i(s^i_t)+l^i_t\mod |S^i|$.⁴⁴ Then he privately sends $l^i_t$ to player $(i+1)\mod n$ and $k^i_t$ to player $(i+2)\mod n$. After this message sharing, they perform secure MPC to draw the recommended action profile from the distribution ${\tilde{p}}$. At the end of the block, players publicly and simultaneously announce the encoded messages that they received during the block, to recover the private signals in this block, and the messages sent and received for the secure MPC, to recover the recommended action profiles in this block. Players are supposed to be truthful and obedient on the main path. Then they compare the frequency of signals they theoretically expected to observe given the recommended actions with the frequency of signals they actually observed. If players follow the prescribed strategies, then the two frequencies will be very close with high probability, i.e., the distance between the two frequencies measured by the sup-norm with respect to all action profiles is less than $\varepsilon _m$, where $(\varepsilon _m)_{m\ge 1}$ is a decreasing sequence of positive real numbers converging to zero, in which case the play stays on the main path and moves to the next block.⁴⁵ If some player has profitably deviated, then it is very likely that the two frequencies will be more different, i.e., the distance between the two frequencies is larger than $\varepsilon _m$; as a result players will switch to the punishment phase.

Next consider the punishment phase. Since $w\in JR$, by Blackwell’s approachability theorem for the stochastic signal case (see Mertens et al. 2014), there exists a communication device ${\tilde{c}}^{a}=({\tilde{c}}^a_t)_{t\ge 1}$ where ${\tilde{c}}^a_t:(A\times S)^{t-1}\rightarrow \Delta (A)$ such that, for any $\varepsilon >0$, there exists $T_0$ such that for any $T>T_0, i\in N$, if all players other than i are truthful and obedient, i.e., $\tau ^j=\tau ^{j*}, \forall j\in N\setminus \{i\}$, then for any $\tau ^i$,

$$\begin{aligned} \frac{1}{T}{\mathbb {E}}\,_{{\tilde{c}}^a, \tau }\left[ \sum _{t=1}^T u^i(a_t) \right] \le w^i+\varepsilon . \end{aligned}$$

Whenever all players agree that a deviation has occurred, the play switches from the main path to the punishment phase immediately. At the beginning of each stage in the punishment phase, players first publicly announce the private signals from the previous stage and then perform secure MPC to generate the recommended action profiles according to ${\tilde{c}}^a$. If a deviation is detected during the computation, then the deviant is identified and is punished by correlated minmax strategies, which again can be implemented by cheap talk.

Finally, we briefly explain that the above strategy profile is an $\varepsilon _0$-equilibrium of the T-period repeated game extend by cheap talk, with $T=\sum _{m=M_0}^{M_1} 2^{q_m}$ for sufficiently large integers $M_0$ and $M_1$. First, if players follow the prescribed strategy profile, then the probability that punishment happens is very small according to Azuma-Hoeffding inequality and hence players’ expected payoffs are within $\varepsilon _0$ of w. Second, notice that with the above constructed cheap talk phases, along the stages within a block each player only learns his own private signals; and only at the end of a block all players learn all the private signals and recommended actions in the block. Therefore, each player has the same amount of information as in the case with a mediator in Renault and Tomala (2004). Following the argument in Renault (2000) or Renault and Tomala (2004), one can show that a profitable deviation by any player will be detected and punished with high probability, in which case no player’s payoff is more than $\varepsilon _0$ compare to his equilibrium payoff.