Multi-player games with LDL goals over finite traces

https://doi.org/10.1016/j.ic.2020.104555Get rights and content

Abstract

Linear Dynamic Logic on finite traces (LDLF) is a powerful logic for reasoning about the behaviour of concurrent and multi-agent systems. In this paper, we investigate techniques for both the characterisation and verification of equilibria in multi-player games with goals/objectives expressed using logics based on LDLF. This study builds upon a generalisation of Boolean games, a logic-based game model of multi-agent systems where players have goals succinctly represented in a logical way. Because LDLF goals are considered, in the settings we study—Reactive Modules games and iterated Boolean games with goals over finite traces—players' goals can be defined to be regular properties while achieved in a finite, but arbitrarily large, trace. In particular, using alternating automata, the paper investigates automata-theoretic approaches to the characterisation and verification of (pure strategy Nash) equilibria, shows that the set of Nash equilibria in multi-player games with LDLF objectives is regular, and provides complexity results for the associated automata constructions.

Introduction

Boolean games (BG [1]) are a logic-based model of multi-agent systems where each agent/player i is associated with a goal, represented as a propositional logic (PL) formula γi, and player i's main purpose is to ensure that γi is satisfied. The strategies and choices for each player i are defined with respect to a set of Boolean variables Φi, drawn from an overall set of variables Φ. Player i is assumed to have unique control over the variables in Φi, in that it can assign truth values to these variables in any way it chooses. Strategic concerns arise in Boolean games as the satisfaction of player i's goal γi can depend on the variables controlled by other players.

Reactive Modules games (RMG [2]) and iterated Boolean games (iBG [3]) generalise Boolean games by making players interact with each other for infinitely many rounds. As in the standard (one-shot or one-round) setting described above, there are n players each of whom uniquely controls a subset of Boolean variables and defines the achievement of a particular goal formula γi satisfied. However, in RMGs and iBGs, players' goals γi are Linear Temporal Logic formulae (LTL [4]), rather than PL formulae, which are naturally interpreted over infinite sequences of valuations of the variables in Φ; thus, in both RMGs and iBGs, such infinite sequences of valuations represent the plays of these games.2

Even though RMGs, iBGs, and conventional Boolean games are logic-based models of multi-agent systems, they capture players' goals—and therefore the desired behaviour of the underlying multi-agent systems—in radically different ways: whereas Boolean games have PL goals (which are naturally evaluated over one-round games), RMGs and iBGs have LTL goals (which are naturally evaluated over games with infinitely many rounds), encompassing two extremes of the landscape when considering repeated games. However, there are games, systems, or situations where goals evaluated after an unbounded, but certainly finite, number of rounds should, or must, be considered [5], [6].

In this paper we fill this gap and define and investigate multi-player games with goals over finite traces, which are games where players' goals can be satisfied/achieved after a finite, but arbitrarily large, number of rounds. More specifically, the goals in these games are given by Linear Dynamic Logic formulae (LDLF ) which are evaluated over finite sequences of valuations of the variables in Φ, that is, over finite traces of valuations, instead of PL formulae (as in BGs) or LTL formulae (as in RMGs and iBGs). Thus, while in game with goals over finite traces a play still is an infinite trace of valuations, the satisfaction of a player's goal may occur after an unbounded but finite number of rounds. This sharply contrasts with the case of goals given by LTL formulae (e.g., as in iBGs and RMGs), where it may be that a player's objective is satisfied only after considering the full infinite trace of valuations. This simple feature has significant implications, since rather complex automata constructions for the analysis of logics and games over infinite traces may become conceptually simpler under this new semantic (logic-based) framework. More importantly, this key observation allows one to define an automata model that exactly characterises the set of Nash equilibria in games with goals given by regular objectives.

There are several reasons to consider LDLF goals. LDLF offers great expressive power to our logic-based framework, which is indeed equivalent to monadic second-order logic (MSO). On the other hand, LTL interpreted on finite traces (LTLF ) is as expressive as first-order logic (FOL) over finite traces [7]. This, in turn, implies that, over finite traces, while with LTLF we can only describe star-free regular languages/properties, with LDLF we can describe all regular languages/properties—that is, the properties and languages that can be described by regular expressions or finite state automata. Nevertheless, the automata-theoretic approach and complexity results for solving their related decision problems are equivalent, showing that the gain in expressiveness is achieved for free. In this paper, we first define multi-player games with LDLF goals and then investigate their main game-theoretic properties using a new automata-theoretic approach to reasoning about Nash equilibria. Our technique to reason about equilibria builds on automata constructions originally defined to reason about LDLF formulae [7], [8]. Using this automata-theoretic technique we show a number of subsequent verification and characterisation results, as follows.

Firstly, we show that checking whether some strategy profile is a Nash equilibrium of a game is a PSPACE-complete problem, thus no harder than LDLF satisfiability [7]. Secondly, we focus on the NE-Nonemptiness problem—which asks for the existence of a Nash equilibrium in a multi-player game succinctly specified by a set of Boolean variables and LDLF formulae—and show that deciding if a multi-player game with LDLF goals (whether RMG or iBG) has a Nash equilibrium can be solved in 2EXPTIME, thus no harder than solving LDLF synthesis [8]. The automata technique we use for this problem also shows that the set of Nash equilibria in these games is ω-regular and can therefore be characterised using alternating automata. Thirdly, we also provide complexity results for the main decision problems related to the equilibrium analysis of these games with respect to extensions and restrictions of the initially studied framework. In particular, we show that a small extension of the goal language, which we call Quantified-Prefix Linear Dynamic Logic (QPLDLF ), has the same automata-theoretic characteristics as LDLF, and so it can be studied using the same techniques. Moreover, LDLF synthesis can be expressed in QPLDLF, ensuring 2EXPTIME-completeness.

Regarding restrictions on the general framework, we first focus on the problem of reasoning with memoryless strategies. We show, using an automata construction, that the set of Nash equilibria for this games is also ω-regular. However, an alternative procedure for this problem, not based on automata, shows that improved complexity can be obtained when compared with the standard automata techniques to reason about LDLF. Another restriction on strategies considered in the paper is the one of myopic strategies (which can be used to define all beneficial deviations in a game), in which players perform actions that are independent of the current state of the game execution. We show that games with such a restriction can be solved in EXPSPACE. We also consider the much more stable solution concept of strong Nash equilibrium, where sets of players in the game are allowed to jointly deviate, and provide an adaptation of the automata-based approach that retains the language characterisation and complexity properties of Nash equilibrium.

A key contribution of this work is that our automata-theoretic approach features two novel properties, within the same reasoning framework. Firstly, it shows that checking the existence of Nash equilibria can be reduced to a number of LDLF synthesis and satisfiability problems—generalising ideas initially used to reason about LTL objectives [9]. Secondly, our automata constructions provide reductions where not only non-emptiness but also language equivalence is preserved. This additionally shows that the set of Nash equilibria in infinite games with regular goals is an ω-regular set, to the best of our knowledge, a semantic characterisation not previously known, and which do not immediately follows from other representations of Nash equilibria—see, e.g., [10], [11], [12], [13], [14].

While studying either multi-player games or LDLF is interesting in itself, from an AI perspective, our main motivation comes from applications to multi-agent systems. In particular, it has been shown that in many scenarios, for instance in the context of planning AI systems [7], [8], while logics like LTL, or even LTL over finite traces (LTLF), can be used to reason about the behaviour of agents in such AI systems, these logics are not powerful enough to express in a satisfactory way the main features of agents in such a context. In order to illustrate the use of LDLF, and motivate even further our work, we will present an example in the next section, where some of the goals either are not expressible in LTL3 or have a more intuitive specification in LDLF than in LTL. Together with applications to planning AI systems (see [7], [8]), this is an example of another instance where one can see an advantage of using a game with LDLF goals over a game with LTL goals, instead.

Moreover, regarding previous work, while our model builds on RMGs [2] and iBGs [3], where goals are given by LTL formulae, there are at least two main differences with such work. Firstly, we study scenarios that consider memoryless and myopic strategies, for which results on iBGs have not been investigated.4 Secondly, and most importantly, the tools developed in this paper to obtain most of our complexity and characterisation results, are technically remarkably different from those used for RMGs and iBGs, specifically, with respect to the techniques used in [16], [9], [2]. To be more precise, for RMGs and iBGs the main question is reduced to rational synthesis [11], whose solution goes via a parity automaton characterising formulae of an extension of Chatterjee et al.'s Strategy Logic [10], which leads to an automata construction that can be further optimised if computing Nash equilibria is the only concern. Instead, in our case, we reduce the problem directly to a question of automata constructed in a different way. As a consequence, we provide a new set of automata constructions which do not rely on nor relate to those used in rational synthesis, i.e., those used to solve RMGs and iBGs. Our automata constructions are also different from those used by De Giacomo and Vardi in [7], [8], [17], as described next.

In [7], [8], [17], De Giacomo and Vardi study the satisfiability and synthesis problems for LDLF, with and without imperfect information. Because of the (game-theoretic) nature of these two problems, their automata constructions deal with two-player zero-sum turn-based scenarios only. Instead, in our case, we deal with multi-player general-sum concurrent scenarios. This difference leads to a completely different technical treatment/manipulation of the automata that can be initially constructed from LDLF formulae. In fact, their automata constructions and ours are the same only up to the point where LDLF formulae are translated into automata—that is, the very first step in a long chain of constructions. Moreover, since De Giacomo and Vardi study synthesis and satisfiability problems (represented by two-player games), whereas we study Nash equilibria (in the context of multi-player games), we are required to have a different technical treatment of the automata involved in the solution of the problems investigated in this paper.

Section snippets

Linear dynamic logic on finite traces

In this paper, we consider Linear Dynamic Logic on Finite Traces (LDLF ), a temporal logic introduced in [7] in order to reason about systems whose behaviour can be characterised by sets of finite traces, that is, finite sequences of valuation for the variables of the system.

Definition 1 Syntax

The syntax of LDLF is as follows:φ:=p|¬φ|φφ|φφ|ρφ|[ρ]φρ:=ψ|φ?|ρ+ρ|ρ;ρ|ρ, where p is an atomic proposition in Φ; ψ denotes a propositional formula over the atomic propositions in Φ.

The symbol ρ denotes path expressions,

NE Membership

In order to address the NE Membership problem, we first provide some preliminary results on automata. An interested reader can find definitions and more details in [24].

Consider a nondeterministic finite word automaton (NFW) A=Σ,S,s0,ϱ,F, recognizing a regular language L(A). Then consider the nondeterministic Büchi word automaton (NBW) A=Σ,S,s0,ϱ,F, where, for all σ and s, we have that ϱ(σ,s)=ϱ(σ,s), if sF, and ϱ(σ,s)={s}, otherwise.

Intuitively, the automaton A mimics the operations

NE non-emptiness and equilibrium checking problems

Now, let us study NE Non-Emptiness for both iBGF and RMGF. We first prove a result for iBGF and then how to adapt it for the case of RMGF. Also in this case we use an automata-theoretic approach. We show how, given a game G, it is possible to construct an alternating automaton ANE(G) such that ANE(G) accepts precisely the set of plays that are generated by the Nash equilibria of G. A distinguishing feature of our automata technique is that it is language preserving, that is, ANE(G) recognizes

Extensions and restrictions

We now investigate on some extensions and restrictions on the problems studied in the previous section. As a first result, we show that an extension of the LDLF language used to represent players' goals can be used to encode LDLF synthesis, studied in [8], as a NE Non-Emptiness problem. Subsequently, we restrict to two classes of strategies, namely memoryless and myopic strategies. With respect to memoryless strategies, we show that our automata-based techniques can be used to show that the set

Concluding remarks

Logic-based multi-player games revisited  In the introduction section it was pointed out that the RMG/iBG and iBGF frameworks rely on different automata techniques, and that RMGF /iBGF is better suited in certain scenarios. However, it is not the case that the RMGF /iBGF frameworks generalise iBGs. Indeed, it should be noted that they are incomparable models. For instance, while iBG cannot be used to reason about games with goals over finite traces, RMGF /iBGF cannot be used to reason about

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This paper is an extended version of [38]. We acknowledge with gratitude the financial support of the ERC Advanced Investigator grant 291528 (“RACE”) at Oxford.

References (38)

  • J. Gutierrez et al.

    Expresiveness and complexity results for strategic reasoning

  • D. Fisman et al.

    Rational synthesis

  • O. Kupferman et al.

    Synthesis with rational environments

  • F. Mogavero et al.

    Reasoning about strategies: on the model-checking problem

    ACM Trans. Comput. Log.

    (2014)
  • O. Kupferman et al.

    Synthesis with rational environments

    Ann. Math. Artif. Intell.

    (2016)
  • J. Gutierrez et al.

    Imperfect information in reactive modules games

  • J. Gutierrez et al.

    Iterated Boolean games

  • G. De Giacomo et al.

    Ltlf and ldlf synthesis under partial observability

  • A. Kučera et al.

    The stuttering principle revisited

    Acta Inform.

    (2005)
  • Cited by (2)

    1

    This work has been done while being affiliated to the University of Oxford.

    View full text