Abstract

ARTICLES

A prototype implementation of a distributed satisfiability modulo theories solver in the TOOLBUS framework

David Déharbe^I; Silvio Ranise^II; Jorgiano Vidal^III

^IUFRN/DIMAp - Campus Universitário, Lagoa Nova - 59072-970 Natal, RN, BRAZIL - david@dimap.ufrn.br

^IILORIA & INRIA-Lorraine - 615, rue du Jardin Botanique - BP 101 - 54602 Villers-les-Nancy, FRANCE - Silvio.Ranise@loria.fr

^IIICEFET-RN - Av. Sen. Salgado Filho, 1559 - Tirol - 59015-000 Natal-RN, BRAZIL - jorgiano@cefetrn.br

ABSTRACT

An increasing number of verification tools (e.g., software model-checkers) require the use of Satisfiability Modulo Theories (SMT) solvers to implement the back-ends for the automatic analysis of specifications and properties. The most prominent approach to build SMT solvers consists in integrating an efficient Boolean solver with decision procedures capable of checking the satisfiability of sets of ground literals in selected theories. Although the problem of checking the satisfiability of arbitrary Boolean combinations of atoms modulo a background theory is NP-hard, there is a strong demand for high-performance SMT-solvers.

In this paper, we describe the design and prototype implementation of—to the best of our knowledge—the first distributed SMT solver. The emphasis is on providing ways to reduce the implementation effort and to make the system easily extensible. This is achieved in two ways: (a) we re-use as much as possible the code of an available sequential SMT solver and (b) we adopt the TOOLBUS architecture for rapid prototyping. The behavior of the distributed SMT solver was tested on a set of problems which are representative of those generated by software verification techniques. The experiments show the possibility to obtain super-linear speed-ups of the distributed SMT solver with respect to its sequential version.

Keywords: Satisfiability Modulo Theories, distributed computing, BDDs, haRVey.

1. INTRODUCTION

An increasing number of verification tools (e.g., software model-checkers [10, 2]) require the use of Satisfiability Modulo Theories (SMT) solvers [26] (in first-order logic) to implement the back-ends for the automatic analysis of specifications and properties. This is so because verification problems require to solve satisfiability problems (e.g., checking if an abstract trace yields a spurious concrete trace can be reduced to a satisfiability problem modulo the theory of the data structures manipulated by the program). So, the availability of efficient SMT solvers becomes a crucial pre-requisite for automating the various verification tasks. To make the situation more complex, most verification problems involve several theories (e.g., programs manipulate composite data structures such as arrays and integers for their indexes), so that methods to combine theories are also required.

There are two prominent approaches to build SMT solvers: eager and lazy. The former (see, e.g., [8]) is based on ad-hoc translations that convert an input formula (and relevant consequences of the background theory) into an equisatisfiable Boolean formula. The approach applies in principle to all theories whose ground satisfiability problem is decidable, possibly at the cost of an exponential blow-up in the translation. The approach is appealing because SAT solvers today are able to quickly process extremely large formulas. The implementation effort is relatively small for being limited to the translator—after that, one can use any off-the-shelf SAT solver. The main disadvantage of the eager approach is that it does not scale up because of the exponential blow-up of the eager translation and the difficulty of combining encodings for different theories.

The lazy approach (see, e.g., [3, 21, 15, 1, 23]), currently the most popular and most successful in terms of run-time performance, consists in building ad-hoc procedures to solve the satisfiability problem in a given background theory. The lure of these specialized procedures is that one can use for them whatever specialized algorithms and data structures are best for the background theory under consideration, which typically leads to better performance. The main disadvantage of this approach is that one has to write an entire solver for each new theory of interest. The standard way to overcome this problem is to reduce a theory solver to its essence by separating generic Boolean reasoning from theory reasoning. The common practice is to write theory solvers just for sets of ground literals (i.e. atomic formulas and their negation). These simple procedures are then integrated with an efficient Boolean solver, allowing the resulting system to accept arbitrary Boolean combinations of ground literals. The key idea is to regard the ground first-order atoms of the input formula as Boolean variables and use the Boolean solver to enumerate all its satisfying assignments. As one of these assignments can be seen as a set of ground first-order literals, the decision procedure for the background theory is used to establish its satisfiability. If one assignment is found satisfiable, we are entitled to conclude the satisfiability of the input formula. Otherwise, if all assignments are unsatisfiable, we conclude the unsatisfiability of the formula.

In the lazy approach, reasoning modules for a background theory obtained as the union of several simpler theories are modularly built by writing procedures for each component theory and then use the solvers cooperatively via well-known combination schemas (see, e.g., [25] for an overview). More recently, a new schema—called Delayed Theory Combination, DTC—combining the procedures directly with the Boolean solver has been put forward and shown more efficient than SMT solvers based on the classic combination schemas [6].

The problem of checking the satisfiability of arbitrary Boolean combinations of atoms modulo a background theory is NP-hard as it subsumes the problem of checking the satisfiability of Boolean formulas. In spite of the computational complexity, there is a strong demand for high-performance SMT-solvers to make a range of verification techniques viable in practice. In this paper, we describe the design and prototype implementation of—to the best of our knowledge—the first distributed SMT solver based on the lazy approach. The emphasis is on providing ways to reduce the implementation effort and to make the system easily extensible. This is achieved in two ways. First, we re-use as much as possible the code of an available sequential SMT solver. We choose [15] as the sequential SMT solver since the first two authors are two of its main developers and hence we were more familiar with its design and implementation structure. Second, we adopt the TOOLBUS [4] architecture in order to obtain a robust implementation of a distributed SMT solver in a short time. This choice is justified by the fact that the TOOLBUS allows one to write wrappers around a piece of available code and turn it into a module that can be executed concurrently in a distributed environment by exchanging messages according to a protocol that can be easily specified by means of a suitable script language. To show the flexibility of the proposed distributed architecture, we show how a distributed version of DTC can be obtained without too much implementation effort.

After the implementation of the distributed version of was completed, its behavior was tested on a set of problems which are representative of those generated by software verification techniques, where the sequential version of the solver was already successfully used (see [15, 11]). The main contribution of this work is to show the possibility to obtain super-linear speed-ups for the distributed version of the SMT solver with respect to its sequential version. This is possible by exploiting the following simple observation, which can be generalized to many situations in distributed computing: if we consider n > 2 Boolean assignments at the same time and invoke n instances of the available procedure on their first-order counterparts, we can hope to significantly reduce the overall running time. As it is well-known in distributed computing, to make this observation practical, special care must be put into choosing a suitable number n of instances of the decision procedures. Another interesting question is whether there is an "optimal" way to choose the n different assignments to be solved in parallel. Our experiments suggest that the best heuristic in this respect consists of randomly selecting the n Boolean assignments. As we will see, this is the only way to obtain the aforementioned super-linear speed-up.

This article extends a paper previously published in the proceedings of the VIII Brazilian Symposium on Formal Methods [17]. It provides additional technical details on the TOOLBUS framework and gives more information on the algorithm and experimental results. Moreover Section 3.4 is completely original and describes a possible evolution of the distributed SMT-solver to handle new advancements in verification, such as delayed theory combination [6].

Plan of the paper. Section 2 provides the background notion on the TOOLBUS (Section 2.1) and the lazy approach to SMT solving (Section 2.2). Section 3 explains how a distributed SMT solver can be derived from its sequential version (Section 3.1 describes the architecture, Section 3.2 the protocol, and Section 3.3 the internal workings of the various concurrently executing modules) as well as the distributed version of DTC (Section 3.4). Section 4 reports on the experiments. Finally, Section 5 concludes and sketches our future directions of research.

Results presented in this paper have been previously published in [17]. This paper adds on the proceedings version [17] by giving a more detailed presentation of the TOOLBUS framework for the implementation of heterogeneous, distributed systems, in addition to fixing several small mistakes and improved writing. A more fundamental contribution found in this paper over [17] is the inclusion of the treatment of DTC in the already existing framework. This is a strong indicator of the flexibility of the proposed approach to incorporate further technical improvements in the implementation of sequential SMT solvers into the proposed distributed algorithm.

2. BACKGROUND

There are two key ingredients to our implementation of a distributed SMT solver: the TOOLBUS [13] and [15]. The former allows us to build a prototype and experiment with concrete benefits of using the TOOLBUS; the latter is our implementation of a lazy (and sequential) SMT solver based on the combination of a Boolean solver and satisfiability procedures for some theories, such as equality, lists, arrays, and their combination. We provide some background notions both on the TOOLBUS and lazy SMT-solving to make the paper self-contained.

2.1. THE TOOLBUS

The construction of (heterogeneous) distributed systems is a challenging task, both from a design and implementation points of view. The TOOLBUS [13] provides an elegant solution to implement robust distributed and heterogeneous systems, using a variation of the Algebra of Communication Processes [5] as a script language to describe the protocol between the different components and a uniform, generic, data type (called ATerms, see below for more details). The TOOLBUS framework has two entity classes: processes and tools. The former are responsible for coordinating actions and synchronizing communications throughout the system, while the latter are the components of the systems which are ultimately responsible for the actual computational work and may be written in a number of different programming languages. All data exchanges are performed using ATerms [29], a term-like data type that features maximal sharing of sub-terms, resulting in a compact representation of symbolic expressions and efficient comparison operators.

The TOOLBUS encourages a methodology whereby programmers write a TOOLBUS script describing the intended interaction protocol between tools. Scripts are then directly executable by the TOOLBUS interpreter. Moreover, the TOOLBUS suite provides utilities to automatically generate the interfaces that each tool has to implement to participate in the protocol. Currently, there is support for two programming languages (JAVA and C) and several adapters are available for Perl, ASF+SDF, UNIX scripts, etc.

A TOOLBUS script defines an interaction protocol between different tools by means of a composition of processes. We will adopt the term tool bus to denote an instance of such a protocol.

The TOOLBUS scripting language provides the classic process algebra constructs: + for choice, . for sequential composition, || for parallel composition, * for repetition, for guarded command, and to represent deadlock. The operator dynamically creates process instances; finally, and spawns and aborts the execution of a tool instance, respectively.

The execution of a tool can be dispatched from within the tool bus, with the command or it can be started externally by issuing a connection request to the tool bus, that may accept such connection requests with the command.

Once connected to the tool bus, tools do not communicate directly. Instead, communication channels are established between tools and processes, or between processes. The communication between processes can be synchronous, using matching and commands, or asynchronous, with the broadcasting command, which can be received using the command from all processes that had previously issued a command on the corresponding label.

Processes use handshaking to communicate with tools. The tool-to-process communication can be either a message (notifies an event) or a message (sends a value), while the communication from a process to a tool can be one of the following three commands: (evaluation request), (action request, i.e. without return value), or (acknowledges a previous event).

All the communication commands may have typed parameters and return results. To distinguish between input and output parameters, the former are decorated with the symbol as suffix.

Let us illustrate all these concepts with a simple example.

Example 1Figure 1 presents the definition of a tool bus gluing a graphical user interface and the (command-line) UNIX calculator . The tool bus is composed of these two tools and the related (two) processes. The first process, named (cf. lines 01-11) mediates requests for numeric computations to a command-line calculator: it spawns the tool and assigns the corresponding identifier to (line 04), then repeatedly receives a message on channel and assigns its value to (line 05), sends it for evaluation to the tool (line 06), gets the answer in variable (line 07), forwards it along channel (line 08), and also broadcasts it to any interested party (line 09). The second process, called (cf. lines 12-24), is responsible for getting expressions to be calculated from the user: it spawns a graphical user-interface tool (line 15), and then repeatedly receives expressions from the interface, transmits them to the calculator, gets the corresponding value, and forwards it back to the user interface (lines 16 to 20), until it gets a command to quit the application (line 21). Finally, the last three lines associate the TOOLBUS actors with actual programs.

2.2. THE LAZY APPROACH TO SMT-SOLVING

Before getting to a detailed explanation of the classic SMT-solving algorithm, we recall the main concepts and properties of first-order logic employed in this approach.

2.2.1. First-order logic: We assume the usual syntactic (such as signature, variable, constant and function symbol, term, atom, literal, formula, sentence, and substitution) and semantic notions (such as interpretation, model, satisfiability, validity, logical consequence, and theory) of first-order logic (see, e.g., [19]). The symbol = is a predefined logical constant. If l and r are terms, then the atom l = r is an equality and the literal l ≠ r is a disequality.

In this paper, we consider first-order theories (i.e. set of first-order sentences) with equality, meaning that the predefined logical constant = is always interpreted as a reflexive, symmetric, transitive relation which is also a congruence. Let be a theory. A formula φ is satisfiable in if it is satisfiable in a model of . The satisfiability problem for amounts to checking whether any given finite and quantifier-free conjunction (or, equivalently, finite set) of literals is satisfiable in . A satisfiability procedure for is an algorithm capable of solving the satisfiability problem of . The satisfiability of a quantifier-free formula φ can be reduced to the satisfiability of several conjunctions of literals by converting φ to disjunctive normal form (DNF), splitting on disjunctions, and then solving the resulting satisfiability problems. For this reason, by abuse of language, we talk about satisfiability problem when considering the problem of establishing the satisfiability of arbitrary quantifier-free formulas. Indeed, the conversion to DNF may result in an exponential blow-up of the size of the formula. A much more efficient way, in practice, to tackle this problem is described in the rest of this section. The problem of checking the satisfiability of arbitrary quantifier-free formulas in is NP-hard, as it subsumes the problem of checking the satisfiability of Boolean formulas.

2.2.2. SMT-solving: An SMT solver takes as input a quantifier-free formula φ and it is capable of checking its (un-)satisfiability in a certain theory . The lazy approach to build SMT solvers consists of using a Boolean enumerator and a decision procedure for . The former enumerates all satisfying Boolean assignments of φ (its atoms are considered as Boolean variables) and then the decision procedure checks if each assignment is satisfiable (or not) in . Indeed, several refinements are needed to make this integration to work efficiently in practice as we may end up invoking the decision procedure exponentially many times in the number of atoms of φ. In the following, we give an abstract view of a lazy SMT solver based on the enumeration of (total) Boolean assignments and the use of "theory conflict clauses" to prune the search space of the Boolean solver. Although this will be sufficient to understand the rest of the paper, we point the reader to, e.g., [6] for more realistic versions of the lazy architecture.

We assume the availability of two simple functions. The first is the propositional abstraction function, i.e. a bijective mapping from atoms to Boolean variables, which is homomorphically extended to arbitrary Boolean combination of atoms. The second is the refinement function, which is the inverse of . In the following, sat and unsat denote the possible values returned by a satisfiability procedure; β^b is used to denote a Boolean assignment; π is used to denote a conjunction (or, equivalently, a set) of literals and π^b its Boolean abstraction; in general, we use the superscript b to denote Boolean expressions.

Figure 2 presents a simple version of the core algorithm underlying any lazy SMT solver. The algorithm enumerates the (total) truth assignments for the Boolean abstraction of φ and checks for satisfiability in . It concludes satisfiability if an assignment is satisfiable in or returns unsatisfiable, otherwise. The function call (φ^b) establishes whether the Boolean formula φ^b is satisfiable or not. The function returns a total assignment to all atoms in φ or equivalently, to all Boolean variables in φ^b = (φ). The function call (β) detects if the set β of literals is satisfiable in the background theory ; if so, it returns (sat, ); otherwise, it returns (unsat, π), where (π) ⊆ β^b and π is an unsatisfiable set in , called a theory conflict set. The negation of (π) is a theory conflict clause and it is used to eliminate—at once—all (total) Boolean assignments sharing the same Boolean abstraction of the theory conflict set π. Indeed, when (π) = β^b, we end up enumerating all possible Boolean assignments of φ^b and performances are likely to be poor. So, in practice, computing conflict sets is the key to speed-up the performances of .

3. DISTRIBUTED SMT SOLVING

In order to design a distributed version of the algorithm in Figure 2, a simple—yet promising—idea would be to modify so as to consider n > 2 Boolean assignments at the same time and let n instances of the decision procedure (encapsulated in the function ) be executed concurrently on the n Boolean assignments. In this way, we can hope to significantly reduce the overall running time of the reasoning system. Indeed, to make this observation practical, particular care must be put in choosing a suitable value for n. In the remaining of this section, we develop this idea by using the TOOLBUS architecture.

3.1. OVERVIEW OF THE ARCHITECTURE

The first step in designing the distributed version of is to identify the various tools (i.e. the modules in TOOLBUS terminology) that can be distributed over a network. In doing this, good software engineering suggests to take into account the following desiderata. First, the distributed version shall be scalable so as to take advantage of having a large or a small amount of available computing resources. Second, the communication overhead in the distributed version shall be minimal; this requires sharing of data between the different tools to be minimal. Third, the distributed and the sequential versions of the algorithm shall have as much code in common as possible, in order to facilitate code maintenance when new features are implemented or changes are made to the code.

We have therefore split into the following components, as illustrated in the interaction diagram depicted in Figure 3.

command =

The interaction between these components is modeled after two well-known architectural patterns in distributed programming: client/server and master/slave. More precisely, the interaction taking place between the modules 'Interface' and 'Boolean reasoning' follows the client/server pattern, while that occurring among the module 'Boolean reasoning' and the various instances of the module 'Theory reasoning' follows the master/slave pattern. Notice that the module 'Boolean reasoning' plays two roles at the same time: it is the server when interacting with the client 'Interface' and it is the master for each instance of the slave 'Theory reasoning.'

3.2. DESCRIPTION OF THE TOOL BUS PROTOCOL

This section describes in detail the processes that form the interaction protocol between the different modules of the distributed version of the SMT-solver. The description starts with the main process and goes on with the different component sub-processes.

3.2.1. The main process: The top-level process () is depicted in Figure 4 and it corresponds to the box labelled with 'Boolean reasoning' in Figure 3. spawns an instance of 'Boolean reasoning' (line 04)—which is a in the sense it plays both roles (see above)—and then repeatedly (lines 05-08) shows one of the following two behaviours:

• the process (line 05 and Figure 5) is activated when a connection request is received from an instance (slave) of 'Theory reasoning' and

• the process (line 07 and Figure 7) is initiated on reception of a new quantifier-free formula from (the client) 'Interface.'

Finally, when emits a quit event (i.e. when it has considered all assignments for satisfiability), the process successfully terminates (lines 09 and 10).

3.2.2. Establishing new master-slave connections: The process (cf. Figure 5) is a sub-process of and it is responsible for the connection between a new instance of a (i.e. an instance of the module 'Theory reasoning') and the instance of the tool .

It sequentially receives a connection request from (line 04), then creates an instance of the process (described below) which is attached to , gets the corresponding process identifier (line 05), and it asynchronously notifies the master that a new slave is available, sending a message parametrized with the value of the process identifier (line 06). From the architectural viewpoint of Figure 3, the execution of this process corresponds to the creation of a new box labelled with 'Theory reasoning' (at the bottom of the Figure) and the establishment of a communication channel between this box and the box marked 'Boolean reasoning.'

3.2.3. Interface with instances of the first-order reasoning tool: The process (cf. Figure 6) is the wrapper around the decision procedure for the background theory , which is required for this to be used in the TOOLBUS architecture. takes as input the identifier for an instance of 'Theory reasoning' and it is responsible for handling two types of events.

3.2.4. Handling proof obligations: Process (cf. Figure 7) mediates the communications and describes the interactions between the three types of tools. Most importantly, it orchestrates the various activities required to check the satisfiability of a given quantifier-free formula. This can be decomposed in two phases. First, accepts a connection request from a client tool (line 08), sends it the message , and gets the parameters of the satisfiability problem: the background theory and goal formula (lines 09 and 10), which are then forwarded to the master-server tool (line 11). The process then enters in the second phase, which consists of the loop at lines 12-19, until the process gets notified by that satisfiability checking has been completed (line 20). Then, it forwards the result (namely, or ) to the client and terminates it (lines 21-22). The body of the loop at lines 12-19 is a choice between two behaviours:

• lines 12—15: can send out a new assignment to be checked for unsatisfiability by some available slave tool . is initialized and notified with the new satisfiability problem. Afterwards, is sent an acknowledgement message, as soon as the activity of satisfiability solving has been started.

• lines 17-18: a slave tool may return the answer to a satisfiability problem that has been previously sent out. The result is then forwarded to .

The activity of satisfiability checking in the slave tools is done asynchronously with respect to the master tool. It is up to the master to create new assignments and dispatch them to slaves that have previously been connected to the tool bus. A detailed description of the internal structure of the tool is presented in the next section.

3.3. DESCRIPTION OF THE COMPONENTS

In this section, we describe the computations carried out in each of the three modules depicted in Figure 3. The module 'Interface' () performs simple operations on the input satisfiability problem, such as recognizing the background theory and the quantifier-free formula ø specified in the input satisfiability problem (besides low level activities such as parsing); it is also responsible of performing the Boolean abstraction/refinement step by invoking the functions and , respectively (see Section 2.2). The module 'Theory reasoning' () simply invokes the function (see again Section 2.2) upon reception of a message labelled .

For the 'Boolean reasoning' component (), the situation is more complex. Its main activity is to generate several distinct Boolean assignments (satisfying the abstraction of the input formula) and dispatch each one of them to a (i.e. an instance of the module 'Theory reasoning') as soon as it becomes idle, in order to minimize latency. To avoid useless computations, it is crucial that the fairly enumerate the Boolean assignments, i.e. without reconsidering the same two or more times. To meet all these requirements, a Boolean solver and some auxiliary data structures monitoring the progress in the satisfiability solving activity are used.

Efficient implementations of Boolean solvers are available, using either carefully engineered variants of the Davis-Putnam-Logeman-Loveland (DPLL) algorithm [12] (SAT-solvers) or Binary Decision Diagrams (BDD) packages. In this paper, we only consider complete solvers, i.e. based on algorithms that are always able to establish the satisfiability or the unsatisfiability of any Boolean formula SAT-solvers are based on the incremental construction of a single satisfying assignment while pruning the search space by using the inconsistencies derived by each truth value assigned, one after the other, to Boolean variables. BDDs compactly encode the disjunctive normal form of a formula, thereby representing all its possible satisfying Boolean assignments. So, SAT-solvers are apt to tackle very large satisfiability problems because they scale up much more significantly than BDDs, which suffer from exponential blow-up in space; but SAT-solvers are not designed to compute several Boolean assignments of a certain formula at once. On the contrary, it is straightforward to extend a BDD package with the capability of generating several distinct Boolean assignments as all possible assignments are readily available.

For simplicity, we have chosen BDDs as the core technique underlying the Boolean enumerator of multiple assignments required by . Also, since we are more interested in SMT-solving for software verification, the issue of the dimension of the formulas to be refuted is less important as their Boolean structure is usually less important than for other (e.g., hardware) verification problems.

In order to avoid reconsidering several times the same Boolean assignment, it is sufficient to assume that the function (see Section 2.2) is fair, i.e. it does not return twice the same Boolean assignment, and that the maintains some data structures to track the progress in the satisfiability solving activity: a set S containing the available s, a set B ⊆ S containing those slaves which are busy, the background theory of the current satisfiability problem, and a BDD of a formula φ^b representing the assignments that have not yet been checked for unsatisfiability.

The pseudo-code of the 'Boolean reasoning' module is shown in Figure 8. The set of s is initialized and the event handling loop is started, namely , which is supposed to handle the following messages. We omit the pseudo-code for as its implementation is automatically generated by the TOOLBUS.

• The message happens when a new instance s of the the 'Theory reasoning' component connects to the tool bus (line 06 of Figure 5). The routine handling this message simply adds the instance s to the set of available slaves (see Figure 9).

• The message occurs when the 'Interface' component () has sent a new satisfiability problem to the tool bus (line 11 of Figure 7). The routine handling this message is given in Figure 10. First it initializes the satisfiability problem (by considering the background theory and the quantifier-free formula φ) and then invokes the auxiliary routine , whose pseudo-code is depicted in Figure 11, which is responsible for dispatching new assignments to idle s. This is implemented by the loop choosing an idle (if any) and considering a not yet dispatched assignment (by re-using the function of the sequential version of the SMT-solving algorithm, cf. Fig. 2), if any Boolean assignments are left to be considered (by invoking the function ). Finally, after updating the set of busy s, the new satisfiability problem is dispatched to the selected (idle) . Otherwise, no assignment has been shown unsatisfiable (see message below) and the result is that the input formula is unsatisfiable in .

• The message happens when a s returns the result of invoking the function on a dispatched assignment (line 11 of Figure 6), i.e. either or . The routine handling this message is given in Figure 12. First, the slave s becomes idle (i.e. it is deleted from the set B of busy s). Then, the outcome of a satisfiability problem is tested: if the assignment is unsatisfiable, then the (Boolean abstraction of the) conflict set π is used to prune φ^b (similar to what was done in the sequential case in Fig. 2) and a new Boolean assignment is considered (cf. the invocation to the function , see also Fig. 11 above). Otherwise, the whole satisfiability solving activity is terminated with a message. In this second case, the conclusion is that the formula φ is satisfiable in .

3.4. FLEXIBILITY OF THE APPROACH: DELAYED THEORY COMBINATION

Program verification often requires to verify proof obligations that are expressed as first-order formulas over a combination of diverse theories that reflect the different data and specification constructs used in program design and implementation artifacts: fragments of arithmetics, set theory, array theory, etc. From a practical viewpoint, such formulas are in expressed in a theory resulting from the union of the component thoeries.

The proposed architecture for the distributed version of an SMT solver can easily be adapted to handle satisfiability problems in combinations of theories, i.e. when the background theory is obtained as the union of several simpler theories. In the following, we assume that is the union of two theories ₁ and ₂; the generalization to more than two theories is straightforward. In order to precisely talk about combination of satisfiability procedures, we need to introduce some basic notions about unions of theories (for more details, the reader is pointed to, e.g., [25]).

3.4.1. Combination: Let be a (finite) set of variables. An identification over is an idempotent substitution from to . Any identification σ over defines a partition of and identifies all the variables in the same equivalence class of the partition with a representative of that class. If σ is an identification over , then () is the conjunction of equalities (disequalities, resp.) of the form¹ 1 We write the application of a substitution in post-fix form. xσ = yσ (xσ ≠ yσ, resp.) for x,y ∈ . Intuitively, expresses the fact that any two variables identified by an identification σ must take identical value, while expresses the fact that any two variables not identified by σ must take distinct values. Hence, the formula ∧ (denoted with ) faithfully represents the identification σ over .

A term in the union of

In order to re-use the satisfiability procedures for

The non-deterministic version of the Nelson-Oppen combination schema [24] can be summarized as follows:

The termination of the schema is obvious as there are only finitely many identifications to be considered over a finite set of interface variables. Its soundness can be shown under the assumption that

The non-deterministic schema above can be turned into a deterministic procedure by case-splitting on the formulas faithfully representing all the identifications over the shared variables and the satisfiability procedures for the component theories can be used to check the satisfiability of each pure conjunction of the split. Indeed, the resulting procedure is far from efficient. A much more efficient way, in practice, to tackle this problem has been proposed in [6] and it is briefly described in the rest of this section.

3.4.2. Delayed Theory Combination: The Delayed Theory Combination (DTC) schema does not require the direct combination of the procedures for ₁ and ₂ as in the Nelson-Oppen schema. Instead, DTC couples the available Boolean solver with each available procedure separately. The Boolean solver enumerates the Boolean assignments of the input formula j extended with a conjunction of equalities and disequalities, faithfully representing one of the possible identifications over the interface variables. This is obtained by adding to the set of Boolean variables abstracting the atoms in φ, the set of all equalities that can be formed over the shared variables. (It is possible to generalize the computation of the set of shared variables from conjunctions of atoms to arbitrary quantifier-free formulas, see [6] for details.) As a consequence, any Boolean assignment is of the form ∧ ∧ ^b, where β_i is a conjunction of atoms in the theory _i, for i = 1,2, and is a conjunction of equalities and disequalities over the interface variables. Then, DTC invokes each available satisfiability procedure for _i on β_i ∧ (i = 1,2): if both return satisfiable, then the satisfiability of φ in the union of ₁ and ₂ is derived; otherwise, at least one of the procedures detected an unsatisfiability, the (Boolean abstraction of the) negation of the corresponding conflict set is added to φ^b, and a new Boolean assignment of the resulting Boolean formula is considered, if any. When the Boolean formula becomes unsatisfiable, φ is declared unsatisfiable in the union of ₁ and ₂. DTC can be implemented by a simple modification of the core algorithm underlying an SMT solver depicted in Figure 2. The argument underlying its correctness is an adaptation of the correctness of the algorithm in Figure 2 and the completeness of the non-deterministic Nelson-Oppen schema (see [6] for details).

3.4.3. Distributed Delayed Theory Combination: As it is possible to derive an implementation of DTC from the algorithm for SMT solving in one theory, it is straightforward to derive a distributed version of DTC by modifying the distributed SMT solver described above (cf. Figure 3). The only delicate point is the handling of the messages returned from the 'Theory reasoning' modules, as they run asynchronously and there is no guarantee that their results are received in the right order by the 'Boolean reasoning' module. More precisely, we must make sure that the answers to the satisfiability problems β₁ ∧ and β₂ ∧ deriving from the same assignment β₁∧ β₂ ∧ are considered together. This is done by decorating each β_i ∧ with the same time-stamp t and propagating t to the result returned by the satisfiability procedure for _i (i = 1,2). Some more details follow.

The 'Boolean reasoning' module is extended so as to compute a purified version of the input formula φ, determine the set of interface variables, and form all possible equalities over . Then, the Boolean solver is instrumented so as to generate Boolean assignments over the atoms in the input formula and the equalities over . When an assignment β₁ ∧ β₂ ∧ is considered, it is split into two conjunctions of i-pure literals (namely, β₁ ∧ and β₂∧ ), each dispatched to an available 'Theory reasoning' module together with the identifier of the related theory _i, for i = 1,2, and a common time-stamp t (i.e. the 'Boolean reasoning' module generates messages containing triples of the form (_i, β_i ∧ , t)) for i = 1,2).

Each 'Theory reasoning' module is associated to one of the background theories

The 'Boolean reasoning' module maintains a set of time-stamps associated to sat results received from 'Theory reasoning' modules. As soon as a sat message is received, the corresponding time-stamp t is tested for membership in : if t is in , then the satisfiability of the input formula φ is derived; otherwise, t is added to . If an unsat message is received, it is also tested for membership in : if t is in , then t is deleted from ; otherwise, t is discarded. After the reception of an unsat message, independently of the fact that t is in or not, the (Boolean abstraction of the) negation of the associated conflict set (for i = 1,2) is passed to the Boolean solver.

4. EXPERIMENTS

As suggested in Section 3, we have implemented the distributed SMT solver by re-using the available functionalities of a sequential SMT solver. Our choice was [15], which is based on the algorithm depicted in Figure 2 and it is being developed by the first two authors and Pascal Fontaine. In , one is free to choose the implementation of the functions and (cf. Figure 2) by either BDDs [7] (the CMU BDD package by Long) or a SAT-solver (MiniSAT [18]). A peculiarity of is that the implementation of the function is done by an automated (first-order) theorem prover, called the E prover [28], which is combined with a decision procedure for a fragment of Linear Arithmetic via the Nelson-Oppen combination schema [24]. The availability of an automated prover for full first-order logic makes it possible to handle also formulas containing quantifiers, as shown in [11], and this is particularly useful for software verification. So far, has been successfully applied to the verification of pointer-based programs [27], B specifications [11], static checking of automatically generated code for aerospatial applications [16] as well as array programs [14].

The distributed version of has been developed along the lines of Section 3 by using TOOLBUS version 0.24, the available BDD package in for the 'Boolean reasoning' module, and the E prover as the 'Theory reasoning' module. It took us 20 man hours to come up with the first running prototype which consists of 105 lines of TOOLBUS and 5900 lines of C code, mostly re-used from the sequential version of , whose source is 5300 lines of C code.

Experiments have been carried out over a 10 Mbps Ethernet network of desktop workstations in a normal working environment, where all the tools had to compete for resources with other user processes. The computing nodes had the following configuration: 2.0GHz Pentium IV processor, with 512MB of RAM, operated under Linux.

As benchmark problems, we have selected 50 proof obligations requiring several interactions between the Boolean solver and the satisfiability procedure from previous experiences with the sequential version of (namely, proof obligations generated from the B methodology [11], the verification of pointer-manipulating programs and Burns protocol [27]). The goal of our experiments was to understand if it were possible to significantly reduce the overall running time by invoking several instances of the satisfiability procedure concurrently.

To understand the rationale underlying the experiments, the following observation is useful. From Figure 2, recall that function chooses an assignment β which is checked for (un-)satisfiability in the background theory. If β is unsatisfiable, a conflict set π is computed and used to prune the search space by adding the Boolean abstraction of np_sumbol_i to the input formula. In this way, all assignments satisfying π are no longer considered.

When using a BDD, to choose an assignment, recursively traverses it from the root to a true leaf (this can be done in time linear in the number of atoms occurring in the BDD). We have considered four ways to perform this traversal, according to how the sub-tree to traverse is chosen:

• the rightmost sub-tree is chosen (this is the simplest heuristic to implement and it is the one implemented in the sequential version of ),

• either the right or the left sub-tree is randomly selected,

• the zigzag heuristic chooses alternatively the right or the left sub-tree, and

• in the alternate heuristics, the traversal proceeds by alternatively following the rightmost or the leftmost sub-tree.

Indeed, if two similar assignments are checked for satisfiability, one after the other (and found unsatisfiable), it is reasonable to expect that their conflict sets be similar (if not identical), thereby pruning essentially the same part of the search space of the Boolean solver. In a distributed environment, it seems desirable to choose a heuristics for that allows us to consider "different" assignments concurrently in the hope that their conflict sets will prune different portions of the search space, thereby (furtherly) reducing the execution time.

Guided by this observation, we present below two experiments. The former (Section 4.1) considers the sequential version of and studies its behaviour according to the four heuristics above to implement . The latter (Section 4.2) consists of repeating the previous experiment with the distributed version in order to investigate the advantages of having several instances of the satisfiability procedure running concurrently. Indeed, the first experiment is conducted so as to be able to compute the speed-up of the distributed version and it is not very interesting per se.

4.1. SEQUENTIAL CASE

The graph in Figure 13 depicts the number of assignments (y-axis) that need to be considered for each of the four choice heuristics on the sequential version of (the x-axis shows the identifiers of the 50 proof obligations in our benchmark). 'Ri' stands for rightmost, 'Ra' for random, 'Zz' for zigzag, and 'Al' for alternate heuristic. As it is apparent, no approach dominates the others. We conclude that the assignment choice heuristics has no measurable impact on the performances of the sequential version of . Again, we emphasize that these results are not interesting per se but rather as a reference for the next experiment.

4.2. DISTRIBUTED CASE

We repeated the previous experiment with the distributed version of in two configurations: (up to) two or four slaves. For each of the 50 proof obligation in the benchmark, we measured the number of Boolean assignments that were considered with two and four slaves, using the four possible assignment choice heuristics (namely, rightmost, random, zigzag, and alternate). To measure the impact of distributing the work-load, we computed the speed-up as follows:

h is one of 'Ri', 'Ra', 'Zz', and 'Al', n is the number of instances of the 'Theory reasoning' modules (i.e. n = 2 or n = 4), is the number of Boolean assignments considered in the sequential version of with h as assignment choice heuristic (cf. the values on the y-axis of the graph in Figure 13), and is the number of Boolean assignments considered in the distributed version of with h as assignment choice heuristic. Notice that the value of is averaged over several runs of the distributed version for each proof obligation. This is necessary because of the concurrent executions of the instances of the 'Theory reasoning' and the essentially random nature of low-level network protocols. In fact, when two instances of the 'Theory reasoning' module deliver their results concomitantly, 'Boolean reasoning' might receive them in a different order for two different executions on a given proof obligation. As a consequence, different sequences of conflict sets might be considered, thereby causing the 'Boolean reasoning' module to consider different assignments at the next iteration, which ultimately explains the difference in the number of generated assignments. In our experiments, we repeated each verification three times and report an average value for the number of Boolean assignment per proof obligation in the benchmark. The speed-ups are reported in Figure 14. For reference purposes, each diagram also contains two lines, corresponding to no speed-up (horizontal solid line, with = 1), and linear speed-up (diagonal dotted line, with = n). The closer the results are to the linear speed-up, the more efficient is the distributed version of . We can clearly visualize that, on our benchmark, the random assignment choice heuristic performs better than the three others (which present similar behaviors). A plausible explanation is that rightmost, zigzag and alternate tend to generate similar successive assignments (in the case of alternate, every other assignment is generated on the same "region" of the graph representing the BDD), which have a higher probability to obtain the same (or similar) prunings of the search space, while this is not the case for random. Indeed, while in the sequential version, the n + 1-th assignment is generated after the pruning of the search space with the conflict set generated from the n-th assignment, this is not necessarily the case in the distributed version. Therefore approaches that tend to generate different consecutive assignments will always tend to perform better in the distributed version.

Finally, note that, in some experiments, the distributed SMT solver achieves super-linear speed-ups. This happens when the conflict set associated to an assignment is so general that it prunes a (relatively) large part of the search space. This may happen for the class of formulas such that the value of a relatively small subset of the atoms causes unsatisfiability. Our experiment shows that the probability of achieving super-linear speed-ups is larger for the random assignment choice heuristic.

5. CONCLUSION

We have presented a distributed version of a lazy SMT solver, based on . This allows us to distribute the work-load over a network of workstations. The feature, unique to , that BDDs can be used to represent the Boolean structure of the formulas has greatly simplified the implementation of the distributed version.

The distributed algorithm has been prototyped by re-organizing the architecture of the sequential version of into three modules whose execution is concurrent: 'Interface,' 'Boolean reasoning,' and 'Theory reasoning.' The last module can be instantiated an arbitrary number of times in the distributed version. The implementation has been realized using the TOOLBUS architecture: a process algebra script describes the module interaction protocol and it is used to generate the code responsible for the communication and synchronization, as well as the interfaces that the modules implement to participate in the interaction. The extension of the architecture to handle a combination of background theories by using a distributed variant of the DTC schema show the flexibility of our approach.

Experiments on a set of representative proof obligations of software verification problems show the possibility to obtain super-linear speed-ups of the distributed SMT solver over its sequential version, thereby showing the viability and the benefits of the proposed architecture.

In the future, we plan to use the interaction protocol as a basis to a grid-based approach to SMT solving (see, e.g., [9] for a grid-based approach to Boolean solving). We also envision to extend or adapt the proposed protocol so that the 'Boolean reasoning' module can be implemented by a SAT-solver, based on some existing parallel implementations of DPLL algorithms (see, e.g., [22, 20]).

ACKNOWLEDGMENTS

This work was partially funded by INRIA/CASSIS project and CNPq projects 490084/2005-2, 477960/2004-9 and 307597/2006-7.

Publication Dates