From LTL to rLTL monitoring: improved monitorability through robust semantics

Abstract

Runtime monitoring is commonly used to detect the violation of desired properties in safety critical cyber-physical systems by observing its executions. Bauer et al. introduced an influential framework for monitoring Linear Temporal Logic (LTL) properties based on a three-valued semantics for a finite execution: the formula is already satisfied by the given execution, it is already violated, or it is still undetermined, i.e., it can still be satisfied and violated by appropriate extensions of the given execution. However, a wide range of formulas are not monitorable under this approach, meaning that there are executions for which satisfaction and violation will always remain undetermined no matter how it is extended. In particular, Bauer et al. report that 44% of the formulas they consider in their experiments fall into this category. Recently, a robust semantics for LTL was introduced to capture different degrees by which a property can be violated. In this paper we introduce a robust semantics for finite strings and show its potential in monitoring: every formula considered by Bauer et al. is monitorable under our approach. Furthermore, we discuss which properties that come naturally in LTL monitoring—such as the realizability of all truth values—can be transferred to the robust setting. We show that LTL formulas with robust semantics can be monitored by deterministic automata, and provide tight bounds on the size of the constructed automaton. Lastly, we report on a prototype implementation and compare it to the LTL monitor of Bauer et al. on a sample of examples.

1 Introduction

Runtime monitoring is nowadays routinely used to assess the satisfaction of properties of systems during their execution. To this end, a monitor, a finite-state device that runs in parallel to the system during deployment, evaluates it with respect to a fixed property. This is especially useful for systems that cannot be verified prior to deployment and, for this reason, can contain hidden bugs. While it is useful to catch and document these bugs during an execution of a system, we find that the current approach to runtime verification based on Linear Temporal Logic (LTL) [14] is not sufficiently informative, especially in what regards a system’s robustness. Imagine that we are monitoring a property $\varphi$ and that this property is violated during an execution. In addition to be alerted to the presence of a bug, there are several other questions we would like to have answered such as: although $\varphi$ was falsified, was there a weaker version of $\varphi$ that was still satisfied or did the system fail catastrophically? Similarly, if we consider a property of the form $\varphi \rightarrow \psi$, where $\varphi$ is an environment assumption and $\psi$ is a system guarantee, and the environment violates $\varphi$ slightly along an execution can we still guarantee that $\psi$ is only slightly violated?

Answering these questions requires a logical formalism for specifying properties that provides meaning to terms such as weaker and slightly. Formalizing these notions within temporal logic, so as to be able to reason about the robustness of a system, was the main impetus behind the definition of robust Linear-time Temporal Logic (rLTL) [58]. While reasoning in LTL yields a binary result, rLTL adopts a five-valued semantics representing different shades of violation. Consider, for example, the specification requiring that b is always satisfied provided a is always satisfied. In LTL, if the premise a is violated in a single position of the trace, then the specification is satisfied vacuously, eliminating all requirements on the system regarding . In this case, rLTL detects a mild violation of the premise and thus allows for a mild violation of the conclusion.

While recent work covers the synthesis [58] and verification problem [5, 6, 58] for rLTL, the runtime verification problem is yet to be addressed, except for a preliminary version of the results in this paper presented in the 2020 International Conference on Hybrid Systems: Computation and Control [47]. Since runtime verification can only rely on finite traces by its nature, interesting theoretical questions open up for rLTL with finite semantics. On the practical side, the very same reasons that make runtime verification for LTL so useful also motivate the need for developing a finite semantics suitable for rLTL runtime verification. To this end, we tackle the problem of evaluating a property over infinite traces based on a finite prefix similarly to Bauer et al. [14]. If the available information is insufficient to declare a specification violated or satisfied, the monitor reports a $\mathord {?}$. This concept is applied to each degree of violation of the rLTL semantics. Thus, the rLTL monitor’s verdict consists of four three-valued bits, as the rLTL semantics is based on four two-valued bits. Each bit represents a degree of violation of the specification in increasing order of severity.

As an example, consider an autonomous drone that may or may not be in a stable state.¹ The specification requires that it remains stable throughout the entire mission. However, if the take-off is shaky due to bad weather, the drone is unstable for the first couple of minutes. An LTL monitor thus jumps to the conclusion that the specification is violated whereas an rLTL monitor only reports a partial violation. As soon as the drone stabilizes, the LTL monitor does not indicate any improvement while the rLTL monitor refines its verdict to also report a partial satisfaction.

Some interesting properties that come naturally with LTL monitoring cannot be seamlessly lifted to rLTL monitoring. While it is obvious that all three truth values for finite trace LTL, i.e. , satisfied, violated, and unknown, can be realized for some prefix and formula, the same does not hold for rLTL. Intuitively, the second and third bit of the rLTL monitor’s four-bit output for the property  represent whether a eventually holds forever or whether it holds infinitely often, respectively. Based on a prefix, a monitor cannot distinguish between these two shades of violation, rendering some monitor outputs unrealizable.

In addition to that, we investigate how the level of informedness of an LTL monitor relates to the one of an rLTL monitor. The first observation is that a verdict of an LTL monitor can be refined at most once, from an unknown to either true or false. With rLTL semantics, however, a monitor can refine its output for a given formula up to four times. Secondly, an LTL monitor can only deliver meaningful verdicts for monitorable [13] properties. Intuitively, a property is monitorable if every prefix can be extended by a finite continuation that gives a definite verdict. We adapt the definition to robust monitoring and show that neither does LTL monitorability imply rLTL monitorability, nor vice versa.

Notwithstanding the above, empirical data suggests that rLTL monitoring indeed provides more information than LTL monitoring: This paper presents an algorithm synthesizing monitors for rLTL specifications. An implementation thereof allows us to validate the approach by replicating the experiments of Bauer et al. [13]. As performance metric, we use LTL and rLTL monitorability. While 44% of the formulas considered by Bauer et al. [13] are not LTL-monitorable, we show all of them to be rLTL-monitorable. This indicates that rLTL monitoring is an improvement over LTL monitoring in terms of monitorability and complements the theoretical results with a practical validation.

This paper is an extended version of the work presented in the 2020 International Conference on Hybrid Systems: computation and control [47]. The main research contributions are a finite trace semantics for rLTL coupled with an investigation of its properties when compared to LTL, as well as an algorithm to synthesize monitors for rLTL specifications. Our construction is doubly-exponential in the size of the formula, showing that rLTL monitoring is no more costly than LTL monitoring. In addition to the original work [47], this article features (i) a more detailed discussion of the properties of our finite trace semantics for rLTL, (ii) a new running example detailing each step of the monitor construction, (iii) a new example illustrating the nesting of rLTL operators, (iv) refined complexity bounds on our monitor construction, and (v) all proofs omitted from the conference paper, which provide important additional insight into the problem of monitoring rLTL properties.

1.1 Related work

In runtime verification [22, 35, 42, 49] the specification is often given in LTL [46]. While properties arguing about the past or current state of a system are always monitorable [34], LTL can also express assumptions on the future that cannot be validated using only a finite prefix of a word. Thus, adaptations of LTL have been proposed which include different notions of a next step on finite words [24, 45], lifting LTL to a three- or four-valued domain [13, 14], or applying predictive measures to rule out impossible extensions of words [60].

Non-binary monitoring has also been addressed by adding quantitative measures such as counting events [9, 48]. Most notably, Bartocci et al. [10] evaluate the “likelihood” that a satisfying or violating continuation will occur. To this end, for a given prefix, they count how long a continuation needs to be such that the specification is satisfied/violated; these numbers are then compared against each other. The resulting verdict is quinary: satisfying/violating, presumably satisfying/violating, or inconclusive. This approach is similar in nature to our work as it assesses the degree of satisfaction or violation of a given prefix. However, the motivation and niche of both approaches differs: Bartocci et al.’s approach computes—intuitively speaking—the amount of work that is required to satisfy or violate a specification, which allows for estimating the likelihood of satisfaction. Our approach, however, focuses on measuring the extent to which a specification was satisfied or violated.

Apart from that, monitoring tools collecting statistics [1, 4, 30] become increasingly popular: Snort [55] is a commercial tool for rule-based network monitoring and computing efficient statistics, Beep Beep 3 [33] is a tool based on a query language allowing for powerful aggregation functions and statistical measures. On the downside, these tools impose the overhead of running a heavy-weight application on the monitored system. In contrast, we generate monitor automata out of an rLTL formula. Such an automaton can easily and automatically be implemented on almost any system with statically determined memory requirements and negligible performance overhead. Similarly, the Copilot [52] framework based on synchronous languages [16, 19] transforms a specification in a declarative data-flow language into a C implementation of a monitor with constant space and time requirements. Lola [2, 19] allows for more involved computations, also incorporating parametrization [27] and real-time capabilities [28] while retaining constant space and time requirements.

Another approach is to enrich temporal logics with quantitative measures such as taking either the edit distance [37], counting the number of possible infinite models for LTL [31, 59], incorporating aggregation expressions into metric first-order temporal logic [11], or using averaging temporal operators that quantify the degree of satisfaction of a signal for a specification by integrating the signal w.r.t. a constant reference signal [3].

Rather than enriching temporal logics with such strong quantitative measures, we consider a robust version of LTL: rLTL [5,6,7, 58]. Robust semantics yields information about to which degree a trace violates a property. We adapt the semantics to work with finite traces by allowing for intermediate verdicts. Here, a certain degree of violation can be classified as “indefinite” and refined when more information becomes available to the monitor. Similarly, for Signal Temporal Logic [43, 44], Fainekos et al. [25] introduced a notion of spacial robustness based on interpreting atomic propositions over the real numbers. The sign of the real number provides information about satisfaction/violation while its absolute value provides information about robustness, i.e., how much can this value be altered without changing satisfaction/violation. This approach is complementary to ours since the notion of robustness in rLTL is related to the temporal evolution of atomic propositions which are interpreted classically, i.e, over the Booleans. Donze et al. [21] introduced a notion of robustness closer to rLTL in the sense that it measures how long we need to wait for the truth value of a formula to change. For this, Cralley et al. [18] presented a convenient toolbox, achieving high efficiency through parallel evaluation. While the semantics of rLTL does not allow for quantifying the exact delay needed to change the truth value of a formula, it allows for distinguishing between the influence that different temporal evolutions, e.g., delays, persistence, and recurrence, have on the truth value of an LTL formula. Closer to rLTL is the work of Radionova et al. [54] (see also [57]) that established an unexpected connection between LTL and filtering through a quantitative semantics based on convolution with a kernel. By using different kernels, one can express weaker or stronger interpretations of the same formula. However, this requires the user to choose multiple kernels and to use multiple semantics to reason about how the degradation of assumptions leads to the degradation of guarantees. In contrast, no such choices are required in rLTL. Finally, it is worth mentioning that extensions similar to rLTL have been proposed for other temporal logics, such as prompt LTL and linear dynamic logic [50, 51].

Another venue for robust monitoring is machine learning. Cheng [17] presents an algorithm for generating monitors evaluating the distance between the input of a neural net and its training data. While neural nets are prone to fragility, the monitor is provably robust in the sense that minor input deviations invariably lead to minor changes in the output. Similarly, Finkbeiner et al. [29] generate monitors for medical cyber-physical systems controlled by machine learned components. Due to the complexity of the underlying specification language, they opt for the simpler task of analyzing the robustness of the specification instead. If the specification is robust, then so will be the generated monitors.

2 Robust Linear Temporal Logic

Throughout this work, we assume basic familiarity with classical LTL and refer the reader to a textbook for more details on the logic (see, e.g. , [8]). Moreover, let us fix some finite set P of atomic propositions throughout the paper and define $\varSigma = 2^P$. We denote the set of finite and infinite words over $\varSigma$ by $\varSigma ^*$ and $\varSigma ^\omega$, respectively. The empty word is denoted by $\varepsilon$ and $\sqsubseteq$ and $\sqsubset$ denote the non-strict and the strict prefix relation, respectively. Moreover, we denote the set of Booleans by $\mathbb {B}= \{ 0,1 \}$.

The logics LTL and rLTL share the same syntax save for a dot superimposed on temporal operators. More precisely, the syntax of rLTL is given by the grammar

where p ranges over atomic propositions in P and the temporal operators , , , and correspond to “next”, “until”, “release”, “eventually”, and “always”, respectively.² The size $| \varphi |$ of a formula $\varphi$ is the number of its distinct subformulas. Furthermore, we denote the set of all LTL and rLTL formulas over P by $\varPhi _{ LTL }$ and $\varPhi _{ rLTL }$, respectively.

The development of rLTL was motivated by the observation that the difference between “minor” and “major” violations of a formula cannot be adequately described in a two-valued semantics. If an LTL formula $\varphi$, for example, demands that the property p holds at all positions of a word $\sigma \in \varSigma ^\omega$, then $\sigma$ violates $\varphi$ even if p does not hold at only a single position, a very minor violation. The semantics of LTL, however, does not differentiate between the $\sigma$ above and a $\sigma '$ in which the property p never holds, a major violation of the property $\varphi$.

In order to alleviate this shortcoming, Tabuada and Neider introduced Robust Linear-time Temporal Logic (rLTL) [58], whose semantics allows for distinguishing various “degrees” to which a word violates a formula. More precisely, the semantics of rLTL are defined over the set $\mathbb {B}_4 = \{ 0000, 0001, 0011, 0111, 1111 \}$ of five truth values, each of which is a monotonically increasing sequence of four bits. We order the truth values in $\mathbb {B}_4$ by $0000< 0001< 0011< 0111 < 1111$.

Intuitively, this order reflects increasingly desirable outcomes. If the specification is , the least desirable outcome, represented by 0000, is that p never holds on the entire trace. A slightly more desirable outcome is that p at least holds sometime but not infinitely often, which results in the value 0001. An even more desirable outcome would be if p holds infinitely often, while also being violated infinitely often, represented by 0011. Climbing up the ladder of desirable outcomes, the next best one requires p to hold infinitely often while being violated only finitely often, represented by the value 0111. Lastly, the optimal outcome fully satisfies , so p holds the entire time, represented by 1111. Thus, the first bit states whether is satisfied, the second one stands for , the third one for , and the fourth one for . If all of them are 0, holds. The robust release is defined analogously.

The robust eventually-operator considers future positions in the trace and returns the truth value with the least degree of violation, which is a maximization with respect to the order defined above. This closely resembles the LTL definition. The robust until is defined analogously.

Based on this, the boolean conjunction and disjunction are defined as $\min$ and $\max$, respectively, w.r.t. the order defined above, which generalizes the classical definition thereof. For the implication, consider a specification , where is an assumption on the environment and is a system guarantee. If the truth value of is greater or equal to the one of , the implication is fully satisfied. Thus, the rLTL semantics takes the violation of the assumption into account and lowers the requirements on the guarantees. However, if the guarantee exhibits a greater violation than the assumptions, the truth value of the implication is the same as the one of the guarantee. Lastly, the intuition behind the negation is that every truth value that is not 1111 constitutes a violation of the specification. Thus, the negation thereof is a full satisfaction (1111). The negation of the truth value representing a perfect satisfaction (1111) is a full violation (0000).

To introduce the semantics, we need some additional notation: for a word $\sigma = \sigma (0) \sigma (1) \sigma (2) \cdots \in \varSigma ^\omega$ and a natural number n, define $\sigma [n,\infty ) = \sigma (n) \sigma (n+1) \sigma (n+2) \cdots$, (i.e. , as the suffix of $\sigma$ obtained by removing the first n letters of $\sigma$). To be able to refer to individual bits of an rLTL truth value $\beta \in \mathbb {B}_4$, we use $\beta [i]$ with $i \in \{ 1, \ldots , 4 \}$ as to denote the i-th bit of $\beta$.

For the sake of a simpler presentation, we denote the semantics of both LTL and rLTL not in terms of satisfaction relations but by means of valuation functions. For LTL, the valuation function $V:\varSigma ^\omega \times \varPhi _{ LTL }\rightarrow \mathbb {B}$ assigns to each infinite word $\sigma \in \varSigma ^\omega$ and each LTL formula $\varphi \in \varPhi _{ LTL }$ the value 1 if $\sigma$ satisfies $\varphi$ and the value 0 if $\sigma$ does not satisfy $\varphi$, and is defined as usual (see, e.g., [8]). The semantics of rLTL, on the other hand, is more complex and formalized next by an valuation function $V_r:\varSigma ^\omega \times \varPhi _{ rLTL }\rightarrow \mathbb {B}_4$ mapping an infinite word $\sigma \in \varSigma ^\omega$ and an rLTL formula $\varphi$ to a truth value in $\mathbb {B}_4$.

• $V_r(\sigma ,p) ={\left\{ \begin{array}{ll} 1111 &{}\text {if}\, p \in \sigma (0),\\ 0000 &{}\text {if}\, p \notin \sigma (0), \end{array}\right. }$

• $V_r(\sigma ,\lnot \varphi ) = {\left\{ \begin{array}{ll} 1111 &{}\text {if}\, V_r(\sigma ,\varphi ) \ne 1111,\\ 0000 &{}\text {if}\,V_r(\sigma ,\varphi ) = 1111, \end{array}\right. }$

• $V_r(\sigma , \varphi _1 \wedge \varphi _2) = \min \{ V_r(\sigma , \varphi _1), V_r(\sigma ,\varphi _2) \}$,

• $V_r(\sigma , \varphi _1 \vee \varphi _2) = \max \{ V_r(\sigma , \varphi _1), V_r(\sigma ,\varphi _2) \}$,

• $V_r(\sigma , \varphi _1 \rightarrow \varphi _2) = {\left\{ \begin{array}{ll} 1111 &{}\text {if}\, V_r(\sigma ,\varphi _1) \le V_r(\sigma , \varphi _2),\\ V_r(\sigma , \varphi _2) &{}\text {if}\, V_r(\sigma ,\varphi _1) > V_r(\sigma , \varphi _2), \end{array}\right. }$

• ,

• with $\beta [i] = \max _{n \ge 0} V_r(\sigma [n,\infty ), \varphi )[i]$ for $i \in \{ 1, \ldots , 4 \}$,

• with

  $$\begin{aligned} \beta [1]&= \min _{n \ge 0} V_r(\sigma [n,\infty ), \varphi )[1],\\ \beta [2]&= \max _{m \ge 0} \min _{n \ge m}V_r(\sigma [n,\infty ), \varphi )[2],\\ \beta [3]&= \min _{m \ge 0} \max _{n \ge m}V_r(\sigma [n,\infty ), \varphi )[3],\\ \beta [4]&= \max _{n \ge 0} V_r(\sigma [n,\infty ), \varphi )[4], \end{aligned}$$

• with

  $$\begin{aligned} \beta [i] = \max _{n \ge 0} \min \{ V_r(\sigma [n,\infty ), \varphi _2 )[i], \min _{0\le n' < n} V_r(\sigma [n',\infty ), \varphi _1)[i] \}, \end{aligned}$$

  for $i \in \{ 1, \ldots , 4 \}$,

• with

  $$\begin{aligned} \beta [1]&= \min _{n \ge 0} \max \{ V_r(\sigma [n,\infty ), \varphi _2 )[1], \max _{0 \le n'< n} V_r(\sigma [n',\infty ), \varphi _1)[1] \},\\ \beta [2]&= \max _{m \ge 0} \min _{n \ge m} \max \{ V_r(\sigma [n,\infty ), \varphi _2 )[2], \max _{0 \le n'< n} V_r(\sigma [n',\infty ), \varphi _1)[2] \},\\ \beta [3]&= \min _{m \ge 0} \max _{n \ge m} \max \{ V_r(\sigma [n,\infty ), \varphi _2 )[3], \max _{0 \le n'< n} V_r(\sigma [n',\infty ), \varphi _1)[3] \}, \text { and} \\ \beta [4]&= \max _{n \ge 0} \max \{ V_r(\sigma [n,\infty ), \varphi _2 )[4], \max _{0 \le n' < n} V_r(\sigma [n',\infty ), \varphi _1)[4] \}. \end{aligned}$$

So as to not clutter this section too much, we refer the reader to the original work by Tabuada and Neider [58] for a thorough introduction and motivation to the preceding semantics. However, we here want to illustrate the definition above and briefly argue that it indeed captures the intuition described at the beginning of this section. To this end, we reconsider the formulas , , in Examples 1, 2, and 3 respectively.

Example 1

Consider the formula and the following five infinite words over the set $P = \{ p \}$ of atomic propositions:

$$\begin{aligned} \sigma _1&= \{ p \}^\omega&(``p\,\text { holds always''}) \\ \sigma _2&= \emptyset \{ p \}^\omega&(``p\,\text { holds almost always''}) \\ \sigma _3&= (\emptyset \{ p \})^\omega&(``p\,\text { holds infinitely often''}) \\ \sigma _4&= \{ p \} \emptyset ^\omega&(``p\,\text { holds finitely often''}) \\ \sigma _5&= \emptyset ^\omega&(``p\,\text { holds never}'') \end{aligned}$$

Let us begin the example with the word $\sigma _1 = \{ p \}^\omega$. It is not hard to verify that because p always holds in $\sigma _1$, i.e., $\min _{n \ge 0} V_r(\sigma [n,\infty ), p)[1] = 1$ for $n \ge 0$. Using the same argument, we also have . Thus, .

As another example, consider the word $\sigma _2 = \emptyset \{ p \}^\omega$. In this case, we have because $V_r(\sigma [0,\infty ), p)[1] = 0$ (p does not hold in the first symbol of $\sigma _2$). However, because p holds almost always, i.e., $\max _{m \ge 0} \min _{n \ge m}V_r(\sigma [n,\infty ), p)[2] = 1$. Moreover, and, therefore, . Similarly, we obtain , , and .

In conclusion, this indeed illustrates that the semantics of the robust always is in accordance with the intuition provided at the beginning of this section. $\square$

Example 2

Let us now consider the more complex formula , where we interpret a to be an assumption on the environment of a cyber-physical system and g one of its guarantees. Moreover, let $\sigma$ be an infinite word over $P = \{ a, g \}$ such that . We now distinguish various cases.

First, let us assume that $\sigma$ is such that , i.e., a always holds. By definition of the robust implication and since , this can only be the case if . Thus, the formula ensures that if the environment assumption a always holds, so does the system guarantee g.

Next, assume that $\sigma$ is such that , i.e., a does not always hold but almost always. By definition of the robust implication and since , this can only be the case if . In this case, the formula ensures that if the environment assumption a holds almost always, then the system guarantee g holds almost always or—even better—always.

It is not hard to verify that we obtain similar results for the cases . In other words, the semantics of rLTL ensures that the violation of the system guarantee g is always proportional to the violation of the environment assumption a (given that evaluates to 1111). Again, this illustrates that the semantics of the implication is in accordance with the intuition provided at the beginning of this section. $\square$

Example 3

As a last example, let us discuss the nesting of temporal operators. Consider the formula  where we interpret q as a request and p as a response.

We have if $\sigma [n,\infty )$ contains a response, otherwise we have . Similarly, we have if $q \in \sigma (n)$ implies that $\sigma [n,\infty )$ contains a response. On the other hand, if $q \in \sigma (n)$ and $\sigma [n,\infty )$ does not contain a response then we have .

From these observations, we can deduce $V_r(\sigma ,\varphi ) = 1111$ if every request in $\sigma$ is followed by a response, which is equivalent to the LTL formula  that expresses a request-response property. Further, we have $V_r(\sigma ,\varphi ) = 0111$ if and only if $\sigma$ violates $\varphi _1$ and if from some point onwards, every request in $\sigma$ is followed by a response. This is equivalent to the LTL formula $\lnot \varphi _1 \wedge \varphi _2$ with , which expresses strong fairness. Similarly, we have $V_r(\sigma ,\varphi ) = 0011$ if and only if $\sigma$ violates $\varphi _2$ and if for infinitely many positions, if there is a request in $\sigma$ at that position, then it is followed by a response. This is equivalent to the LTL formula $\lnot \varphi _2 \wedge \varphi _3$ with , which expresses weak fairness. Moreover, we have $V_r(\sigma ,\varphi ) = 0001$ if and only if $\sigma$ violates $\varphi _3$ and if there is some position such that if there is a request in $\sigma$ at that position, then it is followed by a response. This is equivalent to the LTL formula $\lnot \varphi _3 \wedge \varphi _4$ with , which expresses a very weak notion of fairness. Finally, we have $V_r(\sigma ,\varphi ) = 0000$ if and only if $\sigma$ violates $\varphi _4$.

For $i \in \{ 1,2,3 \}$, the LTL formula $\varphi _i$ implies $\varphi _{i+1}$. Thus, if a trace $\sigma$ violates $\varphi _{i+1}$, it also violates $\varphi _i$. This further illustrates the monotonicity of rLTL. This monotonicity also allows us to only require that $\varphi _{i+1}$ violates $\varphi _i$ in the intuitive explanations above, instead of having to require violations of all $\varphi _{i'}$ with $i' \le i$. $\square$

It is important to note that rLTL is an extension of LTL. In fact, the LTL semantics can be recovered from the first bit of the rLTL semantics (after every implication $\varphi \rightarrow \psi$ has been replaced with $\lnot \varphi \vee \psi$).³

Lemma 1

([58], Proposition 5) Let $\varphi$ be an LTL formula without implications, and let $\varphi '$ be the corresponding rLTL formula (obtained by dotting all temporal operators). Then, we have $V_r(\sigma , \varphi ')[1] = V(\sigma , \varphi )$ for every trace $\sigma$.

To reduce the number of cases we have to consider in our inductive proofs (for instance the one for Lemma 3), we note that the robust eventually and the robust always operator are syntactic sugar. Formally, we say that two rLTL formulas $\varphi _1, \varphi _2$ are equivalent if $V_r(\sigma , \varphi _1) = V_r(\sigma , \varphi _2)$ for every $\sigma \in \varSigma ^\omega$. Now, let $\top = p \vee \lnot p$ and $\bot = p \wedge \lnot p$ for some atomic proposition p. Then, the robust eventually and the robust always are, as usual, expressible in terms of the robust until and the robust release, respectively.

Remark 1

• 1. and are equivalent.

• 2. and are equivalent.

2.1 An alternative definition of robust semantics for LTL

Before we introduce rLTL monitoring, we need to introduce an alternative definition of the semantics of rLTL, which is more convenient to prove some of the results from Sect. 3. This alternative definition has been introduced in later works on rLTL [5, 6].

Definition 1

Let the function $\mathrm {ltl}:\{ 1, \ldots , 4 \} \times \varPhi _{ rLTL }\rightarrow \varPhi _{ LTL }$ be inductively defined as in Table 1. The rLTL semantics is then given as the valuation function $V_r:\varSigma ^\omega \times \varPhi _{ rLTL }\rightarrow \mathbb {B}_4$, where for every $\sigma \in \varSigma ^\omega$, every rLTL formula $\varphi$, and every $i \in \{ 1, \ldots , 4 \}$, the i-th bit of $V_r(\sigma , \varphi )$ is defined as $V_r(\sigma , \varphi )[i] = V\bigl ( \sigma , \mathrm {ltl}(i, \varphi ) \bigr )$ (i.e. , via the semantics of the LTL formulas $\mathrm {ltl}(i, \varphi )$).

Table 1 The function $\mathrm {ltl}:\{ 1, \ldots , 4 \} \times \varPhi _{ rLTL }\rightarrow \varPhi _{ LTL }$

As a consequence of Lemma 1 (cf. [58], Proposition 5), we know that rLTL is at least as expressive as LTL. The latter definition of the semantics of rLTL shows that it is not more expressive than LTL, in the sense that for all rLTL formulas there exist LTL formulas giving the truth values of each of the four bits. However, it is more convenient to work with one formula of rLTL than to work with the four LTL formulas capturing it.

A useful feature of the alternative semantics is the following property: to determine the truth value of an rLTL formula $\varphi$ on $\sigma$, it suffices to determine the truth values of the LTL formulas $\mathrm {ltl}(i, \varphi )$ on $\sigma$. For certain formulas, $\mathrm {ltl}(i, \varphi )$ is obtained from $\varphi$ by a very simple rewriting, as shown below.

Remark 2

Let $\varphi$ be an rLTL formula that has no always in the scope of a negation and only uses negation, conjunction, disjunction, next, eventually, and always. Then,

• $\mathrm {ltl}(1, \varphi )$ is equivalent to the formula obtained from $\varphi$ by replacing every by , every by , and every by ,

• $\mathrm {ltl}(2, \varphi )$ is equivalent to the formula obtained from $\varphi$ by replacing every by , every by , and every by ,

• $\mathrm {ltl}(3, \varphi )$ is equivalent to the formula obtained from $\varphi$ by replacing every by , every by , and every by , and

• $\mathrm {ltl}(4, \varphi )$ is equivalent to the formula obtained from $\varphi$ by replacing every by , every by , and every by .

3 Monitoring robust LTL

In their work on LTL monitoring, Bauer et al. [14] define the problem of runtime monitoring as “check[ing] LTL properties given finite prefixes of infinite [words]”. More formally, given some prefix $u \in \varSigma ^*$ and some LTL formula $\varphi$, it asks whether all, some, or no infinite extension $u\sigma \in \varSigma ^\omega$ of u by some $\sigma \in \varSigma ^\omega$ satisfies $\varphi$. To reflect these three possible results, the authors use the set $\mathbb {B}^{\mathord {?}}= \{ 0, \mathord {?}, 1 \}$ to define a three-valued logic that is syntactically identical to LTL, but equipped with a semantics in form of an evaluation function $V^m :\varSigma ^* \times \varPhi _{ LTL }\rightarrow \mathbb {B}^{\mathord {?}}$ over finite prefixes. This semantics is defined such that $V^m (u, \varphi )$ is equal to 0 (is equal to 1) if no (if every) extension $u \sigma$ of u satisfies $\varphi$. If neither is the case, i.e. , if there is an extension of u that satisfies $\varphi$ and there is an extension of u that does not satisfy $\varphi$, then $V^m (u, \varphi )$ is equal to $\mathord {?}$.

We aim to extend the approach of Bauer et al. to rLTL, whose semantics is based on truth values from the set $\mathbb {B}_4$ (containing the sequences of length four in $0^*1^*$). As a motivating example, let us consider the formula for some atomic proposition s and study which situations can arise when monitoring this formula. Note that the truth value of $\varphi$ can be obtained by concatenating the truth values of the LTL formulas , , , and .

First, consider the empty prefix and its two extensions $\emptyset ^\omega$ and $\{ s \}^\omega$. We have $V_r(\emptyset ^\omega , \varphi ) = 0000$ and $V_r(\{ s \}^\omega , \varphi ) = 1111$. Thus, all four bits can both be equal to 0 and 1. This situation is captured by the sequence $\mathord {?}\mathord {?}\mathord {?}\mathord {?}$ which signifies that for every position i and every bit $b \in \mathbb {B}$, there exists an extension of $\varepsilon$ that has bit b in the i-th position of the truth value with respect to $\varphi$.

Now, consider the prefix $\{ s \}$ for which we have $V_r(\{ s \}\sigma , \varphi )[4] = 1$ for every $\sigma \in \varSigma ^\omega$ as is satisfied on each extension of $\{ s \}$ (s has already occurred). On the other hand, $V_r(\{ s \}\emptyset ^\omega , \varphi ) = 0001$ and $V_r(\{ s \}\{ s \}^\omega , \varphi ) = 1111$, i.e. , the first three bits can both be 0 and 1 by picking an appropriate extension. Hence, the situation is captured by the sequence $\mathord {?}\mathord {?}\mathord {?}1$, signifying that the last bit is determined by the prefix, but the first three are not. Using dual arguments, the sequence $0\mathord {?}\mathord {?}\mathord {?}$ is used for the prefix $\emptyset {}$, signifying that the first bit is determined by the prefix as every extension violates . However, the last three bits are not yet determined by the prefix, hence the trailing $\mathord {?}$’s.

Finally, consider the prefix $\{ s \}\emptyset$. Using the same arguments as for the previous two prefixes, we obtain $V_r(\{ s \}\emptyset \sigma , \varphi )[1] = 0$ and $V_r(\{ s \}\emptyset \sigma , \varphi )[4] = 1$ for every $\sigma \in \varSigma ^\omega$. Also, as before, we have $V_r(\{ s \}\emptyset \emptyset ^\omega , \varphi ) = 0 0 0 1$ and $V_r(\{ s \}\emptyset \{ s \}^\omega , \varphi ) = 0 1 1 1$. Hence, here we obtain the sequence $0 \mathord {?}\mathord {?}1$ signifying that the first and last bit are determined by the prefix, but the middle two are not.

In general, we use truth values of the form $0^*?^*1^*$, which follows from the fact that the truth values of rLTL are in $0^*1^*$. Hence, let $\mathbb {B}^{\mathord {?}}_4$ denote the set of sequences of length four in $0^*?^*1^*$. Based on $\mathbb {B}^{\mathord {?}}_4$, we now formally define the rLTL monitoring semantics as a bitwise generalization of the LTL definition.

Definition 2

The semantics of the robust monitor $V_r^m :\varSigma ^* \times \varPhi _{ rLTL }\rightarrow \mathbb {B}_4^{\mathord {?}}$ is defined as $V_r^m (u, \varphi ) = \beta$ with

$$\begin{aligned} \beta [i] = {\left\{ \begin{array}{ll} 0 &{} \text {if}\, V_r(u\sigma , \varphi )[i] = 0\,\text { for all}\, \sigma \in \varSigma ^\omega ; \\ 1 &{} \text {if}\, V_r(u\sigma , \varphi )[i] = 1\,\text { for all}\, \sigma \in \varSigma ^\omega ;\,\text { and} \\ \mathord {?}&{} \text {otherwise,} \end{array}\right. } \end{aligned}$$

for every $i \in \{ 1, \ldots , 4 \}$, every rLTL formula $\varphi$, and every $u \in \varSigma ^*$.

First, let us remark that our notion of rLTL monitoring indeed refines the notion of LTL monitoring, which follows immediately from Lemma 1.

Remark 3

Let $\varphi$ be an LTL formula without implications, and let $\varphi '$ be the corresponding rLTL formula (obtained by dotting all temporal operators). Then, we have $V_r^m (u, \varphi ')[1] = V^m (u, \varphi )$ for every $u \in \varSigma ^*$.

Using rLTL monitoring semantics, we are able to infer information about the infinite run of a system after having read only a finite prefix thereof. In fact, this robust semantics provides far more information about the degree of violation of the specification than classical LTL monitoring as each bit of the monitoring output represents a degree of violation of the specification: a $\mathord {?}$ turning into a 0 or 1 indicates a deterioration or improvement in the system’s state, respectively. Consider, for instance, an autonomous drone with specification where s denotes a state of stable flight (recall the motivating example on Page 11). Initially, the monitor would output $\mathord {?}\mathord {?}\mathord {?}\mathord {?}$ due to a lack of information. If taking off under windy conditions, the state s is not reached initially, hence the monitor issues a warning by producing $V_r^m (\emptyset ^n,\varphi ) = 0\mathord {?}\mathord {?}\mathord {?}$ for every $n > 0$. Thus, the safety condition is violated temporarily, but not irrecoverably. Hence, mitigation measures can be initiated. Upon success, the monitoring output turns into $V_r^m (\emptyset ^n \{ s \}, \varphi ) = 0\mathord {?}\mathord {?}1$ for every $n > 0$, signaling that flight was stable for some time.

Before we continue, let us first state that the new semantics is well-defined, i.e. , that the sequence $\beta [1]\beta [2]\beta [3]\beta [4]$ in Definition 2 is indeed in $\mathbb {B}^{\mathord {?}}_4$.

Lemma 2

$V_r^m (u, \varphi ) \in \mathbb {B}^{\mathord {?}}_4$ for every rLTL formula $\varphi$ and every $u \in \varSigma ^*$.

Proof

Let $V_r^m (u, \varphi )[i]=0$ and $j<i$. By definition of $V_r^m$, we have $V_r(u\sigma , \varphi )[i] = 0$ for every $\sigma \in \varSigma ^\omega$. Hence, due to the monotonicity of the truth values from $\mathbb {B}_4$ used to define $V_r$, we obtain $V_r(u\sigma , \varphi )[j] = 0$ for every such $\sigma$. Hence, $V_r^m (u, \varphi )[j]=0$.

A dual argument shows that $V_r^m (u, \varphi )[i]=1$ and $j>i$ implies $V_r^m (u, \varphi )[j]=1$. Combining both properties yields $V_r^m (u, \varphi ) \in 0^*\mathord {?}^*1^*$, i.e. , $V_r^m (u, \varphi ) \in \mathbb {B}^{\mathord {?}}_4$. $\square$

After having shown that every possible output of $V_r^m$ is in $\mathbb {B}^{\mathord {?}}_4$, the next obvious question is whether $V_r^m$ is surjective, i.e. , whether every truth value $\beta \in \mathbb {B}^{\mathord {?}}_4$ is realized by some prefix $u \in \varSigma ^*$ and some rLTL formula $\varphi$ in the sense that $V_r^m (u, \varphi ) = \beta$. Recall the motivating example above: The formula realizes at least the following four truth values: $\mathord {?}\mathord {?}\mathord {?}\mathord {?}$ (on $\varepsilon$), $\mathord {?}\mathord {?}\mathord {?}1$ (on $\{ s \}$), $0 \mathord {?}\mathord {?}\mathord {?}$ (on $\emptyset$), and $0 \mathord {?}\mathord {?}1$ (on $\{ s \}\emptyset$). It is not hard to convince oneself that these are all truth values realized by as they represent the following four types of prefixes that can be distinguished: the prefix is empty (truth value $\mathord {?}\mathord {?}\mathord {?}\mathord {?}$), the prefix is in $\{ s \}^+$ (truth value $\mathord {?}\mathord {?}\mathord {?}1$), the prefix is in $\emptyset ^+$ (truth value $0 \mathord {?}\mathord {?}\mathord {?}$), or the prefix contains both an $\{ s \}$ and an $\emptyset$ (truth value $0 \mathord {?}\mathord {?}1$).

For most other truth values, it is straightforward to come up with rLTL formulas and prefixes that realize them. See Table 2 for an overview and recall Remark 2, which is applicable to all these formulas.

Table 2 Realizable truth values. For every truth value $\beta$, the next two columns show prefixes u and formulas $\varphi$ such that $V_r^m (u,\varphi ) = \beta$, or that $\beta$ is unrealizable

For others, such as 0011, it is much harder. Intuitively, to realize 0011, one needs to find an rLTL formula $\varphi$ and a prefix $u \in \varSigma ^*$ such that the formula obtained by replacing all in $\varphi$ by is not satisfied by any extension of u, but the formula obtained by replacing all in $\varphi$ by is satisfied by every extension of u.⁴ Thus, intuitively, the prefix has to differentiate between a property holding almost always and holding infinitely often. It turns out that no such u and $\varphi$ exist. A similar argument is true for 0001, leading to the following theorem.

Theorem 1

All truth values except for 0011 and 0001 are realizable.

The unrealizability results for the truth values 0011 and 0001 are based on the following technical lemma (the reader might want to skip the proof for now and consult it at a later time).

Lemma 3

Let $\varphi$ be an $\text {rLTL}$ formula. Then, the following holds:

• 1. $V_r(u\emptyset ^\omega , \varphi )[2] = V_r(u\emptyset ^\omega , \varphi )[3]$ for all $u \in \varSigma ^*$.

• 2. $V_r(u^\omega , \varphi )[3] = V_r(u^\omega , \varphi )[4]$ for all non-empty $u \in \varSigma ^*$.

• 3. If $\varphi$ does not contain the release operator, then $V_r(u^\omega , \varphi )[1] = V_r(u^\omega , \varphi )[2]$ for all non-empty $u \in \varSigma ^*$.

Proof

The proofs of all three items proceed by induction over the construction of $\varphi$. The induction start and the induction steps for Boolean connectives can be abstracted into the following closure property, which follows easily from the original definition of $V_r$ in Sect. 2:

Let $T \subseteq \mathbb {B}_4$ contain 0000 and 1111. If $V_r(\sigma , \varphi _1)$ and $V_r(\sigma , \varphi _2)$ are in T, then so are $V_r(\sigma , p)$ for atomic propositions p, $V_r(\sigma , \lnot \varphi _1)$, $V_r(\sigma , \varphi _1 \wedge \varphi _2)$, $V_r(\sigma , \varphi _1 \vee \varphi _2)$, and $V_r(\sigma , \varphi _1 \rightarrow \varphi _2)$.

Claim 1 The induction start and the induction step for the Boolean operators follow from the closure property, where we pick T to be the set of truth values from $\mathbb {B}_4$ whose second and third bit coincide. Furthermore, due to Remark 1, we only have to consider the inductive steps for the next, until, and release operator. All three cases rely on the following simple fact: A suffix $u\emptyset ^\omega [n,\infty )$ for some n is again of the form $u'\emptyset ^\omega$, i.e. , the induction hypothesis is applicable to suffixes. Also, if $n \ge | u |$, then $u\emptyset ^\omega [n,\infty ) = \emptyset ^\omega$. In particular, $u\emptyset ^\omega$ has only finitely many distinct suffixes.

So, first consider a formula of the form . Then, we have, for an arbitrary $u \in \varSigma ^*$,

$$\begin{aligned} V_r(u\emptyset ^\omega , \varphi )[2] =&V_r(u\emptyset ^\omega [1,\infty ), \varphi _1)[2]\\ =&V_r(u\emptyset ^\omega [1,\infty ), \varphi _1)[3] =V_r(u\emptyset ^\omega , \varphi )[3], \end{aligned}$$

where the second equality is due to the induction hypothesis being applied to the suffix $u\emptyset ^\omega [1,\infty )$.

Next, consider a formula of the form . Then, we have, for an arbitrary $u \in \varSigma ^*$,

$$\begin{aligned}&V_r(u\emptyset ^\omega , \varphi )[2]\\&\quad =\max _{n \ge 0} \min \{ V_r(u\emptyset ^\omega [n,\infty ), \varphi _2 )[2], \min _{0\le n'< n} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[2] \}\\&\quad =\max _{n \ge 0} \min \{ V_r(u\emptyset ^\omega [n,\infty ), \varphi _2 )[3], \min _{0\le n' < n} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \}\\&\quad =V_r(u\emptyset ^\omega , \varphi )[3], \end{aligned}$$

where the second equality follows from an application of the induction hypothesis to the suffixes $u\emptyset ^\omega [n,\infty )$ and $u\emptyset ^\omega [n',\infty )$.

It remains to consider a formula of the form . Then, we have, for an arbitrary $u \in \varSigma ^*$, that $V_r(u\emptyset ^\omega , \varphi )[2]$ is by definition equal to

$$\begin{aligned}&\max _{m \ge 0} \min _{n \ge m} \max \{ V_r(u\emptyset ^\omega [n,\infty ), \varphi _2 )[2], \max _{0 \le n'< n} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[2] \}\\&\quad =\max _{m \ge 0} \min _{n \ge m} \max \{ V_r(u\emptyset ^\omega [n,\infty ), \varphi _2 )[3], \max _{0 \le n'< n} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \}\\&\quad =\max _{m \ge | u |} \min _{n \ge m} \max \{ V_r(u\emptyset ^\omega [n,\infty ), \varphi _2 )[3], \max _{0 \le n' < n} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \}\\&\quad =\max _{m \ge | u |} \min _{n \ge m} \max \{ V_r(\emptyset ^\omega , \varphi _2 )[3], \max _{0 \le n' \le | u |} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \}\\&\quad =\displaystyle \max \{ V_r(\emptyset ^\omega , \varphi _2 )[3], \max _{0 \le n' \le | u |} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \}, \end{aligned}$$

The first equality follows from twice applying the induction hypothesis. For the second one, observe that

$$\begin{aligned} \min _{n \ge m} \max \{ V_r(u\emptyset ^\omega [n,\infty ), \varphi _2 )[3], \max _{0 \le n' < n} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \} \end{aligned}$$

is increasing in m. For the third one, note that for all $n\ge | u |$, $u\emptyset ^\omega [n,\infty ) = \emptyset ^\omega$, which means that we have eliminated every occurrence of m and n. This explains the last equality. Similarly, $V_r(u\emptyset ^\omega , \varphi )[3]$ is by definition equal to

$$\begin{aligned}&\min _{m \ge 0} \max _{n \ge m} \max \{ V_r(u\emptyset ^\omega [n,\infty ), \varphi _2 )[3], \max _{0 \le n' < n} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \}\nonumber \\&\quad =\displaystyle \max \{ V_r(\emptyset ^\omega , \varphi _2 )[3], \max _{0 \le n' \le | u |} V_r(u\emptyset ^\omega [n',\infty ), \varphi _1)[3] \}, \end{aligned}$$

where the equality again follows from all suffixes $u\emptyset ^\omega [n,\infty )$ with $n \ge | u |$ being equal to $\emptyset ^\omega$. Thus, we have derived the desired equality between $V_r(u\emptyset ^\omega , \varphi )[2]$ and $V_r(u\emptyset ^\omega , \varphi )[3]$.

Claim 2 The induction start and the induction steps for Boolean operators follow from the closure property, where we here pick T to be the set of truth values from $\mathbb {B}_4$ whose third and fourth bit coincide. For $u = u(0) \cdots u(| u |-1)$ and $n < | u |$, we define $\rho (u,n) = u(n) \cdots u(| u |-1)u(0)\cdots u(n-1)$, i.e. , $\rho (u,n)$ is obtained by “rotating” u n times. The induction steps for the temporal operators are based on the following simple fact: The suffix $u^\omega [n,\infty )$ is equal to $(\rho (u,n \bmod | u |))^\omega$, i.e. , the induction hypothesis is applicable to the suffixes. In particular, $u^\omega$ has only finitely many distinct suffixes, which all appear infinitely often in a cyclic order.

Now, the induction steps for the next and until operator are analogous to their counterparts in Item 1, as the only property we require there is that the induction hypothesis is applicable to suffixes. Hence, due to Remark 1, it only remains to consider the inductive step for the release operator.

So consider a formula of the form . Then, we have, for an arbitrary $u \in \varSigma ^*$, that $V_r(u^\omega , \varphi )[3]$ is by definition equal to

$$\begin{aligned}&\min _{m \ge 0} \max _{n \ge m} \max \{ V_r(u^\omega [n,\infty ), \varphi _2 )[3], \max _{0 \le n'< n} V_r(u^\omega [n',\infty ), \varphi _1)[3] \}\\&\quad =\min _{m \ge 0} \max _{n \ge m} \max \{ V_r(u^\omega [n,\infty ), \varphi _2 )[4], \max _{0 \le n'< n} V_r(u^\omega [n',\infty ), \varphi _1)[4] \}\\&\quad =\max _{0 \le n< | u |}\max \{ V_r((\rho (u,n))^\omega , \varphi _2 )[4], \max _{0 \le n' < n} V_r((\rho (u,n'))^\omega , \varphi _1)[4] \}, \end{aligned}$$

where the first equality follows from twice applying the induction hypothesis and the second one is due to all suffixes $u^\omega [n,\infty )$ being equal to $\rho (u,n \bmod | u |)^\omega$, and that there are only finitely many, which all appear infinitely often in a cyclic order among the $(\rho (u,n))^\omega$ for $0 \le n < | u |$.

Similarly, $V_r(u^\omega , \varphi )[4]$ is by definition equal to

$$\begin{aligned}&\max _{n \ge 0} \max \{ V_r(u^\omega [n,\infty ), \varphi _2 )[4], \max _{0 \le n'< n} V_r(u^\omega [n',\infty ), \varphi _1)[4] \}\\&\quad =\max _{0 \le n< | u |}\max \{ V_r((\rho (u,n))^\omega , \varphi _2 )[4], \max _{0 \le n' < n} V_r((\rho (u,n'))^\omega , \varphi _1)[4] \}, \end{aligned}$$

where the equality again follows from all suffixes $u^\omega [n,\infty )$ being equal to $\rho (u,n \bmod | u |)^\omega$, and that there are only finitely many, which appear in a cyclic order: In particular, after the first $| u |$ suffixes, we have seen all of them. Thus, we have derived the desired equality between $V_r(u^\omega , \varphi )[3]$ and $V_r(u^\omega , \varphi )[4]$.

Claim 3 The induction start and the induction steps for Boolean operators are covered by the closure property, where we here pick T to be the set of truth values from $\mathbb {B}_4$ whose first and second bit coincide. The cases of the next and until operator are again analogous to the first and second item. Hence, we only have to consider the inductive step for the always operator, as we here only consider formulas without release.

So, consider a formula of the form . Here, we again rely on the fact that the suffix $u^\omega [n,\infty )$ is equal to $(\rho (u,n \bmod | u |))^\omega$. By definition, $V_r(u^\omega , \varphi )[1]$ is equal to

$$\begin{aligned} \min _{n \ge 0} V_r(u^\omega [n,\infty ), \varphi _1 )[1] =\min _{n \ge 0} V_r(u^\omega [n,\infty ), \varphi _1 )[2] =\min _{0 \le n < | u |} V_r((\rho (u,n))^\omega , \varphi _1 )[2], \end{aligned}$$

where the first equality is due to the induction hypothesis and the second one due to the fact that $u^\omega$ has only finitely many suffixes, which are all already realized by some $u^\omega [n,\infty )$ for $0 \le n < | u |$.

Similarly, $V_r(u^\omega , \varphi )[2]$ is by definition equal to

$$\begin{aligned} \max _{m \ge 0}\min _{n \ge m} V_r(u^\omega [n,\infty ), \varphi _1 )[1]&= \max _{m \ge 0}\min _{n \ge m} V_r(u^\omega [n,\infty ), \varphi _1 )[2] \\&=\min _{0 \le n < | u |} V_r((\rho (u,n))^\omega , \varphi _1 )[2], \end{aligned}$$

where the two equalities follow as before: the first by induction hypothesis and the second one by the fact that $u^\omega$ has only finitely many suffixes, which all appear infinitely often in a cyclic order and which are all already realized by some $u^\omega [n,\infty )$ for $0 \le n < | u |$. Thus, we have derived the desired equality between $V_r(u^\omega , \varphi )[1]$ and $V_r(u^\omega , \varphi )[2]$. $\square$

Now, we are able to prove Theorem 1.

Proof

We begin by showing that 0011 and 0001 are not realizable.

First, towards a contradiction, assume there is an $\text {rLTL}$ formula $\varphi$ and a prefix u such that $V_r^m (u,\varphi ) = 0011$, i.e. , for every extension $u\sigma$, we have $V_r(u\sigma )[2]=0$ and $V_r(u\sigma )[3]=1$. However, by picking $\sigma = \emptyset ^\omega$ we obtain the desired contradiction to Lemma 3.1.

The proof for 0001 is similar. Assume there is an $\text {rLTL}$ formula $\varphi$ and a prefix u such that $V_r^m (u,\varphi ) = 0001$. Due to Lemma 4, we can assume that u is non-empty. Thus, we have $V_r(u^\omega , \varphi ) = 0001$ by definition of $V_r^m$, which contradicts Lemma 3.2.

Finally, applying Lemma 3.3, one can show that no rLTL formula without the release operator realizes 0111. However, we show below that it is realizable by a formula with the release operator.

Next, we show that every other truth value $\beta \notin \{ 0011,0001 \}$ is indeed realizable. The witnessing pairs of prefixes and formulas are presented in Table 2.

First, consider $\beta = 0111$ with prefix $u = \emptyset \{ a \}$ and formula . We have and . Note that is violated by $u \sigma$, for every $\sigma \in \varSigma ^\omega$. Dually, is satisfied by $u \sigma$, for every $\sigma \in \varSigma ^\omega$. Hence, for arbitrary $\sigma \in \varSigma ^\omega$, we have $V_r(u\sigma ,\varphi )[1] = 0$ and $V_r(u\sigma ,\varphi )[2] = 1$. Hence, we have $V_r(u\sigma ,\varphi ) = 0111$ for every $\sigma$, as this is the only truth value that matches this pattern. Hence, by definition, we obtain $V_r^m (u, \varphi ) = 0111$.

The verification for all other truth values is based on Remark 2, which is applicable to all formulas $\varphi$ in the third column witnessing the realization of a truth value $\beta \ne 0111$. Now, for every such truth value $\beta$ and corresponding pair $(u, \varphi )$, one can easily verify the following:

• If $\beta [i] = 0$, then no $u \sigma$ satisfies $\mathrm {ltl}(i, \varphi )$.

• If $\beta [i] = 1$, then every $u \sigma$ satisfies $\mathrm {ltl}(i, \varphi )$.

• If $\beta [i] = ?$, then there are $\sigma ,\sigma '$ such that $u \sigma$ satisfies $\mathrm {ltl}(i, \varphi )$ and such that $u \sigma '$ violates $\mathrm {ltl}(i, \varphi )$. In all such cases, $\sigma ,\sigma ' \in \{ \emptyset ^\omega , \{ a \}^\omega , \{ a \}\emptyset ^\omega , \emptyset \{ a \}^\omega , (\{ a \}\emptyset )^\omega \}$ suffice.

We leave the details of this slightly tedious, but trivial, verification to the reader. $\square$

As shown in Table 2, all of the realizable truth values except for 0111 are realized by formulas using only conjunction, disjunction, negation, eventually, and always. Further, 0111 can only be realized by a formula with the release operator while the truth values 0011 and 0001 are indeed not realizable at all.

Note that the two unrealizable truth values 0011 and 0001 both contain a 0 that is directly followed by a 1. The proof of unrealizability formalizes the intuition that such an “abrupt” transition from definitive violation of a property to definitive satisfaction of the property cannot be witnessed by any finite prefix. Finally, the only other truth value of this form, 0111, is only realizable by using a formula with the release operator.

Going again back to the motivating example , consider the evolution of the truth values on the sequence $\varepsilon , \{ s \}, \{ s \}\emptyset$: They are $\mathord {?}\mathord {?}\mathord {?}\mathord {?}$, $\mathord {?}\mathord {?}\mathord {?}1$, and $0 \mathord {?}\mathord {?}1$, i.e. , 0’s and 1’s are stable when extending a prefix, only a $\mathord {?}$ may be replaced by a 0 or a 1. This property holds in general. To formalize this, say that $\beta ' \in \mathbb {B}^{\mathord {?}}_4$ is more specific than $\beta \in \mathbb {B}^{\mathord {?}}_4$, written as $\beta \preceq \beta '$, if, for all i, $\beta [i] \ne \mathord {?}$ implies $\beta '[i] = \beta [i]$.

Lemma 4

Let $\varphi$ be an rLTL formula and $u,u' \in \varSigma ^*$. If $u \sqsubseteq u'$, then $V_r^m (u, \varphi ) \preceq V_r^m (u', \varphi )$.

Proof

Let $u \sqsubseteq u'$ and assume we have $V_r^m (u,\varphi )[i] \in \{ 0,1 \}$. Thus, by definition, $V_r(u\sigma , \varphi )[i] = V_r^m (u,\varphi )[i]$ for every $\sigma \in \varSigma ^\omega$. Now, as u is a prefix of $u'$, we can decompose $u'$ into $u' = u v$ for some $v \in \varSigma ^*$ and every extension $u' \sigma '$ of $u'$ is the extension $u v \sigma '$ of u. Hence, we have $V_r(u'\sigma ', \varphi )[i] = V_r(uv\sigma ', \varphi )[i] = V_r^m (u,\varphi )[i]$ for every $\sigma ' \in \varSigma ^\omega$. Thus, $V_r^m (u',\varphi )[i] = V_r^m (u,\varphi )[i]$.

As this property holds for every i, we obtain $V_r^m (u, \varphi ) \preceq V_r^m (u', \varphi )$. $\square$

Let us discuss two properties of the semantics: impartiality and anticipation [20]. Impartiality states that a definitive verdict will never be revoked: If $V_r^m (u, \varphi ) [i] \ne \mathord {?}$, then for all finite extensions $v \in \varSigma ^*$, the verdict will not change, so $V_r^m (uv, \varphi )[i] = V_r^m (u, \varphi ) [i]$. This property follows immediately from Lemma 4. Anticipation requires that a definitive verdict is decided as soon as possible, i.e. , if $V_r^m (u, \varphi ) [i] = \mathord {?}$, then u can still be extended to satisfy and to violate $\varphi$ with the i-th bit. Formally, there have to exist infinite extensions $\sigma _0$ and $\sigma _1$ such that $V_r(u\sigma _0, \varphi )[i] =0$ and $V_r(u\sigma _1, \varphi )[i] =1$. Anticipation holds by definition of $V_r^m (u, \varphi )$.

Due to Lemma 4, for a fixed formula, the prefixes of every infinite word can assume at most five different truth values, which are all of increasing specificity. It turns out that this upper bound is tight. To formalize this claim, we denote the strict version of $\preceq$ by $\prec$, i.e. , $\beta \prec \beta '$ if and only if $\beta \preceq \beta '$ and $\beta \ne \beta '$.

Lemma 5

There is an rLTL formula $\varphi$ and prefixes $u_0 \sqsubset u_1 \sqsubset u_2 \sqsubset u_3 \sqsubset u_4$ such that $V_r^m (u_0, \varphi ) \prec V_r^m (u_1, \varphi ) \prec V_r^m (u_2, \varphi ) \prec V_r^m (u_3, \varphi ) \prec V_r^m (u_4, \varphi )$.

Proof

Consider the sequence $\beta _0, \ldots , \beta _4$ with $\beta _j = 0^j \mathord {?}^{4-j}$ and note that we have $\beta _j \prec \beta _{j+1}$ for every $j<4$. Furthermore, let $u_j = \emptyset ^j$ for $j \in \{ 0, \ldots , 4 \}$. We construct a formula $\varphi$ such that $V_r^m (u_j, \varphi ) = \beta _j$ for every $j \in \{ 0, \ldots , 4 \}$.

To this end, let

• ,

• , and

• .

Later, we rely on the following fact about these formulas, which can easily be shown by applying Remark 2: we have $V_r^m (u, \psi _{\beta _j}) = \beta _j$ for every prefix u.

Further, for $j \in \{ 0,1,2,3 \}$, let $\psi _j$ be a formula that requires the proposition a to be violated at the first $j - 1$ positions, but to hold at the j-th position (recall that we start counting at zero), i.e. , . Here, we define the nesting of next operators as usual: and . By definition, we have $V_r(\emptyset ^{j+1}\sigma , \psi _j) = 0000$ for every $\sigma \in \varSigma ^\omega$ ($\dagger$).

Now, we define

$$\begin{aligned} \varphi = \psi _0 \vee \bigvee _{j=1}^3 \left( \psi _{\beta _j} \wedge \psi _j\right) \end{aligned}$$

and claim that it has the desired properties. To this end, we note that property ($\dagger$) implies $V_r(\emptyset ^4\sigma , \varphi ) = 0000$ for every $\sigma \in \varSigma ^\omega$ ($\dagger \dagger$), as every disjunct of $\varphi$ contains a conjunct of the form $\psi _j$ for some $j \le 3$. Also, let us mention that Remark 2 is applicable to $\varphi$.

It remains to prove $V_r^m (u_j, \varphi ) = \beta _j$ for every $j \in \{ 0, \ldots ,4 \}$.

• For $j = 0$, we have $u_0 = \varepsilon$ and $\beta _0 = \mathord {?}\mathord {?}\mathord {?}\mathord {?}$. Hence, it suffices to present $\sigma _0, \sigma _1 \in \varSigma ^\omega$ such that $V_r(\sigma _0, \varphi ) = 0000$ and $V_r(\sigma _1, \varphi ) = 1111$.

  Due to property ($\dagger \dagger$), we can pick $\sigma _0 = \emptyset ^\omega$. To conclude, we pick $\sigma _1 = \{ a \}^\omega$, as we have

  $$\begin{aligned} V_r(\sigma _1, \varphi ) \ge V_r(\sigma _1, \psi _0) = V_r(\{ a \}^\omega , a) = 1111, \end{aligned}$$

  where the first inequality follows from $\psi _0$ being a disjunct of $\varphi$.

• For $j = 1$, we have $u_1 = \emptyset$ and $\beta _1 = 0 \mathord {?}\mathord {?}\mathord {?}$. To show $V_r^m (u_1, \varphi ) = \beta _1$, it suffices to present $\sigma _0,\sigma _1 \in \varSigma ^\omega$ such that $V_r(u_1\sigma _0, \varphi ) = 0000$, $V_r(u_1\sigma _1, \varphi ) = 0111$, and show that $V_r(u_1\sigma , \varphi )[1] = 0$ for every $\sigma \in \varSigma ^\omega$. First, we again pick $\sigma _0 = \emptyset ^\omega$ due to property ($\dagger \dagger$). Now, consider $\sigma _1 = \{ a \}^\omega$. Then,

  $$\begin{aligned} V_r(u_1 \{ a \}^\omega , \psi _{\beta _j} \wedge \psi _j)&= \min \{ V_r(u_1 \{ a \}^\omega , \psi _{\beta _j} ), V_r(u_1 \{ a \}^\omega ,\psi _j) \} \\&= \min \{ 0111, 1111 \}= 0111, \end{aligned}$$

  where $V_r(u_1 \{ a \}^\omega , \psi _{\beta _j} ) = 0111$ can easily be verified using Remark 2. To conclude, using Remark 2, one can easily verify that $\mathrm {ltl}(1, \varphi )$ is not satisfied by $u_1\sigma$ for any $\sigma \in \varSigma ^\omega$.

• The reasoning for $j =2,3$ is along the same lines as the one for $j=1$ and is left to the reader.

• For $j = 4$, we have $u_4 = \emptyset \emptyset \emptyset \emptyset$ and $\beta _4 = 0000$. Hence, our claim follows directly from property ($\dagger \dagger$), which shows $V_r(u_4\sigma , \varphi ) = 0000$ for every $\sigma \in \varSigma ^\omega$.

$\square$

After determining how many different truth values can be assumed by prefixes of a single infinite word, an obvious question is how many truth values can be realized by a fixed formula on different prefixes. It is not hard to combine the formulas in Table 2 to a formula that realizes all truth values not ruled out by Theorem 1.⁵

Lemma 6

There is an rLTL formula $\varphi$ such that for every $\beta \in \mathbb {B}^{\mathord {?}}_4 {\setminus } \{ 0011,0001 \}$ there is a prefix $u_\beta$ with $V_r^m (u_\beta , \varphi ) = \beta$.

Proof

For every $\beta \in \mathbb {B}^{\mathord {?}}_4 \setminus \{ 0011,0001 \}$ let $\varphi _\beta$ be an rLTL formula and $u_\beta '$ be a prefix, both over $\{ a \}$, with $V_r^m (u_\beta ', \varphi _\beta ) = \beta$. Such formulas and prefixes exist as shown in Table 2.

Now, consider the formula

$$\begin{aligned} \varphi = \bigvee _{ \beta \in \mathbb {B}^{\mathord {?}}_4 \setminus \{ 0011,0001 \}} a_\beta \wedge \varphi _\beta \end{aligned}$$

over the propositions $\{ a \} \cup \{ a_\beta \mid \beta \in \mathbb {B}^{\mathord {?}}_4 {\setminus } \{ 0011,0001 \} \}$.

By construction, we have $V_r^m (u_\beta , \varphi ) = \beta$ for every $\beta$, where

$$\begin{aligned} u_\beta = (u_\beta '(0) \cup \{ a_\beta \}) u_\beta '(1) \cdots u_\beta '(| u_\beta ' |-1), \end{aligned}$$

i.e, we obtain $u_\beta$ from $u_\beta '$ by adding the proposition $a_\beta$ to the first letter. Hence, $\varphi$ has the desired properties. $\square$

Finally, let us consider the notion of monitorability [53], an important concept in the theory of runtime monitoring. As a motivation, consider the LTL formula  and an arbitrary prefix $u \in \varSigma ^*$. Then, the extension $u \{ s \}^\omega$ satisfies $\psi$ while the extension $u \emptyset ^\omega$ does not satisfy $\psi$, i.e. , satisfaction of $\psi$ is independent of any prefix u. Hence, we have $V^m (u, \psi ) = \mathord {?}$ for every prefix u, i.e. , monitoring the formula $\psi$ does not generate any information.

In general, for a fixed LTL formula $\varphi$, a prefix $u \in \varSigma ^*$ is called ugly if we have $V^m (uv,\varphi ) = \mathord {?}$ for every finite $v\in \varSigma ^*$, i.e. , every finite extension of u yields an indefinite verdict.⁶ Now, $\varphi$ is LTL-monitorable if there is no ugly prefix with respect to $\varphi$. A wide range of LTL formulas (e.g. , as above) are unmonitorable in that sense. In particular, 44% of the LTL formulas considered in the experiments of Bauer et al. are not LTL-monitorable.

We next generalize the notion of monitorability to rLTL. In particular, we answer whether there are unmonitorable rLTL formulas. Then, in Sect. 5, we exhibit that all LTL formulas considered by Bauer et al.’s experimental evaluation, even the unmonitorable ones, are monitorable under rLTL semantics. To conclude the motivating example, note that the rLTL analogue  of the LTL formula $\psi$ induces two truth values from $\mathbb {B}^{\mathord {?}}_4$ indicating whether s has been true at least once (truth value $\mathord {?}\mathord {?}\mathord {?}1$) or not (truth value $\mathord {?}\mathord {?}\mathord {?}\mathord {?}$). Even more so, every prefix inducing the truth value $\mathord {?}\mathord {?}\mathord {?}\mathord {?}$ can be extended to one inducing the truth value $\mathord {?}\mathord {?}\mathord {?}1$.

Definition 3

Let $\varphi$ be an rLTL formula. A prefix $u \in \varSigma ^*$ is called ugly if we have $V^m (uv,\varphi ) = \mathord {?}\mathord {?}\mathord {?}\mathord {?}$ for every finite $v\in \varSigma ^*$. Further, $\varphi$ is rLTL-monitorable if it has no ugly prefix.

As we have argued above, the formula  has no ugly prefix, i.e. , it is rLTL-monitorable. Thus, we have found an unmonitorable LTL formula whose rLTL analogue (the formula obtained by adding dots to all temporal operators) is monitorable. The converse statement is also true. There is a monitorable LTL formula whose rLTL analogue is unmonitorable. To this end, consider the LTL formula

which is a tautology and therefore monitorable. On the other hand, we claim that $\emptyset \{ s \}$ is an ugly prefix for the rLTL analogue $\varphi$ obtained by adding dots to the temporal operators. To this end note that we have both $V_r(\emptyset \{ s \} v \emptyset ^\omega , \varphi ) = 1111$ and $V_r(\emptyset \{ s \} v \{ s \}^\omega , \varphi ) = 0000$ for every $v \in \varSigma ^*$. Hence, $V_r^m (\emptyset \{ s \}v,\varphi ) = \mathord {?}\mathord {?}\mathord {?}\mathord {?}$ for every such v, i.e. , $\emptyset \{ s \}$ is indeed ugly and $\varphi$ therefore not rLTL-monitorable.

Thus, there are formulas that are unmonitorable under LTL semantics, but monitorable under rLTL semantics and there are formulas that are unmonitorable under rLTL semantics, but monitorable under LTL semantics. Using these formulas one can also construct a formula that is unmonitorable under both semantics.

To this end, fix LTL formulas $\varphi _\ell$ and $\varphi _r$ over disjoint sets of propositions and a fresh proposition p not used in either formula such that

• $\varphi _\ell$ has an ugly prefix $u_\ell$ under LTL semantics, and

• $\varphi _r$ (with dotted operators) has an ugly prefix $u_r$ under rLTL semantics.

We can assume both prefixes to be non-empty, as ugliness is closed under finite extensions. Let $\varphi = (p \wedge \varphi _\ell ) \vee (\lnot p \wedge \varphi _r)$. Then, the prefix obtained from $u_\ell$ by adding the proposition p to the first letter is ugly for $\varphi$ under LTL semantics and $u_r$ is ugly for $\varphi$ (with dotted operators) under rLTL semantics.

As a final example, recall that we have shown that is rLTL-monitorable and consider its negation . It is not hard to see that $V_r^m (u, \varphi ) = \mathord {?}\mathord {?}\mathord {?}\mathord {?}$ holds for every prefix u. Hence, $\varepsilon$ is an ugly prefix for the formula, i.e. , we have found another unmonitorable rLTL formula. In particular, the example shows that, unlike for LTL, rLTL-monitorability is not preserved under negation.

After having studied properties of rLTL monitorability, we next show our main result: The robust monitoring semantics $V_r^m$ can be implemented by finite-state machines.

4 Construction of rLTL monitors

An rLTL monitor is an implementation of the robust monitoring semantics $V_r^m$ in form of a finite-state machine with output. More precisely, an rLTL monitor for an rLTL formula $\varphi$ is a finite-state machine $\mathcal {M}_\varphi$ that on reading an input $u \in \varSigma ^*$ outputs $V_r^m (u, \varphi )$. In this section, we show how to construct rLTL monitors and that this construction is asymptotically not more expensive than the construction of LTL monitors. Let us fix an rLTL formula $\varphi$ for the remainder of this section.

Our rLTL monitor construction is inspired by Bauer et al. [14] and generates a sequence of finite-state machines (i.e. , Büchi automata over infinite words, (non)deterministic automata over finite words, and Moore machines). Underlying these machines are transition structures $\mathcal {T}= (Q, q_I, \varDelta )$ consisting of a nonempty, finite set Q of states, an initial state $q_I\in Q$, and a transition relation $\varDelta \subseteq Q \times \varSigma \times Q$. An (infinite) run of $\mathcal {T}$ on a word $\sigma = a_0 a_1 a_2 \cdots \in \varSigma ^\omega$ is a sequence $\rho = q_0 q_1 \cdots$ of states such that $q_0 = q_I$ and $(q_{j}, a_{j}, q_{j+1}) \in \varDelta$ for $j \in \mathbb N$. Finite runs on finite words are defined analogously. The transition structure $\mathcal {T}$ is deterministic if (a) $(q, a, q') \in \varDelta$ and $(q, a, q'') \in \varDelta$ imply $q' = q''$ and (b) for each $q \in Q$ and $a \in \varSigma$ there exists a $(q, a, q') \in \varDelta$.

We then replace the transition relation $\varDelta$ by a function $\delta :Q \times \varSigma \rightarrow Q$. Finally, we define the size of a transition structure $\mathcal {T}$ as $| \mathcal {T} | = | Q |$ in order to measure its complexity.

Our construction then proceeds in three steps:

• 1. We bring $\varphi$ into an operational form by constructing Büchi automata $\mathcal {A}_\beta ^\varphi$ for each truth value $\beta \in \mathbb {B}_4$ that can decide the valuation $V_r(\sigma , \varphi )$ of infinite words $\sigma \in \varSigma ^\omega$.

• 2. Based on these Büchi automata, we then construct nondeterministic automata $\mathcal {B}_\beta ^\varphi$ that can decide whether a finite word $u \in \varSigma ^*$ can still be extended to an infinite word $u\sigma \in \varSigma ^\omega$ with $V_r(u\sigma , \varphi ) = \beta$.

• 3. We determinize the nondeterministic automata obtained in Step 2 and combine them into a single Moore machine that computes $V_r^m (u, \varphi )$.

Let us now describe each of these steps in detail.

Step 1 We first translate the rLTL formula $\varphi$ into several Büchi automata using a construction by Tabuada and Neider [58], summarized in Theorem 2 below. A (nondeterministic) Büchi automaton (NBA) is a four-tuple $\mathcal {A}= (Q, q_I, \varDelta , F)$ where $\mathcal {T}= (Q, q_I, \varDelta )$ is a transition structure and $F \subseteq Q$ is a set of accepting states. A run $\pi$ of $\mathcal {A}$ on $\sigma \in \varSigma ^\omega$ is a run of $\mathcal {T}$ on $\sigma$, and we say that $\pi$ is accepting if it contains infinitely many states from F. The automaton $\mathcal {A}$ accepts a word $\sigma$ if there exists an accepting run of $\mathcal {A}$ on $\sigma$. The language ${\mathcal {L}}(\mathcal {A})$ is the set of all words accepted by $\mathcal {A}$, and the size of $\mathcal {A}$ is defined as $| \mathcal {A} | = | \mathcal {T} |$.

Theorem 2

(Tabuada and Neider [58]) Given a truth value $\beta \in \mathbb {B}_4$, one can construct a Büchi automaton $\mathcal {A}_\beta ^\varphi$ with $2^{{\mathcal {O}}(|\varphi |)}$ states such that ${\mathcal {L}}(\mathcal {A}_\beta ^\varphi ) = \{ \sigma \in \varSigma ^\omega \mid V_r(\sigma , \varphi ) = \beta \}$. This construction can be performed in $2^{{\mathcal {O}}(|\varphi |)}$ time.

The Büchi automata $\mathcal A_\beta ^\varphi$ for $\beta \in \mathbb {B}_4$ serve as building blocks for the next steps. However, before we proceed, let us illustrate this step with an example.

Example 4

Let us consider the formula , which already served as a running example in Sect. 3. Applying Theorem 2 results in the five nondeterministic Büchi automata $\mathcal {A}_\beta ^\varphi$, one for each $\beta \in \mathbb {B}_4$, shown in Fig. 1. We here use the standard way to represent finite-state machines graphically. States are drawn as circles and transitions are drawn as arrows. Moreover, the initial state has an incoming arrow, while accepting states are indicted by double circles. Finally, note that we use propositional formulas to symbolically define sets of transitions. For instance, a transition labeled with s in Fig. 1a represents all transitions labeled with a symbol from the set $\{ A \subseteq P \mid s \in A \} \subseteq \varSigma$. In particular, $true$ represents all symbols in $\varSigma$. $\square$

Fig. 1

The Büchi automata constructed in Step 1 of our monitor construction

Step 2 For each Büchi automaton $\mathcal {A}^\varphi _\beta$ obtained in the previous step, we now construct a nondeterministic automaton $\mathcal {B}_\beta ^\varphi$ over finite words. This automaton determines whether a finite word $u \in \varSigma ^*$ can be continued to an infinite word $u\sigma \in {\mathcal {L}}(\mathcal {A}_\beta ^\varphi )$ (i.e. , $V_r(u\sigma , \varphi ) = \beta$) and is used later to construct the rLTL monitor.

A nondeterministic finite automaton (NFA) is a four-tuple $\mathcal {A}= (Q, q_I, \varDelta , F)$ that is syntactically identical to a Büchi automaton. The size of $\mathcal {A}$ is defined analogously to Büchi automata. In contrast to Büchi automata, however, NFAs only admit finite runs on finite words, i.e. , a run of $\mathcal {A}$ on $u = a_0 \cdots a_{n-1} \in \varSigma ^*$ is a sequence $q_0 \cdots q_n$ such that $q_0 = q_I$ and $(q_j, a_j, q_{j+1}) \in \varDelta$ for every $j < n$. A run $q_0 \cdots q_n$ is called accepting if $q_n \in F$. Accepted words as well as the language of $\mathcal {A}$ are again defined analogously to the Büchi case. If $(Q, q_I, \varDelta )$ is deterministic, $\mathcal {A}$ is a deterministic finite automaton (DFA). It is well-known that for each NFA $\mathcal {A}$ one can construct a DFA $\mathcal {A}'$ with ${\mathcal {L}}(\mathcal {A}) = {\mathcal {L}}(\mathcal {A}')$ and $| \mathcal {A}' | \in {\mathcal {O}}(2^{| \mathcal {A} |})$.

Given the Büchi automaton $\mathcal {A}_\beta ^\varphi = (Q_\beta , q_{I,\beta }, \varDelta _\beta , F_\beta )$, we first compute the set $F_\beta ^\star = \{ q \in Q_\beta \mid {\mathcal {L}}(\mathcal {A}^\varphi _\beta (q)) \ne \emptyset \}$, where $\mathcal {A}^\varphi _\beta (q)$ denotes the Büchi automaton $\mathcal {A}^\varphi _\beta$ but with initial state q instead of $q_I$. Intuitively, the set $F_\beta ^\star$ contains all states $q \in Q_\beta$ from which there exists an accepting run in $\mathcal {A}^\varphi _\beta$ and, hence, indicates whether a finite word $u \in \varSigma ^*$ reaching a state of $F_\beta ^\star$ can be extended to an infinite word $u\sigma ' \in {\mathcal {L}}(\mathcal {A}_\beta ^\varphi )$. The set $F_\beta ^\star$ can be computed, for instance, using a nested depth-first search [56] for each state $q \in Q_\beta$. Since each such search requires time quadratic in $| \mathcal {A}^\varphi _\beta |$, the set $F_\beta ^\star$ can be computed in time ${\mathcal {O}}(| \mathcal {A}^\varphi _\beta |^3)$.

Using $F_\beta ^\star$, we define the NFA $\mathcal {B}^\varphi _\beta = (Q_\beta , q_{I,\beta }, \varDelta _\beta , F_\beta ^\star )$. It shares the transition structure of $\mathcal {A}_\beta ^\varphi$ and uses $F_\beta ^\star$ as the set of accepting states. Let us illustrate this construction using our running example.

Example 5

Given the NBAs $\mathcal {A}^\varphi _\beta$ from Step 1 of our construction, we now compute the corresponding NFAs $\mathcal {B}^\varphi _\beta$, which are depicted in Fig. 2. Note that the transition structure has remained the same as compared to the preceding step (see Fig. 1). By contrast, the accepting states have changed according to the definition of $F^\star _\beta$, causing all states to be accepting. Note, however, that this does not mean that the resulting NFAs accept any finite word. For instance, the NFA  in Fig. 2a is a counterexample to this claim. $\square$

Fig. 2

The NFAs constructed in Step 2 of our monitor construction

The next lemma now states that $\mathcal {B}_\beta ^\varphi$ indeed recognizes prefixes of words in ${\mathcal {L}}(\mathcal {A}_\beta ^\varphi )$.

Lemma 7

Let $\beta \in \mathbb {B}_4$ and $u \in \varSigma ^*$. Then, $u \in {\mathcal {L}}(\mathcal {B}_\beta ^\varphi )$ if and only if there exists an infinite word $\sigma \in \varSigma ^\omega$ with $V_r(u\sigma , \varphi ) = \beta$.

Proof

We show both directions separately.

From left to right Assume $u \in {\mathcal {L}}(\mathcal {B}_\beta ^\varphi )$. Moreover, let $q \in F^\star _\beta$ be the accepting state reached by $\mathcal {B}_\beta ^\varphi$ on an accepting run on u (which exists since $u \in {\mathcal {L}}(\mathcal {B}^\varphi _\beta )$). By definition of $F^\star _\beta$, this means that ${\mathcal {L}}(\mathcal {A}^\varphi _\beta (q)) \ne \emptyset$, say $\sigma \in {\mathcal {L}}(\mathcal {A}^\varphi _\beta (q))$. Since $\mathcal {A}^\varphi _\beta$ and $\mathcal {B}^\varphi _\beta$ share the same transition structures, the run of $\mathcal {B}^\varphi _\beta$ on u is also a run of $\mathcal {A}^\varphi _\beta$ on u, which both lead to state q. Therefore, $u\sigma \in {\mathcal {L}}(\mathcal {A}^\varphi _\beta )$. By Theorem 2, this is equivalent to $V_r(u\sigma , \varphi ) = \beta$.

From right to left Let $u \in \varSigma ^*$ and $\sigma \in \varSigma ^\omega$ such that $V(u\sigma , \varphi ) = \beta$. By Theorem 2, we have $u\sigma \in {\mathcal {L}}(\mathcal {A}^\varphi _\beta )$. Consider an accepting run of $\mathcal {A}^\varphi _\beta$ on $u\sigma$, and let q be the state that $\mathcal {A}^\varphi _\beta$ reaches after reading the finite prefix u. Since $u\sigma \in {\mathcal {L}}(\mathcal {A}^\varphi _\beta )$, this means that $\sigma \in {\mathcal {L}}(\mathcal {A}^\varphi _\beta (q))$. Thus, $q \in F^\star _\beta$ because ${\mathcal {L}}(\mathcal {A}_\beta ^\varphi (q)) \ne \emptyset$. Moreover, since the run of $\mathcal {A}^\varphi _\beta$ on u is also a run of $\mathcal {B}^\varphi _\beta$ on u, the NFA $\mathcal {B}^\varphi _\beta$ can also reach state q after reading u. Therefore, $u \in {\mathcal {L}}(\mathcal {B}^\varphi _\beta )$ since $q \in F^\star _\beta$. $\square$

Before we continue to the last step in our construction, let us briefly comment on the complexity of computing the NFAs $\mathcal {B}_\beta ^\varphi$. Since $\mathcal {B}_\beta ^\varphi$ and $\mathcal {A}_\beta ^\varphi$ share the same underlying transition structure, we immediately obtain $|\mathcal {B}_\beta ^\varphi | \in 2^{{\mathcal {O}}(| \varphi |)}$. Moreover, the construction of $\mathcal {B}_\beta ^\varphi$ is dominated by the computation of the set $F_\beta ^\star$ and, hence, can be done in time $2^{{\mathcal {O}}(| \varphi |)}$.

Step 3 In the final step, we construct a Moore machine implementing an rLTL monitor for $\varphi$. Formally, a Moore machine is a five-tuple $\mathcal {M}= (Q, q_I, \delta , \varGamma , \lambda )$ consisting of a deterministic transition structure $(Q, q_I, \delta )$, an output alphabet $\varGamma$, and an output function $\lambda :Q \rightarrow \varGamma$. The size of $\mathcal {M}$ as well of runs of $\mathcal {M}$ are defined as for DFAs. In contrast to a DFA, however, a Moore machine $\mathcal {M}$ computes a function $\lambda _\mathcal {M}:\varSigma ^* \rightarrow \varGamma$ that is defined by $\lambda _\mathcal {M}(u) = \lambda (q_n)$ where $q_n$ is the last state reached on the unique finite run $q_0 \cdots q_n$ of $\mathcal {M}$ on its input $u \in \varSigma ^*$.

The first step in the construction of the Moore machine is to determinize the NFAs $\mathcal {B}^\varphi _\beta$, obtaining equivalent DFAs $\mathcal {C}^\varphi _\beta = (Q_\beta ',q_{I,\beta }', \delta _\beta ',F_\beta ')$ of at most exponential size in $| \mathcal {B}^\varphi _\beta |$. Subsequently, we combine these DFAs into a single Moore machine $\mathcal {M}_\varphi$ implementing the desired rLTL monitor. Intuitively, this Moore machine is the product of the DFAs $\mathcal {C}^\varphi _\beta$ for each $\beta \in \mathbb {B}_4$ and tracks the run of each individual DFA on the given input. Formally, $\mathcal {M}_\varphi$ is defined as follows.

Definition 4

Let $\mathbb {B}_4 = \{ \beta _1, \beta _2, \beta _3, \beta _4, \beta _5 \}$. We define $\mathcal {M}_\varphi = (Q, q_I, \varGamma , \delta , \lambda )$ by

• $Q = Q_{\beta _1}' \times Q_{\beta _2}' \times Q_{\beta _3}' \times Q_{\beta _4}' \times Q_{\beta _5}'$;

• $q_I= (q_{I,{\beta _1}}', q_{I,{\beta _2}}', q_{I,{\beta _3}}', q_{I,{\beta _4}}', q_{I,{\beta _5}}')$;

• $\delta \bigl ( (q_1, q_2, q_3, q_4, q_5), a \bigr ) = (q_1', q_2', q_3', q_4', q_5')$ where $q_j' = \delta _{\beta _j}'(q_j, a)$ for each $j \in \{1, \ldots , 5\}$;

• $\varGamma = \mathbb {B}_4^?$; and

• $\lambda \bigl ( (q_1, q_2, q_3, q_4, q_5) \bigr ) = \xi \bigl ( \bigl \{ \beta _j \in \mathbb {B}_4 \mid q_j \in F_{\beta _j}', j \in \{1, \ldots , 5\} \bigr \} \bigr )$,

where the surjective function $\xi :2^{\mathbb {B}_4} \rightarrow \mathbb {B}_4^{\mathord {?}}$ translates sets $B \subseteq \mathbb {B}_4$ of truth values to the robust monitoring semantics as follows: $\xi (B) = \beta ^{\mathord {?}} \in \mathbb {B}_4^?$ with

$$\begin{aligned} \beta ^{\mathord {?}}[j] = {\left\{ \begin{array}{ll} 0 &{} \text {if}\, \beta [j] = 0\,\text { for each}\, \beta \in B; \\ 1 &{} \text {if}\, \beta [j] = 1\,\text { for each}\, \beta \in B;\,\text { and} \\ \mathord {?}&{} \text {otherwise.} \end{array}\right. } \end{aligned}$$

Let us illustrate this last step of our construction by means of our running example.

Example 6

Given the NFAs from Step 2 of our construction, we first apply a standard determinization step. This process results in equivalent DFAs , which are shown in Fig. 3.

The final, minimized monitor , which results from the Cartesian product of all DFAs, is shown in Fig. 4. Note that this monitor has four different verdicts, shown as labels next to each state. These are four of the verdicts used to prove results in Table 2 (on Page 13). $\square$

Fig. 3

The DFAs constructed in Step 3 of our monitor construction

Fig. 4

The final monitor

The main result of this paper now shows that the Moore machine $\mathcal {M}_\varphi$ implements $V_r^m$, i.e. , we have $\lambda _{\mathcal {M}_\varphi }(u) = V_r^m (u, \varphi )$ for every prefix u.

Theorem 3

For every rLTL formula $\varphi$, one can construct an rLTL monitor of size $2^{2^{{\mathcal {O}}(| \varphi |)}}$.

Proof

First observe that $\xi$ indeed produces a valid value of $\mathbb {B}_4^{\mathord {?}}$ (i.e. , a truth value of the form $0^*?^*1^*$). This follows immediately from the definition of $\xi$ and the fact that the truth values of rLTL are sequences in $0^*1^*$.

Next, we observe that $\mathcal {M}_\varphi$ reaches state $(q_1, q_2, q_3, q_4, q_5)$ after reading a word $u \in \varSigma ^*$ if and only if for each $\beta _j \in \mathbb {B}_4$ the DFA $\mathcal {C}^\varphi _{\beta _j}$ reaches state $q_j$ after reading u. A simple induction over the length of inputs fed to $\mathcal {M}_\varphi$ proves this.

Now, let us fix a word $u \in \varSigma ^*$ and assume that $(q_1, q_2, q_3, q_4, q_5)$ is the state reached by $\mathcal {M}_\varphi$ after reading u. This means that each individual DFA $\mathcal {C}^\varphi _{\beta _j} = (Q'_{\beta _j}, q_{I, \beta _j}', \delta '_{\beta _j}, F'_{\beta _j})$ reaches state $q_j$ after reading u. Let now

$$\begin{aligned} B = \bigl \{ \beta _j \in \mathbb {B}_4 \mid q_j \in F_{\beta _j}', j \in \{1, \ldots , 5\} \bigr \} \end{aligned}$$

as in the definition of the output function $\lambda$ of $\mathcal {M}_\varphi$. By applying Lemma 7, we then obtain

$$\begin{aligned} \beta _j \in B \Leftrightarrow q_j \in F'_{\beta _j} \Leftrightarrow u \in L(\mathcal C^\varphi _{\beta _j}) \Leftrightarrow u \in L(\mathcal B^\varphi _{\beta _j}) \Leftrightarrow \exists \sigma \in \varSigma ^\omega :V_r(u\sigma , \varphi ) = \beta _j. \end{aligned}$$

To conclude the proof, it is left to show that $\xi (B) = V_r^m (u, \varphi )$. We show this for each bit individually using a case distinction over the elements of $\mathbb {B}^{\mathord {?}}= \{ 0, ?, 1 \}$. So as to clutter this proof not too much, however, we only discuss the case of $\mathord {?}$ here, while noting that the remaining two cases can be proven analogously. Thus, let $i \in \{ 1, \ldots , 4 \}$. Then,

$$\begin{aligned} \xi (B)[i] = \mathord {?}&\Leftrightarrow \exists \beta , \beta ' \in B :\beta [i] = 0 \text { and } \beta '[i] = 1 \\&\Leftrightarrow \exists \sigma _0, \sigma _1 \in \varSigma ^\omega :V_r(u\sigma _0, \varphi )[i] = 0 \text { and } V_r(u\sigma _1, \varphi )[i] = 1 \\&\Leftrightarrow V_r^m (u, \varphi )[i] = \mathord {?}. \end{aligned}$$

Since $\lambda \bigl ( (q_1, q_2, q_3, q_4, q_5) \bigr ) = \xi (B)$, the Moore machine $\mathcal {M}_\varphi$ indeed outputs $V_r^m (u, \varphi )$ for every word $u \in \varSigma ^*$. Moreover, $\mathcal {M}_\varphi$ has $2^{2^{{\mathcal {O}}(| \varphi |)}}$ states because the DFAs $\mathcal {C}^\varphi _\beta = (Q_\beta ', q_{I,\beta }',\delta _\beta ',F_\beta ')$ are of at most exponential size in $| \mathcal {B}^\varphi _\beta |$, which in turn is at most exponential in $| \varphi |$. In total, this proves Theorem 3. $\square$

In a final post-processing step, we minimize $\mathcal {M}_\varphi$ (e.g., using one of the standard algorithms for deterministic automata). As a result, we obtain the unique minimal monitor for the given rLTL formula.

It is left to determine the complexity of our rLTL monitor construction. Since each DFA $\mathcal {C}^\varphi _\beta$ is in the worst case exponential in the size of the NFA $\mathcal {B}_\beta ^\varphi$, we immediately obtain that $\mathcal {C}^\varphi _\beta$ is at most of size $2^{2^{{\mathcal {O}}(| \varphi |)}}$. Thus, the Moore machine $\mathcal {M}_\varphi$ is at most of size $2^{2^{{\mathcal {O}}(| \varphi |)}}$ as well and can be effectively computed in doubly-exponential time in $| \varphi |$. Note that this matches the complexity bound of Bauer et al.’s approach for LTL runtime monitoring [14]. Moreover, this bound is tight since rLTL subsumes LTL (see Remark 3): Every monitor for an rLTL formula (without implications) can be turned into a monitor for the corresponding LTL formula by projecting every output to its first bit. Thus, the doubly-exponential bound, which is tight for LTL [14, 40], is also tight for rLTL. Hence, robust runtime monitoring asymptotically incurs no extra cost compared to classical LTL runtime monitoring. However, it provides more useful information as we demonstrate next in our experimental evaluation.

5 Experimental evaluation

Besides incorporating a notion of robustness into classical LTL monitoring, our rLTL monitoring approach also promises to provide richer information than its LTL counterpart. In this section, we evaluate empirically whether this promise is actually fulfilled. More precisely, we answer the following two questions on a comprehensive suite of benchmarks:

• 1. How does rLTL monitoring compare to classical LTL monitoring in terms of monitorability?

• 2. For formulas that are both LTL-monitorable and rLTL-monitorable, how do both approaches compare in terms of the size of the resulting monitors and the time required to construct them?

To answer these research questions, we have implemented a prototype, which we named rLTL-mon. Our prototype is written in Java and builds on top of two libraries: Owl [39], a library for LTL and automata over infinite words, as well as AutomataLib (part of LearnLib [36]), a library for automata over finite words and Moore machines. For technical reasons (partly due to limitations of the Owl library and partly to simplify the implementation), rLTL-mon uses a monitor construction that is slightly different from the one described in Sect. 4: Instead of translating an rLTL formula into nondeterministic Büchi automata, rLTL-mon constructs deterministic parity automata. These parity automata are then directly converted into DFAs, thus skirting the need for a detour over NFAs and a subsequent determinization step. Note, however, that this alternative construction produces the same rLTL monitors than the one described in Sect. 4. Moreover, it has the same asymptotic complexity. The sources of our prototype are available online under the MIT license.⁷

5.1 Benchmarks and experimental setup

The starting point of our evaluation was the original benchmark suite of Bauer et al. [14], which is based on a survey by Dwyer on frequently used software specification patterns [23]. This benchmark suite consists of 97 LTL formulas and covers a wide range of patterns, including safety, scoping, precedence, and response patterns. For our rLTL monitor construction, we interpreted each LTL formula in the benchmark suite as an rLTL formula (by treating every operator as a robust operator).

We compared rLTL-mon to Bauer et al.’s implementation of their LTL monitoring approach, which the authors named LTL $_{3}$ tools. This tool uses LTL2BA [32] to translate LTL formulas into Büchi automata and AT&T’s fsmlib as a means to manipulate finite-state machines. Since LTL2BA’s and Owl’s input format for LTL formulas do not match exactly, we have translated all benchmarks into a suitable format using a python script.

We conducted all experiments on an Intel Core i5-6600 @ 3.3 GHz in a virtual machine with 4 GB of RAM running Ubuntu 18.04 LTS. As no monitor construction took longer than $600\,s$, we did not impose any time limit.

5.2 Results

Our evaluation shows that LTL $_{3}$ tools and rLTL-mon are both able to generate monitors for all 97 formulas in Bauer et al.’s benchmark suite.⁸ Aggregated statistics of this evaluation are visualized in Fig. 5.⁹

Fig. 5

Comparison of rLTL-mon and LTL $_{3}$ tools on Bauer et al.’s benchmarks [14]

The histogram in Fig. 5a shows the aggregate number of LTL and rLTL monitors with respect to their number of states. As Bauer et al. already noted in their original work, the resulting LTL monitors are quite small (none had more than six states), which they attribute to Dwyer et al.’s specific selection of formulas [23]. A similar observation is also true for the rLTL monitors: None had more than eight states.

To determine which formulas are monitorable and which are not, we used a different, though equivalent definition, which is easy to check on the monitor itself: an LTL formula (rLTL formula) is monitorable if and only if the unique minimized LTL monitor (rLTL monitor) does not contain a sink-state with universal self-loop that outputs “$\mathord {?}$” (that outputs “$\mathord {?}\mathord {?}\mathord {?}\mathord {?}$”). In other words, even if a finite word does not allow us to infer anything about the satisfaction of the LTL (rLTL) formula by infinite words extending it, it can always be extended into another finite word that does. Bauer et al. report that 44.3LTL monitors (43 out of 97) have this property (in fact, exactly the 43 LTL monitors with a single state), which means that 44.3% of all formulas in their benchmark suite are not LTL-monitorable. By contrast, all these formulas are rLTL-monitorable. Moreover, in 78.4% of the cases (76 out of 97), the rLTL monitor has more distinct outputs than the LTL monitor, indicating that the rLTL monitor provides more fine-grained information of the property being monitored; in the remaining 21.6%, both monitors have the same number of distinct outputs. These results answer our first research question strongly in favor of rLTL monitoring: rLTL monitoring did in fact provide more information than its classical LTL counterpart. In particular, only 55.7% of the benchmarks are LTL-monitorable, whereas 100% are rLTL-monitorable.

Let us now turn to our second research question and compare both monitoring approaches on the 54 formulas that are both LTL-monitorable and rLTL-monitorable. For these formulas, Fig. 5b further provides statistical analysis of the generated monitors in terms of their size (left diagram) as well as the time required to generate them (right diagram). Each box in the diagrams shows the lower and upper quartile (left and right border of the box, respectively), the median (line within the box), and minimum and maximum (left and right whisker, respectively).

Let us first consider the size of the monitors (left diagram of Fig. 5b). The majority of LTL monitors (52) has between two and four states, while the majority of rLTL monitors (45) has between two and five states. For 21 benchmarks, the LTL and rLTL monitors are of equal size, while the rLTL monitor is larger for the remaining 33 benchmarks. On average, rLTL monitors are about 1.5 times larger than the corresponding LTL monitors.

Let us now discuss the time taken to construct the monitors. As the diagram on the right-hand-side of Fig. 5b shows, LTL $_{3}$ tools was considerably faster than rLTL-mon on a majority of benchmarks (around $0.1\,s$ and $2.6\,s$ per benchmark, respectively). For all 54 benchmarks, the rLTL monitor construction took longer than the construction of the corresponding LTL monitor (although there are two non-LTL-monitorable formulas for which the construction of the rLTL monitor was faster). However, we attribute this large runtime gap partly to the overhead caused by repeatedly starting the Java virtual machine, which is not required in the case of LTL $_{3}$ tools. Note that this is not a concern in practice as a monitor is only constructed once before it is deployed.

Finally, our analysis answers our second question: rLTL monitors are only slightly larger than the corresponding LTL monitors and although they require considerably more time to construct, the overall construction time was negligible for almost all benchmarks.

6 Conclusion

We adapted the three-valued LTL monitoring semantics of Bauer et al. to rLTL, proved that the construction of monitors is asymptotically no more expensive than the one for LTL, and validated our approach on the benchmark of Bauer et al.: All formulas are rLTL-monitorable and the rLTL monitor is strictly more informative than its LTL counterpart for 77% of their formulas.

Recall Theorem 1, which states that the truth values 0011 and 0001 are not realizable. This points to a drawback regarding the two middle bits: When considering the formula , the second bit represents and the third bit . A prefix cannot possibly provide enough information to distinguish these two formulas. On the other hand, the truth value ??11 is realizable, which shows that the middle bits can be relevant. In further work, we will investigate the role of the middle bits in rLTL monitoring.

Moreover, the informedness of a monitor can be increased further when attributing a special role to the last position(s) of a prefix. Even though a prefix of the form $\emptyset ^+\{ a \}^+$ does not fully satisfy , neither does it fully violate it. If the system just now reached a state in which $\{ a \}$ always holds, an infinite continuation of the execution would satisfy the specification. So rather than reporting an undetermined result, the monitor could indicate that an infinite repetition of the last position of the prefix would satisfy the specification. Similarly, for a prefix $\{ a \}^+\emptyset$, the specification is undetermined. While an infinite repetition of the last position ($\{ a \}^+\emptyset ^\omega$) does not satisfy the specification, an infinite repetition of the last two positions ($\{ a \}^+(\emptyset \{ a \})^\omega$) would. We plan to investigate an extension of rLTL which takes this observation into account.

Bauer et al. [12] proposed an orthogonal approach with the logic RV-LTL. Here, the specification can contain the strong (weak) next-operator whose operand is consider violated (satisfied) at the last position of the trace. A formula that is undetermined under the strong semantics and satisfied (violated) under the weak semantics is considered potentially true (potentially false). Incorporating one of these approaches into rLTL monitoring could refine its output and thus increase its level of informedness.

Moreover, desired properties for cyber-physical systems often include real-time components such as “touch the ground at most 15 s after receiving a landing command”. Monitors for logics taking real-time into account [15], such as STL [43, 44], induce high computational overhead at runtime when compared to LTL and rLTL monitors. Thus, a real-time extension for rLTL retaining its low runtime cost would greatly increase its viability as specification language.

Experimental results

The following two tables provide detailed results of our experimental evaluation.

Table 3 Summary of the result when comparing the monitor construction of rLTL against LTL; continued in Table 4

Table 4 Summary of the result when comparing the monitor construction of rLTL against LTL; continuation of Table 3