Output analysis for terminating simulations with partial observability

https://doi.org/10.1016/j.simpat.2016.12.002Get rights and content

Highlights

  • It is often difficult to initialize simulation state using observed empirical data.

  • We identify types of partial observability of initial state of simulation.

  • For each type we propose methods that allow for simulation output analysis.

  • In particular, we consider situation when initial simulation state is a rare event.

Abstract

In this paper we propose methods of output analysis for terminating simulations when the researcher is unable to fully specify the initial state of the simulation using observations of the real system. We call this situation “partial observability” and argue that it is common in practice, especially in the case of complex agent-based simulations. We provide classification of situations where not all input parameters or simulation state variables are observable and for each case we propose a method of terminating simulation output analysis. In particular, we focus on the situation where a rare event needs to be analyzed, since it requires careful design of a simulation experiment in order to minimize the computational budget and avoid bias in the output predictions.

Introduction

Complex real-world phenomena that are dynamic and include random behavior are often modeled using stochastic simulations [20], [24]. Two mainstream approaches to analysis of such models concentrate on performance of the system at a given point in time or in a given state (terminating simulation output analysis) or over the long-run (non-terminating or steady state simulation output analysis), see. e.g. Banks et al. [6]. In this text we concentrate on the terminating case. This case is usually considered less challenging statistically than steady state analysis but, as Law [18] emphasizes, it is no less important because practitioners are often interested in the behavior of the modeled system under defined initial conditions.

In a terminating simulation output analysis scenario we want to obtain an evaluation of the simulation output given some well specified initial conditions and a terminating event condition [1], [23]. The key challenge of such an analysis is proper treatment of the initial state of the simulation. Many standard references, e.g. Alexopoulos and Seila [2] do not concentrate on this issue, assuming that the initial state is fully specified. In the texts which discuss the challenges with specifying the initial state of the simulation, two standard recommendations are considered. The first of these is to collect the appropriate empirical data that allows the initial state to be determined and the second one is to use a warmup period [8], [20].

In this text we analyze the situation where we have a simulator S of the real system. We have knowledge about certain characteristics of the real system at time t and want to predict a value of one (or more) of its characteristics at some future moment t+Δt, where Δt is either deterministic or determined by some random terminating event condition. If knowledge of the real system is sufficient to fully specify the initial state of the simulator S at time t then we can follow the standard methods recommended for terminating simulation output analysis [18] to evaluate the output of the simulation at time t+Δt. It is worth highlighting that knowledge of the characteristics of a real system at time t can mean either that the researcher knows deterministic values defining the state or their distribution – both cases are covered by standard methods. However, very often the analyst is unable to collect all the data in the real system that is needed to specify the initial state of simulation S.

In this text we propose methods allowing formulation of simulation output predictions at time t+Δt given that knowledge of the characteristics of the real system is insufficient to fully specify the initial state of simulation S at time t. In some cases, as we show in Section 3, it is possible to make such predictions for any type of simulation. However, if we lack information about the dynamic characteristics of the simulation, some additional information is required to make predictions.

In this text we make an additional assumption that that the simulation is non-terminating and stable. By this we mean that the simulation can be run for an arbitrarily long time and has a steady state distribution, see [5]. A basic example of such a simulation model is a M/M/1 queue with the arrival rate less than the service rate. We wish to make a prediction assuming that the system at time t has been already running for a long time and it is possible to assume that the unconditional distribution of its state at time t follows the steady state distribution (at least approximately). In our M/M/1 queue example this means that we choose such t, that the distribution of number of customers in the system at this time does not depend on the number of customers in the system when the simulation is started.

Using the assumption that the considered simulation at time t is unconditionally in steady state we are able to reason about the distribution of the unknown part of the state at time t conditional on the partial information about the specification of the state at time t which the researcher knows. Later in this section we present two detailed examples of such situations.

The above assumption, in particular, implies that the proposed methodology is not directly applicable to simulations that either do not have steady state or where it is not reasonable to assume that they can reach it in practice, e.g. simulation of one day of operations of a warehouse that only dispatches products during the day so the state of the stock only decreases.

Formally, in terminating simulation output analysis, we measure the output value y at some moment TE given the simulation initial state C at some earlier time t. It is assumed that C and TE are well specified random variables [18]. In consequence it is typically assumed [19] that repeated runs of the simulation generate outputs y that are independent and identically distributed. This reasoning is based on the assumption that it is possible to specify C. We argue that in practice this is often difficult. Following the terminology from control theory [14] we maintain that the state of simulation is observable at time t if it is possible to determine it using information collected from the real system up to time t. In a simulation modeling context we allow for the state to be identified deterministically or as a probability distribution.

If the state of simulation is not observable we specify that it is partially observable, that is we assume that not all state variables are observable, i.e. for some of the variables defining the state of the simulation it is not possible to identify their value using available information about the real system. Examples of such situations are as follows (in these examples we use standard notation and terminology used in queuing theory, see. e.g. [20]).

Example 1

Consider a G/G/1 queuing system with known arrival and service time distributions. Assume that we are able to measure (in the real system) the number of customers in the system Nt at time t and we seek to predict the number of customers Nt+Δt in the system at some well specified future time t+Δt. If the system were memoryless (e.g. M/M/1 queue) then Nt, along with information about arrival and service rates, would constitute a full specification of the simulation state at time t. Thus the parameter Nt is observable and we have no state variables that are unobservable. However, in the case of a G/G/1 queue, full specification of the state would also include the time which elapsed since the last arrival of the customer and the time the current service process has taken so far. Given that we are able to measure Nt alone in the G/G/1 system, the state of the simulation is partially observable.

In this text we propose a method that allows the prediction of Nt+Δt given knowledge of Nt alone, conditional upon the queuing system being stable and that in time t the simulation state distribution is unconditionally (without conditioning on Nt) a steady state distribution, at least approximately. In other words, when t is sufficiently large that the initial conditions, which are set when the system started its operations, do not influence the distribution of the system’s state at time t.

Example 2

The system we want to investigate is an M/M/1 queue with two types of customers A and B arriving with equal probability. Assume that at a given time t we can measure how many customers of each type are in the real system but not in which sequence they have entered it. By At denote the number of customers of type A in the system at time t and analogously define Bt. We are interested in prediction of At+Δt given that we know At and Bt. Observe, that in such a case, the state of simulation at time t can be represented as some sequence of customers of type A and B along with information about arrival and service rates. We assume that we do not know this sequence. The measured values At and Bt, however, as opposed to Nt in Example 1 are not part of the state of the simulation, yet they can be computed given that we knew this state (the sequence of As and Bs). Therefore only the simulation output, i.e. At and Bt, is measurable and the simulation state is not observable. Even though no state variables are observed we still designate this situation as partially observable since we can use knowledge of the simulation output to draw guidance about its state as they are related (though the mapping from outputs to state is not injective).

In this paper we propose a method for predicting At+Δt given that we know that At=α and Bt=β (again assuming that the queuing system is stable and has been already running for a long time at time t).

In fact, we argue that in complex simulation models the problem with partial observability is common. Consider for example an agent-based macroeconomic simulation model that allows for heterogeneous households [10], [22]. If we sought to use such a model for prediction we are faced with the following challenge: in a real economic system we are able to measure only aggregate characteristics of the households (e.g. mean income, Gini coefficient of income distribution). However, in the simulation model these values are outputs that are not part of simulation state. On the other hand variables that constitute the state – i.e. individual incomes of households – are in practice not measurable on an individual household level. A similar situation is encountered in many other simulations of complex processes e.g. traffic patterns [12], where the simulation state consists of all variables describing individual drivers and in real systems we are able to measure only macro-level aggregate simulation outputs, called KPIs by Hara et al. [12].

Before proceeding further it is worth highlighting that partial observability is a qualitatively richer concept than the empirical data collection or warmup period methods discussed in the simulation literature [8], [20]. We will present this difference using the textbook example proposed by Law [20] (section 9.4.3, p. 510).

Example 3

Assume that we would like to simulate the operations of a bank that opens at 9 A.M. and that we want to analyze its performance during the period from 12 noon to 1 P.M.

If we wanted to use the empirical data method we would need to collect the historical information about the number of customers at 12 noon for several days and then use the obtained empirical distribution to initialize the simulation. This scenario efficiently falls under the class of full observability, with the sole notion that the initial state C is not deterministic but it is a random variable.

If we wanted to use the warmup period method then the simplest approach would be to start the simulation at 9 A.M. with a well specified initial condition (the bank is empty) and collect the output data during the period of interest. Thus, we once again have full observability of the simulation state. Law [20] considers a modification of this scenario that is closer to the partial observability case. He assumes that we start a simulation at 11 A.M. and are unable to measure the number of customers at this moment (in contrast to 9 A.M. where it is measurable by definition of the process). In such a case the initial state of the simulation at 11 A.M. is unobservable but we assume that by running it until 12 noon distribution of its state will reach (at least approximately) a steady state distribution. Observe that in both examples of the warmup period method we assume that at 12 noon the system state is not observable at all (no information about the true system is measured at this moment).

In this paper we are interested in situations that lie between the extremes of historical and warmup methods, i.e. the cases where i) we have to start running the simulation at 11 A.M. without full knowledge of the initial state or ii) we are starting it at 9 A.M. with an empty system but in both cases where we are able to measure some information describing the state of the real system at 12 noon and wish to take it into consideration when predicting the simulation output until 1 P.M.

Apart from the discussion above, in the literature there exist three streams of research related to the proposed approach. The first is a methodology proposed by Poropudas and Virtanen [27], [28] (respectively for continuous and discrete time analysis). Similarly to our approach they concentrate on the modeling of dynamic dependency between simulation state and simulation output, i.e. “(...) how the fixed simulation state affects the following simulation” [28], and employ a metamodeling [7], [15] approach. The limitations of this approach are as follows. Firstly, the proposed methodology does not explain how to select variables tracked by the metamodel. That is – using terminology proposed in this text – there is no discussion of the explicit distinction between state and output variables and which of them should be used to build a metamodel. Secondly, the proposed metamodeling approach uses dynamic Bayesian networks and in consequence it: (a) requires discretization of all variables of interest and (b) requires explicit specification of the simulation time span to which the metamodel is fitted. Finally, as Poropudas and Virtanen [28] themselves observe, the proposed method does not efficiently handle the cases in which the simulation state of interest is a rare event. In this paper we propose a framework that removes these limitations and in particular allows results of the simulation to be analyzed either by using standard statistical procedures [4], or by fitting an arbitrary suitable metamodel [16].

The second stream of research is discussed by Hara et al. [12], who proposed using the intermediate outputs of an agent-based simulation to predict its final outputs. They observe that the intermediate state of a complex simulation is usually of such high dimensionality that it is not feasible to be directly used to predict the final simulation output. Thus, they introduce a concept of compact features, which are aggregates of simulation state variables and can be used to build a metamodel which predicts outputs of interest. The motivation of Hara et al. [12] to introduce compact features is of a technical nature – it makes building the metamodel simpler. In this text we argue that such variables (classified as observable output variables) should be and are used in such situations not only for the sake of computational convenience but because it is often the only way to anchor a simulation system to the empirical data. For instance consider the example of Hara et al. [12], who discuss a traffic simulator. The state of the simulator consists of the origins and destinations of all the agents, their personality, route, speed, etc.[25]. In practice we may not be able to count on having all this information – it is not observable. However, we might know a number of cars in a given region, their average speed etc. – variables that can be measured but are simulation output, not state variables. Thus the concept of compact features is a special case of the more general framework discussed in this text.

The third stream of research is related to simulation based methods studied in computational econometrics literature, see e.g. Gouriéroux and Monfort [11]. However, in this class of methods it is assumed that the part of the simulation state which cannot be measured in the real system consists of fixed parameters. Referring to such a case, we also concentrate on dynamic state variables that are unobservable.

The remainder of the paper is structured as follows. In Section 2 we formally define a notion of partial observability of simulation state variables and propose classification of simulation models depending on which types of variables are present in them. Next in Section 3 we discuss how simulation analysis should be performed in partial observability situations. Section 4 presents numerical examples of proposed procedures and Section 5 provides a conclusion. Following the standard requirements for computational research [26], in order to ensure reproducibility of the results all the source code and data used to generate the presented results is available for download at http://bogumilkaminski.pl/pub/dynamet.zip.

Section snippets

Proposed approach to partial observability

In this section we firstly define the classification of variables in the simulation model taking into account partial observability of the simulation state. Secondly, in this context we formulate a terminating simulation output prediction problem.

Prediction methods

In the given framework the types of prediction are classified in Fig. 2 and are consequently described in the following subsections.

Numeric examples

We shall use a single server queuing model as an example of the proposed methodology. We cover cases B2a), B2b) and B3).

Concluding remarks

In this text we have proposed a framework for the prediction of terminating simulation output, given that not all elements of the simulation state are observable. The key issue in the discussed methodology is whether the hidden variables are input parameters or dynamic state variables. In the text we cover all possible combinations of the cases (purely input parameters, purely dynamic state variables and both). A limitation of the proposed methodology is that when hidden variables represent a

Acknowledgments

The research was financed by the funds obtained from National Science Centre, Poland, granted following the decision number DEC-2013/11/B/HS4/02120.

References (29)

  • C. Currie et al.

    A Practical Introduction to Analysis of Simulation Output Data

  • A.C. Davison et al.

    Bootstrap Methods and Their Application

    (1997)
  • C. Gouriéroux et al.

    Simulation-Based Econometric Methods

    (1996)
  • S. Hara et al.

    Predicting Halfway through Simulation: Early Scenario Evaluation Using Intermediate Features of Agent-based Simulations

  • View full text