Adaptive threshold computation for CUSUM-type procedures in change detection and isolation problems

https://doi.org/10.1016/j.csda.2008.01.026Get rights and content

Abstract

Statistical methods dealing with change detection and isolation in dynamical systems are based on algorithms deriving from hypothesis testing. As for any statistical test, the problem of threshold choice has to be addressed by taking into account the constraints fixed by the supervisors and the nonstationary nature of the stochastic systems under supervision. A procedure for obtaining adaptive thresholds in change detection or diagnosis algorithms of CUSUM-type rules is proposed. This procedure is carried out through a large number of simulations. The advantage of such an adaptive threshold, when compared with a fixed threshold, is its adaptation to the time evolution of the probability distribution of the test statistic, in order to guarantee constant rates of false alarm or false diagnosis, fixed by the supervisor.

Introduction

Process supervision is crucial in many industries (car, food, waste-water treatment, etc.), notably with respect to quality control. One step in this supervision is the detection and diagnosis of faults corresponding to model changes. The aim of change detection is to implement a decision rule which is capable of detecting, as rapidly as possible, a change from an in-control state, called Hypothesis 0, to an out-of-control state (i.e. fault) called Hypothesis 1, corresponding to an unforeseeable change of certain process-related parameters. When applying such a decision rule, the supervisor hopes that the time elapsing before detection will be short and that there will be few false alarms.

Control theory and artificial intelligence are the approaches most frequently adopted to tackle this challenge–see Blanke et al. (2003) and references therein. The theory of sequential statistical tests also offers interesting alternatives and brings a complement to these traditional approaches–see Basseville and Nikiforov (1993) or more recently, Frisen (2007), for an overview. This theory proposes well-adapted tools dealing with rates of false alarms and of non-detection.

The most frequently met sequential tests are based on the log-likelihood ratio between hypotheses H1 and H0. Functional states are characterized by a vector of parameters θ which may vary from a known value θ0 in the in-control mode H0 to a value θ1 in the fault mode H1. The best known algorithm, introduced by Page (1954), is the cumulative sum (CUSUM) algorithm which deals with a known value θ1 and an unknown but nonrandom change time. The CUSUM statistic has been generalized and used in other tests according to different assumptions on the system under supervision. For example, the generalized likelihood ratio statistics (GLR, see Willsky and Jones (1976)) assumes that θ1 is unknown but belongs to a known compact set. Several optimality results have been obtained for CUSUM and GLR rules (Lai, 1998, Tartakovsky et al., 2006).

All these test statistics are compared at each time point with a threshold, an alarm being triggered as soon as this threshold is exceeded. The choice of this threshold results from a trade-off between the time delay to detection and the rate of false alarms. If the threshold is too high, detection will be delayed, while if it is too low, the rate of false alarms will be excessive. Traditionally, constant thresholds are used. In this case, the relevance of the decision is largely affected by the fluctuations of the test statistic in the fault free case H0. These fluctuations are the most important with nonstationary systems, like the industrial controlled systems under supervision which were mentioned above. The detection can be strongly improved if the thresholds are adapted to the time evolution of the test statistic probability distributions. This leads to time-varying thresholds, also known as adaptive thresholds. This notion is often refered to in the field of process control (see Frank (1996), for example) which mainly uses tools from applied deterministic mathematics. When considering stochastic perturbations, the advantage of adaptive thresholds is that they can guarantee a constant rate of false alarms throughout the process. This idea has recently been adopted in the process control community. Le et al. (1997) use a multi-step ahead system output prediction which they compare to a confidence bound. But the noise assumptions are rather restrictive. Shi et al. (2005) also obtain confidence intervals in approximating the test statistic to a Gaussian distribution in spite of a nonstationary stochastic process. Ding et al. (2003) try to link the computations of adaptive threshold and of false alarm rate for a given linear state-space system. The test statistic is a norm-based residual evaluation.

Genarally, methods of threshold computation are mostly developed for distribution-free statistical process control charts, in the case of independent variables, identically distributed before and after the change time (Knoth, 2006) and even for autocorrelated data (Kim et al., 2007). For model-based approach, it is often impossible to obtain an approximation of the law of the test statistic. The thresholds, usually fixed, are then determined by experimentation (see for example El Falou et al. (2007)). There are rather few references in the case of dependent and nonstationary data that is at stake when considering industrial dynamic systems. For this type of processes, a fixed threshold is not adapted.

In the present paper we propose an adaptive threshold estimation scheme in detection algorithms of CUSUM-type. This estimation scheme is based on a criterion that ensures a constant rate of false alarms throughout the process. We then extend this approach to an algorithm of change diagnosis by considering the generalization of the CUSUM proposed by Nikiforov, 2000, Nikiforov, 2003 to the problem of multi-hypotheses change detection. Our purpose is not to find the theoretical value of the threshold in a detection or diagnosis procedure, but rather to propose an algorithm, easy to carry out, that provides threshold values which guarantee the satisfaction of constraints fixed by the supervisor, in spite of fluctuations of the test statistic distribution in the fault free case.

The paper is organized as follows. In Section 2, we remind the CUSUM detection rule in the i.i.d. and dependent cases. The choice of the threshold is also discussed according to optimal properties. The adaptive threshold estimation scheme is then presented in Section 3. The procedure is extended to mode change diagnosis in Section 4. Finally, two simulated case study trials are presented in Section 5 and show the relevance of the proposed adaptive threshold estimation scheme.

Section snippets

Optimality of the CUSUM procedure for i.i.d. observations

The CUSUM rule is one of the most famous change detection algorithms. It was first proposed by Page (1954) who treats the case of independent and identically distributed observations before and after the change time. Denote tp this change time. Let X1,,Xtp1 be independent random variables with a common density function pθ0 and Xtp,Xtp+1, be independent with density pθ1. Denote P(tp) the probability measure for this distribution and E(tp) the expectation associated with this measure. The aim

A new constraint

In order to characterize the adaptive threshold, let us first consider a constraint other than that of Lorden. Let us consider the conditional false alarm probability and choose, at time n, the threshold hn such that: n=1,Pθ0[g1h1]=αandn2,Pθ0[gnhn|g1<h1,,gn1<hn1]=α, where α is fixed by the supervisor. At each time step n, hn is chosen such that the probability to have a false alarm at time n while there were none before, is equal to α. The value hn which satisfies the constraint (7) is

Extension to change diagnosis

In the previous section the problem of parameter change detection was studied. The parameter characterizing the change takes only two values: θ0 for the in-control mode and θ1 for the out-of-control mode. However, system monitoring does not have to be limited to the detection of only one change. Several types of change may occur on a process, some being more serious than others and statistical methods dealing with system monitoring have also to be able to deal with the change diagnosis problem.

Simulations

Two simulated case studies have been performed in Matlab®: the first one is concerned with a change detection in a strongly non stationary model. The second deals with a more realistic model of biotechnological process. Both simulations show the relevance of the adaptive threshold.

Conclusion

A computational method for the estimation of adaptive thresholds of change detection algorithms like the CUSUM rule has been presented. Moreover this method has been generalized to the CUSUM extension proposed by Nikiforov for diagnosis problems. The particularity of the thresholds thus obtained is that they adapt to the time variations of the test statistic distributions, in order to guarantee constant rates of false alarms or false diagnoses throughout the process. This appears to be of great

References (25)

  • S.-H. Kim et al.

    A distribution-free tabular CUSUM chart for autocorrelated data

    IIE Trans.

    (2007)
  • T.L. Lai

    Sequential changepoint detection in quality control and dynamical systems

    J. Roy. Statist. Soc. Ser. B

    (1995)
  • Cited by (39)

    • Sensitivity clustering and ROC curve based alarm threshold optimization

      2020, Process Safety and Environmental Protection
      Citation Excerpt :

      Meanwhile, combined with 1 as the basic weight for FAR and MAR, a balance can be achieved between FAR and MAR, which can avoid excessive effect produced by the remaining weights (ω1 and ω2). In this work, with advantages of high precision and fast convergence, the particle swarm optimization (PSO) algorithm is used to optimize the abovementioned objective functions in addition to derivation methods, which has been proved to be an effective approach in describing optimal multilevel thresholds (Gao et al., 2009), designing and controlling curing processes (Lu et al., 2010), and other fields. Meanwhile, for the sake of the following optimization and multivariable joint optimization in the future, the shortcomings of the standard PSO algorithm are also carefully considered in the design process of the PSO algorithm to avoid the premature convergence problem:

    • PCA weight and Johnson transformation based alarm threshold optimization in chemical processes

      2018, Chinese Journal of Chemical Engineering
      Citation Excerpt :

      Current threshold setting methods for process variables can be classified into three general categories: model based method, knowledge based method, and process history based method. Although process history based method is not well suitable for real-time alarm management due to its high-capacity historical data requirement, its implementation is easy and efficient compared with the model based [4–10] and knowledge based [11–17] methods. With massive data from DCS, this method has been widely used in process control field.

    • Alarm clustering analysis and ACO based multi-variable alarms thresholds optimization in chemical processes

      2018, Process Safety and Environmental Protection
      Citation Excerpt :

      There have appeared many optimization methods for setting process variable threshold in chemical process. For example, in terms of FAR and MAR, Verdier et al. (2008) improved the change detection algorithm CUSUM to obtain adaptive alarm threshold with constant FAR in process. Izadi et al. (2009a,b) presented a framework about the best filter based on FAR, MAR, time delay, and deadband.

    • Change detection for uncertain autoregressive dynamic models through nonparametric estimation

      2016, Statistical Methodology
      Citation Excerpt :

      The resulting test statistic is then not stationary and the use of a fixed threshold (like it is traditionally the case for the CUSUM) is no more adapted. The computational procedure proposed in Verdier et al. [37] in order to obtain a constant probability of false alarm is then used.

    View all citing articles on Scopus
    View full text