Adaptive threshold computation for CUSUM-type procedures in change detection and isolation problems
Introduction
Process supervision is crucial in many industries (car, food, waste-water treatment, etc.), notably with respect to quality control. One step in this supervision is the detection and diagnosis of faults corresponding to model changes. The aim of change detection is to implement a decision rule which is capable of detecting, as rapidly as possible, a change from an in-control state, called Hypothesis 0, to an out-of-control state (i.e. fault) called Hypothesis 1, corresponding to an unforeseeable change of certain process-related parameters. When applying such a decision rule, the supervisor hopes that the time elapsing before detection will be short and that there will be few false alarms.
Control theory and artificial intelligence are the approaches most frequently adopted to tackle this challenge–see Blanke et al. (2003) and references therein. The theory of sequential statistical tests also offers interesting alternatives and brings a complement to these traditional approaches–see Basseville and Nikiforov (1993) or more recently, Frisen (2007), for an overview. This theory proposes well-adapted tools dealing with rates of false alarms and of non-detection.
The most frequently met sequential tests are based on the log-likelihood ratio between hypotheses and . Functional states are characterized by a vector of parameters which may vary from a known value in the in-control mode to a value in the fault mode . The best known algorithm, introduced by Page (1954), is the cumulative sum (CUSUM) algorithm which deals with a known value and an unknown but nonrandom change time. The CUSUM statistic has been generalized and used in other tests according to different assumptions on the system under supervision. For example, the generalized likelihood ratio statistics (GLR, see Willsky and Jones (1976)) assumes that is unknown but belongs to a known compact set. Several optimality results have been obtained for CUSUM and GLR rules (Lai, 1998, Tartakovsky et al., 2006).
All these test statistics are compared at each time point with a threshold, an alarm being triggered as soon as this threshold is exceeded. The choice of this threshold results from a trade-off between the time delay to detection and the rate of false alarms. If the threshold is too high, detection will be delayed, while if it is too low, the rate of false alarms will be excessive. Traditionally, constant thresholds are used. In this case, the relevance of the decision is largely affected by the fluctuations of the test statistic in the fault free case . These fluctuations are the most important with nonstationary systems, like the industrial controlled systems under supervision which were mentioned above. The detection can be strongly improved if the thresholds are adapted to the time evolution of the test statistic probability distributions. This leads to time-varying thresholds, also known as adaptive thresholds. This notion is often refered to in the field of process control (see Frank (1996), for example) which mainly uses tools from applied deterministic mathematics. When considering stochastic perturbations, the advantage of adaptive thresholds is that they can guarantee a constant rate of false alarms throughout the process. This idea has recently been adopted in the process control community. Le et al. (1997) use a multi-step ahead system output prediction which they compare to a confidence bound. But the noise assumptions are rather restrictive. Shi et al. (2005) also obtain confidence intervals in approximating the test statistic to a Gaussian distribution in spite of a nonstationary stochastic process. Ding et al. (2003) try to link the computations of adaptive threshold and of false alarm rate for a given linear state-space system. The test statistic is a norm-based residual evaluation.
Genarally, methods of threshold computation are mostly developed for distribution-free statistical process control charts, in the case of independent variables, identically distributed before and after the change time (Knoth, 2006) and even for autocorrelated data (Kim et al., 2007). For model-based approach, it is often impossible to obtain an approximation of the law of the test statistic. The thresholds, usually fixed, are then determined by experimentation (see for example El Falou et al. (2007)). There are rather few references in the case of dependent and nonstationary data that is at stake when considering industrial dynamic systems. For this type of processes, a fixed threshold is not adapted.
In the present paper we propose an adaptive threshold estimation scheme in detection algorithms of CUSUM-type. This estimation scheme is based on a criterion that ensures a constant rate of false alarms throughout the process. We then extend this approach to an algorithm of change diagnosis by considering the generalization of the CUSUM proposed by Nikiforov, 2000, Nikiforov, 2003 to the problem of multi-hypotheses change detection. Our purpose is not to find the theoretical value of the threshold in a detection or diagnosis procedure, but rather to propose an algorithm, easy to carry out, that provides threshold values which guarantee the satisfaction of constraints fixed by the supervisor, in spite of fluctuations of the test statistic distribution in the fault free case.
The paper is organized as follows. In Section 2, we remind the CUSUM detection rule in the i.i.d. and dependent cases. The choice of the threshold is also discussed according to optimal properties. The adaptive threshold estimation scheme is then presented in Section 3. The procedure is extended to mode change diagnosis in Section 4. Finally, two simulated case study trials are presented in Section 5 and show the relevance of the proposed adaptive threshold estimation scheme.
Section snippets
Optimality of the CUSUM procedure for i.i.d. observations
The CUSUM rule is one of the most famous change detection algorithms. It was first proposed by Page (1954) who treats the case of independent and identically distributed observations before and after the change time. Denote this change time. Let be independent random variables with a common density function and be independent with density . Denote the probability measure for this distribution and the expectation associated with this measure. The aim
A new constraint
In order to characterize the adaptive threshold, let us first consider a constraint other than that of Lorden. Let us consider the conditional false alarm probability and choose, at time , the threshold such that: where is fixed by the supervisor. At each time step , is chosen such that the probability to have a false alarm at time while there were none before, is equal to . The value which satisfies the constraint (7) is
Extension to change diagnosis
In the previous section the problem of parameter change detection was studied. The parameter characterizing the change takes only two values: for the in-control mode and for the out-of-control mode. However, system monitoring does not have to be limited to the detection of only one change. Several types of change may occur on a process, some being more serious than others and statistical methods dealing with system monitoring have also to be able to deal with the change diagnosis problem.
Simulations
Two simulated case studies have been performed in Matlab®: the first one is concerned with a change detection in a strongly non stationary model. The second deals with a more realistic model of biotechnological process. Both simulations show the relevance of the adaptive threshold.
Conclusion
A computational method for the estimation of adaptive thresholds of change detection algorithms like the CUSUM rule has been presented. Moreover this method has been generalized to the CUSUM extension proposed by Nikiforov for diagnosis problems. The particularity of the thresholds thus obtained is that they adapt to the time variations of the test statistic distributions, in order to guarantee constant rates of false alarms or false diagnoses throughout the process. This appears to be of great
References (25)
Analytical and qualitative model based fault diagnosis — A survey and some new results
Eur. J. Control
(1996)Computation of the ARL for CUSUM- schemes
Comput. Statis. Data Anal.
(2006)- et al.
The development of an adaptive threshold for model-based fault detection of a nonlinear electro-hydraulic system
Control Eng. Practice
(2005) - et al.
Detection of intrusions in information systems by sequential change-point methods
Stat. Methodol.
(2006) - et al.
Detection of Abrupt Changes: Theory and Applications
(1993) - et al.
On-line Estimation and Adaptive Control of Bioreactors
(1990) - et al.
Diagnosis and Fault-Tolerant Control
(2003) - Ding, S.X., Zhang, P., Frank, P.M., 2003. Application of probabilistic robustness technique to the fault detection...
- et al.
Automatic threshold determination for a local approach of change detection in long-term signal recordings
EURASIP Journal on Advances in Signal Processing
(2007) - Frisen, M., 2007. Optimal sequential surveillance for finance, public health, and other areas. Research report,...
A distribution-free tabular CUSUM chart for autocorrelated data
IIE Trans.
Sequential changepoint detection in quality control and dynamical systems
J. Roy. Statist. Soc. Ser. B
Cited by (39)
Sensitivity clustering and ROC curve based alarm threshold optimization
2020, Process Safety and Environmental ProtectionCitation Excerpt :Meanwhile, combined with 1 as the basic weight for FAR and MAR, a balance can be achieved between FAR and MAR, which can avoid excessive effect produced by the remaining weights (ω1 and ω2). In this work, with advantages of high precision and fast convergence, the particle swarm optimization (PSO) algorithm is used to optimize the abovementioned objective functions in addition to derivation methods, which has been proved to be an effective approach in describing optimal multilevel thresholds (Gao et al., 2009), designing and controlling curing processes (Lu et al., 2010), and other fields. Meanwhile, for the sake of the following optimization and multivariable joint optimization in the future, the shortcomings of the standard PSO algorithm are also carefully considered in the design process of the PSO algorithm to avoid the premature convergence problem:
PCA weight and Johnson transformation based alarm threshold optimization in chemical processes
2018, Chinese Journal of Chemical EngineeringCitation Excerpt :Current threshold setting methods for process variables can be classified into three general categories: model based method, knowledge based method, and process history based method. Although process history based method is not well suitable for real-time alarm management due to its high-capacity historical data requirement, its implementation is easy and efficient compared with the model based [4–10] and knowledge based [11–17] methods. With massive data from DCS, this method has been widely used in process control field.
Alarm clustering analysis and ACO based multi-variable alarms thresholds optimization in chemical processes
2018, Process Safety and Environmental ProtectionCitation Excerpt :There have appeared many optimization methods for setting process variable threshold in chemical process. For example, in terms of FAR and MAR, Verdier et al. (2008) improved the change detection algorithm CUSUM to obtain adaptive alarm threshold with constant FAR in process. Izadi et al. (2009a,b) presented a framework about the best filter based on FAR, MAR, time delay, and deadband.
Change detection for uncertain autoregressive dynamic models through nonparametric estimation
2016, Statistical MethodologyCitation Excerpt :The resulting test statistic is then not stationary and the use of a fixed threshold (like it is traditionally the case for the CUSUM) is no more adapted. The computational procedure proposed in Verdier et al. [37] in order to obtain a constant probability of false alarm is then used.
Combining FAP, MAP and correlation analysis for multivariate alarm thresholds optimization in industrial process
2016, Journal of Loss Prevention in the Process Industries