Development of soft-sensor using locally weighted PLS with adaptive similarity measure
Introduction
In various industrial processes, it is necessary to measure and control product quality to produce high-quality, competitive products. However, online measurement is not always available due to unacceptable expenses of analytical instruments or long measurement/analysis delay. To solve this problem, inferential models using online measured variables as predictor variables have been adopted in many fields such as chemical, bioprocess, steel, and pharmaceutical [1], [2]. According to the recent questionnaire survey of process control in the chemical industry in Japan [3], 90% of the inferential models are constructed by using linear regression methods such as multiple regression analysis (MRA) and partial least squares (PLS). This fact shows that linear models are practically useful. In some cases, nonlinear models are required to achieve high estimation accuracy for processes having strong nonlinearity. Thus, nonlinear modeling methods such as neural networks [4], [5], [6], [7], support vector regression [8], [9], [10] and polynomial functions [11], [12], [13] have been used to construct nonlinear inferential models.
The above-mentioned questionnaire survey revealed that the most important problem of current inferential models is how to cope with changes in process characteristics and keep high estimation accuracy for a long period of time, i.e., model maintenance [3]. The importance of this problem was also pointed out in [1], [14]. To cope with changes in process characteristics, many kinds of recursive modeling methods, which update models by prioritizing newer samples, have been developed [15]. When process characteristics change gradually, the prioritized samples are supposed to be similar to a query sample, for which an output estimation is required. For such a case, recursive methods can cope with gradual changes in process characteristics. However, they cannot cope with an abrupt change in process characteristics caused by replacement of a catalyst, cleaning of equipment, etc., because a query sampled just after an abrupt change becomes significantly different from the prioritized samples.
Locally weighted regression (LWR) [16], which is also called just-in-time learning, lazy learning or model-on-demand, constructs a local model by prioritizing samples in a database according to the similarity between them and a query sample. Hence, LWR can cope with abrupt changes as well as gradual ones in contrast to the recursive methods introduced in [15]. Furthermore, it can cope with nonlinearity since it builds a local model repeatedly. To build an accurate model with LWR, the similarity needs to be properly defined. In general, similarity is defined on the basis of the Euclidean distance or the Mahalanobis distance [10], [17], [18], [19], [20], [21], [22]. Other similarity measures proposed so far include the angle [14], [23], [24], the distance between an output estimate for a query sample derived by a global model and output measurements for samples in a database [25], the correlation [26], [27] and the weighted Euclidean distance [28], [29], [30]. In addition to define the similarity properly, it is crucial to update a database when new data become available in order to cope with changes in process characteristics. More detailed explanation and review of the problem of the changes in process characteristics and LWR can be found in [31].
This study focuses on the problem of nonlinearity and the definition of the similarity, and does not deal with the problem of changes in process characteristics. The similarity based on the weighted Euclidean distance is further investigated for its simplicity. PLS is adopted for local modeling since it can cope with collinearity and has been widely accepted in various fields. The main contribution of this paper is to discuss how the weight of each input should be determined and to propose a method for deriving appropriate weights from operation data stored in a database.
The rest of this paper is organized as follows. In Section 2, the algorithm of locally weighted PLS (LW-PLS) is explained. Section 3 discusses how to determine the weight of each input, and a method for deriving the appropriate weights from operation data is proposed. Section 4 shows the effectiveness of the proposed method through numerical examples. In Section 5, an application result of the proposed method to an industrial distillation process is reported. Finally, this research is concluded in Section 6.
Section snippets
Locally weighted partial least squares
The nth sample (n = 1,2,⋯,N) of input and output variables is denoted bywhere M is the number of input variables, L is the number of output variables and the superscript T denotes the transpose of a vector or matrix. and are input and output variable matrices whose nth rows are xnT and ynT, respectively. N is the number of samples.
LW-PLS is a just-in-time (JIT) modeling method. X and Y are stored in a database in order to construct a local PLS model. When
How should weights be determined?
In the present work, it is assumed that the number of output variables is one, and the following form of the similarity ω is investigated:where σd is a standard deviation of dn (n = 1,2,⋯,N) and φ is a localization parameter; the similarity decreases steeply when φ is small and gradually when φ is large. In addition, is a weighting matrix and θm is a weight of the mth input variable.
Fig. 1 shows a simple example, in which a relationship between a
Numerical example
In this section, the proposed method is compared with the conventional methods in two numerical examples. The following four methods are compared.
- LW-PLS 1)
LW-PLS with θm = 1.
- LW-PLS 2)
LW-PLS with θm defined as the absolute value of the mth variable's regression coefficient of a global MRA model [28].
- LW-PLS 3)
LW-PLS with θm defined as the absolute value of the mth variable's regression coefficient of an LW-PLS model constructed by LW-PLS 1 [29].
- LW-PLS 4)
LW-PLS with θm defined by the proposed method.
Application to an industrial distillation process
In this section, an application result of the proposed method to an industrial distillation process is reported. A soft-sensor for estimating the aroma concentration was constructed in order to realize highly efficient operation of a cracked gasoline (CGL) fractionator of an ethylene production process at the Showa Denko K.K. (SDK) Oita plant in Japan. Aroma denotes the generic name for benzene, toluene, xylene and styrene, etc. In this case study, linear PLS, LW-PLS 1, 2, 3 and 4 were
Conclusion
To construct highly accurate locally weighted partial least squares (LW-PLS) models, an adaptive similarity measure was proposed. In the proposed method, weights of input variables are determined through iterative calculation by using the weighted variance of the regression coefficients. The results of the case studies showed that the proposed method could adaptively derive the appropriate weights and more accurate models than the conventional methods in numerical examples. Furthermore, root
Acknowledgments
This work was partially supported by the Japan Society for the Promotion of Science (JSPS), Grant-in-Aid for Scientific Research (C) 21560793.
References (32)
- et al.
Data-driven soft sensors in the process industry
Computers and Chemical Engineering
(2009) - et al.
Multivariate data analysis in pharmaceutics: a tutorial review
International Journal of Pharmaceutics
(2011) - et al.
The state of the art in chemical process control in Japan: good practice and questionnaire survey
Journal of Process Control
(2010) - et al.
Non-linear projection to latent structures revisited (the neural network PLS algorithm)
Computers and Chemical Engineering
(1999) - et al.
Nonlinear PLS modeling using neural networks
Computers and Chemical Engineering
(1992) - et al.
Nonlinear PLS modeling
Chemometrics and Intelligent Laboratory Systems
(1989) Contributions to the problem of approximation of non-linear data with linear PLS in an absorption spectroscopic context
Chemometrics and Intelligent Laboratory Systems
(1999)- et al.
Review of adaptation mechanisms for data-driven soft sensors
Computers and Chemical Engineering
(2011) - et al.
Identification of trends in annual maximum sea levels using robust locally weighted regression
Estuarine, Coastal and Shelf Science
(1983) - et al.
Locally weighted regression for desulphurisation intelligent decision system modeling
Simulation Modelling Practice and Theory
(2004)
A comparative study of just-in-time-learning based methods for online soft sensor modeling
Chemometrics and Intelligent Laboratory Systems
A new data-based methodology for nonlinear process modeling
Chemical Engineering Science
Online monitoring of nonlinear multiple mode processes based on adaptive local model approach
Control Engineering Practice
Development of correlation-based clustering method and its application to software sensing
Chemometrics and Intelligent Laboratory Systems
Optimum quality design system for steel products through locally weighted regression model
Journal of Process Control
Estimation of active pharmaceutical ingredients content using locally weighted partial least squares and statistical wavelength selection
International Journal of Pharmaceutics
Cited by (91)
Just-in-time based soft sensors for process industries: A status report and recommendations
2023, Journal of Process ControlReinforcement learning for soft sensor design through autonomous cross-domain data selection
2023, Computers and Chemical EngineeringSemi-supervised adaptive PLS soft-sensor with PCA-based drift correction method for online valuation of NOx emission in industrial water-tube boiler
2023, Process Safety and Environmental ProtectionEnhancing the reliability and accuracy of data-driven dynamic soft sensor based on selective dynamic partial least squares models
2022, Control Engineering Practice
- 1
Present address: Nippon Steel Corporation, Tokyo 1008071, Japan.