Abstract
In this paper, we propose an efficient method for handling large datasets in linear parameter-varying (LPV) model identification. The method is based on least-squares support vector machine (LS-SVM) identification in the primal space. To make the identification computationally feasible, even for very large datasets, we propose estimating a finite-dimensional feature map. To achieve this, we propose a two-step method to reduce the computational effort. First, we define the training set as a fixed-size subsample of the entire dataset, considering collision entropy for subset selection. The second step involves approximating the feature map through the eigenvalue decomposition of the kernel matrices. This paper considers both autoregressive with exogenous input (ARX) and state-space (SS) model forms. By comparing the problem formulation in the primal and dual spaces in terms of accuracy and computational complexity, the main advantage of the proposed technique is the reduction in space and time complexity during the training stage, making it preferable for handling very large datasets. To validate our proposed primal approach, we apply it to estimate LPV models using provided inputs, outputs, and scheduling signals for two nonlinear benchmarks: the parallel Wiener-Hammerstein system and the Silverbox system. The performances of our proposed approach are compared with the dual LS-SVM approach and the kernel principal component regression.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M. Espinoza, K. Pelckmans, L. Hoegaerts, J. A. Suykens, and B. De Moor, “A comparative study of LS-SVM’s applied to the silver box identification problem,” IFAC Proceedings Volumes, vol. 37, no. 13, pp. 369–374, 2004.
K. De Brabanter, P. Dreesen, P. Karsmakers, K. Pelckmans, J. De Brabanter, J. Suykens, and B. De Moor, “Fixed-size LS-SVM applied to the Wiener-Hammerstein benchmark,” IFAC Proceedings Volumes, vol. 42, no. 10, pp. 826–831, 2009.
P. L. dos Santos and T. A. Perdicoúlis, “A kernel principal component regressor for LPV system identification,” IFAC-PapersOnLine, vol. 52, no. 28, pp. 7–12, 2019.
L. Cavanini, L. Ciabattoni, F. Ferracuti, E. Marchegiani, and A. Monteriù, “A comparative study of driver torque demand prediction methods,” IET Intelligent Transport Systems, 2022.
S. Ijaz, M. T. Hamayun, H. Anwaar, L. Yan, and M. K. Li, “LPV modeling and tracking control of dissimilar redundant actuation system for civil aircraft,” International Journal of Control, Automation and Systems, vol. 17, pp. 705–715, 2019.
F. Ma, J. Li, L. Wu, and D. Yuan, “Tensor product based polytopic lpv system design of a 6-dof multi-strut platform,” International Journal of Control, Automation, and Systems, vol. 20, no. 1, pp. 137–146, 2022.
J. Che, Y. Zhu, M. V. Basin, and D. Zhou, “Active fault-tolerant control for discrete-time Markov jump LPV systems via time-varying hidden markov model approach,” International Journal of Control, Automation, and Systems, vol. 20, no. 6, pp. 1785–1799, 2022.
J. A. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. P. Vandewalle, Least Squares Support Vector Machines, World Scientific, 2002.
L. Cavanini, G. Ippoliti, and E. F. Camacho, “Model predictive control for a linear parameter varying model of an uav,” Journal of Intelligent & Robotic Systems, vol. 101, no. 3, pp. 1–18, 2021.
R. Tóth, V. Laurain, W. X. Zheng, and K. Poolla, “Model structure learning: A support vector machine approach for LPV linear-regression models,” Proc. of 50th IEEE Conference on Decision and Control and European Control Conference, IEEE, pp. 3192–3197, 2011.
S. Z. Rizvi, J. Mohammadpour, R. Tóth, and N. Meskin, “A kernel-based approach to MIMO LPV state-space identification and application to a nonlinear process system,” IFAC-PapersOnLine, vol. 48, no. 26, pp. 85–90, 2015.
M. Mejari, D. Piga, and A. Bemporad, “Regularized least square support vector machines for order and structure selection of LPV-ARX models,” Proc. of European Control Conference (ECC), pp. 1649–1654, 2016.
D. Piga and R. Tóth, “LPV model order selection in an LS-SVM setting,” Proc. of 52nd IEEE Conference on Decision and Control, pp. 4128–4133, 2013.
R. Duijkers, R. Tóth, D. Piga, and V. Laurain, “Shrinking complexity of scheduling dependencies in LS-SVM based LPV system identification,” Proc. of 53rd IEEE Conference on Decision and Control, IEEE, pp. 2561–2566, 2014.
L. Cavanini, F. Ferracuti, S. Longhi, E. Marchegiani, and A. Monteriù, “Sparse approximation of LS-SVM for LPV-ARX model identification: Application to a power-train subsystem,” Proc. of AEIT International Conference of Electrical and Electronic Technologies for Automotive (AEIT AUTOMOTIVE), pp. 1–6, 2020.
L. Cavanini, F. Ferracuti, S. Longhi, and A. Monteriù, “LS-SVM for LPV-ARX identification: Efficient online update by low-rank matrix approximation,” Proc. of International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1590–1595, 2020.
M. Mejari, B. Mavkov, M. Forgione, and D. Piga, “Direct identification of continuous-time LPV state-space models via an integral architecture,” Automatica, vol. 142, 110407, 2022.
J. A. Suykens, C. Alzate, and K. Pelckmans, “Primal and dual model representations in kernel-based learning,” Statistics Surveys, vol. 4, pp. 148–183, 2010.
M. Schoukens, A. Marconato, R. Pintelon, G. Vandersteen, and Y. Rolain, “Parametric identification of parallel Wiener-Hammerstein systems,” Automatica, vol. 51, pp. 111–122, 2015.
T. Wigren and J. Schoukens, “Three free data sets for development and benchmarking in nonlinear system identification,” Proc. of European Control Conference (ECC), 2013, pp. 2933–2938.
S. Z. Rizvi, J. M. Velni, F. Abbasi, R. Tóth, and N. Meskin, “State-space LPV model identification using kernelized machine learning,” Automatica, vol. 88, pp. 38–47, 2018.
M. Girolami, “Orthogonal series density estimation and the kernel eigenvalue problem,” Neural computation, vol. 14, no. 3, pp. 669–688, 2002.
E. J. Nyström, “Über die praktische Auflösung von Integralgleichungen mit Anwendungen auf Randwertaufgaben,” Acta Mathematica, vol. 54, pp. 185–204, 1930.
C. Williams and M. Seeger, “Using the Nyström method to speed up kernel machines,” Advances in Neural Information Processing Systems, vol. 13, 2000.
R. Tóth, H. Hjalmarsson, and C. R. Rojas, “Order and structural dependence selection of LPV-ARX models revisited,” Proc. of IEEE 51st IEEE Conference on Decision and Control (CDC), pp. 6271–6276, December 2012.
R. Tóth, P. Heuberger, and P. van den Hof, “LPV system identification using series expansion models,” Linear Parameter-varying System Identification: New Developments and Trends, World Scientific, pp. 259–294, 2012.
J. Chen and Y. Saad, “Lanczos vectors versus singular vectors for effective dimension reduction,” IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 8, pp. 1091–1103, 2008.
A. Rahimi and B. Recht, “Random features for large-scale kernel machines,” Advances in Neural Information Processing Systems, vol. 20, 2007.
Q. Le, T. Sarlós, and A. Smola, “Fastfood: Approximating kernel expansions in loglinear time,” Proc. of the 30th International Conference on International Conference on Machine Learning - Volume 28, ser. ICML’13, JMLR.org, p. III-244–III-252, 2013.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors declare that there is no competing financial interest or personal relationship that could have appeared to influence the work reported in this paper.
Additional information
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Luca Cavanini received his Ph.D. degree in automation, information, and management engineering from Università Politecnica delle Marche, Ancona, Italy, in 2018. He works as a Technical Consultant at Industrial Systems and Control Ltd. His research activity includes model predictive control, autonomous mobile robotics and artificial intelligence, and machine learning techniques for control systems.
Riccardo Felicetti received his master’s degree cum laude in computer and automation engineering and a Ph.D. degree cum laude in information engineering from Università Politecnica delle Marche, in 2016 and 2021, respectively. Since 2019, he has been a postdoctoral research fellow with Università Politecnica delle Marche. His main research interests are fault detection and diagnosis, fault tolerant control, and optimization with applications to unmanned vehicles and energy management systems.
Francesco Ferracuti received his Ph.D. degree in automation, information and management engineering from Università Politecnica delle Marche, Ancona, Italy, in 2014. He is a researcher at Università Politecnica delle Marche. His research interests include model-based and data-driven fault diagnosis, signal processing, statistical pattern recognition, and machine learning and their applications in industry.
Andrea Monteriù received his M.Sc. degree cum laude in electronic engineering and a Ph.D. degree in artificial intelligence systems from Università Politecnica delle Marche, Italy, in 2003 and 2006, respectively. Currently, he is an associate professor at Università Politecnica delle Marche. His research interests mainly focus on the areas of fault diagnosis and fault tolerant control applied on robotic, and unmanned and artificial intelligent systems.
Rights and permissions
About this article
Cite this article
Cavanini, L., Felicetti, R., Ferracuti, F. et al. Fixed-size LS-SVM LPV System Identification for Large Datasets. Int. J. Control Autom. Syst. 21, 4067–4079 (2023). https://doi.org/10.1007/s12555-023-0062-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-023-0062-y