On the ill-conditioning of subspace identification with inputs☆
Introduction
Subspace methods for the identification of linear systems have been object of much research in the last 10 years. In particular subspace methods for time series (no observable inputs) have been thoroughly analyzed in the literature (Van Overschee & De Moor, 1993; Bauer, 2002; Lindquist & Picci, 1996) and it seems fair to say that they are perhaps the most efficient and accurate methods available today for multivariable time series identification. The situation is not the same for identification of systems with observable inputs. Although it is generally admitted that subspace methods with inputs offer substantial advantages over traditional PEM identification (Ljung, 1997), especially for identification of multivariable systems, there is experimental evidence that standard subspace methods (e.g., the N4SID method) perform poorly in certain experimental conditions, in particular when the past signals (past inputs and outputs) and future input spaces are nearly parallel (Chiuso & Picci, 1999; Kawauchi, Chiuso, Katayama, & Picci, 1999). Subspace methods operating on joint input–output data involve the solution of a multiple regression problem and the reason for the poor behavior may be attributed to ill-conditioning of the regression problem, which may occur when the regressors are nearly parallel. Although this phenomenon is well-known in multivariate statistics (Stewart, 1987; Belsley, 1991), it does not seem to have been noticed and analyzed in the subspace identification literature. The study of numerical conditioning of the regression problem, besides the effect of numerical roundoff errors, which are clearly irrelevant in the present context, yields information on the sensitivity of the parameter and transfer function estimates to noise in the data, which is instead important to assess the performance of identification methods. It should be intuitively clear, and it will be demonstrated formally in the companion paper (Chiuso & Picci, 2004b), that bad numerical conditioning generally implies a large variance of the estimates. We believe that this aspect should be taken into account in the design and comparison of subspace algorithms.
In this paper, which expands on work presented in previous conference papers (Chiuso & Picci, 1999; Kawauchi et al., 1999), we shall provide an elementary error analysis of some well-known subspace methods. We shall see that the sensitivity to random errors of a subspace method can be measured by the condition number of the conditional cross-covariance matrices , and of the state, given the future inputs, and of the future inputs given the current state, which will be introduced later in the paper. We shall relate the conditioning of these two matrices to the “near parallelism” of the state and future input spaces and thereby see that there are two main possible causes of ill-conditioning. The first is due to the cross-correlation between the state and future inputs, while the second is inherent in the dynamical structure of the input signal (and there is not much one can do about it, if identification has to be based on experiments performed during normal operation of the plant). Here we make contact and extend the analysis of the papers (Jansson and Wahlberg 1997, Jansson and Wahlberg 1998); where it has been shown that the singularity of is a cause of lack of consistency of subspace methods with inputs. In a sense, we study also the effects of near-singularity of this matrix.
A main motivation of this paper is to compare the performance of some widely used subspace methods from the literature. To this purpose, we undertake a comparative analysis of the conditioning of N4SID, “Robust” N4SID and PO-MOESP. The analysis is first directed to recasting the various algorithms into a common setting using ideas from stochastic realization theory. A result of this analysis is that, at least for the estimation of the (A,C) parameters, the “Robust” N4SID and PO-MOESP methods are equivalent. As expected, the original N4SID of (Van Overschee & De Moor, 1994), is generally worse.
The structure of the paper is as follows:
- •
In Section 2 we review the basic ideas of subspace identification, discuss the finite-interval stochastic realization problem, describe the basic “ideal” Kalman filter model which should be used in identification with inputs and discuss some difficulties which prevent constructing the state from finite input–output data. This explains why there is a multitude of subspace identification methods with inputs and indicates a common background for their analysis.
- •
In Section 3 we do some error analysis, comparing the conditioning of the identification (regression) problem based on the “ideal model”, with the regression problem occurring in the N4SID method.
- •
In Section 4 we compare the “Robust N4SID” and PO-MOESP methods. We prove that these two methods produce exactly the same estimates of (A,C) and hence the same conditioning analysis holds for these two methods.
- •
Section 5 contains some conclusions.
Section snippets
A review of subspace identification
Letbe observed input–output trajectories of an unknown system, which we want to identify. For the moment we shall pretend that the trajectories are infinitely long. We shall assume that the data are sample paths of a pair of zero-mean second-order stationary true random processes , having a rational spectral density; in other words, data (2.1) are generated by a linear stochastic system of the form
Error analysis of subspace methods
Ideally, the first step of subspace identification should be of constructing the state of a transient Kalman-filter type realization (2.16). Naturally, with only finite data available, the state vector must be approximated by a finite tail matrix and will be affected by random errors. Hence the parameter estimates, obtained by regressing the next state and output variables on the estimated state (and on the observed input) will also be affected by errors. In this section we shall show that the
Conditioning of the robust N4SID and of MOESP-type methods
In this section we shall see that the computation of the asymptotic estimates of (A,C) in the “robust” N4SID method of Van Overschee and De Moor (1996) and in the so-called PO-MOESP method of Verhaegen, 1994, is described, except for a change of basis in the state space, by the same formulas found for N4SID. Therefore the same conditioning analysis which applies to N4SID (in particular Proposition 7) also applies to these two methods.
In the process of doing this, we shall actually establish
Conclusions
In this paper we have presented an error analysis which applies to some commonly used subspace identification methods with inputs. We have shown that in presence of collinearity of the regressors these methods may lead to inaccurate estimates of the system parameters (and of the relative transfer function). We have also demonstrated that some of the most well-known algorithms in the literature (in particular robust N4SID and PO-MOESP) are equivalent as far as estimation of the matrices (A,C) is
Alessandro Chiuso received his D.Ing. degree Cum Laude in 1996, and the Ph.D. degree in Systems Engineering in 2000 both from the University of Padova. In 1998/99 he was a Visiting Research Scholar with the Electronic Signal and Systems Research Laboratory (ESSRL) at Washington University, St. Louis. From March 2000 to July 2000 he has been Visiting Post-Doctoral (EU-TMR) fellow with the Division of Optimization and System Theory, Department of Mathematics, KTH, Stockholm, Sweden. Since March
References (34)
- et al.
Analysis of the asymptotic properties of the moesp type of subspace algorithms
Automatica
(2000) - et al.
The asymptotic variance of subspace estimates
Journal of Econometrics
(2004) - et al.
On consistency of subspace methods for system identification
Automatica
(1998) - et al.
Realization of stochastic systems with exogenous inputs and subspace system identification methods
Automatica
(1999) - et al.
Canonical correlation analysis, approximate covariance extension and identification of stationary time series
Automatica
(1996) - et al.
Stochastic realization with exogenous inputs and “subspace methods” identification
Signal Processing
(1996) - et al.
Subspace algorithms for the stochastic identification problem
Automatica
(1993) - et al.
N4SIDSubspace algorithms for the identification of combined deterministic-stochastic systems
Automatica
(1994) Identification of the deterministic part of mimo state space models given in innovations form from input–output data
Automatica
(1994)- Bauer, D. (2002). Comparing the cca subspace method to pseudo maximum likelihood methods in the case of no exogenous...
Some facts about the choice of the weighting matrices in larimore type of subspace algorithm
Automatica
Conditioning diagnostics, collinearity and weak data regression
Cited by (56)
Markov Parameter Identification via Chebyshev Approximation
2023, IFAC-PapersOnLineStrategy for performance improvement in piezoelectric semi-active structural system identification by excluding switching failures using pseudo-state feedback
2023, Mechanical Systems and Signal ProcessingAn iterative state-space identification method with data correlation for MIMO systems with measurement noise
2022, Journal of the Franklin InstituteCitation Excerpt :Subspace methods made relevant contributions starting from the work of Van Overschee and De Moor [29,30], with a highlight to the N4SID as the principal subspace identification method. However, some drawbacks of subspace methods were appointed as in the work of Chiuso et al. [31], mentioning collinearity of the regressors that may lead to inaccurate estimates of the system parameters. Our main contribution here is the use of feedbacking the Markov parameters in the identification algorithm to minimize the noise effects, which is distinct from the works [16,17] where the closed-loop is in the system plant.
A novel subspace identification approach with passivity enforcement
2021, AutomaticaCitation Excerpt :Despite its many strengths, several drawbacks have been recognized. In general, estimates from 4SIDs are not as accurate as those from prediction error methods, and under closed-loop conditions, some traditional 4SID algorithms are biased thus requiring further methods to cope with this problem (Chiuso & Picci, 2004a, 2004b). Subspace-based methods were originally proposed in the late 90s by Overschee and Moor (1991), then three different algorithms emerged: CVA (Larimore, 1996), MOESP (Verhaegen & Dewilde, 1992) and N4SID (Overschee & Moor, 1994), which were later unified in Overschee and Moor (1995).
Identification of High-Tech Motion Systems: An Active Vibration Isolation Benchmark
2015, IFAC-PapersOnLine
Alessandro Chiuso received his D.Ing. degree Cum Laude in 1996, and the Ph.D. degree in Systems Engineering in 2000 both from the University of Padova. In 1998/99 he was a Visiting Research Scholar with the Electronic Signal and Systems Research Laboratory (ESSRL) at Washington University, St. Louis. From March 2000 to July 2000 he has been Visiting Post-Doctoral (EU-TMR) fellow with the Division of Optimization and System Theory, Department of Mathematics, KTH, Stockholm, Sweden. Since March 2001 he is Research Faculty (“Ricercatore”) with the Dept. of Information Engineering, University of Padova. In the summer 2001 he has been visiting researcher with the Dept. of Computer Science, University of California Los Angeles. His research interests are mainly in Identification and Estimation Theory, System Theory and Computer Vision.
Giorgio Picci holds a full professorship with the University of Padova, Italy, Department of Information Engineering, since 1980. He graduated (cum laude) from the University of Padova in 1967 and since then has held several long-term visiting appointments with various American and European universities among which Brown University, M.I.T., the University of Kentucky, Arizona State University, the Center for Mathematics and Computer Sciences (C.W.I.) in Amsterdam, the Royal Institute of Technology, Stockholm Sweden, Kyoto University and Washington University in St. Louis, Mo.
He has been contributing to Systems and Control theory mostly in the area of modeling, estimation and identification of stochastic systems and published over 100 papers and edited three books in this area. Since 1992 he has been active also in the field of Dynamic Vision and scene and motion reconstruction from monocular vision.
He has been involved in various joint research projects with industry and state agencies. He is currently general coordinator of the italian national project New techniques for identification and adaptive control of industrial systems, funded by MIUR (the Italian ministery for higher education), has been project manager of the italian team for the Commission of the European Communities Network of Excellence System Identification (ERNSI) and is currently general project manager of the Commission of European Communities IST project RECSYS, in the fifth Framework Program.
Giorgio Picci is a Fellow of the IEEE, past chairman of the IFAC Technical Committee on Stochastic Systems and a member of the EUCA council.
- ☆
This work has been supported in part by the ERNSI TMR network System Identification and by the national project Identification and adaptive control of industrial systems funded by MIUR. Part of this work has been done while the first author (A.C.) was a post-doctoral researcher with the Division of Optimization and Systems Theory, KTH, Stockholm, supported by ERNSI. This paper was recommended for publication in revised form by Associate Editor Brett Niness under the direction of Editor Torsten Söderström.