We usually assume that the true underlying model of the observed process is a finite AR, MA or a mixed ARMA equation. However, this assumption can be rarely justified in practice. A common approach is to approximate the true model by finite AR models whose orders are chosen by information criterions such as AIC, BIC, PLS and whose parameters may be estimated by the ordinary least-squares method, the Yule–Walker method and others. In this paper we investigate a limiting case for approximating by finite AR models, i.e. an AR model of infinite order. We focus on the strong consistency properties of the regularized least-squares estimates. The regularizer may be interpreted as the smoothing operator applied to the number of the AR coefficients being estimated, and constitutes a link to the model selection criterions.

The rate of a.s. convergence of the infinite LS estimates is established. In addition, a complimentary result on the convergence of almost semi-martingales, which is also of independent interest, is presented. The proposed identification procedure is evaluated by numerical examples.