Skip to main content
SpringerLink
Log in

System Identification with GP Models

  • Chapter
  • First Online:
Modelling and Control of Dynamic Systems Using Gaussian Process Models

Part of the book series: Advances in Industrial Control ((AIC))

Abstract

This chapter talks about system identification with GP models. After outlining the complete procedure for system identification, the chapter is focused on issues that are specific to modelling based on GPs. The issues emphasised here are the setting-up of the model, model selection and validation of the identified model. The system identification is illustrated on the bioreactor benchmark model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The data is available from http://cdiac.esd.ornl.gov/ftp/trends/co2/maunaloa.co2.

References

  1. Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Delyon, B., Glorennec, P.Y., Hjalmarsson, H., Juditsky, A.: Nonlinear black-box modelling in system identification: a unified overview. Automatica 31(12), 1691–1724 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  2. Ljung, L.: System Identification - Theory for the User, 2nd edn. Prentice Hall, Upper Saddle River, NJ (1999)

    Google Scholar 

  3. Bai, E.W.: Prediction error adjusted Gaussian process for nonlinear non-parametric system identification. In: 16th IFAC Symposium on System Identification, pp. 101–106. IFAC, Brussels (2012)

    Google Scholar 

  4. Johansen, T.A., Shorten, R., Murray-Smith, R.: On the interpretation and identification of dynamic Takagi-Sugeno fuzzy models. IEEE Trans. Fuzzy Syst. 8, 297–313 (2000)

    Article  Google Scholar 

  5. Murray-Smith, R., Johansen, T.A. (eds.): Multiple Model Approaches to Modelling and Control. Taylor and Francis, London (1997)

    Google Scholar 

  6. Murray-Smith, R., Johansen, T.A., Shorten, R.: On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures. In: Proceedings of European Control Conference, pp. BA-14. Karlsruhe (1999)

    Google Scholar 

  7. Suykens, J.A.K., Gestel, T.V., Brabanteer, J.D., Moor, B.D., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)

    Book  MATH  Google Scholar 

  8. Snelson, E., Rasmussen, C.E., Ghahramani, Z.: Warped Gaussian processes. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16, pp. 337–344 (2004)

    Google Scholar 

  9. Lazáro-Gredilla, M.: Bayesian warped Gaussian processes. Advances in Neural Information Processing Systems, vol. 26. MIT Press, Cambridge, MA (2013)

    Google Scholar 

  10. Nelles, O.: Nonlinear System Identification. Springer, Berlin (2001)

    Book  MATH  Google Scholar 

  11. Goodwin, G.C.: Identification: experiment design. In: Singh, M.G. (ed.) Systems and Control Encyclopedia, vol. 4 (I–L), pp. 2257–2264. Pergamon Press, Oxford (1987)

    Google Scholar 

  12. Norgaard, M., Ravn, O., Poulsen, N.K., Hansen, L.K.: Neural Networks for Modelling and Control of Dynamic Systems. A Practitioner’s Handbook. Advanced Textbooks in Control and Signal Processing. Springer, London (2000)

    Book  Google Scholar 

  13. Pronzato, L.: Optimal experimental design and some related control problems. Automatica 44(2), 303–325 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  14. Kocijan, J., Přikryl, J.: Soft sensor for faulty measurements detection and reconstruction in urban traffic. In: Proceedings 15th IEEE Mediterranean Electromechanical Conference (MELECON), pp. 172–177. Valletta (2010)

    Google Scholar 

  15. Haykin, S.: Neural Networks, A Comprehensive Foundation. Macmillan College Publishing Company, New York, NY (1994)

    Google Scholar 

  16. Ackermann, E.R., de Villiers, J.P., Cilliers, P.J.: Nonlinear dynamic systems modeling using Gaussian processes: predicting ionospheric total electron content over South Africa. J. Geophys. Res. A: Space Phys. 116(10) (2011)

    Google Scholar 

  17. Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with Gaussian processes. In: Troch, I., Breitenecker, F. (eds.) Proceedings of 4th IMACS Symposium on Mathematical Modelling (MathMod), pp. 776–784. Vienna (2003)

    Google Scholar 

  18. Kocijan, J., Petelin, D.: Output-error model training for Gaussian process models. In: Dobnikar, A., Lotrič, U., Šter, B. (eds.) Adaptive and Natural Computing Algorithms. Lecture Notes in Computer Science, vol. 6594, pp. 312–321. Springer, Berlin (2011)

    Chapter  Google Scholar 

  19. Frigola, R., Rasmussen, C.E.: Integrated pre-processing for Bayesian nonlinear system identification with Gaussian processes. In: 52nd IEEE Conference on Decision and Control (CDC) (2013)

    Google Scholar 

  20. Frigola, R., Lindsten, F., Schön, T.B., Rasmussen, C.E.: Bayesian inference and learning in Gaussian process state-space models with particle MCMC. In: L. Bottou, C. Burges, Z. Ghahramani, M. Welling, K. Weinberger (eds.) Advances in Neural Information Processing Systems, vol. 26, pp. 3156–3164 (2013)

    Google Scholar 

  21. Wang, J., Fleet, D., Hertzmann, A.: Gaussian process dynamical models. Adv. Neural Inf. Process. Syst. 18, 1441–1448 (2005)

    Google Scholar 

  22. Levine, W.S. (ed.): The Control Handbook. CRC Press, IEEE Press, Boca Raton (1996)

    MATH  Google Scholar 

  23. Ko, J., Fox, D.: GP-Bayesfilters: Bayesian filtering using Gaussian process prediction and observation models. Auton. Robots 27(1), 75–90 (2009)

    Article  Google Scholar 

  24. Deisenroth, M.P., Turner, R.D., Huber, M.F., Hanebeck, U.D., Rasmussen, C.E.: Robust filtering and smoothing with Gaussian processes. IEEE Trans. Autom. Control 57(7), 1865–1871 (2012). doi:10.1109/TAC.2011.2179426

    Article  MathSciNet  Google Scholar 

  25. Deisenroth, M.P., Huber, M.F., Hannebeck, U.D.: Analytic moment-based Gaussian process filtering. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 225–232. Montreal (2009)

    Google Scholar 

  26. Tong, C.H., Furgale, P., Barfoot, T.D.: Gaussian process Gauss–Newton: non-parametric state estimation. In: 2012 Ninth Conference on Computer and Robot Vision, pp. 206–213. Toronto (2012)

    Google Scholar 

  27. Turner, R., Rasmussen, C.E.: Model based learning of sigma points in unscented Kalman filtering. Neurocomputing 80, 47–53 (2012)

    Article  Google Scholar 

  28. Wang, Y., Chaib-draa, B.: A marginalized particle Gaussian process regression. In: Neural Information Processing Systems, vol. 25 (2012)

    Google Scholar 

  29. Wang, Y., Chaib-draa, B.: An adaptive nonparametric particle filter for state estimation. In: 2012 IEEE International Conference on Robotics and Automation, pp. 4355–4360. IEEE (2012)

    Google Scholar 

  30. Peltola, V., Honkela, A.: Variational inference and learning for non-linear state-space models with state-dependent observation noise. In: Proceedings of the 2010 IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2010, pp. 190–195 (2010)

    Google Scholar 

  31. Turner, R., Deisenroth, M.P., Rasmussen, C.E.: System identification in Gaussian process dynamical systems. Nonparametric Bayes Workshop at NIPS. Whistler (2009)

    Google Scholar 

  32. Turner, R., Deisenroth, M.P., Rasmussen, C.E.: State-space inference and learning with Gaussian processes. In: Proceedings of 13th International Conference on Artificial Intelligence and Statistics, vol. 9, pp. 868–875. Sardinia (2010)

    Google Scholar 

  33. Snelson, E., Ghahramani, Z.: Sparse Gaussian processes using pseudo-inputs. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems, vol. 18, pp. 1257–1264. MIT Press, Cambridge, MA (2006)

    Google Scholar 

  34. Hartikainen, J., Riihimäki, J., Särkkä, S.: Sparse spatio-temporal Gaussian processes with general likelihoods. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) Artificial Neural Networks and Machine Learning - ICANN 2011. Lecture Notes in Computer Science, vol. 6791, pp. 193–200. Springer, Berlin (2011)

    Chapter  Google Scholar 

  35. Hartikainen, J., Seppänen, M., Särkkä, S.: State-space inference for non-linear latent force models with application to satellite orbit prediction. In: Proceedings of the 29th International Conference on Machine Learning (ICML-12), pp. 903–910. Edinburgh (2012)

    Google Scholar 

  36. Särkkä, S., Hartikainen, J.: Infinite-dimensional kalman filtering approach to spatio-temporal Gaussian process regression. In: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS). JMLR: W&CP, vol. 22, pp. 993–1001 (2012)

    Google Scholar 

  37. Särkkä, S., Solin, A., Hartikainen, J.: Spatiotemporal learning via infinite-dimensional Bayesian filtering and smoothing: a look at Gaussian process regression through Kalman filtering. IEEE Signal Process. Mag. 30(4), 51–61 (2013). doi:10.1109/MSP.2013.2246292

    Article  Google Scholar 

  38. Chiuso, A., Pillonetto, G., De Nicolao, G.: Subspace identification using predictor estimation via Gaussian regression. In: Proceedings of the IEEE Conference on Decision and Control (2008)

    Google Scholar 

  39. Blum, A.L., Langley, P.: Selection of relevant features and examples in machine learning. Artif. Intell. 97(1–2), 245–271 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  40. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  41. Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1–2), 273–324 (1997)

    Article  MATH  Google Scholar 

  42. May, R., Dandy, G., Maier, H.: Review of input variable selection methods for artificial neural networks (Chap). Artificial Neural Networks - Methodological Advances and Biomedical Applications, pp. 19–44. InTech, Rijeka (2011)

    Google Scholar 

  43. Lind, I., Ljung, L.: Regressor selection with the analysis of variance method. Automatica 41, 693–700 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  44. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer Science + Business Media, New York, NY (2006)

    Google Scholar 

  45. He, X., Asada, H.: A new method for identifying orders of input-output models for nonlinear dynamical systems. In: Proceedings of the American Control Conference, San Francisco, CA, pp. 2520–2523 (1993)

    Google Scholar 

  46. Bomberger, J.D., Seborg, D.E.: Determination of model order for NARX models directly from input-output data. J. Process Control 8(5–6), 459–468 (1998)

    Article  Google Scholar 

  47. Broer, H., Takens, F.: Dynamical Systems and Chaos. Epsilon Uitgaven, Utrecht (2009)

    MATH  Google Scholar 

  48. Stark, J., Broomhead, D.S., Davies, M.E., Huke, J.: Delay embeddings for forced systems: II stochastic forcing. J. Nonlinear Sci. 13(6), 519–577 (2003)

    Article  MathSciNet  Google Scholar 

  49. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA (2006)

    Google Scholar 

  50. Thompson, K.: Implementation of Gaussian process models for non-linear system identification. Ph.D. thesis, University of Glasgow, Glasgow (2009)

    Google Scholar 

  51. Deisenroth, M.P.: Efficient reinforcement learning using Gaussian processes. Ph.D. thesis, Karlsruhe Institute of Technology, Karlsruhe (2010)

    Google Scholar 

  52. Murray-Smith, R., Girard, A.: Gaussian process priors with ARMA noise models. In: Proceedings of Irish Signals and Systems Conference, pp. 147–152. Maynooth (2001)

    Google Scholar 

  53. Ažman, K., Kocijan, J.: Identifikacija dinamičnega sistema z znanim modelom šuma z modelom na osnovi Gaussovih procesov. In: Zajc, B., Trost, A. (eds.) Zbornik petnajste elektrotehniške in računalniške konference (ERK), pp. 289–292. Portorož (2006). (In Slovene)

    Google Scholar 

  54. MAC Multi-Agent Control: Probabilistic reasoning, optimal coordination, stability analysis and controller design for intelligent hybrid systems (2000–2004). Research Training Network, 5th EU framework

    Google Scholar 

  55. Neal, R.M.: Bayesian Learning for Neural Networks. Lecture Notes in Statistics, vol. 118. Springer, New York, NY (1996)

    Google Scholar 

  56. Stein, M.L.: Interpolation of Spatial Data. Springer, New York, NY (1999)

    Google Scholar 

  57. Duvenaud, D.: The kernel cookbook: advice on covariance functions (2013)

    Google Scholar 

  58. Álvarez, M.A., Luengo, D., Lawrence, N.D.: Latent force models. J. Mach. Learn. Res. - Proc. Track 5, 9–16 (2009)

    Google Scholar 

  59. Álvarez, M.A., Peters, J., Schölkopf, B., Lawrence, N.D.: Switched latent force models for movement segmentation. In: Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010. Proceedings of a meeting held 6–9 December 2010, Vancouver, pp. 55–63 (2010)

    Google Scholar 

  60. Honkela, A., Girardot, C., Gustafson, E.H., Liu, Y.H., Furlong, E.M.F., Lawrence, N.D., Rattray, M.: Model-based method for transcription factor target identification with limited data. Proc. Natl. Acad. Sci. USA 107(17), 7793–7798 (2010)

    Article  Google Scholar 

  61. Neo, K.K.S., Leithead, W.E., Zhang, Y.: Multi-frequency scale Gaussian regression for noisy time-series data. In: UKACC International Control Conference. Glasgow (2006)

    Google Scholar 

  62. Nguyen-Tuong, D., Peters, J.: Using model knowledge for learning inverse dynamics. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 2677–2682 (2010)

    Google Scholar 

  63. Zhang, Y., Leithead, W.E.: Exploiting Hessian matrix and trust region algorithm in hyperparameters estimation of Gaussian process. Appl. Math. Comput. 171(2), 1264–1281 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  64. Petelin, D., Filipič, B., Kocijan, J.: Optimization of Gaussian process models with evolutionary algorithms. In: Dobnikar, A., Lotrič, U., Šter, B. (eds.) Adaptive and Natural Computing Algorithms. Lecture Notes in Computer Science, vol. 6594, pp. 420–429. Springer, Berlin (2011)

    Chapter  Google Scholar 

  65. Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Natural Computing Series. Springer, Heidelberg (2003)

    Book  MATH  Google Scholar 

  66. Hachino, T., Kadirkamanathan, V.: Multiple Gaussian process models for direct time series forecasting. IEEJ Trans. Electr. Electron. Eng. 6(3), 245–252 (2011)

    Article  Google Scholar 

  67. Hachino, T., Takata, H.: Identification of continuous-time nonlinear systems by using a Gaussian process model. IEEJ Trans. Electr. Electron. Eng. 3(6), 620–628 (2008)

    Article  Google Scholar 

  68. Storn, R., Price, K.: Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J. Global Opt. 11, 341–359 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  69. Price, K., Storn, R., Lampinen, J.: Differential Evolution. Natural Computing Series. Springer, Heidelberg (2005)

    MATH  Google Scholar 

  70. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, pp. 1942–1948. IEEE Press. (1995)

    Google Scholar 

  71. Kennedy, J., Eberhart, R., Shi, Y.: Swarm Intelligence. Morgan Kaufmann, San Francisco, CA (2001)

    Google Scholar 

  72. Hachino, T., Yamakawa, S.: Non-parametric identification of continuous-time Hammerstein systems using Gaussian process model and particle swarm optimization. Artif Life Robotics 17(1), 35–40 (2012)

    Article  Google Scholar 

  73. Birge, B.: Particle swarm optimization toolbox. http://www.mathworks.com/matlabcentral/fileexchange/7506-particle-swarm-optimization-toolbox

  74. Pohlheim, H.: GEATbx - the genetic and evolutionary algorithm toolbox for Matlab. http://www.geatbx.com/

  75. McHutchon, A., Rasmussen, C.E.: Gaussian process training with input noise. In: J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, K. Weinberger (eds.) Advances in Neural Information Processing Systems, vol. 24, pp. 1341–1349 (2011)

    Google Scholar 

  76. Shi, J.Q., Choi, T.: Gaussian Process Regression Analysis for Functional Data. Chapman and Hall/CRC, Taylor & Francis group, Boca Raton, FL (2011)

    Google Scholar 

  77. Seeger, M.W., Kakade, S.M., Foster, D.P.: Information consistency of nonparametric Gaussian process methods. IEEE Trans. Inf. Theory 54(5), 2376–2382 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  78. Trobec, R., Vajteršic, M., Zinterhof, P. (eds.): Parallel Computing: Numerics, Applications, and Trends. Springer, London (2009)

    Google Scholar 

  79. Kurzak, J., Bader, D.A., Dongarra, J. (eds.): Scientific Computing with Multicore and Accelerators. Chapman & Hall/CRC Computational Science Series, 1st edn. CRC Press, Boca Raton, FL (2011)

    Google Scholar 

  80. Shen, J.P., Lipasti, M.H.: Modern Processor Design: Fundamentals of Superscalar Processors. McGraw-Hill Series in Electrical and Computer Engineering, 1st edn. McGraw-Hill, New York, NY (2004)

    Google Scholar 

  81. Kirk, D.B., Hwu, W.W.: Programming Massively Parallel Processors A Hands-on Approach, 1st edn. Morgan Kaufmann, San Francisco, CA (2010)

    Google Scholar 

  82. Tomov, S., Dongarra, J., Baboulin, M.: Towards dense linear algebra for hybrid GPU accelerated manycore systems. Parallel Comput. 36(5), 232–240 (2010)

    Article  MATH  Google Scholar 

  83. NVIDIA Corporation, Santa Clara, CA: Cuda Programming Guide Version 2.3.1 (2009)

    Google Scholar 

  84. Advanced Micro Devices, Inc.: AMD Accelerated Parallel Processing OpenCL Programming Guide. Sunnyvale, CA (2011)

    Google Scholar 

  85. Srinivasan, B.V., Duraiswami, R.: Scaling kernel machine learning algorithm via the use of GPUs. In: GPU Technology Conference, NVIDIA Research Summit. NVIDIA (2009)

    Google Scholar 

  86. Srinivasan, B.V., Hu, Q., Duraiswami, R.: GPUML: graphical processors for speeding up kernel machines. In: Workshop on High Performance Analytics - Algorithms, Implementations, and Applications, SIAM Conference on Data Mining. SIAM (2010)

    Google Scholar 

  87. Musizza, B., Petelin, D., Kocijan, J.: Accelerated learning of Gaussian process models. In: Proceedings of the 7th EUROSIM Congress on Modelling and Simulation (2010)

    Google Scholar 

  88. Blackford, L., Petitet, A., Pozo, R., Remington, K., Whaley, R., Demmel, J., Dongarra, J., Duff, I., Hammarling, S., Henry, G., et al.: An updated set of basic linear algebra subprograms (BLAS). ACM Trans. Math. Softw. 28(2), 135–151 (2002)

    Article  Google Scholar 

  89. Volkov, V., Demmel, J.: LU, QR and Cholesky factorizations using vector capabilities of GPUs. Technical Report UCB/EECS-2008-49, EECS Department, University of California, Berkeley, CA (2008). http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-49.html

  90. Humphrey, J.R., Price, D.K., Spagnoli, K.E., Paolini, A.L., J.Kelmelis, E.: CULA: hybrid GPU accelerated linear algebra routines. In: Kelmelis, E.J. (ed.) Proceeding of SPIE: Modeling and Simulation for Defense Systems and Applications V, vol. 7705. SPIE (2010)

    Google Scholar 

  91. Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide, 3rd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA (1999)

    Google Scholar 

  92. Smola, A.J., Bartlett, P.L.: Sparse greedy Gaussian process regression. Advances in Neural Information Processing Systems, vol. 13, pp. 619–625. MIT Press, Cambridge, MA (2001)

    Google Scholar 

  93. Wahba, G.: Spline Models for Observational Data. Society for Industrial and Applied Mathematics, Philadelphia, PA (1990)

    Google Scholar 

  94. Williams, C.K.I., Seeger, M.: Using the Nyström method to speed up kernel machines. Advances in Neural Information Processing Systems, pp. 682–688. MIT Press, Cambridge, MA (2001)

    Google Scholar 

  95. Quiñonero-Candela, J., Rasmussen, C.E.: A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939–1959 (2005)

    MATH  MathSciNet  Google Scholar 

  96. Quiñonero-Candela, J., Rasmussen, C.E., Williams, C.K.I.: Approximation methods for Gaussian process regression (Chap). Large Scale Learning Machines, pp. 203–223. MIT Press, Cambridge, MA (2007)

    Google Scholar 

  97. Chalupka, K., Williams, C.K.I., Murray, I.: A framework for evaluating approximation methods for Gaussian process regression. J. Mach. Learn. Res. 14, 333–350 (2013)

    MATH  MathSciNet  Google Scholar 

  98. Murray, I.: Gaussian processes and fast matrix-vector multiplies. In: Presented at the Numerical Mathematics in Machine Learning workshop at the 26th International Conference on Machine Learning (ICML 2009), Montreal (2009). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.153.7688

  99. Leithead, W.E., Zhang, Y., Leith, D.J.: Time-series Gaussian process regression based on Toeplitz computation of O(N2) operations and O(N) level storage. In: Joint 44th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC). Sevilla (2005)

    Google Scholar 

  100. Leithead, W.E., Zhang, Y.: O(N-2)-operation approximation of covariance matrix inverse in Gaussian process regression based on quasi-Newton BFGS method. Commun. Stat.-Simul. Comput. 36(2), 367–380 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  101. Zhang, Y., Leithead, W.E.: Approximate implementation of the logarithm of the matrix determinant in Gaussian process regression. J. Stat. Comput. Simul. 77(4), 329–348 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  102. Lawrence, N.D., Seeger, M., Herbrich, R.: Fast sparse Gaussian process methods: the informative vector machine. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 609–616. MIT Press, Cambridge, MA (2003)

    Google Scholar 

  103. Csató, L., Opper, M.: Sparse online Gaussian processes. Neural Comput. 14(3), 641–668 (2002)

    Article  MATH  Google Scholar 

  104. Sathiya Keerthi, S., Chu, W.: A matching pursuit approach to sparse Gaussian process regression. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems, vol. 18, pp. 643–650. MIT Press, Cambridge, MA (2006)

    Google Scholar 

  105. Seeger, M., Williams, C.K.I., Lawrence, N.D.: Fast forward selection to speed up sparse Gaussian process regression. In: Ninth International Workshop on Artificial Intelligence and Statistics, Society for Artificial Intelligence and Statistics (2003)

    Google Scholar 

  106. Lazáro-Gredilla M., Quiñonero-Candela J., Figueiras-Vidal A.: Sparse spectral sampling. Technical report, Microsoft Research, Cambridge (2007)

    Google Scholar 

  107. Lazáro-Gredilla, M., Quiñonero-Candela, J., Rasmussen, C.E., Figueiras-Vidal, A.R.: Sparse spectrum Gaussian process regression. J. Mach. Learn. Res. 11, 1865–1881 (2010)

    MATH  MathSciNet  Google Scholar 

  108. Titsias, M.: Variational learning of inducing variables in sparse Gaussian processes. In: The 12th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 5, pp. 567–574 (2009)

    Google Scholar 

  109. Ni, W., Tan, S.K., Ng, W.J., Brown, S.D.: Moving-window GPR for nonlinear dynamic system modeling with dual updating and dual preprocessing. Ind. Eng. Chem. Res. 51(18), 6416–6428 (2012)

    Article  Google Scholar 

  110. Oba, S., Sato, M., Ishii, S.: On-line learning methods for Gaussian processes. In: Dorffner, G., Bischof, H., Hornik, K. (eds.) Artificial Neural Networks (ICANN 2001). Lecture Notes in Computer Science, vol. 2130, pp. 292–299. Springer, Berlin (2001). doi:10.1007/3-540-44668-0

    Chapter  Google Scholar 

  111. Grbić, R., Slišković, D., Kadlec, P.: Adaptive soft sensor for online prediction based on moving window Gaussian process regression. In: 2012 11th International Conference on Machine Learning and Applications, pp. 428–433 (2012)

    Google Scholar 

  112. Ranganathan, A., Yang, M.H., Ho, J.: Online sparse Gaussian process regression and its applications. IEEE Trans. Image Process. 20, 391–404 (2011)

    Article  MathSciNet  Google Scholar 

  113. Nguyen-Tuong, D., Seeger, M., Peters, J.: Real-time local GP model learning. From Motor Learning to Interaction Learning in Robots, pp. 193–207. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  114. Tresp, V.: A Bayesian committee machine. Neural Comput. 12, 2719–2741 (2000)

    Article  Google Scholar 

  115. Shi, J.Q., Murray-Smith, R., Titterington, D.M.: Hierarchical Gaussian process mixtures for regression. Statist. Comput. 15(1), 31–41 (2005)

    Article  MathSciNet  Google Scholar 

  116. Gregorčič, G., Lightbody, G.: Local model identification with Gaussian processes. IEEE Trans. Neural Netw. 18(5), 1404–1423 (2007)

    Article  Google Scholar 

  117. Petelin, D., Kocijan, J.: Control system with evolving Gaussian process model. In: Proceedings of IEEE Symposium Series on Computational Intelligence, SSCI 2011. IEEE, Paris (2011)

    Google Scholar 

  118. Angelov, P., Filev, D.P., Kasabov, N.: Evolving Intelligent Systems: Methodology and Applications. IEEE Press Series on Computational Intelligence. Wiley-IEEE Press, New York, NY (2010)

    Google Scholar 

  119. Åström, K.J., Wittenmark, B.: Computer Controlled Systems: Theory and Design. Prentice Hall, Upper Saddle River, NJ (1984)

    Google Scholar 

  120. Isermann, R., Lachman, K.H., Matko, D.: Adaptive Control Systems. Systems and Control Engineering. Prentice Hall International, Upper Saddle River, NJ (1992)

    Google Scholar 

  121. Fritzke, B.: Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural Netw. 7(9), 1441–1460 (1994)

    Article  Google Scholar 

  122. Angelov, P., Buswell, R.: Evolving rule-based models: a tool for intelligent adaptation. In: Proceedings of the Joint 9th NAFIPS International Conference, pp. 1062–1066. IEEE Press (2001)

    Google Scholar 

  123. Kasabov, N.K.: Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines. Springer, New York, NY (2002)

    Google Scholar 

  124. Abusnina, A., Kudenko, D.: Adaptive soft sensor based on moving Gaussian process window, pp. 1051–1056. IEEE (2013)

    Google Scholar 

  125. Petelin, D., Grancharova, A., Kocijan, J.: Evolving Gaussian process models for the prediction of ozone concentration in the air. Simul. Modell. Pract. Theory 33(1), 68–80 (2013)

    Article  Google Scholar 

  126. Deisenroth, M.P., Rasmussen, C.E.: PILCO: A model-based and data-efficient approach to policy search. In: Proceedings of the 28th International Conference on Machine Learning (ICML 2011). Bellevue, WA (2011)

    Google Scholar 

  127. Ni, W., Tan, S.K., Ng, W.J.: Recursive GPR for nonlinear dynamic process modeling. Chem. Eng. J. 173(2), 636–643 (2011)

    Article  Google Scholar 

  128. Ni, W., Wang, K., Chen, T., Ng, W.J., Tan, S.K.: GPR model with signal preprocessing and bias update for dynamic processes modeling. Control Eng. Pract. 20(12), 1281–1292 (2012)

    Article  Google Scholar 

  129. Duvenaud, D., Lloyd, J.R., Grosse, R., Tenenbaum, J.B., Ghahramani, Z.: Structure discovery in nonparametric regression through compositional kernel search. In: Proceedings of the 30th International Conference on Machine Learning (2013)

    Google Scholar 

  130. Seeger, M.: Low rank updates for the Cholesky decomposition. Technical report, University of California, Berkeley, CA (2008)

    Google Scholar 

  131. Murray-Smith, D.J.: Methods for the external validation of continuous system simulation models: a review. Math. Comput. Modell. Dyn. Syst. 4(1), 5–31 (1998)

    Article  MATH  Google Scholar 

  132. Cawley, G.C., Talbot, N.L.C.: On over-fitting in model selection and subsequent selection bias in performance evaluation. J. Mach. Learn. Res. 11, 2079–2107 (2010)

    MATH  MathSciNet  Google Scholar 

  133. Hvala, N., Strmčnik, S., Šel, D., Milanić, S., Banko, B.: Influence of model validation on proper selection of process models — an industrial case study. Comput. Chem. Eng. 29, 1507–1522 (2005)

    Article  Google Scholar 

  134. Kocijan, J., Girard, A., Banko, B., Murray-Smith, R.: Dynamic systems identification with Gaussian processes. Math. Comput. Modell. Dyn. Syst. 11(4), 411–424 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  135. Girard, A.: Approximate methods for propagation of uncertainty with Gaussian process models. Ph.D. thesis, University of Glasgow, Glasgow (2004). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.63.8313

  136. Girard, A., Rasmussen, C.E., Candela, J.Q., Murray-Smith, R.: Gaussian process priors with uncertain inputs - application to multiple-step ahead time series forecasting. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 542–552. MIT Press, Cambridge, MA (2003)

    Google Scholar 

  137. Groot, P., Lucas, P., van den Bosch, P.: Multiple-step time series forecasting with sparse Gaussian processes. In: Proceedings of the 23rd Benelux Conference on Artificial Intelligence (BNAIC 2011), pp. 105–112. Ghent (2011)

    Google Scholar 

  138. Gutjahr, T., Ulmer, H., Ament, C.: Sparse Gaussian processes with uncertain inputs for multi-step ahead prediction. In: 16th IFAC Symposium on System Identification, pp. 107–112. Brussels, (2012)

    Google Scholar 

  139. Girard, A., Rasmussen, C., Murray-Smith, R.: Gaussian process priors with uncertain inputs: multiple-step ahead prediction. Technical report DCS TR-2002-119, University of Glasgow, Glasgow (2002)

    Google Scholar 

  140. Kocijan, J., Likar, B.: Gas-liquid separator modelling and simulation with Gaussian-process models. Simul. Modell. Pract. Theory 16(8), 910–922 (2008)

    Article  Google Scholar 

  141. Ažman, K., Kocijan, J.: Application of Gaussian processes for black-box modelling of biosystems. ISA Trans. 46, 443–457 (2007)

    Article  Google Scholar 

  142. Cho, J., Principe, J.C., Erdogmus, D., Motter, M.A.: Quasi-sliding model control strategy based on multiple-linear models. Neurocomputing 70, 960–974 (2007)

    Article  Google Scholar 

  143. Gauthier, J.P., Hammouri, H., Othman, S.: A simple observer for nonlinear systems applications to bioreactors. IEEE Trans. Autom. Control 6, 875–880 (1992)

    Article  MathSciNet  Google Scholar 

  144. Lind, I.: Regressor selection in system identification using ANOVA. Licentiate thesis, University of Linköping, Linköping (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Systems and Control, Jožef Stefan Institute, Ljubljana, Slovenia

    Juš Kocijan

  2. Centre for Systems and Information Technologies, University of Nova Gorica, Nova Gorica, Slovenia

    Juš Kocijan

Authors
  1. Juš Kocijan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juš Kocijan .

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kocijan, J. (2016). System Identification with GP Models. In: Modelling and Control of Dynamic Systems Using Gaussian Process Models. Advances in Industrial Control. Springer, Cham. https://doi.org/10.1007/978-3-319-21021-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-21021-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-21020-9

  • Online ISBN: 978-3-319-21021-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation