Skip to main content
SpringerLink
Log in

Evaluation of Prioritized Deep System Identification on a Path Following Task

  • Regular Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper revisits system identification and shows how new paradigms from machine learning can be used to improve it in the case of non-linear systems modeling from noisy and unbalanced dataset. We show that using importance sampling schemes in system identification can provide a significant performance boost in modeling, which is helpful to a predictive controller. The performance of the approach is first evaluated on simulated data of a Unmanned Surface Vehicle (USV). Our approach consistently outperforms baseline approaches on this dataset. Moreover we demonstrate the benefits of this identification methodology in a control setting. We use the model of the Unmanned Surface Vehicle (USV) in a Model Predictive Path Integral (MPPI) controller to perform a track following task. We discuss the influence of the controller parameters and show that the prioritized model outperform standard methods. Finally, we apply the Model Predictive Path Integral (MPPI) on a real system using the know-how developed here.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., et al.: Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467 (2016)

  2. Akpan, V.A., Hassapis, G.D.: Nonlinear model identification and adaptive model predictive control using neural networks. ISA Trans. 50(2), 177–194 (2011)

    Article  Google Scholar 

  3. Alain, G., Lamb, A., Sankar, C., Courville, A., Bengio, Y.: Variance reduction in sgd by distributed importance sampling. arXiv:1511.06481 (2015)

  4. Amini, A., Schwarting, W., Soleimany, A., Rus, D.: Deep evidential regression. Advances in Neural Information Processing Systems. 33 (2020)

  5. Dentler, J., Kannan, S., Mendez, M.A.O., Voos, H.: A tracking error control approach for model predictive position control of a quadrotor with time varying reference. In: Robotics and Biomimetics (ROBIO), 2016 IEEE International Conference On, pp. 2051–2056 (2016)

  6. Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)

  7. Gonzalez, J., Yu, W.: Non-linear system modeling using lstm neural networks. IFAC-PapersOnLine 51(13), 485–489 (2018)

    Article  Google Scholar 

  8. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  9. Horgan, D., Quan, J., Budden, D., Barth-Maron, G., Hessel, M., Van Hasselt, H., Silver, D.: Distributed prioritized experience replay. arXiv:1803.00933 (2018)

  10. Hwangbo, J., Sa, I., Siegwart, R., Hutter, M.: Control of a quadrotor with reinforcement learning. IEEE Robot. Autom. Lett. 2(4), 2096–2103 (2017)

    Article  Google Scholar 

  11. Katharopoulos, A., Fleuret, F.: Biased importance sampling for deep neural network training. arXiv:1706.00043 (2017)

  12. Katharopoulos, A., Fleuret, F.: Not all samples are created equal: Deep learning with importance sampling. arXiv:1803.00942 (2018)

  13. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv:1412.6980(2014)

  14. Lattanzi, D., Miller, G.: Review of robotic infrastructure inspection systems. J Infrastruct. Syst. 23(3), 04017004 (2017)

    Article  Google Scholar 

  15. Lenain, R., Thuilot, B., Cariou, C., Martinet, P.: High accuracy path tracking for vehicles in presence of sliding: Application to farm vehicle automatic guidance for agricultural tasks. Auton. Robots 21(1), 79–97 (2006)

    Article  Google Scholar 

  16. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. arXiv:1509.02971 (2015)

  17. Ljung, L.: System identification. In: Signal Analysis and Prediction, pp 163–173. Springer (1998)

  18. Loshchilov, I., Hutter, F.: Online batch selection for faster training of neural networks. arXiv:1511.06343 (2015)

  19. Lucet, E., Lenain, R., Grand, C.: Dynamic path tracking control of a vehicle on slippery terrain. Control Eng. Pract. 42, 60–73 (2015)

    Article  Google Scholar 

  20. Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: Proc. Icml, vol. 30, p. 3 (2013)

  21. Mahé, A., Pradalier, C., Geist, M.: Trajectory-control using deep system identication and model predictive control for drone control under uncertain load. In: 2018 22Nd International Conference on System Theory, Control and Computing (ICSTCC), pp. 753–758. https://doi.org/10.1109/ICSTCC.2018.8540719 (2018)

  22. Mahé, A., Richard, A., Mouscadet, B., Pradalier, C., Geist, M.: Importance sampling for deep system identification. In: 2019 19Th International Conference on Advanced Robotics (ICAR), pp. 43–48. IEEE (2019)

  23. Malinin, A., Gales, M.: Predictive uncertainty estimation via prior networks. In: Advances in Neural Information Processing Systems, pp. 7047–7058 (2018)

  24. Naegeli, T., Alonso-Mora, J., Domahidi, A., Rus, D., Hilliges, O.: Real-time motion planning for aerial videography with dynamic obstacle avoidance and viewpoint optimization. IEEE Robot. Automat. Lett. 2(3), 1696–1703 (2017). https://doi.org/10.1109/LRA.2017.2665693

    Article  Google Scholar 

  25. Pannocchia, G.: Offset-free tracking Mpc: a tutorial review and comparison of different formulations. In: Control Conference (ECC), 2015 European, pp. 527–532. IEEE (2015)

  26. Qin, S.J., Badgwell, T.A.: A survey of industrial model predictive control technology. Control Eng. Pract. 11(7), 733–764 (2003)

    Article  Google Scholar 

  27. Schaal, S., Atkeson, C.G., Vijayakumar, S.: Real-time robot learning with locally weighted statistical learning. In: Robotics and Automation, 2000. Proceedings. ICRA’00. IEEE International Conference On, vol. 1, pp. 288–293 (2000)

  28. Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized experience replay. arXiv:1511.05952 (2015)

  29. Williams, G., Drews, P., Goldfain, B., Rehg, J.M., Theodorou, E.A.: Aggressive driving with model predictive path integral control. In: Robotics and Automation (ICRA), 2016 IEEE International Conference On, pp. 1433–1440 (2016)

  30. Williams, G., Wagener, N., Goldfain, B., Drews, P., Rehg, J.M., Boots, B., Theodorou, E.A.: Information theoretic mpc for model-based reinforcement learning (2017)

  31. Williams, G., Wagener, N., Goldfain, B., Drews, P., Rehg, J.M., Boots, B., Theodorou, E.A.: Information theoretic Mpc for model-based reinforcement learning. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 1714–1721 (2017)

  32. Yaghoubi, S., Akbarzadeh, N.A., Bazargani, S.S., Bazargani, S.S., Bamizan, M., Asl, M.I.: Autonomous robots for agricultural tasks and farm assignment and future trends in agro robots. Int. J. Mech. Mechatronics Eng. 13(3), 1–6 (2013)

    Google Scholar 

  33. Zhang, T., Kahn, G., Levine, S., Abbeel, P.: Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search. ArXiv e-prints (2015)

  34. Zhang, T., Kahn, G., Levine, S., Abbeel, P.: Learning deep control policies for autonomous aerial vehicles with Mpc-guided policy search. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 528–535 (2016)

Download references

Funding

This work is done under the Grande Region rObotique aerienNE(GRoNe) project, funded by a European Union Grant thought theFEDER INTERREG VAinitiative and the french “Grand Est” Région.

Author information

Authors and Affiliations

  1. CentraleSupélec, Université de Lorraine, CNRS, LORIA, Villers-lès-Nancy, France

    Antoine Mahé

  2. Georgia Institute of Technology, Atlanta, 30332–0250, GA, USA

    Antoine Richard

  3. IRL 2958 GT-CNRS, 2 rue Marconi, 57070, Metz, France

    Stéphanie Aravecchia & Cédric Pradalier

  4. Google France, 8 rue de Londres, 75009, Paris, France

    Matthieu Geist

Authors
  1. Antoine Mahé
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antoine Richard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stéphanie Aravecchia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthieu Geist
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cédric Pradalier
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Antoine Mahé: Simulation experiments, writing, coding

Antoine Richard: Simulation/Field experiments, writing, coding

Stéhanie Aravecchia: Field experiments, writing, coding

Matthieu Geist: Supervision, writing, review

Cédric Pradalier: Supervision, writing, review

Corresponding author

Correspondence to Antoine Richard.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is done under the Grande Region rObotique aerienNE (GRoNe) project, funded by a European Union Grant thought the feder interreg va initiative and the french “Grand Est” Région.

Antoine Mahé and Antoine Richard contributed equally to this work

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahé, A., Richard, A., Aravecchia, S. et al. Evaluation of Prioritized Deep System Identification on a Path Following Task. J Intell Robot Syst 101, 78 (2021). https://doi.org/10.1007/s10846-021-01341-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01341-1

Keywords

Search

Navigation