Skip to main content
SpringerLink
Log in

Event-triggered \(\textrm{H}_\infty \) consensus control for input-constrained multi-agent systems via reinforcement learning

  • Research Article
  • Published:
Control Theory and Technology Aims and scope Submit manuscript

Abstract

This article presents an event-triggered \(\textrm{H}_\infty \) consensus control scheme using reinforcement learning (RL) for nonlinear second-order multi-agent systems (MASs) with control constraints. First, considering control constraints, the constrained \(\textrm{H}_\infty \) consensus problem is transformed into a multi-player zero-sum game with non-quadratic performance functions. Then, an event-triggered control method is presented to conserve communication resources and a new triggering condition is developed for each agent to make the triggering threshold independent of the disturbance attenuation level. To derive the optimal controller that can minimize the cost function in the case of worst disturbance, a constrained Hamilton–Jacobi–Bellman (HJB) equation is defined. Since it is difficult to solve analytically due to its strongly non-linearity, reinforcement learning (RL) is implemented to obtain the optimal controller. In specific, the optimal performance function and the worst-case disturbance are approximated by a time-triggered critic network; meanwhile, the optimal controller is approximated by event-triggered actor network. After that, Lyapunov analysis is utilized to prove the uniformly ultimately bounded (UUB) stability of the system and that the network weight errors are UUB. Finally, a simulation example is utilized to demonstrate the effectiveness of the control strategy provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The data that support the findings of this study are available upon request from the corresponding author dclrce@gmail.com.

References

  1. Jafari, M., & Xu, H. (2019). A biologically-inspired distributed fault tolerant flocking control for multi-agent system in presence of uncertain dynamics and unknown disturbance. Engineering Applications of Artificial Intelligence, 79, 1–12.

    Article  Google Scholar 

  2. Xue, Y., Luo, Y., & Zhu, M. (2020). Uav formation control method based on consistency strategy. IOP Conference Series Earth and Environmental Science, 440, 052084.

    Article  Google Scholar 

  3. Dou, C., Yue, D., Guerrero, J. M., Xie, X., & Hu, S. (2016). Multiagent system-based distributed coordinated control for radial dc microgrid considering transmission time delays. IEEE Transactions on Smart Grid, 8(5), 2370–2381.

    Article  Google Scholar 

  4. Dai, L., Hao, Y., Xie, H., Sun, Z., & Xia, Y. (2022). Distributed robust MPC for nonholonomic robots with obstacle and collision avoidance. Control Theory and Technology, 20(1), 32–45.

    Article  MathSciNet  Google Scholar 

  5. Wen, G., Chen, C. P., Liu, Y.-J., & Liu, Z. (2016). Neural network-based adaptive leader-following consensus control for a class of nonlinear multiagent state-delay systems. IEEE Transactions on Cybernetics, 47(8), 2151–2160.

    Article  Google Scholar 

  6. Li, H., Zhang, X., & Pan, W. (2022). Consensus control of feedforward nonlinear multi-agent systems: a time-varying gain method. Control Theory and Technology, 20(1), 46–53.

    Article  MathSciNet  Google Scholar 

  7. Rezaee, H., & Abdollahi, F. (2020). Adaptive leaderless consensus control of strict-feedback nonlinear multiagent systems with unknown control directions. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 51(10), 6435–6444.

    Article  Google Scholar 

  8. Rezaei, V., & Stefanovic, M. (2018). Distributed output feedback stationary consensus of multi-vehicle systems in unknown environments. Control Theory and Technology, 16(2), 93–109.

    Article  MathSciNet  Google Scholar 

  9. Li, G., Ren, C.-E., & Chen, C. P. (2021). Preview-based leader-following consensus control of distributed multi-agent systems. Information Sciences, 559, 251–269.

    Article  MathSciNet  Google Scholar 

  10. Zhang, X., Zhu, Q., & Liu, X. (2016). Consensus of second order multi-agent systems with exogenous disturbance generated by unknown exosystems. Entropy, 18(12), 423.

    Article  ADS  Google Scholar 

  11. Lin, P., & Jia, Y. (2010). Robust H-infinity consensus analysis of a class of second-order multi-agent systems with uncertainty. IET Control Theory & Applications, 3(4), 487–498.

    Article  Google Scholar 

  12. Liu, D., Li, H., & Wang, D. (2013). Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing, 110, 92–100.

    Article  Google Scholar 

  13. Sassano, M., & Astolfi, A. (2012). Dynamic approximate solutions of the HJ inequality and of the HJB equation for input-affine nonlinear systems. IEEE Transactions on Automatic Control, 57(10), 2490–2503.

    Article  MathSciNet  Google Scholar 

  14. Huang, Z., Li, Y., Zhang, C., Wu, G., Liu, Y., & Chen, Y. (2018). A data-driven approximate solution to the model-free HJB equation. Optimal Control Applications and Methods, 39(2), 835–844.

    Article  MathSciNet  Google Scholar 

  15. Luo, B., Wu, H.-N., Huang, T., & Liu, D. (2015). Reinforcement learning solution for HJB equation arising in constrained optimal control problem. Neural Networks, 71, 150–158.

    Article  PubMed  Google Scholar 

  16. Zhang, H., Zhao, X., Wang, H., Zong, G., & Xu, N. (2022). Hierarchical sliding-mode surface-based adaptive actor-critic optimal control for switched nonlinear systems with unknown perturbation. IEEE Transactions on Neural Networks and Learning Systems, 2, 2.

    CAS  Google Scholar 

  17. Liu, D., Wei, Q., Wang, D., Yang, X., & Li, H. (2017). Adaptive dynamic programming with applications in optimal control. Berlin: Springer.

    Book  Google Scholar 

  18. Zhao, Y., Niu, B., Zong, G., Xu, N., & Ahmad, A. M. (2023). Event-triggered optimal decentralized control for stochastic interconnected nonlinear systems via adaptive dynamic programming. Neurocomputing, 539, 126163.

    Article  Google Scholar 

  19. Yang, X., & He, H. (2019). Adaptive critic learning and experience replay for decentralized event-triggered control of nonlinear interconnected systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 50(11), 4043–4055.

    Article  Google Scholar 

  20. Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics, 5, 834–846.

    Article  Google Scholar 

  21. Wen, G., Chen, C. P., & Li, B. (2019). Optimized formation control using simplified reinforcement learning for a class of multiagent systems with unknown dynamics. IEEE Transactions on Industrial Electronics, 67(9), 7879–7888.

    Article  Google Scholar 

  22. Wen, G., Chen, C. P., Feng, J., & Zhou, N. (2017). Optimized multi-agent formation control based on an identifier-actor-critic reinforcement learning algorithm. IEEE Transactions on Fuzzy Systems, 26(5), 2719–2731.

    Article  Google Scholar 

  23. Wen, G., & Chen, C. P. (2021). Optimized backstepping consensus control using reinforcement learning for a class of nonlinear strict-feedback-dynamic multi-agent systems. IEEE Transactions on Neural Networks and Learning Systems, 2, 2.

    Google Scholar 

  24. Zhang, H., Wang, H., Niu, B., Zhang, L., & Ahmad, A. M. (2021). Sliding-mode surface-based adaptive actor-critic optimal control for switched nonlinear systems with average dwell time. Information Sciences, 580, 756–774.

  25. Liu, L., Liu, Y.-J., & Tong, S. (2018). Neural networks-based adaptive finite-time fault-tolerant control for a class of strict-feedback switched nonlinear systems. IEEE Transactions on Cybernetics, 49(7), 2536–2545.

    Article  PubMed  Google Scholar 

  26. Anta, A., & Tabuada, P. (2010). To sample or not to sample: Self-triggered control for nonlinear systems. IEEE Transactions on Automatic Control, 55(9), 2030–2042.

    Article  MathSciNet  Google Scholar 

  27. Liu, W., & Huang, J. (2018). Cooperative global robust output regulation for a class of nonlinear multi-agent systems by distributed event-triggered control. Automatica, 93, 138–148.

    Article  MathSciNet  Google Scholar 

  28. Dong, L., Zhong, X., Sun, C., & He, H. (2016). Event-triggered adaptive dynamic programming for continuous-time systems with control constraints. IEEE Transactions on Neural Networks and Learning Systems, 28(8), 1941–1952.

    Article  MathSciNet  PubMed  Google Scholar 

  29. Zhao, J., Gan, M., & Zhang, C. (2019). Event-triggered H-infinity optimal control for continuous-time nonlinear systems using neurodynamic programming. Neurocomputing, 360, 14–24.

  30. Zhu, Y., Zhao, D., He, H., & Ji, J. (2016). Event-triggered optimal control for partially unknown constrained-input systems via adaptive dynamic programming. IEEE Transactions on Industrial Electronics, 64(5), 4101–4109.

    Article  Google Scholar 

  31. Yang, D., Li, T., Zhang, H., & Xie, X. (2019). Event-trigger-based robust control for nonlinear constrained-input systems using reinforcement learning method. Neurocomputing, 340, 158–170.

    Article  Google Scholar 

  32. Yang, X., & He, H. (2020). Event-driven H-infinity-constrained control using adaptive critic learning. IEEE Transactions on Cybernetics, 51(10), 4860–4872.

    Article  Google Scholar 

  33. Zhao, W., Yu, W., & Zhang, H. (2019). Event-triggered optimal consensus tracking control for multi-agent systems with unknown internal states and disturbances. Nonlinear Analysis: Hybrid Systems, 33, 227–248.

    MathSciNet  Google Scholar 

  34. Luo, Y., & Zhu, W. (2021). Event-triggered H-infinity finite-time consensus control for nonlinear second-order multi-agent systems with disturbances. Advances in Difference Equations, 2021(1), 1–19.

    Article  ADS  MathSciNet  Google Scholar 

  35. Vamvoudakis, K. G. (2014). Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA Journal of Automatica Sinica, 1(3), 282–293.

    Article  Google Scholar 

  36. Vamvoudakis, K. G., Mojoodi, A., & Ferraz, H. (2017). Event-triggered optimal tracking control of nonlinear systems. International Journal of Robust and Nonlinear Control, 27(4), 598–619.

    Article  MathSciNet  Google Scholar 

  37. Abu-Khalaf, M., Huang, J., & Lewis, F. L. (2006). Nonlinear H2/H-infinity constrained feedback control: a practical design approach using neural networks. Berlin: Springer.

    Google Scholar 

  38. Lewis, F., Jagannathan, S., & Yesildirak, A. (1998). Neural network control of robot manipulators and non-linear systems. London: CRC Press.

    Google Scholar 

  39. Bhasin, S., Kamalapurkar, R., Johnson, M., Vamvoudakis, K. G., Lewis, F. L., & Dixon, W. E. (2013). A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica, 49(1), 82–92.

    Article  MathSciNet  Google Scholar 

  40. Wen, G., & Li, B. (2021). Optimized leader-follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52(9), 5546–5555.

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Jinxuan Zhang and Chang-E. Ren contributed equally to this work.

Authors and Affiliations

  1. College of Information Engineering, Capital Normal University, Beijing, 100089, China

    Jinxuan Zhang & Chang-E Ren

Authors
  1. Jinxuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chang-E Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chang-E Ren.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Ren, CE. Event-triggered \(\textrm{H}_\infty \) consensus control for input-constrained multi-agent systems via reinforcement learning. Control Theory Technol. 22, 25–38 (2024). https://doi.org/10.1007/s11768-023-00177-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11768-023-00177-4

Keywords

Search

Navigation