Skip to main content
SpringerLink
Log in

DRJOA: intelligent resource management optimization through deep reinforcement learning approach in edge computing

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Mobile edge computing (MEC) can enhance the computation capabilities of smart mobile devices for computation-intensive mobile applications via supporting computation offloading efficiently. However, the limitation of wireless resources and computational resources of edge servers often becomes the bottlenecks to realizing the developments of MEC. In order to address the computation offloading problem in the time-varying wireless networks, the offloading decisions and the allocation of radio and computation resources need to be jointly managed. Traditional optimization methods are challenging to deal with the combinatorial optimization problem in complex real-time dynamic network environments. Therefore, we propose a deep reinforcement learning (DRL)-based optimization approach, named DRJOA, which jointly optimizes offloading decisions, computation, and wireless resources allocation. The optimization algorithm based on DRL has the advantages of fast solving speed and strong generalization ability, which makes it possible to solve combinatorial optimization problems online. Simulation results show that our proposed DRJOA in this study dramatically outperforms the benchmark methods for offloading decisions and system utility.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Li, K.: A game theoretic approach to computation offloading strategy optimization for non-cooperative users in mobile edge computing, IEEE Trans. Sustain. Comput. pp. 1–1 (2018)

  2. Xu, X., Zhang, X., Gao, H., Xue, Y., Qi, L., Dou, W.: Become: blockchain-enabled computation offloading for iot in mobile edge computing. IEEE Trans. Ind. Inform. 16(6), 4187–4195 (2020)

    Article  Google Scholar 

  3. Arthur Sandor, V.. K., Lin, Y., Li, X., Lin, F., Zhang, S.: Efficient decentralized multi-authority attribute based encryption for mobile cloud data storage. J. Netw. Comput. Appl. 129, 25–36 (2019)

    Article  Google Scholar 

  4. Long, C., Cao, Y., Jiang, T., Zhang, Q.: Edge computing framework for cooperative video processing in multimedia IOT systems. IEEE Trans. Multimed. 20, 1126–1139 (2018)

    Article  Google Scholar 

  5. Liu, C., Li, K., Liang, J., Li, K.: COOPER-MATCH: Job offloading with a cooperative game for guaranteeing strict deadlines in mec. IEEE Trans. Mobile Comput. pp. 1–1 (2019)

  6. Yi, C., Cai, J., Su, Z.: A multi-user mobile computation offloading and transmission scheduling mechanism for delay-sensitive applications. IEEE Trans. Mobile Comput. 19(1), 29–43 (2020)

    Article  Google Scholar 

  7. Wang, C., Liang, C., Yu, F.R., Chen, Q., Tang, L.: Computation offloading and resource allocation in wireless cellular networks with mobile edge computing. IEEE Trans. Wirel. Commun. 16(8), 4924–4938 (2017)

    Article  Google Scholar 

  8. Chen, M., Hao, Y.: Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J. Sel. Areas Commun. 36(3), 587–597 (2018)

    Article  MathSciNet  Google Scholar 

  9. Zhao, J., Li, Q., Gong, Y., Zhang, K.: Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks. IEEE Trans. Veh. Technol. 68(8), 7944–7956 (2019)

    Article  Google Scholar 

  10. Zhou, W., Chen, L., Tang, S., Lai, L., Xia, J., Zhou, F., Fan, L.: Offloading strategy with PSO for mobile edge computing based on cache mechanism. Clust. Comput. 25(4), 2389–2401 (2022)

    Article  Google Scholar 

  11. Bacanin, N., Antonijevic, M., Bezdan, T., Zivkovic, M., Venkatachalam, K., Malebary, S.: Energy efficient offloading mechanism using particle swarm optimization in 5g enabled edge nodes, Clust. Comput. pp. 1–12 (2022)

  12. Lyu, X., Tian, H., Sengul, C., Zhang, P.: Multiuser joint task offloading and resource optimization in proximate clouds. IEEE Trans. Veh. Technol. 66, 3435–3447 (2017)

    Article  Google Scholar 

  13. Tran, T.X., Pompili, D.: Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Trans. Veh. Technol. 68, 856–868 (2019)

    Article  Google Scholar 

  14. Du, J., Yu, F.R., Chu, X., Feng, J., Lu, G.: Computation offloading and resource allocation in vehicular networks based on dual-side cost minimization. IEEE Trans. Veh. Technol. 68(2), 1079–1092 (2019)

    Article  Google Scholar 

  15. Li, H., Xu, H., Zhou, C., Lü, X., Han, Z.: Joint optimization strategy of computation offloading and resource allocation in multi-access edge computing environment. IEEE Trans. Veh. Technol. 69(9), 10214–10226 (2020)

    Article  Google Scholar 

  16. Zhang, D., Tang, J., Du, W., Ren, J., Yu, G.: Joint optimization of computation offloading and ul, dl resource allocation in mec systems. In: IEEE 29th annual international symposium on personal. Indoor Mobile Radio Commun. (PIMRC), pp. 1–6 (2018)

  17. Huang, P.-Q., Wang, Y., Wang, K., Liu, Z.-Z.: A bilevel optimization approach for joint offloading decision and resource allocation in cooperative mobile edge computing. IEEE Trans. Cybern. 50(10), 4228–4241 (2020)

    Article  Google Scholar 

  18. Narendra, P., Fukunaga, K.: A branch and bound algorithm for feature subset selection. IEEE Trans. Comput. 26, 917–922 (1977)

    Article  MATH  Google Scholar 

  19. Bertsekas, D.: Dynamic programming and optimal control (1995)

  20. Bi, S., Zhang, Y.: Computation rate maximization for wireless powered mobile-edge computing with binary computation offloading. IEEE Trans. Wirel. Commun. 17, 4177–4190 (2018)

    Article  Google Scholar 

  21. Li, Z., Chen, S., Zhang, S., Jiang, S., Gu, Y., Nouioua, M.: FSB-EA: fuzzy search bias guided constraint handling technique for evolutionary algorithm. Expert Syst. Appl. 119, 20–35 (2019)

    Article  Google Scholar 

  22. Guo, S., Xiao, B., Yang, Y., Yang, Y.: Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. In: IEEE INFOCOM 2016—the 35th Annual IEEE International Conference on Computer Communications, pp. 1–9 (2016)

  23. Dinh, T.Q., Tang, J., La, Q., Quek, T.Q.S.: Offloading in mobile edge computing: task allocation and computational frequency scaling. IEEE Trans. Commun. 65, 3571–3584 (2017)

    Google Scholar 

  24. Liang, W., Li, Y., Xie, K., Zhang, D., Li, K.-C., Souri, A., Li, K.: Spatial-temporal aware inductive graph neural network for c-its data recovery. In: IEEE Transactions on Intelligent Transportation Systems, pp. 1–12 (2022)

  25. Diao, C., Zhang, D., Liang, W., Li, K.-C., Hong, Y., Gaudiot, J.-L.: A novel spatial-temporal multi-scale alignment graph neural network security model for vehicles prediction. In: IEEE Transactions on Intelligent Transportation Systems, pp. 1–11 (2022)

  26. Zhao, P., Tian, H., Qin, C., Nie, G.: Energy-saving offloading by jointly allocating radio and computational resources for mobile edge computing. IEEE Access 5, 11255–11268 (2017)

    Article  Google Scholar 

  27. Chen, M.-H., Dong, M., Liang, B.: Joint offloading decision and resource allocation for mobile cloud with computing access point. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3516–3520 (2016)

  28. Li, J., Gao, H., Lv, T., Lu, Y.: Deep reinforcement learning based computation offloading and resource allocation for mec. In: 2018 IEEE Wireless Communications and Networking Conference (WCNC), pp. 1–6 (2018)

  29. Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., Bennis, M.: Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet Things J. 6, 4005–4018 (2019)

    Article  Google Scholar 

  30. Huang, L., Bi, S., Zhang, Y.: Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Trans. Mobile Comput. 19, 2581–2593 (2020)

    Article  Google Scholar 

  31. Zhan, Y., Guo, S., Li, P., Zhang, J.: A deep reinforcement learning based offloading game in edge computing. IEEE Trans. Comput. 69, 883–893 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  32. Du, J., Yu, F.R., Lu, G., Wang, J., Jiang, J., Chu, X.: MEC-assisted immersive VR video streaming over terahertz wireless networks: A deep reinforcement learning approach. IEEE Internet Things J. 7(10), 9517–9529 (2020)

    Article  Google Scholar 

  33. Mustafa, E., Shuja, J., Bilal, K., Mustafa, S., Maqsood, T., Rehman, F. et al.: Reinforcement learning for intelligent online computation offloading in wireless powered edge networks. Clust. Comput. pp. 1–10 (2022)

  34. Cuervo, E., Balasubramanian, A., ki Cho, D., Wolman, A., Saroiu, S., Chandra, R., Bahl, P.:MAUI: making smartphones last longer with code offload, in: MobiSys ’10, (2010)

  35. Yang, L., Cao, J., Yuan, Y., Li, T., Han, A., Chan, A.: A framework for partitioning and execution of data stream applications in mobile cloud computing. In: 2012 IEEE Fifth International Conference on Cloud Computing pp. 794–802 (2012)

  36. Sesia, S., Toufik, I., Baker, M.: LTE-the UMTS long term evolution: From theory to practice. (2011)

  37. Wen, Y., Zhang, W., Luo, H.: Energy-optimal mobile application execution: taming resource-poor mobile devices with cloud clones. In: 2012 Proceedings IEEE INFOCOM. pp. 2716–2720 (2012)

  38. Miettinen, A. P. , Nurminen, J.: Energy efficiency of mobile clients in cloud computing. In: HotCloud (2010)

  39. Chen, X.: Decentralized computation offloading game for mobile cloud computing. IEEE Trans. Parall. Distrib. Syst. 26, 974–983 (2015)

    Article  Google Scholar 

  40. Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations

  41. Yang, L., Zhang, H., Li, M., Guo, J., Ji, H.: Mobile edge computing empowered energy efficient task offloading in 5G. IEEE Trans. Veh. Technol. 67(7), 6398–6409 (2018)

    Article  Google Scholar 

Download references

Funding

This work was partially supported by National Key Research and Development Program of China (No. 2018YFB1308604), National Natural Science Foundation of China (Nos. U21A20518, 61976086, 61906065), State Grid Science and Technology Project (No. 5100-202123009A), Special Project of Foshan Science and Technology Innovation Team (No. FS0AA-KJ919-4402-0069), and Hunan Natural Science Foundation (No. 2020JJ5200).

Author information

Authors and Affiliations

  1. College of Computer Science and Electronic Engineering, Hunan University, Changsha, 410082, China

    Yifan Chen & Zhiyong Li

  2. Key Laboratory for Embedded and Network Computing of Hunan Province, Changsha, 410082, China

    Yifan Chen & Zhiyong Li

  3. School of Computer Science and Engineering, Hunan University of Science and Technology, Xiangtan, 411201, China

    Shaomiao Chen & Wei Liang

  4. Department of Computer Science and Information Engineering, Providence University, Taichung, 43301, Taiwan

    Kuan-Ching Li

Authors
  1. Yifan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaomiao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kuan-Ching Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhiyong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YC conceived the presented idea. YC and SC designed the model and the computational framework, and analyzed the problem models and design algorithms. YC conducted simulation experiments and analyzed the results. YC and SC wrote the manuscript with input from all authors. KL, WL and ZL conceived the study and were in charge of overall direction and planning.

Corresponding authors

Correspondence to Wei Liang or Zhiyong Li.

Ethics declarations

Conflict of interest

The authors declared that they do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Human and animal rights

This work did not include humans or animal participants.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Chen, S., Li, KC. et al. DRJOA: intelligent resource management optimization through deep reinforcement learning approach in edge computing. Cluster Comput 26, 2897–2911 (2023). https://doi.org/10.1007/s10586-022-03768-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-022-03768-z

Keywords

Search

Navigation