Federica Filippini
Valeria Cardellini
ABSTRACT
The widespread adoption of Artificial Intelligence applications to analyze data generated by Internet of Things sensors leads to the development of the edge computing paradigm. Deploying applications at the periphery of the network effectively addresses cost and latency concerns associated with cloud computing. However, it generates a highly distributed environment with heterogeneous devices, opening the challenges of how to select resources and place application components. Starting from a state-of-the-art design-time tool, we present in this paper a novel framework based on Reinforcement Learning, named FIGARO (reinForcement learnInG mAnagement acRoss the computing cOntinuum). It handles the runtime adaptation of a computing continuum environment, dealing with the variability of the incoming load and service times. To reduce the training time, we exploit the design-time knowledge, achieving a significant reduction in the violations of the response time constraint.
- Zeinab Akhavan et al. 2022. Deep reinforcement learning for online latency aware workload offloading in mobile edge computing. In IEEE GLOBECOM'22, 2218--2223.Google Scholar
- James Bergstra et al. 2013. Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In ICML'13. Vol. 28, 115--123.Google Scholar
- Zequn Cao and Xiaoheng Deng. 2023. Dependent task offloading in edge computing using GNN and deep reinforcement learning. (2023). arXiv: 2303.17100 [cs.DC].Google Scholar
- Valeria Cardellini et al. 2018. Decentralized self-adaptation for elastic data stream processing. Future Gener. Comput. Syst., 87, 171--185.Google ScholarDigital Library
- FORTUNE Business Insights. 2023. Cloud computing market size, growth & COVID-19 impact analysis, 2023--2030. https://www.fortunebusinessinsights.com/cloud-computing-market-102697. (2023).Google Scholar
- Shilu Li et al. 2020. Joint optimization of caching and computation in multi-server NOMA-MEC system via reinforcement learning. IEEE Access, 8, 112762--112771.Google ScholarCross Ref
- Yihong Li et al. 2023. Task placement and resource allocation for edge machine learning: A GNN-based multi-agent reinforcement learning paradigm. (2023). arXiv: 2302.00571 [cs.MA].Google Scholar
- Volodymyr Mnih et al. 2013. Playing Atari with deep reinforcement learning. (2013). arXiv: 1312.5602 [cs.LG].Google Scholar
- Malathy Navaneetha Krishnan and Revathi Thiyagarajan. 2022. Multi-objective task scheduling in fog computing using improved gaining sharing knowledge based algorithm. Concurr. Comput., 34, 24, e7227.Google ScholarCross Ref
- Xin Peng et al. 2023. Deep reinforcement learning for shared offloading strategy in vehicle edge computing. IEEE Syst. J., 17, 2.Google Scholar
- Edgar Ramos et al. 2019. Distributing intelligence to the edge and beyond [research frontier]. Comp. Intell. Mag., 14, 4, 65--92.Google ScholarDigital Library
- Sebastián Risco et al. 2021. Serverless workflows for containerised applications in the cloud continuum. J. Grid Comput., 19, 3, 1--18.Google Scholar
- Fabiana Rossi et al. 2019. Horizontal and vertical scaling of container-based applications using reinforcement learning. In IEEE CLOUD'19, 329--338.Google Scholar
- Hamta Sedghani et al. 2021. A random greedy based design time tool for AI applications component placement and resource selection in computing continua. In IEEE EDGE'21, 32--40.Google Scholar
- Yi Su et al. 2023. Joint DNN Partition and Resource Allocation Optimization for Energy-Constrained Hierarchical Edge-Cloud Systems. IEEE Trans. Veh. Technol., 72, 3, 3930--3944.Google ScholarCross Ref
- Yaohua Sun et al. 2022. Enabling mobile virtual reality with open 5G, fog computing and reinforcement learning. IEEE Network, 36, 6, 142--149.Google ScholarCross Ref
- Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT Press.Google ScholarDigital Library
- Bao Trinh and Gabriel-Miro Muntean. 2022. A deep reinforcement learning-based offloading scheme for multi-access edge computing-supported extended reality systems. IEEE Trans. Veh. Technol., 72, 1, 1254--1264.Google ScholarCross Ref
- Wenting Wei et al. 2023. Multi-dimensional resource allocation in distributed data centers using deep reinforcement learning. IEEE Trans. Netw. Service Manag., 20, 2, 1817--1829.Google ScholarDigital Library
- Somayeh Yeganeh et al. 2023. A novel Q-learning-based hybrid algorithm for the optimal offloading and scheduling in mobile edge computing environments. J. Netw. Comput. Appl., 214, 103617.Google ScholarDigital Library
- Seonghoon Yoo et al. 2023. Hybrid UAV-enabled secure offloading via deep reinforcement learning. IEEE Wirel. Commun. Lett., 12, 6, 972--976.Google ScholarCross Ref
- Zhi Zhou et al. 2021. Deep reinforcement learning for intelligent cloud resource management. In IEEE INFOCOM'21 Workshops, 1--6.Google Scholar
Index Terms
- FIGARO: reinForcement learnInG mAnagement acRoss the computing cOntinuum
Recommendations
Task Offloading in Computing Continuum Using Collaborative Reinforcement Learning
Internet of ThingsAbstractOne of the challenges in the Computing Continuum paradigm is the optimal distribution of the generated tasks between the devices in each layer (cloud-fog-edge). In this paper, we propose to use Reinforcement Learning (RL) to solve the Task ...
Deep learning, reinforcement learning, and world models
AbstractDeep learning (DL) and reinforcement learning (RL) methods seem to be a part of indispensable factors to achieve human-level or super-human AI systems. On the other hand, both DL and RL have strong connections with our brain functions ...
Infrastructure Manager: A TOSCA-Based Orchestrator for the Computing Continuum
AbstractThe edge-to-cloud continuum involves heterogeneous computing resources, including low-power physical devices, Virtual Machines (VMs) in cloud management platforms and serverless computing services based on the FaaS (Functions as a Service) model. ...
Comments