Tomasz Szydlo
ABSTRACT
We observe significant interest in reinforcement learning methods for real-world sensing-control scenarios driven by the sensor data streams. However, the delay introduced to the data by the communication channels may degrade the system's performance. It is especially crucial in the internet of things (IoT), where devices with constraint resources and low throughput networks are used.
We demonstrate TinyRL framework, a different approach to this problem, by transferring RL algorithms knowledge to resource-limited devices. Our initial experiments point towards a successful demonstration of our technique using common microcontrollers used in IoT systems. Such devices have limited resource capability, and their regulation by processing data directly on devices without their transmission to the cloud can play a crucial role in their lifespan and usefulness.
Supplemental Material
- Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, Carlsbad, CA, 578--594.Google Scholar
- Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized Neural Networks. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. Curran Associates, Inc.Google Scholar
- Lei Lei, Yue Tan, Kan Zheng, Shiwen Liu, Kuan Zhang, and Xuemin Shen. 2020. Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges. IEEE Communications Surveys Tutorials, Vol. 22, 3 (2020), 1722--1760.Google ScholarCross Ref
- Norbert Mitschke, Michael Heizmann, Klaus-Henning Noffz, and Ralf Wittmann. 2019. A Fixed-Point Quantization Technique for Convolutional Neural Networks Based on Weight Scaling. In 2019 IEEE International Conference on Image Processing (ICIP). 3836--3840.Google ScholarCross Ref
- Muhammad Shafique, Theocharis Theocharides, Vijay Janapa Reddy, and Boris Murmann. 2021. TinyML: Current Progress, Research Challenges, and Future Roadmap. In 2021 58th ACM/IEEE Design Automation Conference (DAC). 1303--1306.Google ScholarDigital Library
- Jiecao Yu, Andrew Lukefahr, David J. Palframan, Ganesh S. Dasika, Reetuparna Das, and Scott A. Mahlke. 2017. Scalpel: Customizing DNN pruning to the underlying hardware parallelism. 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA) (2017), 548--560. iGoogle Scholar
Index Terms
- TinyRL: Towards Reinforcement Learning on Tiny Embedded Devices
Recommendations
HoneyIoT: Adaptive High-Interaction Honeypot for IoT Devices Through Reinforcement Learning
WiSec '23: Proceedings of the 16th ACM Conference on Security and Privacy in Wireless and Mobile NetworksAs IoT devices are becoming widely deployed, there exist many threats to IoT-based systems due to their inherent vulnerabilities. One effective approach to improving IoT security is to deploy IoT honeypot systems, which can collect attack information ...
Recent Reinforcement Learning and Blockchain Based Security Solutions for Internet of Things: Survey
AbstractUsers’ security is one of the most important issues in Internet of Things (IoT) due to the high number of IoT devices involved in different applications. Security threats are evolving at a rapid pace that make the current security and privacy ...
Reward Shaping in Episodic Reinforcement Learning
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent SystemsRecent advancements in reinforcement learning confirm that reinforcement learning techniques can solve large scale problems leading to high quality autonomous decision making. It is a matter of time until we will see large scale applications of ...
Comments