Francesco Fraternali
Bharathan Balaji
Rajesh K. Gupta
ABSTRACT
Batteryless energy-harvesting sensing systems are attractive for low maintenance but face challenges in real-world applications due to low quality of service from sporadic and unpredictable energy availability. To overcome this challenge, recent data-driven energy management techniques optimize energy usage to maximize application performance even in low harvested energy scenarios by learning energy availability patterns in the environment. These techniques require prior knowledge of the environment in which the sensor nodes are deployed to work correctly. In the absence of historical data, the application performance deteriorates.
To overcome this challenge, we describe here the use of meta reinforcement learning to increase the application performance of newly deployed batteryless sensor nodes without historical data. Our system, called Marble, exploits information from other sensor node locations to expedite the learning of newly deployed sensor nodes, and improves application performance in the initial period of deployment. Our evaluation using real-world data traces shows that Marble detects up to 66% more events in low lighting conditions, and up to 25.6% more events on average on the first 3 days of deployment compared to the state-of-the-art.1
- Fayçal Ait Aoudia at al. 2018. RLMan: An Energy Manager Based on Reinforcement Learning for Energy Harvesting Wireless Sensor Networks. IEEE Transactions on Green Communications and Networking (2018).Google Scholar
- Bradford Campbell and Prabal Dutta. 2014. An Energy-Harvesting Sensor Architecture and Toolkit for Building Monitoring and Event Detection. In Proceedings of the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings (BuildSys '14). Association for Computing Machinery.Google ScholarDigital Library
- Roy Chaoming Hsu, Cheng-Ting Liu, and Wei-Ming Lee. 2009. Reinforcement Learning-Based Dynamic Power Management for Energy Harvesting Wireless Sensor Network. In Next-Generation Applied Intelligence, Been-Chian Chien, Tzung-Pei Hong, Shyi-Ming Chen, and Moonis Ali (Eds.).Google Scholar
- Qingping Chi, Hairong Yan, Chuan Zhang, Zhibo Pang, and Li Da Xu. 2014. A Reconfigurable Smart Sensor Interface for Industrial WSN in IoT Environment. IEEE Transactions on Industrial Informatics (2014).Google Scholar
- Man Chu, Hang Li, Xuewen Liao, and Shuguang Cui. 2018. Reinforcement learning-based multiaccess control and battery prediction with energy harvesting in IoT systems. IEEE Internet of Things Journal 6, 2 (2018), 2009--2020.Google ScholarCross Ref
- Sanio Semiconductor CO. 2007. https://media.digikey.com/pdf/Data%20Sheets/Sanyo%20Energy/Amorphous_Br.pdf.Google Scholar
- Samuel DeBruin, Bradford Campbell, and Prabal Dutta. 2013. Monjolo: An Energy-harvesting Energy Meter Architecture. In Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems (Roma, Italy) (SenSys '13). ACM, New York, NY, USA, Article 18, 14 pages. Google ScholarDigital Library
- Gabriel Martins Dias, Maddalena Nurchis, and Boris Bellalta. 2016. Adapting sampling interval of sensor networks using on-line reinforcement learning. In 2016 IEEE 3rd World Forum on Internet of Things (WF-IoT).Google Scholar
- echelon.com. 2018. https://www.echelon.com/.Google Scholar
- Joshua F Ensworth and Matthew S Reynolds. 2017. Ble-backscatter: ultralow-power IoT nodes compatible with bluetooth 4.0 low energy (BLE) smartphones and tablets. IEEE Transactions on Microwave Theory and Techniques 65, 9 (2017).Google ScholarCross Ref
- R. C. Hsu et al. 2014. A Reinforcement Learning-Based ToD Provisioning Dynamic Power Management for Sustainable Operation of Energy Harvesting Wireless Sensor Node. IEEE Transactions on Emerging Topics in Computing (2014).Google Scholar
- Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400 [cs.LG]Google Scholar
- Francesco Fraternali, Bharathan Balaji, Yuvraj Agarwal, Luca Benini, and Rajesh K. Gupta. 2018. Pible: Battery-Free Mote for Perpetual Indoor BLE Applications. In Proceedings of the 5th ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Building (BuildSys '18). ACM.Google Scholar
- Francesco Fraternali, Bharathan Balaji, Yuvraj Agarwal, and Rajesh K. Gupta. 2020. ACES: Automatic Configuration of Energy Harvesting Sensors with Reinforcement Learning. ACM Trans. Sen. Netw. 16, 4, Article 36 (July 2020), 31 pages. Google ScholarDigital Library
- Francesco Fraternali, Bharathan Balaji, and Rajesh Gupta. 2018. Scaling Configuration of Energy Harvesting Sensors with Reinforcement Learning. In Proceedings of the 6th International Workshop on Energy Harvesting & Energy-Neutral Sensing Systems (ENSsys '18). Google ScholarDigital Library
- Francesco Fraternali, Bharathan Balaji, Dhiman Sengupta, Dezhi Hong, and Rajesh K. Gupta. 2020. Ember: Energy Management of Batteryless Event Detection Sensors with Deep Reinforcement Learning (SenSys '20). Association for Computing Machinery, New York, NY, USA.Google Scholar
- Josiah Hester, Lanny Sitanayah, and Jacob Sorber. 2015. Tragedy of the Coulombs: Federating Energy Storage for Tiny, Intermittently-Powered Sensors (SenSys '15).Google Scholar
- Josiah Hester and Jacob Sorber. 2017. Flicker: Rapid Prototyping for the Batteryless Internet-of-Things (SenSys '17). Association for Computing Machinery.Google ScholarDigital Library
- Josiah Hester, Kevin Storer, and Jacob Sorber. 2017. Timely Execution on Intermittently Powered Batteryless Sensors. In Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems (SenSys '17).Google ScholarDigital Library
- Jason Hsu, Sadaf Zahedi, Aman Kansal, Mani Srivastava, and Vijay Raghunathan. 2006. Adaptive Duty Cycling for Energy Harvesting Systems. In ISLPED'06 Proceedings of the 2006 Int Symposium on Low Power Electronics and Design.Google Scholar
- Jui-Yang Hsu, Yuan-Jui Chen, and Hung-yi Lee. 2020. Meta learning for end-to-end low-resource speech recognition. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 7844--7848.Google ScholarCross Ref
- R. C. Hsu, C. T. Liu, K. C. Wang, and W. M. Lee. 2009. QoS-Aware Power Management for Energy Harvesting Wireless Sensor Network Utilizing Reinforcement Learning. In 2009 International Conference on Computational Science and Engineering, Vol. 2. 537--542. Google ScholarDigital Library
- Hrishikesh Jayakumar, Kangwoo Lee, Woo Suk Lee, Arnab Raha, Younghyun Kim, and Vijay Raghunathan. 2014. Powering the internet of things. In Proceedings of the 2014 international symposium on Low power electronics and design. ACM.Google ScholarDigital Library
- Aman Kansal, Jason Hsu, Sadaf Zahedi, and Mani B. Srivastava. 2007. Power Management in Energy Harvesting Sensor Networks. ACM Trans. Embed. Comput. Syst. 6, 4, Article 32 (Sept. 2007). Google ScholarDigital Library
- Bryce Kellogg, Aaron Parks, Shyamnath Gollakota, Joshua R Smith, and David Wetherall. 2014. Wi-Fi backscatter: Internet connectivity for RF-powered devices. In ACM SIGCOMM Computer Communication Review, Vol. 44. ACM, 607--618.Google ScholarDigital Library
- Azam Khan and Kasper Hornbæk. 2011. Big Data from the Built Environment. In Proceedings of the 2nd International Workshop on Research in the Large. Association for Computing Machinery.Google ScholarDigital Library
- Victor Lawson and Lakshmish Ramaswamy. 2015. Data Quality and Energy Management Tradeoffs in Sensor Service Clouds. In Big Data (Big Data Congress), 2015 IEEE International Congress on. IEEE, 749--752.Google Scholar
- Eric Liang, Richard Liaw, Philipp Moritz, Robert Nishihara, Roy Fox, Ken Goldberg, Joseph E. Gonzalez, Michael I. Jordan, and Ion Stoica. 2018. RLlib: Abstractions for Distributed Reinforcement Learning. arXiv:1712.09381 [cs.AI]Google Scholar
- Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018).Google Scholar
- Brandon Lucia, Vignesh Balaji, Alexei Colin, Kiwan Maeng, and Emily Ruppel. 2017. Intermittent computing: Challenges and opportunities. In 2nd Summit on Advances in Programming Languages (SNAPL 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.Google Scholar
- Yubo Luo et al. 2019. Spoton: Just-in-time active event detection on energy autonomous sensing systems. Brief Presentations Proceedings (RTAS 2019) (2019).Google Scholar
- Kiwan Maeng, Alexei Colin, and Brandon Lucia. 2017. Alpaca: Intermittent execution without checkpoints. Proceedings of the ACM on Programming Languages 1, OOPSLA (2017), 1--30.Google ScholarDigital Library
- Paul Martin, Zainul Charbiwala, and Mani Srivastava. 2012. DoubleDip: Leveraging Thermoelectric Harvesting for Low Power Monitoring of Sporadic Water Use. In 10th ACM Conference on Embedded Network Sensor Systems (SenSys '12).Google ScholarDigital Library
- Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 2017. A simple neural attentive meta-learner. arXiv preprint arXiv:1707.03141 (2017).Google Scholar
- C. Moser, L. Thiele, D. Brunelli, and L. Benini. 2010. Adaptive Power Management for Environmentally Powered Systems. IEEE Trans. Comput. (2010).Google Scholar
- Abdulmajid Murad, Kerstin Bach, Frank Alexander Kraemer, and Gavin Taylor. 2019. IoT Sensor Gym: Training Autonomous IoT Devices with Deep Reinforcement Learning. In Proceedings of the 9th International Conference on the Internet of Things. Association for Computing Machinery.Google ScholarDigital Library
- Abdulmajid Murad, Frank Alexander Kraemer, Kerstin Bach, and Gavin Taylor. 2019. Autonomous Management of Energy-Harvesting IoT Nodes Using Deep Reinforcement Learning. arXiv preprint arXiv:1905.04181 (2019).Google Scholar
- Abdulmajid Murad, Frank Alexander Kraemer, Kerstin Bach, and Gavin Taylor. 2020. Information-Driven Adaptive Sensing Based on Deep Reinforcement Learning. In 10th International Conference on the Internet of Things (IoT '20).Google Scholar
- Saman Naderiparizi and et al. 2015. Self-localizing Battery-free Cameras. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15).Google Scholar
- Saman Naderiparizi, Aaron N Parks, Zerina Kapetanovic, Benjamin Ransford, and Joshua R Smith. 2015. WISPCam: A battery-free RFID camera. In RFID (RFID), 2015 IEEE International Conference on. IEEE, 166--173.Google ScholarCross Ref
- Matteo Nardello, Harsh Desai, Davide Brunelli, and Brandon Lucia. 2019. Camaroptera: A Batteryless Long-Range Remote Visual Sensing System (ENSsys'19).Google ScholarDigital Library
- Panasonic. 2018. https://media.digikey.com/pdf/Data%20Sheets/Panasonic%20Electronic%20Components/SG_Series_LowTemp.pdf.Google Scholar
- products.currentbyge.com. 2017. https://products.currentbyge.com/sites/products.currentbyge.com/files/documents/document_file/DT106-GE-Wireless-Occupancy-Sensor-Wall-Mount-Installation-Guide.pdf.Google Scholar
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. (2017).Google Scholar
- Shaswot Shresthamali and et al. 2017. Adaptive Power Management in Solar Energy Harvesting Sensor Node Using Reinforcement Learning. ACM Transactions on Embedded Computing Systems (TECS) (2017).Google Scholar
- Jake Snell, Kevin Swersky, and Richard S Zemel. 2017. Prototypical networks for few-shot learning. arXiv preprint arXiv:1703.05175 (2017).Google Scholar
- Sujesha Sudevalayam and Purushottam Kulkarni. 2011. Energy harvesting sensor nodes: Survey and implications. IEEE Communications Surveys & Tutorials (2011).Google Scholar
- Richard S Sutton, Andrew G Barto, Francis Bach, et al. 1998. Reinforcement learning: An introduction. MIT press.Google Scholar
- Nima Tajbakhsh, Jae Y Shin, Suryakanth R Gurudu, R Todd Hurst, Christopher B Kendall, Michael B Gotway, and Jianming Liang. 2016. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE transactions on medical imaging 35, 5 (2016), 1299--1312.Google Scholar
- Vamsi Talla, Bryce Kellogg, Shyamnath Gollakota, and Joshua R Smith. 2017. Battery-free cellphone. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (2017), 25.Google ScholarDigital Library
- Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, et al. 2019. Meta-dataset: A dataset of datasets for learning to learn from few examples. arXiv preprint arXiv:1903.03096 (2019).Google Scholar
- Adrian Udenze and Klaus McDonald-Maier. 2009. Direct reinforcement learning for autonomous power configuration and control in wireless networks. In Adaptive Hardware and Systems, 2009. AHS 2009. NASA/ESA Conference on. IEEE.Google ScholarDigital Library
- Sennur Ulukus, Aylin Yener, Elza Erkip, Osvaldo Simeone, Michele Zorzi, Pulkit Grover, and Kaibin Huang. 2015. Energy harvesting wireless communications: A review of recent advances. IEEE Journal on Selected Areas in Communicat (2015).Google Scholar
- utc.com. 2018. https://www.utc.com/.Google Scholar
- Christopher M Vigorito, Deepak Ganesan, and Andrew G Barto. 2007. Adaptive control of duty cycling in energy-harvesting wireless sensor networks. In 2007 4th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks. IEEE, 21--30.Google ScholarCross Ref
- Christopher JCH Watkins and Peter Dayan. 1992. Q-learning. Machine learning 8, 3--4 (1992), 279--292.Google Scholar
- Thomas Weng, Bharathan Balaji, Seemanta Dutta, Rajesh Gupta, and Yuvraj Agarwal. 2011. Managing plug-loads for demand response within buildings. In Proceedings of the Third ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings. 13--18.Google ScholarDigital Library
- Kok-Lim Alvin Yau and et al. 2012. Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues. Journal of Network and Computer Applications (2012).Google Scholar
Index Terms
- Marble: collaborative scheduling of batteryless sensors with meta reinforcement learning
Recommendations
Marble: collaborative scheduling of batteryless sensors with meta reinforcement learning
BuildSys '21: Proceedings of the 8th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and TransportationBatteryless energy-harvesting sensing systems are attractive for low maintenance but face challenges in real-world applications due to the low quality of service from sporadic and unpredictable energy availability. To overcome this challenge, recent ...
ACES: Automatic Configuration of Energy Harvesting Sensors with Reinforcement Learning
Many modern smart building applications are supported by wireless sensors to sense physical parameters, given the flexibility they offer and the reduced cost of deployment. However, most wireless sensors are powered by batteries today, and large ...
Pible: battery-free mote for perpetual indoor BLE applications
BuildSys '18: Proceedings of the 5th Conference on Systems for Built EnvironmentsSmart building applications require a large-scale deployment of sensors distributed across the environment. Recent innovations in smart environments are driven by wireless networked sensors as they are easy to deploy. However, replacing these batteries ...
Comments