He Zhang
John Carroll
ABSTRACT
Both sensor networks and data fusion are essential foundations for developing the smart home Internet of Things (IoT) and related fields. We proposed a multi-channel sensor network construction method involving hardware, acquisition, and synchronization in the smart home environment and a smart home data fusion method (SHDFM) for multi-modal data (position, gait, voice, pose, facial expression, temperature, and humidity) generated in the smart home environment to address the configuration of a multi-channel sensor network, improve the quality and efficiency of various human activities and environmental data collection, and reduce the difficulty of multi-modal data fusion in the smart home. SHDFM contains 5 levels, with inputs and outputs as criteria to provide recommendations for multi-modal data fusion strategies in the smart home. We built a real experimental environment using the proposed method in this paper. To validate our method, we created a real experimental environment — a physical setup in a home-like scenario where the multi-channel sensor network and data fusion techniques were deployed and evaluated. The acceptance and testing results show that the proposed construction and data fusion methods can be applied to the examples with high robustness, replicability, and scalability. Besides, we discuss how smart homes with multi-channel sensor networks can support digital twins.
- Silvana M Azcarate, Rocío Ríos-Reina, José M Amigo, and Héctor C Goicoechea. 2021. Data handling in data fusion: methodologies and applications. TrAC Trends in Analytical Chemistry 143 (2021), 116355. https://doi.org/10.1016/j.trac.2021.116355Google Scholar
Cross Ref
- Altaf QH Badar and Amjad Anvari-Moghaddam. 2022. Smart home energy management system–a review. Advances in Building Energy Research 16, 1 (2022), 118–143. https://doi.org/10.1080/17512549.2020.1806925Google ScholarCross Ref
- Alexander Benlian, Johannes Klumpe, and Oliver Hinz. 2020. Mitigating the intrusive effects of smart home assistants by using anthropomorphic design features: A multimethod investigation. Information Systems Journal 30, 6 (2020), 1010–1042. https://doi.org/10.1111/isj.12243Google ScholarCross Ref
- Federico Castanedo. 2013. A review of data fusion techniques. The scientific world journal 2013 (2013), 1–19. https://doi.org/10.1155/2013/704504Google ScholarCross Ref
- Guo Chen, Zhigui Liu, Guang Yu, and Jianhong Liang. 2021. A New View of Multisensor Data Fusion: Research on Generalized Fusion. Mathematical Problems in Engineering 2021 (2021), 1–21. https://doi.org/10.1155/2021/5471242Google ScholarCross Ref
- B.V. Dasarathy. 1997. Sensor fusion potential exploitation-innovative architectures and illustrative applications. Proc. IEEE 85, 1 (1997), 24–38. https://doi.org/10.1109/5.554206Google ScholarCross Ref
- Paul Dourish. 2006. Re-Space-Ing Place: "Place" and "Space" Ten Years On. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work (Banff, Alberta, Canada) (CSCW ’06). Association for Computing Machinery, New York, NY, USA, 299–308. https://doi.org/10.1145/1180875.1180921Google ScholarDigital Library
- Hugh Durrant-Whyte and Thomas C Henderson. 2016. Multisensor data fusion. Springer handbook of robotics (2016), 867–896. https://doi.org/10.1007/978-3-319-32552-1_35Google ScholarCross Ref
- Benjamin Eckstein, Eva Krapp, Anne Elsässer, and Birgit Lugrin. 2019. Smart substitutional reality: Integrating the smart home into virtual reality. Entertainment Computing 31 (2019), 100306. https://doi.org/10.1016/j.entcom.2019.100306Google ScholarCross Ref
- Berry Eggen, Gerard Hollemans, and Richard van de Sluis. 2003. Exploring and enhancing the home experience. Cognition, Technology & Work 5, 1 (2003), 44–54. https://doi.org/10.1007/s10111-002-0114-7Google ScholarCross Ref
- Jaime Esteban, Andrew Starr, Robert Willetts, Paul Hannah, and Peter Bryanston-Cross. 2005. A review of data fusion models and architectures: towards engineering guidelines. Neural Computing & Applications 14, 4 (2005), 273–281. https://doi.org/10.1007/s00521-004-0463-7Google ScholarDigital Library
- Dina Fawzy, Sherin Moussa, and Nagwa Badr. 2021. The Spatiotemporal Data Fusion (STDF) approach: IoT-based data fusion using big data analytics. Sensors 21, 21 (2021), 7035. https://doi.org/10.3390/s21217035Google ScholarCross Ref
- Xinyi FU, He ZHANG, Cheng XUE, and Tongxin SUN. 2023. A review of the frontier research on future smart home. Science & Technology Review 41, 8, Article 36 (2023), 16 pages. https://doi.org/10.3981/j.issn.1000-7857.2023.08.004Google ScholarCross Ref
- V Gopinath, A Srija, and C Neethu Sravanthi. 2019. Re-design of smart homes with digital twins. In Journal of Physics: Conference Series, Vol. 1228. IOP Publishing, 012031. https://doi.org/10.1088/1742-6596/1228/1/012031Google ScholarCross Ref
- D.L. Hall and J. Llinas. 1997. An introduction to multisensor data fusion. Proc. IEEE 85, 1 (1997), 6–23. https://doi.org/10.1109/5.554205Google ScholarCross Ref
- Tom Hargreaves, Charlie Wilson, and Richard Hauxwell-Baldwin. 2018. Learning to live in a smart home. Building Research & Information 46, 1 (2018), 127–139. https://doi.org/10.1080/09613218.2017.1286882Google ScholarCross Ref
- Guosheng Hu, Yu’ao Wang, Mengyuan Mao, and Yi Zhao. 2022. Remote Care and Collaboration for Empty Nest Family: Smart Home, Digital Twin and Mixed Reality. In 2022 8th International Conference on Virtual Reality (ICVR). IEEE, 126–134. https://doi.org/10.1109/ICVR55215.2022.9847779Google ScholarCross Ref
- Stephen S Intille, Kent Larson, J Beaudin, E Munguia Tapia, Pallavi Kaushik, Jason Nawyn, and Thomas J McLeish. 2005. The PlaceLab: A live-in laboratory for pervasive computing research (video). Proceedings of PERVASIVE 2005 Video Program (2005).Google Scholar
- Rebecca Jamwal, Hannah K Jarman, Eve Roseingrave, Jacinta Douglas, and Dianne Winkler. 2022. Smart home and communication technology for people with disability: a scoping review. Disability and Rehabilitation: Assistive Technology 17, 6 (2022), 624–644. https://doi.org/10.1080/17483107.2020.1818138Google ScholarCross Ref
- S. R. Jino Ramson and D. Jackuline Moni. 2017. Applications of wireless sensor networks — A survey. In 2017 International Conference on Innovations in Electrical, Electronics, Instrumentation and Media Technology (ICEEIMT). IEEE, 325–329. https://doi.org/10.1109/ICIEEIMT.2017.8116858Google ScholarCross Ref
- Bahador Khaleghi, Alaa Khamis, Fakhreddine O Karray, and Saiedeh N Razavi. 2013. Multisensor data fusion: A review of the state-of-the-art. Information fusion 14, 1 (2013), 28–44. https://doi.org/10.1016/j.inffus.2011.08.001Google ScholarDigital Library
- Cheonshik Kim, Bryan Kim, Joel JPC Rodrigues, and James CN Yang. 2018. Advanced technology for smart home automation and entertainment., 2 pages. https://doi.org/10.1007/s00779-017-1102-5Google ScholarDigital Library
- Sunyoung Kim and Abhishek Choudhury. 2021. Exploring older adults’ perception and use of smart speaker-based voice assistants: A longitudinal study. Computers in Human Behavior 124 (2021), 106914. https://doi.org/10.1016/j.chb.2021.106914Google ScholarDigital Library
- Rachel C King, Emma Villeneuve, Ruth J White, R Simon Sherratt, William Holderbaum, and William S Harwin. 2017. Application of data fusion techniques and technologies for wearable health monitoring. Medical engineering & physics 42 (2017), 1–12.Google Scholar
- Elena Kornyshova, Rebecca Deneckere, Kaoutar Sadouki, Eric Gressier-Soudan, and Sjaak Brinkkemper. 2022. Smart Life: Review of the Contemporary Smart Applications. In International Conference on Research Challenges in Information Science. Springer, 302–318. https://doi.org/10.1007/978-3-031-05760-1_18Google ScholarCross Ref
- Erik Kučera, Oto Haffner, and Štefan Kozák. 2018. Connection between 3D engine unity and microcontroller arduino: A virtual smart house. In 2018 Cybernetics & Informatics (K&I). 1–8. https://doi.org/10.1109/CYBERI.2018.8337531Google ScholarCross Ref
- Jeremiah Lasquety-Reyes. 2019. Smart Home Report 2021. Retrieved October 19, 2022 from https://www.statista.com/study/42112/smart-home-report/Google Scholar
- Carson K Leung, Peter Braun, and Alfredo Cuzzocrea. 2019. AI-based sensor information fusion for supporting deep supervised learning. Sensors 19, 6 (2019), 1345. https://doi.org/10.3390/s19061345Google ScholarCross Ref
- Chua Boon Liang, Mujahid Tabassum, Saad Bin Abul Kashem, Zulfiqar Zama, P Suresh, and U Saravanakumar. 2021. Smart home security system based on Zigbee. In Advances in Smart System Technologies. Springer, 827–836. https://doi.org/10.1007/978-981-15-5029-4_71Google ScholarCross Ref
- Martin Liggins II, David Hall, and James Llinas. 2017. Handbook of multisensor data fusion: theory and practice. CRC press.Google Scholar
- Haipeng Liu, Yuheng Wang, Anfu Zhou, Hanyue He, Wei Wang, Kunpeng Wang, Peilin Pan, Yixuan Lu, Liang Liu, and Huadong Ma. 2020. Real-Time Arm Gesture Recognition in Smart Home Scenarios via Millimeter Wave Sensing. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4, Article 140 (dec 2020), 28 pages. https://doi.org/10.1145/3432235Google ScholarDigital Library
- Jie Liu, Dan Luo, Xinyi Fu, Qi Lu, and Karen Yixin Kang. 2023. Design Strategy of Multimodal Perception System for Smart Environment. In Internet of Things for Smart Environments. Springer, 93–115. https://doi.org/10.1007/978-3-031-09729-4_6Google ScholarCross Ref
- Davit Marikyan, Savvas Papagiannidis, and Eleftherios Alamanos. 2019. A systematic review of the smart home literature: A user perspective. Technological Forecasting and Social Change 138 (2019), 139–154. https://doi.org/10.1016/j.techfore.2018.08.015Google ScholarCross Ref
- Shashi Mehrotra, Shweta Sinha, and Sudhir Kumar Sharma. 2021. Overview of the Internet of Things and Ubiquitous Computing. In Blockchain Technology for Data Privacy Management. CRC Press, 1–19.Google Scholar
- David Mills, Jim Martin, Jack Burbank, and William Kasch. 2010. Network time protocol version 4: Protocol and algorithms specification. Technical Report. http://www.rfc-editor.org/info/rfc5905Google Scholar
- YANG Ming-Hao and TAO Jian-Hua. 2019. Data fusion methods in multimodal human computer dialog. Virtual Reality & Intelligent Hardware 1, 1 (2019), 21–38. https://doi.org/10.3724/SP.J.2096-5796.2018.0010Google ScholarCross Ref
- Raz Kamaran Radha. 2022. Flexible smart home design: Case study to design future smart home prototypes. Ain Shams Engineering Journal 13, 1 (2022), 101513. https://doi.org/10.1016/j.asej.2021.05.027Google ScholarCross Ref
- Iqbal H Sarker. 2021. Data science and analytics: an overview from data-driven smart computing, decision-making and applications perspective. SN Computer Science 2, 5 (2021), 1–22. https://doi.org/10.1007/s42979-021-00765-8Google ScholarDigital Library
- Monika Schak, Rainer Blum, and Birgit Bomsdorf. 2022. Smart Home for the Elderly-A Survey of Desires, Needs, and Problems. In International Conference on Human-Computer Interaction. Springer, 107–121. https://doi.org/10.1007/978-3-031-05654-3_7Google ScholarDigital Library
- Eugene Seo, Sihwa Bae, Hyunchul Choi, and Donghyeog Choi. 2021. Preference and usability of smart-home services and items-a focus on the smart-home living-lab-. Journal of Asian Architecture and Building Engineering 20, 6 (2021), 650–662. https://doi.org/10.1080/13467581.2020.1812397Google ScholarCross Ref
- Faisal Karim Shaikh and Sherali Zeadally. 2016. Energy harvesting in wireless sensor networks: A comprehensive review. Renewable and Sustainable Energy Reviews 55 (2016), 1041–1054. https://doi.org/10.1016/j.rser.2015.11.010Google ScholarCross Ref
- Maximilian Speicher, Brian D. Hall, and Michael Nebeling. 2019. What is Mixed Reality?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3290605.3300767Google ScholarDigital Library
- X-J Sun and Z-L Deng. 2009. Information fusion Wiener filter for the multisensor multichannel ARMA signals with time-delayed measurements. IET Signal Processing 3, 5 (2009), 403–415. https://doi.org/10.1049/iet-spr.2008.0096Google ScholarCross Ref
- Haseeb Touqeer, Shakir Zaman, Rashid Amin, Mudassar Hussain, Fadi Al-Turjman, and Muhammad Bilal. 2021. Smart home security: challenges, issues and solutions at different IoT layers. The Journal of Supercomputing 77, 12 (2021), 14053–14089. https://doi.org/10.1007/s11227-021-03825-1Google ScholarDigital Library
- Ahmed Mohmmad Ullah, Md. Rashedul Islam, Sayeda Farzana Aktar, and S K Alamgir Hossain. 2012. Remote-touch: Augmented reality based marker tracking for smart home control. In 2012 15th International Conference on Computer and Information Technology (ICCIT). IEEE, 473–477. https://doi.org/10.1109/ICCITechn.2012.6509774Google ScholarCross Ref
- Franklin E White. 1991. Data fusion lexicon. Technical Report. Joint Directors of Labs Washington DC. Retrieved May 31, 2023 from https://apps.dtic.mil/sti/citations/ADA529661Google Scholar
- Song Zheng, Qi Zhang, Rong Zheng, Bi-Qin Huang, Yi-Lin Song, and Xin-Chu Chen. 2017. Combining a multi-agent system and communication middleware for smart home control: A universal control platform architecture. Sensors 17, 9 (2017), 2135. https://doi.org/10.3390/s17092135Google ScholarCross Ref
Index Terms
- Multi-channel Sensor Network Construction, Data Fusion and Challenges for Smart Home
Recommendations
Enablement of IoT Based Context-Aware Smart Home with Fog Computing
The recent advent of Internet of Things IoT, has given rise to a plethora of smart verticals-smart homes being one of them. Smart Home is a classic example of IoT, wherein smart appliances connected via home gateways constitute a local home network to ...
Smart Home
The smart home service is a key part of the smart grid consumption. It is a real-time interactive response between the power grid and users, and enhances the comprehensive service capability of the power grid, also realizes the intelligent and ...
Smart Home Technologies: A Preliminary Review
ICIT '18: Proceedings of the 6th International Conference on Information Technology: IoT and Smart CityIn recent years, smart homes have become increasingly popular with the deployment of Internet of Things (IoT). Rapid diffusion of sensing technology also enabled advancement in smart homes. The advancements in these technologies surroundings smart homes ...
Comments