Smart control of window and air cleaner for mitigating indoor PM2.5 with reduced energy consumption based on deep reinforcement learning
Introduction
Numerous epidemiologic studies have shown that exposure to PM2.5 (particulate matter with aerodynamic diameter less than 2.5 μm) is strongly associated with adverse health effects such as respiratory infections [1], lung cancer [2], chronic obstructive pulmonary disease (COPD) [3] and cardiovascular disease [4], and thus leads to large numbers of premature deaths [3]. Since people spend most of their daily lives in indoor environments [5], there is a great need to eliminate indoor exposure to PM2.5 and the related diseases and deaths.
In China, people often ventilate their apartments naturally by opening the windows [6]. However, with the frequent severe ambient PM2.5 pollution, the use of air cleaners has become more and more popular for reducing indoor PM2.5 pollution. During the past decade, the fraction of residences with home air cleaners has increased by ten times in China [6,7]. Therefore, it is worthwhile to investigate the control of windows and air cleaners for effective reduction of PM2.5 pollution in apartments that are naturally ventilated.
In naturally ventilated apartments, opening the windows can reduce the concentration of indoor PM2.5 generated by indoor activities such as cooking [[8], [9], [10]], smoking [11,12] and printing [13]. On the other hand, closing the windows reduces the ventilation rate and thus the entry of outdoor PM2.5 into the indoor environment [14]. When air cleaners equipped with high-efficiency particulate air (HEPA) filters are used, the indoor PM2.5 concentration can be significantly reduced as long as the clean air delivery rate (CADR) is sufficiently large [15]. However, the use of air cleaners results in higher energy consumption and the need for regular replacement of filtration media [6,15]. Thus, it is crucial to develop an approach that controls a window and an air cleaner simultaneously, in a bid to effectively mitigate indoor PM2.5 pollution while reducing the energy consumed by the air cleaner.
Two commonly used control approaches, closed-loop and model predictive control, can be considered for the goal of reducing indoor PM2.5 pollution. For closed-loop control, a typical method is to define a setpoint for indoor PM2.5 concentration in an air cleaner with on/off control. However, this method does not consider the influence of window status or the contributions of indoor emission and outdoor infiltration. Thus, the energy consumption of the air cleaner is not necessarily minimized. For model predictive control, prior knowledge of the specific environment is needed; i.e., all the parameters of the environment must be accurately measured in advance for prediction and optimization. However, it is challenging to measure key parameters such as air exchange rate, CADR, and indoor source emission rates in real time. Therefore, these conventional control methods cannot be utilized in practical applications if both health and energy issues are to be considered. Note that control of the window and the air cleaner is a sequential decision-making process. This kind of control problem can be effectively solved by reinforcement learning [16,17]. Hence, the reinforcement learning approach could potentially be employed to simultaneously control a window and air cleaner in order to reduce indoor PM2.5 pollution with less energy consumption.
Reinforcement learning has been applied in the control of smart buildings in many investigations [[18], [19], [20], [21], [22], [23], [24], [25], [26]]. For instance, Chen et al. [18] proposed a Q-learning control strategy for windows and an HVAC system to save energy and reduce thermal discomfort in virtual environments located in Miami and Los Angeles. Nagy et al. [20] developed a deep reinforcement learning algorithm for controlling space heating that could reduce the cost by 5–10% when compared with a rule-based method. Heo et al. [22] trained a controller for the mechanical ventilation system of a subway station using the deep Q-network algorithm. Testing in a simulator showed that the control strategy maintained the indoor PM10 at an acceptable level and reduced energy use by 14.37% when compared with a rule-based method. These studies have provided great insight into the application of reinforcement learning in building systems. However, most of these investigations tested reinforcement learning algorithms for smart building control in computer simulations rather than experimentally. Furthermore, for naturally ventilated apartments with air cleaners, there are few existing studies that have utilized reinforcement learning for control of a window and air cleaner.
Therefore, this investigation attempted to develop an approach that trains a controller using the deep Q-network (DQN) algorithm, a reinforcement learning method, to control the operation of a window and an air cleaner in order to reduce indoor PM2.5 concentration with lower energy consumption. The proposed approach first trained the smart controller using the DQN in a simulated virtual environment. The virtual environment was constructed with the use of a particle dynamics model with typical building parameters. To test the trained DQN controller experimentally, this study constructed two small laboratory chambers, each with a window, an air cleaner, and a control system. The controller was then integrated into the control system to smartly control the window actuator and the air cleaner. Both the indoor PM2.5 concentrations and the operation times of the air cleaner were compared between the trained DQN controller and different benchmark controllers with various outdoor PM2.5 levels under different chamber conditions in order to assess the controller performance. Note that the inputs for the trained controller were the indoor and outdoor PM2.5 concentrations. These real-time inputs can be easily monitored by sensors available on the market. Therefore, the proposed control approach can easily be applied in practical scenarios.
Section snippets
Control objectives and inputs
This investigation centered on naturally ventilated apartments equipped with air cleaners. The aim was to mitigate the health risks attributable to indoor PM2.5 exposure while reducing the energy consumption of the air cleaner. The actuators for indoor PM2.5 control were the window that provides natural ventilation and the air cleaner that provides indoor PM2.5 filtration. The objective of the window and air cleaner controller was to achieve a balance between PM2.5-related health risks and the
Experimental setup
In this investigation, a series of experiments were conducted in two laboratory chambers to test the performance of the trained DQN controller. Fig. 5 shows the schematic of the experimental setup, which mainly consisted of two identical small testing chambers that simulated the indoor environments. The size of each chamber was 0.4 m0.5 m0.4 m. The two identical chambers were used to compare the performance of the trained DQN control algorithm and benchmark algorithms. The chambers were
Sensitivity analysis of building parameters utilized in DQN training
The sensitivity analysis in this section aimed to test if the trained DQN controller could still outperform the benchmark controllers when the building parameters ( and ) of the virtual environment utilized for training were changed. The and of the virtual environment that trained the DQN controller were adjusted in a ±10% range based on the typical parameters utilized for training in Section 2.2.2 (denoted as “Typical” in Table 6). Two new virtual environments for training,
Conclusions
This study developed a smart controller that can automatically control a window and an air cleaner in a naturally ventilated apartment to reduce both the health risks attributed to indoor PM2.5 and the energy consumption of the air cleaner. The controller was developed with the use of the DQN algorithm, a deep reinforcement learning method. Offline training of the controller was conducted in a virtual apartment constructed on the basis of a particle dynamics model with typical building
CRediT authorship contribution statement
Yuting An: Writing – original draft, Software, Project administration, Methodology, Investigation, Data curation, Conceptualization. Zhuolun Niu: Resources. Chun Chen: Writing – review & editing, Supervision, Funding acquisition, Conceptualization.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgement
This work was partially supported by the General Research Fund of Research Grants Council of Hong Kong SAR, China (Grant No. 14204520).
References (46)
- et al.
The association between lung cancer incidence and ambient air pollution in China: a spatiotemporal analysis
Environ. Res.
(2016) - et al.
Operating behavior and corresponding performance of portable air cleaners in residential buildings, China, Build
Environ. Times
(2019) - et al.
Reducing human exposure to PM2.5 generated while cooking typical Chinese cuisine
Build. Environ.
(2020) - et al.
Indoor environmental quality in social housing: a literature review
Build. Environ.
(2018) - et al.
Emission rates of ultrafine and fine particles generated from human smoking of Chinese cigarettes
Atmos. Environ.
(2018) - et al.
Evolution of pressure drop across electrospun nanofiber filters clogged by solid particles and its influence on indoor particulate air pollution control
J. Hazard Mater.
(2021) - et al.
Optimal control of HVAC and window systems for natural ventilation through reinforcement learning
Energy Build.
(2018) - et al.
Whole building energy model for HVAC optimal control: a practical framework based on deep reinforcement learning
Energy Build.
(2019) - et al.
A reinforcement learning approach for control of window behavior to reduce indoor PM2.5 concentrations in naturally ventilated buildings
Build. Environ.
(2021) - et al.
Reinforcement learning for energy conservation and comfort in buildings
Build. Environ.
(2007)
Fusing TensorFlow with building energy simulation for intelligent energy management in smart cities
Sustain. Cities Soc.
Impacts of implementing Healthy Building guidelines for daily PM2.5 limit on premature deaths and economic losses in urban China: a population-based modeling study
Environ. Int.
Indoor air quality and occupants' ventilation habits in China: seasonal measurement and long-term monitoring
Build. Environ.
Deposition, resuspension, and penetration of particles within a residence
Atmos. Environ.
Review of relationship between indoor and outdoor particles: I/O ratio, infiltration factor and penetration factor
Atmos. Environ.
A methodology for predicting particle penetration factor through cracks of windows and doors for actual engineering application
Build. Environ.
Modifications of exposure to ambient particulate matter: tackling bias in using ambient concentration as surrogate with particle infiltration factor and ambient exposure factor
Environ. Pollut.
Exploring the feasibility of predicting contaminant transport using a stand-alone Markov chain solver based on measured airflow in enclosed environments
Build. Environ.
A Markov chain model for predicting transient particle transport in enclosed environments
Build. Environ.
Exposure and health impact evaluation based on simultaneous measurement of indoor and ambient PM2.5 in Haidian, Beijing
Environ. Pollut.
A novel model for regional indoor PM2.5 quantification with both external and internal contributions included
Environ. Int.
Window opening behaviour modelled from measurements in Danish dwellings, Build
Environ. Times
Occupants' window opening behaviour: a literature review of factors influencing occupant behaviour and models
Build. Environ.
Cited by (8)
Achieving better indoor air quality with IoT systems for future buildings: Opportunities and challenges
2023, Science of the Total Environment