Building energy model calibration: A detailed case study using sub-hourly measured data
Introduction
With more than 40% of the energy use and 36% of the CO2 emissions [1], the European building sector is one of the main contributors to global warming. It also represents a tremendous potential for improvement through the retrofit of the existing building stock and the construction of new efficient buildings. Building energy simulation (BES) became an unmissable and powerful tool to overcome this challenge and now plays a significant role in the building industry. Many BES tools exists, each of them having different capabilities and limits [2]. As law-driven models (also known as physical models or white-box models), BES models rely on the thermodynamics rules that govern a system to predict its behavior given its properties and external conditions. Most of them are based on the nodal approach [3], which consists in considering each zone of the building as an homogeneous volume characterized by discrete values of state variables such as temperature, pressure, or humidity. Each zone represents one node of the mesh, in which the thermal transfer equations are solved over each time-step. This method can be considered as a one-dimensional approach. Those models require large computation time and a thorough knowledge of the building to be modeled. They are able to model a system given a set of previously unobserved initial conditions. However, detailed data may not be available during the design stage of a building, which forces the user to make different assumptions. This could result in an inaccurate simulation leading to a well-known problem called the “performance gap” (i.e. the difference between the simulated and measured values). Several studies pointed out significant discrepancies between the BES program outputs and the actual measured value [4], [5], discrediting those tools and their users [6] and questioning their usefulness in the building sector.
The performance gap results from the existence of a variety of uncertainties in the process of building modeling. Their study is the subject of specific researches, and two main fields can be distinguished, uncertainty and sensitivity analysis. While uncertainty analysis aims at calculating the likely variation in the outputs due to the variations in the inputs, the sensitivity analysis, as stated by Saltelli et al [7], aims at studying “how the uncertainty in the output of a model can be apportioned to different sources of uncertainty in the model input”. In other words, the aim of sensitivity analysis is to identify the input parameters to which the output is sensitive. As a result, the model output may be sensitive to an input but, if this parameter is well known, it will not be a critical parameter in an uncertainty analysis [8]. A complete inventory of sensitivity analysis methods applied to the study of BES models is made in [9], [10], while a review of uncertainty analysis results and techniques for building energy assessment can be found in [11].
According to Imam et al.[6], the performance gap find its origin in the three phases of a building's life: design, construction and operation. Many studies already identified uncertainty sources occurring at different stages [12], [13], [14], classified as follow:
- •
Specification uncertainties: Incomplete, improper or inaccurate parameters specification such as building geometry or HVAC (Heating, Ventilation, and Air Conditioning) systems properties. This can be due to a user's lack of experience or negligence, or to an incomplete knowledge of inputs parameters.
- •
Modelling uncertainties: Simplifications and assumptions made in the models used by the BES program to represent the physical phenomenon (nodal model, space and time discretization, etc.). This does not directly rely on the user, but he can choose to reduce this uncertainty by for example reducing the time-step or choosing the most appropriate model to represent a given phenomenon.
Discrepancies can also occur during the construction process, identified in [6] as the “construction gap”. Even if a very particular attention is given during this phase to produce a high quality building, it will not be as perfect as it can be in a BES program, which will lead in unavoidable differences. Finally, during the operational stage occurs the last source of discrepancies with the predictions, identified as the scenario uncertainties source.
- •
Scenario uncertainties: Errors resulting from the assumptions made on external parameters such as weather data or occupants’ behavior. The stochastic nature of those parameters inevitably lead to a form of uncertainty. However, it can be drastically reduced during the design phase, yet it entirely relies on the user's knowledge. Even if representative weather files are now easily accessible, poor occupants’ scenario assumptions (occupancy, internal loads, etc.) can still be made and will inevitably impact the simulation performance.
Many researchers have worked to identify the most influential input parameters along with their associated uncertainty to quantify their effect on the model outputs. As an example Menberg et al. [15] compared several methods for sensitivity analysis of BES models. They studied the influence of eleven parameters that are typically selected for sensitivity analysis, and identified set point temperature, infiltration, building thermal properties, and ventilation related parameters as influential for building energy models. They stated that these findings are in good agreement with the conclusions of previous studies. However, they did not consider weather data in their analysis, even though this was identified as one of the sources of uncertainty in BES models [11].
Specification uncertainties have been widely studied in the literature. Based on the work of Clarke et al. [16], Macdonald [8] established detailed uncertainty data, especially for conductivity, density, specific heat capacity, or emissivity and absorptivity of construction materials. Other key parameters of building envelope such as infiltration rate [17] or thermal bridges [18] have also been studied. As previously mentioned, specification uncertainty can also be related to HVAC systems properties. Wang et al. [19] studied the influence of different building operation strategies related to HVAC systems on the energy consumption of an office building. They showed that the combined uncertainties of those parameters could result in an uncertainty on the annual energy consumption ranging from −15.8% to 70.3%.
Although less frequently studied, it is possible to find studies dealing with modeling uncertainties. As an example, the effect of simulation time-step has been studied in [20]. The author simulated the same model at various time-steps (1, 5, 10, 15, 20, 30, and 60 min) and found differences up to 3.6% in daily cooling load when simulating at a 60 min time-step, compared to 1 min time-step simulation. When considering daily peak load (which is an often used sizing factor for building design), discrepancies goes up to 75%. Dos Santos and Mendes [21] found other results with zone temperature differences up to 4 °C and 10% variations in humidity ratio between 1 s time-step and 3600 s time-step simulations. They conclude that a 60 min time-step can lead to strong discrepancies in indoor air temperature and consequently in building load calculation.
Scenario uncertainty, such as weather data, have also been studied in the literature. Thus, Wang et al. [19] investigated the influence of weather data among other parameters on the energy use in an office building. They considered four different cities for which they gathered 10 to 15 years of weather data. They found that the energy could vary from − 4.0% to 6.1% depending on the city and year compared to the typical meteorological year that is usually considered.
Together, these studies provide insight into the various sources of uncertainty and their effects on the model outputs. However, these results are highly dependent on the building under consideration. Therefore, they remain trends that can be expected, but are by no means directly transferable to any given building.
The uncertainty on the metered data used for the model performance assessment due to the sensors' accuracy can also explain a part of the performance gap. Although this may be negligible regarding the discrepancies usually observed, it should be considered or at least mentioned.
The complexity of BES programs and the complete freedom the user benefit in their utilization make the modeler the main reason of a potential “performance gap”. This statement can be illustrated in the work of Imam et al. [6] where the authors asked 108 modellers to comment the importance of 21 common modelling input variables such as U-values, ventilation rate or glazing ratio. They found very heterogeneous results with no correlation between modellers’ answers since many of them identified parameters to be important when others thought they were irrelevant. In addition, the authors found little correlation between the variables which were thought to be important by the modellers and which proved to be objectively important. Those results corroborate Guyon findings [22] where 12 users where asked to model the same residential house using the same software. The author found energy consumption predictions ranging from −41% to +39% around the average value. As mentioned by Imam et al. [6], modellers rarely compare energy consumption results with measured values of the building during its operational stage. Consequently, there is no feedback about good or wrong practices, causing the knowledge to drift over time. To overcome this issue, calibration process should be conducted regularly by modellers. However, this implies to have highly monitored buildings available and to give a real value to a calibrated model since the process is very time consuming.
The calibration process aims at closing, or at least reducing the performance gap. It consists in tuning the different unknown input parameters within a defined range in order to match the simulated and measured values. Reddy [12] proposed four classes to classify the different calibration approaches:
- •
Manual, iterative and pragmatic intervention
- •
A suite of informative graphical comparative displays
- •
Special tests and analytical procedures
- •
Analytical/Mathematical methods of calibration
The details of each category as well as examples of applications can be found in his review. Those different approaches for BES models calibration have been broadly classified in two main categories [13]:
- •
Manual: This gathers every method that does not rely on automated procedures, these are the most commonly used. They are conducted by the user, in an iterative and “trial and error” manner, and entirely rely on the user's experience, knowledge and judgment. Those methods can be considered as subjective. The majority of these methods involve the use of graphical representations [23] and comparative displays including time-series plots, 3D comparative plots [24], [25], signature analysis methods [26], or statistical displays [24].
- •
Automated: This include all automated approaches based on mathematical or statistical tools that cannot be considered as user driven. Most of these techniques require to identify a set of influential parameters to be calibrated as well as their maximum range of variation using sensitivity analysis. The objective is then to find, among the multitude of possible configurations, the set of input parameters that minimizes a cost function, usually representing the error between measurement and simulation [27], [28]. Another approach regularly used is the Bayesian calibration, which has the advantage of naturally taking into account the uncertainties in the calibration process [29].
Detailed explanations as well as examples of the application of each of the above methods can be found in [13], [30]. Regardless of the calibration technique used, the process remains time-consuming and costly. Indeed, manual techniques require the time of an experienced human to analyze the results of the multiple simulations that will be conducted. Similarly, automatic methods also require the time of a human expert to be developed, as well as the computing power and time needed to perform a large number of simulations. According to Fabrizio et al. [30], there are five different calibration complexity levels, depending on the data available for comparison, from simple utility bills comparison to long-term monitoring:
- •
Level 1: The first level of calibration is based on energy bills only. This type of calibration will therefore only be based on monthly or annual energy consumption data. As the dynamic aspect is not taken into account, the calibration can be qualified as static.
- •
Level 2: The data available for a level 1 calibration may be supplemented by visual inspections on site to verify some of the design data (equipment installed, internal geometry of the building, zones occupancy, etc.). The building manager may also be questioned. Calibration will therefore always be based on monthly energy consumption data, and the dynamic aspect will not be considered.
- •
Level 3: In addition to the information available for the previous levels, information can be obtained from detailed on-site audits through the use of measuring instruments (oscilloscope, luxmeter, balometer, temperature probe, etc.). These measurements may be carried out at different times of the day but remain punctual, as no instrument is left on site. The dynamic aspect may be considered, essentially for systems, over short periods of time only, around the hour.
- •
Level 4: At this level, measuring instruments are installed on site to measure different variables (temperatures, thermal or electrical energy, humidity, etc.) over a relatively short period of time (typically several weeks to a few months), at short time intervals (from minutes to hours). Here, calibration can be done on other variables than the simple monthly energy consumption, such as zone temperatures or the thermal or electrical powers. Because the measurement period has become sufficiently long, the building dynamics can be considered over specific periods of time only.
- •
Level 5: This last level of calibration involves an almost exhaustive instrumentation, which is permanently installed on the building. The instrumentation was therefore designed beforehand, during the design phase of the building. There are no limits to the duration of the monitoring, meaning that several years of data are usually available. The recording interval is small enough to capture the dynamics of rapid transient phenomena (of the magnitude of a minute). The data is usually accessible remotely and the building is controlled through a BEMS (Building Energy Management System) which also allows its monitoring along with the modification of its operation in real time. Calibration can be carried out in a totally dynamic way, in order to reproduce the dynamic behavior of the building over the entire period under consideration by working on variables such as thermal and electrical power or the temperature and humidity of the zones.
Evaluating a model's accuracy asks the question of the indicators that have to be used. For the sake of uniformity and standardization, statistical indices summarized in Table 1 have been selected by three international organizations [31], [32], [33] to assess whether a model can be considered as being calibrated or not [13], [30]. The maximum value of each criterion varies depending on whether the model is calibrated on an hourly or monthly basis. Note that the hourly NMBE criterion of the IPMVP guideline is more restrictive than the monthly criterion, which goes against the trend observed for the other two guidelines. According to every guideline, the performance calculation is mostly made on energy consumption data by comparing the simulated and the measured values.
A confusion between the Mean Bias Error (MBE) and the Normalized Mean Bias Error (NMBE) has been identified in the guidelines. The origins of this error, the potential consequences on the results, and the articles concerned are reported in [34]. According to the authors, the most common mistake is the use of the abbreviation “MBE” when referring to the “NMBE”.
- •
NMBE: It corresponds to the normalization of the mean bias error (MBE) using the mean of the measured values to make the results comparable regardless of the unit used. It indicates the overall bias of the model. However, it suffers from cancellation effect, when negative and positive error values offset each other. For this reason, NMBE cannot be considered alone, another indicator has to be introduced.
Where:
is the hourly or monthly measured value of energy consumption
is the hourly or monthly simulated value of energy consumption
is the number of data points (; )
is the number of adjustable model parameters
The number of adjustable model parameters is suggested to be set to zero for calibration procedures [34], simplifying NMBE formula to:
- •
CvRMSE (Coefficient of variation of the Root Mean Squared Error): This corresponds to the RMSE (Root Mean Squared Error) divided by the averaged value of the measured data. Because of squaring, it tends to maximize the impact of important errors in comparison to smaller errors. It is always positive and allows to determine how well a model fits the reality. The lower the CvRMSE, the more accurate the model.
In spite of the equifinality of BES models (i.e. multiple model configurations may produce the same results [35]), calibration performance assessment is rarely made on other variables like zones temperatures or humidity. This means for example that a model can be considered as being calibrated with no regards to the temperature evolution in the different zones, even if BES is also meant to evaluate the building's thermal comfort. Moreover, to the authors’ knowledge, there is no guideline regarding calibrations conducted on a sub-hourly time-step.
Section snippets
Building description
The case study is the Hikari project located in Lyon, France (45°44′31.1″N 4°49′09.5″E, altitude 240 m), represented in Fig. 1. It has been designed to be the first net zero energy city block in Europe and is operating since July 2015. The project of 12 310 m2 is composed of three distinct mixed-use buildings (from left to right in Fig. 1: Nishi, Minami and Higashi) built on a common basement gathering a parking lot and the mutual technical facilities providing the energy needs. Hikari is fully
Calibration methodology
Due to the large amount of monitoring data available and the unrestricted access to the design data (plans, diagrams, data sheets, calculation notes, etc.), we chose to conduct a manual calibration [13] of level 5 [30]. The calibration process was conducted in three main steps:
- •
Update of the geometry of the building: the model created during the design phase has been revised by drawing more precisely some facade elements that could results in solar masks. Other close masks such as non-existing
Results analysis
Results will first be shown at the building scale considering annually and monthly energy needs for both heating and cooling. Based on those results, a first calibration performance assessment is made on statistical indicators recommended in the guidelines previously mentioned. Then, a second performance evaluation is done through a dynamic physical analysis of the heating and cooling powers. In a third part, the model performance is also assessed through a statistical and a dynamical analysis
Conclusion
This paper details the manual calibration of a 5 434 m2 office building model simulated over one year at a five minutes time-step. Thanks to a complete instrumentation, the majority of input parameters where known, making a level 5 [30] calibration possible. Thus, the approach simply consists in feeding the model with as much measured data as possible instead of day-typed schedules without tuning unknown parameters to match measured data. The authors followed a similar approach as the one
CRediT authorship contribution statement
Dimitri Guyot: Conceptualization, Methodology, Investigation, Formal analysis, Writing - original draft, Visualization. Florine Giraud: Conceptualization, Writing - review & editing. Florian Simon: Methodology, Software. David Corgier: Resources, Project administration, Supervision. Christophe Marvillet: Project administration, Supervision. Brice Tremeac: Conceptualization, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (38)
- et al.
A review on applications of ANN and SVM for building electrical energy consumption forecasting
Renew. Sustain. Energy Rev.
(2014) - et al.
Contrasting the capabilities of building energy performance simulation programs
Build. Environ.
(2008) - et al.
State of the art in building modelling and energy performances prediction: A review
Renew. Sustain. Energy Rev.
(2013) Do LEED-certified buildings save energy? Not really
Energy Build.
(2009)- et al.
The role of sensitivity analysis in the building performance analysis: A critical review
Energy Build.
(2020) - et al.
A review of uncertainty analysis in building energy assessment
Renew. Sustain. Energy Rev.
(2018) - et al.
A review of methods to match building energy simulation models to measured data
Renew. Sustain. Energy Rev.
(2014) - et al.
Analysis of uncertainty in building design evaluations and its implications
Energy Build.
(2002) - et al.
Sensitivity analysis methods for building energy models: Comparing computational costs and extractable information
Energy Build.
(2016) - et al.
Calibration of building energy models for retrofit analysis under uncertainty
Energy Build.
(2012)
Uncertainties in energy consumption introduced by building operations and weather for a medium-size office building
Energy Build.
A rapid calibration procedure and case study for simplified simulation models of commonly used HVAC systems
Build. Environ.
Building energy model calibration using automated optimization-based algorithm
Energy Build.
An automated optimization method for calibrating building energy simulation models with measured data: Orientation and a case study
Appl. Energy
Bayesian calibration of building energy models with large datasets
Energy Build.
A manifesto for the equifinality thesis
J. Hydrol.
Calibrating whole building energy models: An evidence-based methodology
Energy Build.
Calibrating whole building energy models: Detailed case study using hourly measured data
Energy Build.
Estimating the influence of occupant behavior on building heating and cooling energy in one simulation run
Appl. Energy
Cited by (28)
Building energy model calibration: A review of the state of the art in approaches, methods, and tools
2024, Journal of Building EngineeringTwo-stage multi-step energy model calibration of the cooling systems of a large-space commercial building
2023, Applied Thermal EngineeringInvestigation of control characteristics in multi-user building heating system based on demand-side adjustment
2023, Applied Thermal Engineering