Related work

ABMs have been used throughout the COVID-19 pandemic to forecast a variety of outcomes, including demand for non-ICU beds and ICU beds, with varying levels of geographic specificity. Examples include Donker et al. [9], who used data from Italy to provide early insights for Germany; Weissman et al. [5], who developed the CHIME ABM tool for the Philadelphia region; Tuomisto et al. [10], who developed an ABM called REINA to identify destructive pandemic policies for broad geographic areas; Fort et al. [11], who focused specifically on an ABM for hospitals in the New Orleans region; and Truszkowska [12], who applied an ABM to the outbreak in New Rochelle, New York (NY). In these ABMs, agents become infected with SARS-CoV-2 and can be hospitalized for COVID-19. Model outputs, including estimates of the non-ICU and ICU beds needed by the agents for the duration of the ABM, have been used to inform decisions such as increasing hospital bed capacity and purchasing additional personal protective equipment [5].

A key parameter for estimating these outputs is the probability of hospitalization among COVID-19 agents. This probability has a direct impact on the outputs because an increased probability will lead to more hospitalizations. However, Gao and Dong [13] describe how this probability is challenging to derive in practice. If the number of all positive SARS-CoV-2 infections were known, one could calculate the probability of hospitalization as the number of COVID-19-associated hospitalizations among all SARS-CoV-2 infections. However, the true number of SARS-CoV-2 infections is unknown; in July 2020, CDC estimated that there were 10 times as many SARS-CoV-2 infections as laboratory-confirmed and reported COVID-19 cases [7] An approximate hospitalization rate can be calculated as the number of persons hospitalized with COVID-19 compared to the total number of reported COVID-19 cases, but this number is likely a substantial overestimate of likelihood of hospitalization if used to estimate the probability of hospitalization for all persons with COVID-19. Although Gao and Dong [13] discuss how testing may impact the probability of hospitalization, the ABMs reviewed for this analysis did not specifically use the probability of COVID-19 testing in determining the probability of hospitalization associated with COVID-19.

The literature also strongly suggests that not all persons infected with SARS-CoV-2 have the same probability of hospitalization with COVID-19; specifically, older individuals and individuals with comorbidities (e.g., chronic obstructive pulmonary disease, obesity) appear to have higher likelihoods of requiring hospitalization [6]. Attributes like age and comorbidities can be assigned to agents in ABMs [14]. The parameter for probability of hospitalization can be conditionally assigned for each COVID-19 agent according to the agent’s age and comorbidity status. Kashyap et al. [15] suggested that models that do not use patient age distribution are more likely to overestimate the resource burden of COVID-19.

When specifying the parameter for the probability of hospitalization, several ABMs used the same probability for all agents with COVID-19, regardless of age or comorbidities. Based on data collected at Ochsner Health hospitals in New Orleans in March and early April 2020, Fort et al. [11] used a 9% hospitalization rate. Weismann et al. [5] used internal March 2020 data from Pennsylvania to estimate a hospitalization rate of 2.5% (CI 1%– 5.1%). Donker et al. [9] calculated the hospital admission rate as the cumulative number of hospital admissions divided by the cumulative number of confirmed cases, although no numeric estimate was provided. Hoertel et al. [16] forecasted ICU occupancy and estimated it as four times the mortality rate observed in the ICU.

In March 2020, Verity et al. [17] released age-specific probabilities of hospitalization given SARS-CoV-2 infection. In the same month, Ferguson et al. [18] updated these probabilities to account for non-uniform attack rates by age. When applied to the UK population, these estimates suggested an overall hospitalization rate of 4.4%. Uncertainty estimates were not provided. Many subsequent ABMs throughout 2020 and into 2021 used the direct estimates or modifications of the estimates by Verity et al. [17] or Ferguson et al. [18] as shown in Table 1, including Tuomisto et al. [10], Silva et al. [19], Kerr et al. [20], and Truszkowska et al. [12]. Tuomisto et al. [10], Silva et al [19]. and Kerr et al. [20] models were location-agnostic, while Truszkowska et al. [12] explicitly modeled the small town of New Rochelle, NY. Many models recognized that importance of incorporating age-specific parameters, but also assumed applicability across geographies which may not reflect regional realities.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Age-specific probabilities of hospitalization given COVID-19 infection.

https://doi.org/10.1371/journal.pone.0264704.t001

Fewer models explicitly incorporated comorbidities into the probability of hospitalization. Gao and Dong [13] explored which comorbidities were most highly related to the probability of hospitalization and Qian et al. [21] estimated the profile of comorbidities among those hospitalized. Only Kerr et al. [20] explicitly used the comorbidities of each agent to modify the probability of developing severe COVID-19 symptoms that would require hospitalization.

The models that considered age- or comorbidity-specific probabilities of hospitalization generally relied on conditional probabilities alone. Gao and Dong [13,22] extended the use of conditional probabilities by proposing a Bayesian framework with which to estimate the probability of hospitalization.

Throughout the COVID-19 pandemic, stakeholders including government officials have utilized agent-based models for key policy decisions [23]. The re-use of age estimates and the lack of use of comorbidity or testing attributes estimates in many of these models reflect the lack of available information in the early months of the COVID-19 pandemic as well as a need by stakeholders for rapid model development. However, these omissions conflict with a stakeholder demand for interpretability to facilitate transparent decisions and the reuse of parameters in different geographies may affect model reliability. A lack of interpretability and reliability of COVID-19 ABMs have contributed to critiques of their use for policymaking [24].

ABM design

We developed an ABM in Python to generate 30-day forecasts for the demand for ICU and non-ICU beds for all agents (both with and without COVID-19) in North Carolina [25]. Estimates were provided at the state level and for the 10 Local Health Director (LHD) regions in North Carolina from March 2020 through December 2020. The ABM modeled agent movement between the community, short term care facilities, long-term acute-care hospitals (LTACHs), and nursing homes. The ABM utilized all agents from NC in the RTI SynthPop [26]. The RTI SynthPop is a virtual, anonymized synthetic population that is representative of real people and can be used for simulation efforts like this one. Each person in the synthetic population is called an agent, and agent attributes included age and a binary flag for presence of comorbidities that was previously developed for an ABM of Clostridioides difficile Infection [14]. Note that one limitation of the comorbidity assignments is that they were only estimated for three age ranges (0–49, 50–64, and 65+ years), with no agents under 50 being assigned comorbidities [14]. The ABM utilized a susceptible-exposed-infected-recovered (SEIR) model for SARS-CoV-2 with different probabilities of infection by age, and agents could be hospitalized with COVID-19 during the “Infected” stage.

When determining the probability of hospitalization with COVID-19, we initially used a single point estimate for all infected agents similar to ABMs developed by Fort et al. [11], Weismann et al. [5], Donker et al. [9], and Hoertel et al. [16]. This value was set to 2.2%. This was half of the value estimated from Ferguson et al. [18] and was equivalent to the CDC estimate of 4.6 hospitalizations per 100,000 persons [27]. However, literature contradicted the assumption that all infected agents should have the same probability of hospitalization with COVID-19.

We started with age and comorbidity status, which were available in the underlying synthetic population, as well as testing status which was estimated in the ABM. A testing status of “tested” meant that a person had received any type of test for COVID-19, while a testing status of “untested” meant that a person had not received any type of test for COVID-19. To incorporate these attributes in the probability of hospitalization with COVID-19, we proposed a Bayesian framework. While Gao and Dong focus on approximating a Bayesian posterior with beta distributions, we used a straightforward application of the Bayes’ theorem. We aimed to use local NC hospital data to capture the conditional relationships between age, comorbidities, and hospitalization from symptoms of COVID-19 while also accounting for unreported SARS-CoV-2 infections. With this method, we were able to use NC-specific data rather than relying on estimates derived from different populations that may not reflect NC.

Bayesian specification

Our proposed method was intentionally designed to require few inputs that were reasonably accessible given the pandemic and to be interpretable for stakeholders. A similar method had been used by Gao and Dong [13,22]. To estimate the conditional probability of hospitalization associated with COVID-19 p(hosp) given an agent’s age and comorbidity status (i.e., presence or absence of comorbidities), we use the following formula for Bayes’ theorem: The same equation can be used separately to obtain estimates for tested infections (i.e., laboratory confirmed and reported COVID-19 cases) and untested infections. It is important to note that “untested” is not an official status in the ABM but is understood to mean “untested prior to hospitalization” because it is likely that patients presenting with possible COVID-19 at a hospital would be tested upon admission. This does not impact the model because the probability of hospitalization is what the ABM uses to identify agents for hospital admission, so subsequent testing at the hospital would not impact this parameter. At the time that the model was initially constructed in June 2020, it was estimated that 10% of SARS-CoV-2 infections were reported as COVID-19 cases [7].

Table 2 highlights differences in the methods used to calculate each component of the Bayesian equation between tested and untested infections. One key difference is that the probability of hospitalization for agents for COVID-19 agents is expected to be higher among tested infections than untested infections. This is based on a comparison with the CDC overall estimate of probability of hospitalization, and the suggested estimate that untested infections are larger in number than tested infections, particularly in the early months of the pandemic in 2020 when testing was limited.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Methods for calculating portions of Bayesian equation by COVID-19 testing status.

https://doi.org/10.1371/journal.pone.0264704.t002

Another key difference is the joint probability of age and comorbidity status. These estimates were calculated from those agents estimated to have COVID-19 in the ABM. The ABM included a separate calculation of probability of SARS-CoV-2 infection through a SEIR model that accounted for differences in likelihood of infection by age.

Data considerations

One important advantage of our method is that it requires few data inputs for ease of calculation and interpretation. We obtained the following aggregated data quickly from the North Carolina Department of Health and Human Services (NC DHHS) which was crucial in a setting where access to data was limited, and record-level patient data would have been extremely difficult to access:

• Count of hospitalizations disaggregated by age group and COVID-19 status
• Count of hospitalizations disaggregated by comorbidities and COVID-19 status
• Count of COVID-19 tests by result and by age group

The results from the Bayesian calculation were used directly in the ABM. Each agent with COVID-19 on a given day received a probability of hospitalization given their age and comorbidity status and was assigned a hospitalization status based on this probability. Disease progression including admission to the ICU was handled in the ABM itself.

Non-public, deidentified data were obtained and fully anonymized prior to access. The RTI IRB reviewed this study and determined that informed consent was not required as it does not constitute research with human subjects.

Limitations

A variety of limitations exist for this work. First, as discussed in Section 4, the specific Bayesian probabilities in this analysis should only be interpreted in the context of the NC synthetic population used in this project. Comorbidity status is only assigned to adults aged 50 and older in the synthetic population, and the comorbidity status assignment was based on health care-associated risk, which is slightly different than the one published by CDC for COVID-19 hospitalization [6,14]. A second limitation is that in this initial analysis, separate probabilities of age given hospitalization with COVID-19 and comorbidity status given hospitalization with COVID-19 were provided by NC DHHS. The calculation of the joint probability of age and comorbidity status given hospitalization with COVID-19 assumed that the underlying probabilities were independent, an assumption that may not be true. In later versions of the ABM, we subsequently made updates to this calculation and used data that included the joint distribution of age and comorbidity status given hospitalization with COVID-19. A third limitation is that the calculations did not account for differences in specificity and sensitivity for different types of COVID-19 tests and may overestimate the quality of testing results. Finally, the assumption was made that although age impacted the likelihood of infection with COVID-19, neither age nor comorbidity status influenced the likelihood of receiving a COVID-19 test once infected with SARS-CoV-2.