Article Text
PDF
Abstract
Objective To examine the value of health systems data as indicators of emerging COVID-19 activity.
Design Observational study of health system indicators for the COVID Hotspotting Score (CHOTS) with prospective validation.
Setting and participants An integrated healthcare delivery system in Northern California including 21 hospitals and 4.5 million members.
Main outcome measures The CHOTS incorporated 10 variables including four major (cough/cold calls, emails, new positive COVID-19 tests, COVID-19 hospital census) and six minor (COVID-19 calls, respiratory infection and COVID-19 routine and urgent visits, and respiratory viral testing) indicators assessed with change point detection and slope metrics. We quantified cross-correlations lagged by 7–42 days between CHOTS and standardised COVID-19 hospital census using observational data from 1 April to 31 May 2020 and two waves of prospective data through 21 March 2021.
Results Through 30 September 2020, peak cross-correlation between CHOTS and COVID-19 hospital census occurred with a 28-day lag at 0.78; at 42 days, the correlation was 0.69. Lagged correlation between medical centre CHOTS and their COVID-19 census was highest at 42 days for one facility (0.63), at 35 days for nine facilities (0.52–0.73), at 28 days for eight facilities (0.28–0.74) and at 14 days for two facilities (0.73–0.78). The strongest correlation for individual indicators was 0.94 (COVID-19 census) and 0.90 (new positive COVID-19 tests) lagged 1–14 days and 0.83 for COVID-19 calls and urgent clinic visits lagged 14–28 days. Cross-correlation was similar (0.73) with a 35-day lag using prospective validation from 1 October 2020 to 21 March 2021.
Conclusions Passively collected health system indicators were strongly correlated with forthcoming COVID-19 hospital census up to 6 weeks before three successive COVID-19 waves. These tools could inform communities, health systems and public health officials to identify, prepare for and mitigate emerging COVID-19 activity.
- COVID-19
- health informatics
- public health
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The study included routinely available health systems data and previously described algorithms to confirm their value for COVID-19 hotspotting across a large and diverse population of 4.5 million patients cared for at 20 medical centres and >250 medical offices.
A composite score, implemented in June 2020, using these indicators showed strong correlation with COVID-19 census across two prospective COVID-19 waves with peak correlationwith a lead time of 4–6 weeks at regional and hospital levels.
Although model generalisability will likely vary across settings and geographies, a reduced form of the model, excluding regional call centre and email data, showed similar performance.
Continuously changing routine healthcare practices and patterns attributable to COVID-19 will require ongoing score re-examination and the potential use of other statistical approaches for refinement.
The COVID Hotspotting Score performance may change with large-scale vaccination programmes and the development of effective new treatments.
Introduction
COVID-19 is an unprecedented, dynamic and persistent threat to global health.1 Since initial reports implicated the SARS-CoV-2 virus in late 2019,2 COVID-19 disease has affected 126 million people worldwide resulting in nearly 3 million deaths.3 4 Efforts to contain SARS-CoV-2 spread have seen mixed success; numerous locales that withstood an initial wave of COVID-19 disease activity are now witnessing viral resurgence.5 These examples confirm epidemiological and simulation models that predicted ongoing waves of COVID-19 disease over the next year.6–11
Given the expectation of ongoing surges of COVID-19 disease, focus has turned towards data-driven approaches to identify the earliest signals of re-emergent viral activity.12–20 With enough lead time, communities with COVID-19 clusters or hospitals with anticipated growth (‘hotspots’) could intervene to mitigate and prepare for resurgence.1 21 22 Mitigation strategies augment existing contact tracing programmes with local proactive testing, syndromic surveillance and/or social distancing policy reinforcement to suppress spread.23 24 Preparation strategies can help secure adequate hospital space, healthcare personnel and key supplies like ventilators, medications or personal protective equipment.21 25 26 Without adequate lead time, COVID-19 surges can result in critical shortages in healthcare capacity and lagging public health policies that risk further destabilising vulnerable economic activity and community well-being.26–32
Numerous efforts are underway to evaluate promising approaches to identify and predict COVID-19 hotspots using aggregated social media, viral testing patterns, mobility, biometric and symptoms data.6 12–20 In this study, we investigated the development of a composite index to identify emerging hospital COVID-19-related activity using passively collected daily electronic health record (EHR) data from a large, regional integrated healthcare system. We further quantified the potential lead time that such data—aggregated as the COVID Hotspotting Score (CHOTS)—might offer to health systems and communities by using observational data and two periods of prospective validation data across three COVID-19 waves in Northern California.
Methods
The target population was all members within the Kaiser Permanente Northern California (KPNC) integrated healthcare delivery system which serves 4.5 million members across 21 hospitals and >250 medical offices, supported by a regional Appointment and Advice Call Center available around the clock and a single comprehensive EHR systemwide.
Candidate health systems indicators
Because our health system had experienced only a single surge of significant COVID-19 hospital activity by 8 May 2020,33 34 we evaluated candidate leading indicators retrospectively from a convenience sample of health plan data drawn from 1 January 2015 through 8 May 2020 to identify temporal patterns present in prior seasonal influenza as well as for COVID-19. We denoted the COVID-19 period as 1 March 2020 forward. Based on existing literature, clinical judgement and expert opinion, we considered diverse health system indicators across KPNC related to healthcare utilisation, diagnosis codes, antimicrobial medication use, viral testing, patient communication with providers and respiratory or COVID-19-specific utilisation. For each indicator, we generated their daily count at the regional level as well as within each of 20 medical centres (local-level). We excluded one medical centre because their typical practice was to transfer their COVID-19 patients to a larger, jointly run neighbouring hospital. There were no missing or imputed values since all the indicators are counts with values >0. Online supplemental appendix table 1 describes baseline demographic data in each of six subregions in KPNC.
Supplemental material
For healthcare utilisation, we identified potential leading indicators including all outpatient ambulatory visits with a ‘respiratory infection’ diagnosis based on Healthcare Cost and Utilization Project Clinical Classification Software single-level groupers for pneumonia (122), influenza (123), acute bronchitis (125) and other respiratory infections (126).35 Respiratory infection ambulatory visits were grouped as in-person, telephone or video visit and denoted as routine or urgent. We evaluated utilisation indicators for emergency department (ED) influenza-like illness (ILI) visits based on primary symptoms of cough, dyspnoea and/or fever. We assessed COVID-19-specific utilisation based on urgent and routine clinic visits as well as hospital, intensive care unit and mechanical ventilator encounters containing COVID-19 diagnoses. For antimicrobial treatment, we evaluated outpatient antibiotic and oseltamivir prescriptions. For viral testing, we quantified tests ordered and positive results for SARS-CoV-2/COVID-19 tests, 14-panel respiratory viral panel PCR testing (RVP 14), and a combined rapid test for influenza A/B and respiratory syncytial viruses. For patient communication, we evaluated daily counts of regional call centre data focusing on patient calls which activated regional ‘cough and cold’ or ‘COVID-19’ protocols.36 37 We also identified patient-initiated ILI email communications based on subject headers containing terms similar to sore throat, shortness of breath, fever, cough, chest discomfort, chills, influenza and/or COVID-19 (removing those related to vaccination).38 39
Temporal patterns and aggregation
For each indicator with data available prior to 1 March 2020, we visually assessed changes in standardised counts and slopes from prior seasonal influenza surges and from the first wave of COVID-19 disease to evaluate their temporal association with increases in hospital census and health system utilisation. We identified historical periods of high regional influenza-related utilisation based on the top fifth percentile of daily summed values of influenza tests, oseltamivir prescriptions and outpatient antibiotic use. Because COVID-19 emergence caused significant changes in healthcare utilisation patterns and practices (eg, transitions from in-person to virtual visits; reduced availability of swabs for routine non-COVID-19 viral testing; decreased non-COVID-19 hospital census), we aggregated several individual data elements. For example, we grouped in-person, telephone and video respiratory infection visits together because remote visits largely supplanted on-site visits after COVID-19 onset. We also aggregated data for children (age <18 years) and adults (age ≥18 years) within each indicator.
Identifying temporal changes in leading indicators
To identify statistically significant changes over time in each indicator we used two methods: change point detection and moving average 7-day slopes. We used the change point analysis (CPA) algorithm to identify changes from a fixed initial baseline mean and from the last change point mean.40 CPA algorithms can detect changes in the mean values of time-series data and to identify periods marked by a significant change in the mean. They have been used previously to assess for changes in seasonal influenza data. Because we focused on identifying emerging hotspots of increasing COVID-19 activity, we down-weighted statistically significant changes in the new mean compared with the last change point mean after 14 days. We also calculated the slope of the 3-day moving average over the prior 7 days to identify statistically significant positive slope deflections.
To account for historical patterns among indicators with pre-COVID-19 data available, we estimated expected values based on seasonality and day of week via ordinary least squares using data from 1 January 2015 through 31 December 2019. Starting from 1 January 2020, we calculated the residuals as the difference between observed and expected values and assessed for significant changes in CPA (vs baseline and last value) and slope using the residual, rather than actual, values. Thus, at the regional and medical centre levels, each indicator underwent three significance tests on each day. We scored each combination of these three significance tests and summed them together to generate a daily CHOTS, focusing on simple points-based calculation that could be instantiated by June 2020 to prepare for the next wave of COVID-19 activity.
Because of the urgent need to establish a hotspotting tool in our health system to prepare for forthcoming COVID-19 waves, we used visual inspection and association analysis of potential indicators with prior seasonal influenza patterns as well as clinical judgement and heuristics to identify the final leading indicators and relevant score components. Final score rules are described later in the text and in online supplemental appendix tables 2 and 3; significance testing code is available in online supplemental file 1. Because we designed the CHOTS to focus on detecting emerging activity rather than on attempting to predict absolute hospital census, our health system also implemented and used more traditional infectious disease epidemiology and fitted curve models to predict shorter-term absolute hospital census estimates.6 34 41
Supplemental material
Prospective evaluation
Because COVID-19 activity is often measured by future COVID-19 hospital census increases, we prospectively evaluated the performance of the CHOTS developed by June 2020 in two successive waves: using prospective data through 30 September 2020 (wave 2) and in a temporally independent prospective sample from 1 October 2020 through 21 March 2021 (wave 3). Within each time period, we calculated the cross-correlation between daily CHOTSs and standardised COVID-19-specific hospital census at regional and medical centre levels when the CHOTS was lagged by 7–42 days (ie, when the CHOTSs were examined against standardised census values from 7 to 42 days later). For comparison, we also examined the cross-correlation of each of the individual indicators with COVID-19 hospital census to evaluate their performance. Finally, we also generated ‘reduced’ CHOTS that included all indicators except for call centre data and secure email messages, since those might not be routinely available in all health system settings. We calculated the correlation confidence intervals via the Fisher transformation.42
Patient and public involvement
This study did not include patient or public involvement.
All analyses were conducted with SAS V.14.5 or R V.3.6.2.
Results
We examined potential COVID-19 indicators within 23 summary groups (table 1) that included a total of 35 086 325 data elements in our initial validation period through 30 September 2020. The highest count totals were seen for indicators available retrospectively from January 2015 through September 2020 including respiratory infection clinic visits (n=9 459 882) and ED encounters (n=7 423 168). Among COVID-19-relevant predictors available from 1 March 2020 forward, the most common included COVID-19 tests ordered (n=936 360) and COVID-19-related call centre calls (n=531 765).
Health system indicators for COVID-19 activity evaluated across the Kaiser Permanente Northern California integrated healthcare delivery system sorted by total counts, including those incorporated within the COVID Hotspotting Score (grey highlights)
Figure 1 displays the seasonal patterns evident among selected key indicators during prior periods of increased seasonal influenza activity through September 2020. Figure 1A exhibits the temporal relationship between standardised counts of influenza testing and outpatient oseltamivir and antibiotics with the top fifth percentile of days with the highest influenza-related utilisation (blue bars). Other key health system indicators related to healthcare utilisation, patient-initiated communication and testing also showed similar increases timed with high influenza-related activity (figure 1B). The onset of the COVID-19 period in our health system on 1 March 2020 (red line) resulted in notable changes in nearly all indicator patterns relative to prior seasonal patterns.
Each plot displays daily standardised counts of each indicator aggregated at the regional level. The blue bands indicate periods of higher influenza seasonal activity indicative of days where the aggregate sum of influenza A/B and respiratory syncytial virus testing, oseltamivir prescriptions and antibiotic use were above the fifth percentile for all daily values. The red line marks 1 March 2020, which was the first date of significant COVID-19 regional impact in the Kaiser Permanente Northern California healthcare system.
By examining prior temporal patterns among pre-COVID-19 indicators and assessing the correlation among COVID-19-specific indicators with hospital census through May 2020, we used visual inspection and clinical judgement to select 10 variables for calculating the CHOTS including 4 major and 6 minor variables (table 1). Major variables included: (1) cough and cold calls; (2) ILI-like email message subject headers; (3) new positive COVID-19 test rates and (4) COVID-19-specific hospital census. Minor variables contributed half the weight of major variables and included: (1) COVID-19-specific call centre calls; (2) respiratory infection routine clinic visits; (3) respiratory infection urgent visits; (4) COVID-19 clinic visits; (5) COVID-19 urgent clinic visits and (6) RVP 14 tests ordered. Online supplemental appendix figure 1 exhibits an example of CPA and slope significance tests among each of the indicators from 1 January 2020 through 30 September 2020. We assigned points to each combination of the three significance tests for each indicator and summed them together to produce daily scores for the region and for each medical centre (online supplemental appendix table 1). An example of the CHOTS calculation is provided in online supplemental appendix table 3.
Figure 2A displays the CHOTS lagged by 28 days overlaid atop the regional COVID-19 hospital census through 30 September 2020; online supplemental appendix figure 2A shows the same plots at each medical centre. At the regional level, the correlation between the CHOTS and COVID-19 hospital census peaked when CHOTS was lagged 28 days reaching a high value of 0.78 (95% CI 0.71 to 0.84). With a 42-day lag, the CHOTS and COVID-19 census cross-correlation was 0.69 (95% CI 0.59 to 0.77). Table 2 exhibits the lagged correlation between medical centre CHOTSs and their COVID-19 hospital census with the lagged cross-correlation highest at 42 days for one facility (0.63), at 35 days for nine facilities (range, 0.52–0.73), at 28 days for eight facilities (range, 0.28–0.74) and at 14 days for two facilities (range, 0.73–0.78).
Standardised values of the 28-day lagged COVID Hotspotting Score (CHOTS; red dotted line and right y-axis) and COVID-19 hospital census (black line and left x-axis) at the regional level from 1 April 2020 to 30 September 2020 (waves 1 and 2, A) and standardised values of the 35-day lagged CHOTS (red dotted line and right y-axis) and COVID-19 hospital census (black line and left x-axis) at the regional level from 1 October 2020 to 21 March 2021 (wave 3, B). Data are smoothed using a locally weighted smoothing LOESS model. The smoothing parameter is the proportion of data in the local neighbourhood: a value near 0 results in a curve that nearly interpolates the data whereas a value near 1 is nearly a straight line.
Correlation between the COVID-19-specific hospital census and lagged COVID Hotspotting Score between 7 and 42 days at the regional and medical centre (facility values A through T) levels
Among individual indicators, the cross-correlations with COVID-19 census over the next 7–14 days were highest for current COVID-19 census (0.94; 95% CI 0.93 to 0.96) and the count of new positive COVID-19 tests (0.90; 95% CI 0.86 to 0.92—figure 3). For indicators lagged 14–28 days, the correlation was highest for COVID-19 urgent visits (0.83; 95% CI 0.78 to 0.87) and COVID-19 call centre calls (0.83; 95% CI 0.77 to 0.87). Beyond 28 days, the lagged CHOTS showed the highest correlation with COVID-19 census and remained >0.60 when lagged up to 49 days.
The horizontal dotted line shows a correlation coefficient ≥60%. The correlation between lagged COVID Hotspotting Score (CHOTS) and hospital census increases as lag increases, peaking at 28 days while the cross-correlation of individual indicators decreases. Indicators displayed in this figure are those that have an average positive correlation with COVID-19 hospital census. ILI, influenza-like illness.
In a temporally independent prospective validation sample (coinciding with the third wave from 1 October 2020 through 21 March 2021), the CHOTS displayed a maximum cross-correlation of 0.73 with a 35-day lag (figure 2B, table 3 and online supplemental appendix figure 2B). The highest lagged cross-correlation occurred at 42 days for one facility (0.65), at 35 days for six facilities (range, 0.59–0.68), at 28 days for nine facilities (0.52–0.75) and at 21 days for four facilities (0.55–0.77). In a ‘reduced’ CHOTS, which removed call centre and secure email messages from inclusion, the regional correlation was of similar magnitude (0.74–0.75; online supplemental appendix table 4) with a maximum lag at 28 days.
Correlation between the COVID-19-specific hospital census and lagged COVID Hotspotting Score between 7 and 42 days at the regional and medical centre (facility values A through T) levels
Discussion
Many countries and locales are now facing new waves of COVID-19-related infections highlighting the need for early warning systems that can alert communities, hospitals and public health officials to prepare for an impending rise in COVID-19 impact, including hospitalisations.5 In this study, we examined passively collected health system EHR data available with a single day lag to evaluate how changes in these indicator data could identify impending increases in COVID-19 hospital census. We found that, even before COVID-19, many of these EHR data showed significant temporal variability indicative of seasonal influenza’s impact on utilisation. We then developed a score, comprising 10 daily data elements, which showed strong correlation with COVID-19-specific hospital census up to 4–6 weeks in advance of two subsequent COVID-19 waves in Northern California. Even at the subregional level, where COVID-19 infections and hospitalisations have exhibited substantial heterogeneity in timing and size, cross-correlation varied but remained strong at most individual medical centres over the same periods.
Strengths and weakness in relation to other studies
Over the course of the COVID-19 pandemic, numerous forecasting and simulation tools have been developed which combine diverse types of data (eg, COVID-19 testing rates, COVID-19-specific hospitalisation or death rates, symptom surveillance, evidence of community-level social distancing, biometrics) to identify emerging hotspots.8 12–20 24 34 43 While these tools have proven valuable at an aggregate level, they often lack the geographic and temporal specificity that reflects heterogeneous emerging patterns within local communities. The most reliable indicators, like COVID-19 hospitalisations or death rates, are also known to lag significantly behind emerging disease activity. For these reasons, such tools have had limited value for informing our local medical centres about their medium-term preparation and mitigation activities prior to COVID-19 surge.41 In addition, these tools often focus on trying to predict the precise COVID-19 hospital census, which has been shown to be highly variable across serial COVID-19 waves as well as across waves of other pandemic disease including 1918 influenza and 2009 H1N1. This variability is attributable to many factors including the social distancing behaviours of individuals and communities, the policies enacted by local and national governments, the dynamic biology of the virus itself, and other factors that have yet to be elucidated.
To address these limitations, in this study, we focused on passively collected EHR data readily available within our health system that would maximally reflect daily local patterns and were amenable to urgent development, testing and deployment. We chose to use existing algorithms for identifying temporal change in indicators rather than more advanced, and potentially more complex, machine learning approaches to facilitate development and instantiation. Because the CHOTS is designed to inform medium-term decisions, we also chose not to build a model to generate precise predictions of hospital census and instead used other curve-fitting and epidemiological models to predict absolute hospital census numbers over short intervals.6 34 41 Finally, we used a points-based scoring system that reflected our intuitive approach to generating the CHOTS, a necessity driven by the paucity of reliable historical data available in June 2020 after only a single wave of significant COVID-19 activity in Northern California.
Importantly, we found that individual COVID-19 disease indicators showed very strong correlation with subsequent hospital census when these data were lagged between 1 and 4 weeks over three successive waves of activity through March 2021. For example, the number of new positive COVID-19 tests was highly correlated with forthcoming hospital census over the following 2 weeks. Similarly, call centre volume was strongly correlated with hospital census at 3–4 weeks. ILI-like email communication between patients and providers showed lower correlation with COVID-19 census. Thus, our data suggest that single leading indicators like these can be effectively used to inform short-term hospital census predictions or to identify emerging COVID-19 activity with a lead time of 3–4 weeks. Indeed, in our health system, high-value indicators like call centre data have been used to direct our response to emerging seasonal influenza activity for many years.
However, the correlation of individual indicators with census decreased steadily beyond 2–4 weeks, while the correlation of the aggregate score continued to increase and remain strong through 6 weeks. While any lead time to prepare before COVID-19 hospitalisations increase has tremendous value, narrower windows may hamper effective mitigation and adequate preparation for hospitals and public health agencies. With adequate lead time, individual hospitals can focus on ensuring that the additional staff, supplies and space needed to care for a large number of COVID-19 patients in advance of their expected need.26–31 Public health officials can also titrate social distancing policies to target expected, rather than current or lagging, COVID-19 activity. A ‘reduced’ form CHOTS also showed robust performance suggesting that the score could have value in health systems without ready access to patient-initiated data from call centres or emails.
Implications for clinicians and health system leaders
The CHOTS has been in use in our health system since June 2020 and is updated on a daily basis in a variety of dashboards that are accessible to our health system and hospital leadership. After KPNC’s COVID-19 census began to ebb following wave 2, the alarming increase in the CHOTS before wave 3 was used to inform the reopening of daily Regional COVID-19 Command Center operations. The CHOTS tool has also been used to inform decisions about health system staffing and resource allocation as well as clinical care, based on the expected rise, stabilisation or fall of COVID-19 activity across different subregions and individual medical centres. Finally, the CHOTS tools have also informed decisions about the urgency of health system communications with members, communities and public health agencies, particularly during periods when the easing of social distancing behaviours occurred concurrently with the emergence of increasing COVID-19 hotspotting signals.
Strengths and limitations of the current study
The major strength of this study was its use of comprehensive and diverse EHR indicator data across a large and diverse integrated healthcare delivery system to demonstrate the value of these data for COVID-19 preparation. Because we experienced significant heterogeneity across medical centres with respect to COVID-19 impact, we were also able to compare its performance within individual subregional communities. By focusing on a parsimonious set of indicators and available algorithms, without developing a traditional predictive model, we were able to rapidly deploy our tool which proved extremely valuable for COVID-19 preparation and planning.44 45
There are several limitations to this study. Most importantly, our data come from a single integrated healthcare system in a single region of the USA. Thus, we were likely able to capture more comprehensive data within our EHR and data systems since our patients receive the overwhelming majority of their care within our facilities. Thus, the generalisability of our tool may vary across settings and geographies, particularly for health systems which may lack robust call centre and/or email communications systems, protocols and data. Our data also lack indicators of local COVID-19 activity that does not occur in our members but that still likely impacts hospitalisation rates. Second, routine healthcare patterns have changed dramatically owing to COVID-19 including the shift from in-person to virtual care as well as the deferral of routine healthcare including surgical procedures.46 We attempted to aggregate diverse data—–for example, consolidating in-person, telephonic and video visits among adults and children for respiratory diseases—into single indicators to minimise the impact of practice changes on score calculation. However, the patterns we identified during the study period are likely to continue to change and require ongoing re-examination and refinement.
Third, we also did not attempt to develop a predictive model designed to precisely estimate absolute hospital census, instead focusing on a hotspotting approach designed to give the earliest signals of incipient COVID-19 activity that might impact hospitalisation. We have built and used other models for absolute census prediction but have found that their accuracy is greatest over very short intervals like 1–2 weeks, limiting their longer-range use in health system preparation. Fourth, we generated and deployed the CHOTS during a time of great uncertainty following the first wave of COVID-19 activity in California. As a result of the extreme urgency to prepare our health system, we depended on clinical judgement and heuristics, in addition to prior health system influenza patterns, to develop our score. With the luxury of time, more advanced machine learning or statistical techniques may have produced different calculations. Small sample sizes in each facility may have also impacted statistical significance testing. Nonetheless, the CHOTS continued to show very strong performance through the third wave of COVID-19 in Northern California. Finally, we examined the performance of the score during a period of low regional influenza activity; these patterns may continue to change based on seasonal influenza.
Unanswered questions and future research
The implications of our findings on future research remain dynamic because of the tremendous uncertainties owing to the biology and impact of COVID-19 worldwide and differences in regional responses driven by health policies and treatments for the disease. However, tools which evaluate passively collected leading indicators beyond only positive COVID-19 case counts, hospital census or deaths, will continue to provide strong utility to inform health system decisions about preparation, mitigation and suppression of this pandemic. Even with some effective vaccination campaigns underway in 2021, there is persistent concern that the impact of new variants or incomplete and waning herd immunity will mean that ongoing COVID-19 activity will continue on a seasonal or intermittent basis. In our health system, the CHOTS will continue to inform our health system response, particularly entering the fall of this year. Additional external validation of this tool will also be important to assure its generalisability beyond our system. We will make our CPA and slope algorithm available for others who are interested in applying them to their own daily health system data to assess for informative leading indicators of local COVID-19 activity. Undoubtedly, additional improvements to this tool can likely be realised by incorporating non-health system community-level data across diverse domains indicative of disease spread including mobility, wastewater, biometrics, COVID-19 genotyping, and symptoms.
Conclusion
In conclusion, we used 10 health system indicators of potential COVID-19-related disease activity to generate an aggregate score that was strongly correlated with forthcoming hospital census at 4–6 weeks at both a regional and local level. While individual indicators showed very strong cross-correlation with impending COVID-19 hospital census over a 1–4 weeks timeframe, this hotspotting tool could potentially extend the lead time for local communities, health systems, and public health officials to prepare for and mitigate emerging COVID-19 activity.
Data availability statement
No data are available.
Ethics statements
Ethics approval
This study was approved by the Kaiser Permanente Northern California (KPNC) Institutional Review Board (FWA# 00002344, IRB# 00001045). Informed consent was waived by the IRB. The research involved no more than minimal risk to the subjects and could not practicably be carried out without the requested waiver.
Acknowledgments
The authors would like to acknowledge the contributions of Marc Flagg, Debbie Amaral, Jamila Gul, Mei Lee, Sri Chimmula, Daniel Ng, Chinmay Pandit, Jay Soule, Miriam Espiritu, Reen Samady and Yasser Attiga with data preparation and processing. The authors do not plan to make study data available. The lead author affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted and that any discrepancies from the study as originally planned have been explained. The authors have no competing interests to report nor have they published, posted or submitted any related papers from the same study. All authors have completed the ICMJE uniform disclosure form and declare: no support from any organisation for the submitted work other than the lead author VL as indicated in the submission; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years, no other relationships or activities that could appear to have influenced the submitted work. Dissemination of study results to study participants and or patient organisations is not possible/applicable.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors VL, KKT, JG and PK contributed to the conception and design of the study; KKT, JG, LDG and PK contributed to the acquisition of data; and VL, KKT, JG, LDG, LM, SP, Y-FIC, NG, GJE and PK contributed to the analysis and/or interpretation of findings. VL, KKT and PK contributed to drafting the manuscript and VL, KKT, JG, LDG, LM, SP, Y-FIC, NG, GJE and PK contributed to revising the manuscript for critically important intellectual content. VL, KKT, JG, LDG, LM, SP, Y-FIC, NG, GJE and PK approved the final version of the manuscript.
Funding This work was funded by The Permanente Medical Group, Kaiser Foundation Hospitals, NIGMS R35128672.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.