Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

An evaluation of the impact of social and structural determinants of health on forgone care during the COVID-19 pandemic in Baltimore, Maryland

  • Diane Meyer ,

    Roles Conceptualization, Formal analysis, Funding acquisition, Methodology, Writing – original draft, Writing – review & editing

    dmeyer10@jhmi.edu

    Affiliations Center for Infectious Disease and Nursing Innovation, Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America, Center for Health Security, Johns Hopkins University, Bloomberg School of Public Health, Baltimore, MD, United States of America

  • Kelly Lowensen,

    Roles Investigation, Writing – review & editing

    Affiliation Center for Infectious Disease and Nursing Innovation, Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America

  • Nancy Perrin,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America

  • Ayana Moore,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation FHI 360, Durham, NC, United States of America

  • Shruti H. Mehta,

    Roles Methodology, Writing – review & editing

    Affiliation Department of Epidemiology, Johns Hopkins University, Bloomberg School of Public Health, Baltimore, MD, United States of America

  • Cheryl R. Himmelfarb,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America

  • Thomas V. Inglesby,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Center for Health Security, Johns Hopkins University, Bloomberg School of Public Health, Baltimore, MD, United States of America

  • Jacky M. Jennings,

    Roles Investigation, Writing – review & editing

    Affiliation Department of Pediatrics, Johns Hopkins University, School of Medicine, Baltimore, MD, United States of America

  • Alexandra K. Mueller,

    Roles Investigation, Writing – review & editing

    Affiliation Department of Pediatrics, Johns Hopkins University, School of Medicine, Baltimore, MD, United States of America

  • Jessica N. LaRicci,

    Roles Investigation, Writing – review & editing

    Affiliation Center for Infectious Disease and Nursing Innovation, Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America

  • Woudase Gallo,

    Roles Investigation, Writing – review & editing

    Affiliation Center for Infectious Disease and Nursing Innovation, Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America

  • Adam P. Bocek,

    Roles Investigation, Writing – review & editing

    Affiliation Center for Infectious Disease and Nursing Innovation, Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America

  • Jason E. Farley

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Center for Infectious Disease and Nursing Innovation, Johns Hopkins University, School of Nursing, Baltimore, MD, United States of America

Abstract

Evidence suggests that reductions in healthcare utilization, including forgone care, during the COVID-19 pandemic may be contributing towards excess morbidity and mortality. The objective of this study was to describe individual and community-level correlates of forgone care during the COVID-19 pandemic. We conducted a cross-sectional, secondary data analysis of participants (n = 2,003) who reported needing healthcare in two population-representative surveys conducted in Baltimore, MD in 2021 and 2021–2022. Abstracted data included the experience of forgone care, socio-demographic data, comorbidities, financial strain, and community of residence. Participant’s community of residence were linked with data acquired from the Baltimore Neighborhood Indicators Alliance relevant to healthcare access and utilization, including walkability and internet access, among others. The data were analyzed using weighted random effects logistic regression. Individual-level factors found to be associated with increased odds for forgone care included individuals age 35–49 (compared to 18–34), female sex, experiencing housing insecurity during the pandemic, and the presence of functional limitations and mental illness. Black/African American individuals were found to have reduced odds of forgone care, compared to any other race. No community-level factors were significant in the multilevel analyses. Moving forward, it will be critical that health systems identify ways to address any barriers to care that populations might be experiencing, such as the use of mobile health services or telemedicine platforms. Additionally, public health emergency preparedness planning efforts must account for the unique needs of communities during future crises, to ensure that their health needs can continue to be met. Finally, additional research is needed to better understand how healthcare access and utilization practices have changed during versus before the pandemic.

Introduction

Evidence suggests that reductions in healthcare utilization during the Coronavirus disease 2019 (COVID-19) pandemic may be contributing towards excess morbidity and mortality [13]. Much of this evidence consists of studies describing changes in the volume of services rendered, such as trends in hospital admissions and emergency department usage [1, 4, 5]. Fewer studies have evaluated COVID-19 related changes in healthcare utilization through the lens of patient-reported forgone care, which is defined as healthcare that is perceived as needed by the person but not received, and includes delayed, missed, or skipped visits with a healthcare provider [6, 7].

Existing studies have identified several individual-level social and structural determinants of health (SSDoH) as important correlates of forgone care during the pandemic, although the findings have been at times contradictory, likely related to the heterogeneity of sampling [819]. Czeisler et al., using a nationally representative sample of US adults, found the prevalence ratio of forgone care in non-Hispanic Black adults to be 1.6 times that of non-Hispanic White adults in June 2020 [10]. However, another nationally-representative survey of forgone care from March to July 2020 among adults did not identify any statistically significant difference in the frequency of forgone care by race/ethnicity [9]. A separate study that evaluated forgone care among Medicare beneficiaries found a higher prevalence among non-Hispanic White beneficiaries (22.5%), compared to non-Hispanic Black beneficiaries (12.6%) in the summer of 2020, although these differences dissipated when looking at forgone care in the fall of 2020 and winter of 2021 [16]. Other factors identified as being significantly correlated with forgone care, although at times with mixed results, include experiencing food or income insecurity; education level; having comorbid conditions, including mental health conditions and functional disabilities; both older and younger age; having health insurance; identifying as a member of the LGBTQ+ community; and experiencing racial discrimination [813, 1517, 19]. These studies may indicate that certain individuals and communities experienced disproportionate impacts on healthcare access and utilization during the pandemic, leading to forgone care.

Community-specific SSDoH, such as Area Deprivation Index, internet coverage, walkability, and ethnic segregation are related to health outcomes [2026]. Studies have identified geographic inequities related to the COVID-19 pandemic, including COVID-19 case counts and mortality rates [27, 28] and inequitable access to testing and treatment [2931], but few have acknowledged these factors and their associations with forgone care. Similar geographic inequities may have occurred in who experienced forgone care, such as in communities with high rates of poverty, those with ethnic segregation, and those without resources such as internet coverage. For example, several community-level factors have been cited in the literature as reasons for forgoing care during the pandemic, including closures of local health facilities, transportation challenges, and lack of resources required for telemedicine [1, 9, 32, 33]. Also concerning is that many of the communities that suffered higher rates of SARS-CoV-2 infection also likely needed additional healthcare services as a result.

Forgone care can have drastic consequences on individual and community health. Delayed and cancelled preventive care, including immunizations, dental cleanings, and cancer screenings, will have long-term health implications that go well beyond the end of the COVID-19 pandemic. For example, one study identified decreases in screenings for breast, lung, colon, cervical, and prostate cancer ranging from 60% to 82% at one large health center in Massachusetts [34]. Delayed treatment for emergent conditions such as strokes, heart attacks, and orthopedic trauma likely have led to worse prognoses and more severe outcomes [3537]. Furthermore, studies conducted prior to the pandemic found that those with chronic conditions are more likely to forgo care [12]. Given that the pandemic introduced or exacerbated existing barriers to healthcare access, these individuals could be at higher risk for severe downstream consequences of interruptions in care. This is reflected in mortality data published by the US Centers for Disease Control and Prevention, which suggests a large increase in excess deaths from chronic conditions such as heart disease and diabetes, highlighting the importance of continued access to care and the potential consequences of forgone care [38].

The aim of this study was to evaluate the impact of individual and community-level SSDoH on healthcare utilization during the COVID-19 pandemic by specifically looking at forgone care among a sample of adults living in Baltimore, Maryland. The objective was to determine whether community-level correlates accounted for forgone care above and beyond any individual participant-level correlates. Understanding the intersections of forgone care during the COVID-19 pandemic with SSDoH is critical to providing a comprehensive view of the health impacts of the pandemic and to identifying which individuals and communities experienced the greatest interruptions in care.

Methods

Study design and data sources

This was a secondary data analysis using baseline data abstracted from two parent studies funded by the National Institute of Allergy and Infectious Diseases. The combined analytic sample leveraged the strengths of two robust sampling strategies used in the parent studies, increasing the overall sample size, and helping to ensure adequate representation of economically disadvantaged and historically excluded communities that may not be included in the current COVID-19 forgone care literature.

This nested study used a cross-sectional design to examine individual and community-level factors associated with forgone care during the COVID-19 pandemic in Baltimore, MD. Data abstracted from the parent studies included responses to survey questions regarding demographics, comorbidities, healthcare needs, and forgone care during the COVID-19 pandemic. For the purposes of this study, forgone care was defined as healthcare that was skipped, missed, or delayed during the COVID-19 pandemic. The two parent studies are described in more detail below.

The Community Collaborative to Combat COVID-19 study.

The Community Collaborative to Combat COVID-19 (C-Forward) parent study was a two-phased comparative effectiveness trial. Enrollment of participants occurred from February 2021 to December 2022. The study aimed to evaluate three different SARS-CoV-2 testing modalities (fixed site, mobile, and home-based) using a representative sample of Baltimore households. The study enrolled households (HH) using a multi-staged sampling approach, organized by census block groups that were stratified by socioeconomic status and race/ethnicity, with oversampling of historically excluded and under-resourced populations (e.g., Hispanic/LatinX populations, low-income whites).

Household recruitment was multi-modal and included doorhangers, mailings, telephone calls, and online methods. The study included enrollment of one eligible member per HH, 18 years of age or older, who was able to provide informed consent (designated as the “HH index member”). The HH index member completed a baseline survey that covered demographics; history of COVID-19 symptoms, testing and treatment; adoption of preventive behaviors; comorbidities and health care access and utilization (including forgone care); COVID-19 impact and pandemic stress; mental health and substance abuse; and knowledge and attitude towards COVID-19. This survey was completed electronically, by mail, or via telephone with a trained research assistant and entered into REDCap. HH index members also received SARS-CoV-2 antigen and antibody testing and other serum testing (e.g., chemistry profiles). A total of n = 1,978 HH index members were enrolled. Additional information about this study, including other study phases that were not included in this secondary data analysis, can be found on clinicaltrials.gov (clinical trial identifier # NCT04673292).

The COVID-19 Prevention Network 5002 study.

The COVID-19 Prevention Network 5002 (CoVPN 5002) parent study was a multi-site, cross-sectional SARS-CoV-2 prevalence study conducted from March to July 2021. The study used a venue-time sampling strategy, with random selection of venues and enrollment times. Venues included low-income housing, transitional housing, shelters, soup kitchens, places of worship, local federally qualified health centers, and rehabilitation centers, among others [39].

All individuals accessing the selected venue during the randomly selected time/location slot were approached to assess interest and eligibility to enroll in the study. Adults and children greater than 2 months of age who were willing and able to provide consent (assent for those <18 years of age, with the consent of a guardian) were eligible. Participants completed a paper-based survey with a research assistant that included questions on demographics; medical history (including forgone care); COVID-19 history included exposure; knowledge, attitudes, and behavior about COVID-19; economic impacts of COVID-19; and willingness to take the COVID-19 vaccine. All survey responses were later entered into Medidata. Individuals also received SARS-CoV-2 antigen and antibody testing. A total of n = 1,022 individuals enrolled in the study at the Baltimore, Maryland site. Additional information about this study can be found on clinicaltrials.gov (clinical trial identifier # NCT04658121). The secondary data analysis presented in this manuscript describes only data collected from the Baltimore research site.

Secondary data analysis

Inclusion and exclusion criteria.

All parent study participants who reported needing any care type during the COVID-19 pandemic in the parent study survey, including chronic, preventive, or emergent care or a major medical or dental procedure were included (Fig 1); all other parent study participants were dropped (n = 888). Parent study questions can be located in Supplement 1. Participants under the age of 18 were also dropped regardless of whether or not they needed any type of care (n = 22), as C-Forward only included enrollment of HH index members who were ≥ 18 years of age.

Individuals were also excluded if it could not be ascertained as to whether they needed any type of care during the pandemic (e.g., they left these survey questions blank or responses were discordant; n = 44) or it was unclear whether they had experienced the outcome of forgone care (n = 42). Surveys that had seven or more incomplete demographic questions were also dropped (n = 1).

Measures

Outcome variable.

The outcome variable for this study was participant-reported forgone care on enrollment in either of the parent studies. This included any forgone care experienced since the beginning of the COVID-19 pandemic, measured from March 1, 2020 until completion of the survey. Questions on forgone care assessed the need for and receipt of chronic or preventive care, emergent care, and major medical or dental procedures. All C-Forward participants responded to questions on needing/forgoing chronic, preventive, and emergent care and elective/dental procedures. For the CoVPN 5002 study, a supplemental survey on needing/forgoing emergent care and elective/dental procedures was added in April 2021 to the Baltimore, MD research site only. Prior to that date, participants only responded to survey questions on needing/forgoing chronic/preventive care.

Independent variables—Individual level.

The selection of individual-level variables was informed by the Gelberg Behavioral Model for Vulnerable Populations [40]. This framework, a revision of the Andersen and Newman Model of Healthcare Utilization [41], was chosen as it includes determinants of healthcare system access and utilization relevant to underserved populations living in Baltimore, which has higher rates of poverty, homelessness, crime, and drug use in comparison to other cities of its size as well as widespread disparities across the SSDoH [4245]. Our adapted model included individual and community-level factors related to healthcare utilization and organizes them into three categories: predisposing—factors that may increase the propensity to seek care; enabling/impeding—factors that may enable or impede use of the health system; and need—factors that reflect perceived need for care [40, 41]. For this study, predisposing factors included age, sex at birth, race, marital status, employment status, disability status, and education level. Enabling/impeding factors included income, loss of housing during the pandemic, and worrying about paying rent/mortgage during the pandemic. Need factors included number of comorbidities, presence of functional limitations, mental illness, and alcohol/substance and injection drug use. Each of these variables were measured using survey data from the two parent studies (see supplement 1 for parent surveys). Responses were consolidated across the two surveys by sociodemographic topic (e.g., sex, age) and sub-categories (e.g., female, male) were created to be inclusive of the wording of the parent studies.

Independent variables—Community level.

Participants were assigned community-level factors based on their community statistical area (CSA) of residence. CSA’s are clusters of neighborhoods that were developed by Baltimore City’s Planning Department and are based on city neighborhoods [46]. There are 55 CSA’s in the city of Baltimore, with boundaries that align directly to census tracts (46). The population of a given CSA ranges from 5,000 to 20,000 people and consist of anywhere from 1–8 Census Tracts [47]. C-Forward participants were matched to a CSA based on their census block group at enrollment. CoVPN 5002 participants were matched to a CSA based on the address of the venue in which they enrolled, as their actual physical address was not collected for the study.

Seven continuous CSA-level variables that have previously been associated with forgone care (prior to the COVID-19 pandemic) were included in the community-level analysis of forgone care. All CSA-level data was acquired from the Baltimore Neighborhood Indicators Alliance (BNIA) [48]. These variables included the below (year of data collection according to BNIA is notated in parenthesis):

  • Percent of households with no internet at home (2020): individuals living in communities with poor internet coverage have been found to be less likely to utilize telehealth services offered during the COVID-19 pandemic [22, 23].
  • Walkability score (2017): individuals living in communities with poor walkability have been found to be associated with decreased healthcare access [27]. Challenges associated with poor walkability were likely compounded by the COVID-19 pandemic with closures of public transportation and other nearby resources.
  • Percent of population that is Hispanic (2020): segregation of ethnic minorities are associated with “place” disparities, including access to healthcare [24].
  • Components of the Area Deprivation Index (ADI): ADI is a census-based index that tracks community socioeconomic status across 17 indicators, including measures of education, employment, quality of housing, and poverty [49]. Higher ADI scores have been previously associated with poorer health outcomes [25, 50]. Unfortunately, ADI scores were unavailable by Baltimore CSA, so 4 related indicators were selected for this analysis that had data available at the CSA-level. These included:
    1. ○ Percentage of households that pay more than 30% of their total household income on rent and related expenses (2020).
    2. ○ Percentage of 12th graders in a school year that successfully completed high school (2020/2021).
    3. ○ Percentage of households whose income fell below the poverty threshold (2020).
    4. ○ Percentage of population who are not in the labor force (age 16–64 years) (2020).

Covariates.

Two “pandemic wave” binary variables were added to the models to help account for the different enrollment periods of the parent studies and what waves of the pandemic each participant experienced that may have impacted their ability or desire to seek healthcare. All participants who enrolled on or after August 1, 2021 were coded as having experienced the delta COVID-19 wave; those who enrolled on or after December 1, 2021 were coded as having experienced the omicron COVID-19 wave. A binary variable was also created to account for which parent study the data were acquired from.

Statistical analyses

Imputation of missing data.

Patterns of missingness were explored to determine whether data were missing at random. Multiple imputation was then used to impute missing data [51] using Stata version 16.0 (StatCorp, College Station, TX). The sequential imputation using chained equations method was employed for this study, using the individual-level independent variables, the outcome of forgone care, and “study” variable as predictors of missing values. Five total imputed datasets were created. A sensitivity analysis was conducted at the completion of the model-building process to check for biases introduced during multiple imputation. For the sensitivity analysis, the final model was run using only participants with complete data. The magnitude of the resulting coefficients was then compared with those obtained from running the same model using the imputed dataset.

Model building—Level-one model.

Weighted bivariate logistic regressions were conducted for each individual-level independent variable, with the binary outcome of any type of forgone care. Each observation was weighted in the regression model for the total amount of years the individual had experienced in the pandemic, measured from March 1, 2020 until the date of survey completion. Any individual-level variable significant at p≤0.10 was carried over to the final level-one multivariable model. The multivariable weighted level-one model also included the “study” and “pandemic wave” variables. Collinearity of independent variables was also checked using variation inflation factor, with none noted.

Model building—Level-two multi-level model.

The level-two model included weighted random effects logistic regressions with participants clustered within Baltimore CSAs. The model contained all significant level-one variables, and the level-two continuous community-level variables were tested one at a time. Any CSA-level variables significant at p ≤0.10 were incorporated into the final multivariable model. The significance of variables in this final model were set a p≤0.05.

Power analysis.

A power analysis based on preliminary data assumed an event rate of 0.41 [9, 10], an alpha of 0.05, 80% power, an estimated sample size of n = 1,662, and 30 Baltimore CSAs. Assuming these parameters and an intraclass correlation coefficient (ICC) of 0.010, the study was powered to detect odds ratios that ranged from 1.34 to 1.45, as the distribution of the predictor variable ranged from 20/80 split to a 50/50 split.

Ethical considerations

This study was approved by the Johns Hopkins University School of Medicine (JHU SOM) Institutional Review Board (IRB) on December 19, 2022 (IRB# 00348695). The C-Forward parent study was approved by the JHU SOM IRB on June 2, 2020 (IRB# 00250298). The CoVPN 5002 parent study was approved by the JHU SOM IRB on January 28, 2021 (IRB#00262004) through a reliance agreement with Advarra IRB, which was the IRB of record for the larger CoVPN 5002 network study. This secondary data analysis was granted a consent waiver as participants who enrolled in either parent study completed consent forms that covered the conduct of this research. During data analysis, authors of this article had access to a limited dataset provided by the parent study data management teams.

Results

Participant characteristics

Seventy-one percent of C-Forward participants (1,405 of 1,978) met inclusion criteria for this secondary analysis. A slightly smaller percentage of those enrolled in the CoVPN 5002 study (598 of 1,022; 59%) met inclusion criteria. This yielded a total sample size of n = 2,003 (see Fig 1). Nearly 65% of the sample reported their sex at birth as female. Most (87.6%) reported not being of Hispanic or Latino origin and 52.7% reported their race as Black or African American. Nearly 40% of the sample reported being married or partnered, slightly less than half (45.1%) were employed part or full-time, and 15.5% reported being on disability status. Twenty percent of participants reported being worried about their ability to pay or being unable to pay their rent or mortgage during the pandemic, nearly 8% reported losing their housing during the pandemic, and 30% reported making less than $25,000 per year. Nearly 65% reported having at least one co-morbidity, including a cancer diagnoses, diabetes, renal disease, sickle cell anemia, respiratory disease, cardiac disease, or immune disease (Table 1).

In total, 567 participants reported experiencing any type of forgone care. This included 495 participants who reported forgoing chronic or preventive care, 33 participants who reported forgoing emergent care, and 94 participants who reported delaying an elective surgery or dental procedure. Nearly 57% of those who reported forgone care were over the age of 50 years, 67.9% were female, and over half (50.3%) reported being Black of African American.

CSA characteristics

The sample included participants from 48 of the 55 CSAs in Baltimore. The number of participants per CSA ranged from 2 to 174. The calculated ICC was 0.09. Table 2 summarizes the mean and standard deviation values for each of the seven CSA-level variables for those who did and did not report forgone care.

Correlates of forgone care

Results—Individual-level correlates of forgone care.

Table 3 presents the results from the level-one weighted bivariate analyses examining the association of each individual level variable with forgone care. Several pre-disposing factors were found to be associated with forgone care. Those in the age group 35–49 years were found to have a 1.48 higher odds of forgone care, compared to the age group 18–34 (p = 0.001); conversely, those in the ≥65 age group were found to have a 25% lower odds of forgone care, compared to individuals 18–34 years (p = 0.020). The odds of forgone care were also higher in those reporting female sex at birth (OR 1.24, p = 0.010) and those on disability status (OR 1.43, p = 0.001). Individuals reporting Black or African American race had a 15% lower odds of forgone care (p = 0.047).

Two enabling factors were found to be associated with higher odds of forgone care, including those who reported losing their housing during the COVID-19 pandemic (OR 1.90, p = <0.001) and those who reported being worried about paying their rent or mortgage (OR 1.69, p = <0.001). Finally, significant need level factors with higher odds of forgone care included those with functional limitations (OR 1.66, p<0.001), those with mental illness (OR 1.87, p<0.001), those who reported using substances (OR 1.43, p = 0.004), and those with two or more comorbidities, compared to no comorbidities (OR 1.24, p = 0.023).

Results—Final multivariable model.

The results of the final weighted multivariable model can be found in Table 3, adjusted for the pandemic wave and study variables and age category, sex, race, disability status, loss of housing, worrying about rent/mortgage, presence of functional limitations, mental illness, substance abuse, and comorbidity number. Predisposing factors that were statistically significant in the final model included those who reported being ages 35–49 years, who were found to have a higher odds of forgone care compared to those in the 18–34 years age group (OR 1.51, p = 0.001). Individuals who reported their sex at birth as female were also found to have higher odds of forgone care, compared to those who reported their sex at birth as male (OR 1.21, p = 0.028). Individuals who reported their race as Black/African American had a 32% reduction in their odds of forgone care, compared to individuals of any other race (OR 0.68, p = ≤0.001). Significant enabling factors included those who reported losing their house during COVID-19, compared to those who did not (OR 1.47, p = 0.014) and those who reported being worried about paying their rent or mortgage, compared to those who were not (OR 1.37, p = 0.005). Need level factors that remained significant in the final model included those with functional limitations, compared to those without (OR 1.46, p = ≤0.001) and those who reported having a mental illness, compared to those without (OR 1.44, p = ≤0.001).

Results—Community-level correlates of forgone care.

None of the continuous CSA-level variables were found to be significant at the p≤0.10 level after controlling for the individual level variables of age category, sex, race, disability status, loss of housing, worrying about rent/mortgage, presence of functional limitations, mental illness, substance abuse, and comorbidity number. Thus, none of the CSA-level variables were added to the model with the individual level factors.

Discussion

Several studies published during the COVID-19 pandemic on forgone care indicate that economically disadvantaged and historically excluded populations experienced a greater prevalence of forgone care [810, 12, 14, 19]. However, few studies have investigated correlates of forgone care that go beyond social and economic factors, such as examining the influence of the local community in which people live. Given the widespread impacts of the COVID-19 pandemic on both individuals and communities, it is important to explore the myriad, multilevel SSDoH that may influence forgone care [52]. However, our study found that none of the community-level determinants were significantly associated with forgone care in the multilevel model. This finding differs from previously published literature that has identified community-level disparities in COVID-19-related outcomes, although the geographic unit of analyses used in these studies differ from what was used in our study [2731].

In this racially and economically diverse sample, we found that Black/African Americans living in Baltimore were less likely to experience forgone care, which is in opposition to some previously published work on forgone care during the COVID-19 pandemic [10, 13, 14]. However, this finding was consistent with Tsuzaki & Taira, who found that non-Hispanic/LatinX Black Medicare beneficiaries were less likely to forgo care in the summer of 2020 compared to White beneficiaries [16]. These findings warrant additional exploration, including comparisons of healthcare utilization practices prior to versus during the pandemic. There are long-standing, systemic inequities in healthcare access and utilization within the Black/African American community that were further exacerbated during the COVID-19 pandemic. For example, a 2016 report by the University of Maryland found that Black/African American Maryland residents were 1.9 times more likely to be unable to afford a visit with a healthcare provider than White residents and were much more likely to die from heart disease, diabetes, stroke, and asthma [53]. Black/African American individuals included in our study sample may have reported less episodes of forgone care because they have historically experienced systemic barriers to access within the healthcare system. Therefore, they may have been less likely to use the healthcare system even prior to the pandemic, and thus less likely to report forgone care.

In our analysis of individual-level correlates of forgone care, we identified those who have experienced housing insecurity during the pandemic, as evidenced by loss of housing or worrying about paying for housing, as having higher odds of forgone care. Income inequities and financial instability have previously been identified as critical determinants of an individual’s healthcare utilization practices and the experience of forgone care [54]. Unfortunately, the pandemic impacted the economic stability of millions of American households, particularly lower-income families, and many lost jobs as a result [55]. This may have drove increased rates of forgone care within individuals experiencing financial hardship and housing instability. Anderson et al., for example, found that 52% of respondents who reported missing medical care did so due to “financial repercussions of the COVID-19 pandemic” [9]. Notably, several safety net programs, such as the Coronavirus Aid, Relief, and Economic Security (CARES) Act, which provided unemployment benefits and additional financial support to American families, and the continuous Medicaid enrollment provision under the Families First Coronavirus Response Act, were available to qualifying individuals early in the pandemic [56, 57]. However, a study published in 2020 on access and enrollment in safety net programs during the pandemic found gaps in awareness for several programs, including health insurance exchanges [58]. Our study also found that individuals reporting female sex at birth had higher odds of forgone care compared to those reporting male sex at birth. During the pandemic, many women had to take on additional caretaking roles, including homeschooling children and caring for ill family members [59]. These additional responsibilities may have impacted their ability to seek healthcare when needed. Further, it was well established prior to the pandemic that men seek healthcare less than women [60], and it is possible this was carried forward in this sample.

The odds of forgone care were also higher in individuals reporting functional limitations. These individuals likely have a higher reliance on support services such as accessible transportation, and interruptions in these services may have disproportionately impacted their ability to access the health system. Those reporting a mental illness also reported an increased odds of forgone care, a finding that aligns with other published COVID-19 forgone care literature [12]. Individuals suffering from mental illness have previously been shown to be at higher risk for forgone care [61], and exacerbations in mental health crises during the pandemic may have disproportionately led to the increased risk of forgone care within this population.

In a world where pandemics and other humanitarian emergencies are increasingly likely due to population growth, interactions at the human-animal interface, climate change, and global travel, it is critical that we strengthen models of care so that health systems can continue to support individual and community health during future public health emergencies. One potential strategy is to increase the use of mobile-based healthcare services. Mobile health clinics have previously been shown to be effective in providing urgent care, preventive health, and chronic disease management and have been shown to increase healthcare access for underserved communities, including the economically disadvantaged [62]. Some of the benefits of mobile health clinics include the elimination of barriers such as lack of transportation, inconvenient operating hours, and long wait times and the ability to form trusting relationships within communities, particularly those who experience stigma such as people living with mental illness [62]. Furthermore, leveraging mobile clinics in the midst of an emergency have been shown to reduce disruptions in care, as they are already well-integrated into the community and can be effectively and quickly deployed [62]. Leveraging of mobile clinics during COVID-19 have been captured in the literature to-date. For example, co-location of COVID-19 vaccines within existing harm reduction mobile services has been demonstrated to be an effective way to deliver vaccines to underserved populations [63]. More widespread use of these types of clinics now and in the future could help reduce forgone care, particularly among those who already experience barriers to healthcare access.

Another strategy that could help address inequities in forgone care during COVID-19 and future health emergencies would be adoption of programs that could extend telehealth services to those who currently lack access. This should include not only the resources required for telehealth, such as internet coverage and smartphones, but also changes in insurance policies that increase coverage and better education for consumers on how to utilize this technology [64]. For example, participants in one study reported that a major disadvantage of telehealth was their “unfamiliarity” with the platform and distrust that their information was secure and private [65]. Healthcare providers also expressed gaps in knowledge on how to use telemedicine and what services were covered by insurance [65], indicating additional opportunities for education to ensure more consistent and widespread use. Additionally, while important policy changes have been implemented to expand telehealth during COVID-19, permanent adoption of some of these changes, such as compensation for primary healthcare services, could help ensure continued access and would already be in place in the event of a new health emergency [66].

Limitations

This study has several limitations. First, missing data for participants across several independent variables necessitated imputation of missing values for the model-building process, which may have led to biased results. However, a sensitivity analysis using only participants with complete data revealed similar magnitudes of odds ratios for all variables found to have significance at p≤0.05 in the final, imputed dataset. Second, low sample size amongst participants reporting Hispanic/LatinX origin prohibited incorporation into the final model. Third, the different enrollment periods of the parent studies covered different time periods within the pandemic that may have led to forgone care. We have addressed this limitation by using models weighted by years in the pandemic at time of parent study enrollment. Fourth, because the CoVPN 5002 study did not collect participant’s address, we had to assume their residence was within the venue of enrollment. While most venues included low-income housing developments, their assigned CSA may not have been accurately reflected in the data. Fifth, because this study is based upon participant-reported survey data, it may be subject to recall bias. Finally, there is the potential for selection bias which may impact the representativeness of the Baltimore population—for example, the unemployment rate within our sample (52.3%) was higher than that of Baltimore City (29.0%), according to 2021 BNIA data [67], which may be due to greater representation of individuals from CSAs with higher unemployment rates. However, our final study sample was relatively similar across other characteristics, including sex (study sample-54.5% female; Baltimore-53.1% female), race (study sample-52.7% Black/African American; Baltimore-61.6% Black/African American), ethnicity (study sample-3.7% Hispanic; Baltimore-5.6% Hispanic), and poverty (study sample-30% with income <%25,000 per year; Balitmore-20.3% live below the federal poverty level), when comparing our study sample to 2022 Baltimore census data [42]. Additionally, the CoVPN 5002 parent study did not collect data on participant’s health insurance status, which likely impacted healthcare utilization. However, it could be inferred to be similar to the 2022 census data (approximately 6.4% of population under age 65 years is uninsured in Baltimore) [42].

Conclusions

Despite these limitations, this study has important implications for addressing unmet healthcare needs that have occurred during the pandemic. However, additional research will be critical to further understanding how healthcare access and utilization practices differed during versus before the pandemic and to more clearly identify those who are at highest risk of the downstream consequences of forgone care. It is also critical that the health system identify ways to address any new barriers to care that under-resourced populations might be experiencing, such as the increased use of mobile health services or telemedicine platforms. Additionally, moving forward, public health emergency preparedness planning efforts must account for the unique needs of these communities during crises, to ensure that their health needs can continue to be met. This will require additional studies that seek to better understand the perspectives and experiences of these populations during the pandemic, and how models of care can be adapted to serve them better in the future.

Supporting information

Acknowledgments

We would like to acknowledge the participants of the C-Forward and CoVPN 5002 studies for their participation. We would also like to acknowledge the team at the Center for Infectious Disease and Nursing Innovation for their tireless work in recruitment of participants in both of the parent protocols.

References

  1. 1. Roy CM, Bollman EB, Carson LM, Northrop AJ, Jackson EF, Moresky RT. Assessing the indirect effects of COVID-19 on healthcare delivery, utilization and health outcomes: a scoping review. Eur J Public Health [Internet]. 2021 Mar 23 [cited 2021 Jun 21];(ckab047). Available from: pmid:33755130
  2. 2. McGough M, Amin K, Cox C. Peterson-KFF Health System Tracker. 2023 [cited 2023 Mar 22]. How has healthcare utilization changed since the pandemic? Available from: https://www.healthsystemtracker.org/chart-collection/how-has-healthcare-utilization-changed-since-the-pandemic/
  3. 3. Stokes AC, Lundberg DJ, Bor J, Elo IT, Hempstead K, Preston SH. Association of Health Care Factors With Excess Deaths Not Assigned to COVID-19 in the US. JAMA Netw Open. 2021 Sep 13;4(9):e2125287. pmid:34515787
  4. 4. Birkmeyer JD, Barnato A, Birkmeyer N, Bessler R, Skinner J. The Impact Of The COVID-19 Pandemic On Hospital Admissions In The United States. Health Aff (Millwood). 2020 Nov;39(11):2010–7. pmid:32970495
  5. 5. Jeffery MMD’Onofrio G, Paek H, Platts-Mills TF, Soares WE, Hoppe JA, et al. Trends in Emergency Department Visits and Hospital Admissions in Health Care Systems in 5 States in the First Months of the COVID-19 Pandemic in the US. JAMA Intern Med. 2020 Oct;180(10):1328–33. pmid:32744612
  6. 6. Li X, Chen M, Wang Z, Si L. Forgone care among middle aged and elderly with chronic diseases in China: evidence from the China Health and Retirement Longitudinal Study Baseline Survey. BMJ Open. 2018 Mar 16;8(3):e019901. pmid:29549207
  7. 7. Röttger J, Blümel M, Köppen J, Busse R. Forgone care among chronically ill patients in Germany—Results from a cross-sectional survey with 15,565 individuals. Health Policy. 2016 Feb 1;120(2):170–8. pmid:26806678
  8. 8. Bertoldo J, Wolfson JA, Sundermeir SM, Edwards J, Gibson D, Agarwal S, et al. Food Insecurity and Delayed or Forgone Medical Care During the COVID-19 Pandemic. Am J Public Health. 2022 May;112(5):776–85. pmid:35417213
  9. 9. Anderson KE, McGinty EE, Presskreischer R, Barry CL. Reports of Forgone Medical Care Among US Adults During the Initial Phase of the COVID-19 Pandemic. JAMA Netw Open. 2021 Jan 4;4(1):e2034882. pmid:33475757
  10. 10. Czeisler MÉ. Delay or Avoidance of Medical Care Because of COVID-19–Related Concerns—United States, June 2020. MMWR Morb Mortal Wkly Rep [Internet]. 2020 [cited 2021 Jun 21];69. Available from: https://www.cdc.gov/mmwr/volumes/69/wr/mm6936a4.htm pmid:32915166
  11. 11. Giannouchos TV, Brooks JM, Andreyeva E, Ukert B. Frequency and factors associated with foregone and delayed medical care due to COVID-19 among nonelderly US adults from August to December 2020. J Eval Clin Pract. 2022;28(1):33–42. pmid:34910347
  12. 12. Park S, Stimpson JP. Trends in Self-reported Forgone Medical Care Among Medicare Beneficiaries During the COVID-19 Pandemic. JAMA Health Forum. 2021 Dec 30;2(12):e214299. pmid:35977302
  13. 13. Zhang D, Li G, Shi L, Martin E, Chen Z, Li J, et al. Association between racial discrimination and delayed or forgone care amid the COVID-19 pandemic. Prev Med. 2022 Sep;162:107153. pmid:35810933
  14. 14. Gonzalez D, Karpman M, Kenney G, Zuckerman S. Delayed and Forgone Health Care for Nonelderly Adults during the COVID-19 Pandemic [Internet]. Urban Institute; 2021 Feb. Available from: https://www.urban.org/research/publication/delayed-and-forgone-health-care-nonelderly-adults-during-covid-19-pandemic
  15. 15. Xie Z, Hong YR, Tanner R, Marlow NM. People With Functional Disability and Access to Health Care During the COVID-19 Pandemic: A US Population-based Study. Med Care. 2023 Feb 1;61(2):58–66. pmid:36040096
  16. 16. Tsuzaki K. Forgone Health Care for Non–COVID-19–Related Needs Among Medicare Beneficiaries During the COVID-19 Pandemic, Summer 2020–Winter 2021. Prev Chronic Dis [Internet]. 2022 [cited 2023 Apr 6];19. Available from: https://www.cdc.gov/pcd/issues/2022/22_0110.htm pmid:36227851
  17. 17. Werner P, Tur-Sinai A, AboJabel H. Examining Dementia Family Caregivers’ Forgone Care for General Practitioners and Medical Specialists during a COVID-19 Lockdown. Int J Environ Res Public Health. 2021 Apr 1;18(7):3688. pmid:33916152
  18. 18. Findling MG, Blendon RJ, Benson JM. Delayed Care with Harmful Health Consequences—Reported Experiences from National Surveys During Coronavirus Disease 2019. JAMA Health Forum. 2020 Dec 14;1(12):e201463. pmid:36218479
  19. 19. Tabler J, Schmitz RM, Charak R, Propst A. Forgone Care among LGBTQ and Non-LGBTQ Americans during the COVID-19 Pandemic: The Role of Health, Social Support, and Pandemic-Related Stress. South Med J. 2022 Oct;115(10):752–9. pmid:36191911
  20. 20. Mohnen SM, Schneider S, Droomers M. Neighborhood characteristics as determinants of healthcare utilization–a theoretical model. Health Econ Rev. 2019 Mar 6;9(1):7. pmid:30840211
  21. 21. Samantha Artiga EHP. Beyond Health Care: The Role of Social Determinants in Promoting Health and Health Equity [Internet]. Kaiser Family Foundation. 2018 [cited 2022 May 27]. Available from: https://www.kff.org/racial-equity-and-health-policy/issue-brief/beyond-health-care-the-role-of-social-determinants-in-promoting-health-and-health-equity/
  22. 22. Okoye SM, Mulcahy JF, Fabius CD, Burgdorf JG, Wolff JL. Neighborhood Broadband and Use of Telehealth Among Older Adults: Cross-sectional Study of National Survey Data Linked With Census Data. J Med Internet Res. 2021 Jun 14;23(6):e26242. pmid:34125071
  23. 23. Wilcock AD, Rose S, Busch AB, Huskamp HA, Uscher-Pines L, Landon B, et al. Association Between Broadband Internet Availability and Telemedicine Use. JAMA Intern Med. 2019 Nov 1;179(11):1580–2. pmid:31355849
  24. 24. Gaskin DJ, Dinwiddie GY, Chan KS, McCleary R. Residential Segregation and Disparities in Healthcare Services Utilization. Med Care Res Rev. 2012 Apr;69(2):158–75. pmid:21976416
  25. 25. Singh GK. Area Deprivation and Widening Inequalities in US Mortality, 1969–1998. Am J Public Health. 2003 Jul;93(7):1137–43. pmid:12835199
  26. 26. Chaiyachati KH, Hom JK, Hubbard RA, Wong C, Grande D. Evaluating the association between the built environment and primary care access for new Medicaid enrollees in an urban environment using Walk and Transit Scores. Prev Med Rep. 2017 Dec 8;9:24–8. pmid:29276668
  27. 27. Kakolyris A, DelaCruz JJ, Giannikos CI. COVID-19, Race/Ethnicity, and Age: the Role of Telemedicine to Close the Gaps on Health Disparities. J Econ Race Policy. 2021 Aug 26;1–11.
  28. 28. Chen JT, Krieger N. Revealing the Unequal Burden of COVID-19 by Income, Race/Ethnicity, and Household Crowding: US County Versus Zip Code Analyses. J Public Health Manag Pract JPHMP. 2021 Feb;27 Suppl 1, COVID-19 and Public Health: Looking Back, Moving Forward:S43–56. pmid:32956299
  29. 29. Lieberman-Cribbin W, Alpert N, Flores R, Taioli E. Analyzing disparities in COVID-19 testing trends according to risk for COVID-19 severity across New York City. BMC Public Health. 2021 Sep 21;21(1):1717. pmid:34548041
  30. 30. Hendricks B, Paul R, Smith C, Wen S, Kimble W, Amjad A, et al. Coronavirus testing disparities associated with community level deprivation, racial inequalities, and food insecurity in West Virginia. Ann Epidemiol. 2021 Jul 1;59:44–9. pmid:33812965
  31. 31. Servick Kelly. ‘Huge hole’ in COVID-19 testing data makes it harder to study racial disparities. Science [Internet]. 2020 Jul 10 [cited 2022 May 27]; Available from: https://www.science.org/content/article/huge-hole-covid-19-testing-data-makes-it-harder-study-racial-disparities
  32. 32. Gerberding JL. Measuring Pandemic Impact: Vital Signs From Vital Statistics. Ann Intern Med. 2020 Dec 15;173(12):1022–3. pmid:32915634
  33. 33. Rossen LM. Excess Deaths Associated with COVID-19, by Age and Race and Ethnicity—United States, January 26–October 3, 2020. MMWR Morb Mortal Wkly Rep [Internet]. 2020 [cited 2021 Feb 13];69. Available from: https://www.cdc.gov/mmwr/volumes/69/wr/mm6942e2.htm
  34. 34. Bakouny Z, Paciotti M, Schmidt AL, Lipsitz SR, Choueiri TK, Trinh QD. Cancer Screening Tests and Cancer Diagnoses During the COVID-19 Pandemic. JAMA Oncol. 2021 Mar 1;7(3):458–60. pmid:33443549
  35. 35. Messac L, Knopov A, Horton M. Delayed care-seeking for non-COVID illnesses in Rhode Island. R I Med J 2013. 2020 May 1;103(4):10–1. pmid:32357585
  36. 36. Garcia S, Albaghdadi MS, Meraj PM, Schmidt C, Garberich R, Jaffer FA, et al. Reduction in ST-Segment Elevation Cardiac Catheterization Laboratory Activations in the United States During COVID-19 Pandemic. J Am Coll Cardiol. 2020 Jun 9;75(22):2871–2. pmid:32283124
  37. 37. Friedman AB, Barfield D, David G, Diller T, Gunnarson C, Liu M, et al. Delayed emergencies: The composition and magnitude of non‐respiratory emergency department visits during the COVID‐19 pandemic. J Am Coll Emerg Physicians Open. 2021 Jan 14;2(1):e12349. pmid:33490998
  38. 38. Woolf SH, Chapman DA, Sabo RT, Weinberger DM, Hill L. Excess Deaths From COVID-19 and Other Causes, March-April 2020. JAMA. 2020 Aug 4;324(5):510–3. pmid:32609307
  39. 39. Zangeneh SZ, Skalland T, Yuhas K, Emel L, Tapsoba JDD, Reed D, et al. Adaptive Time Location Sampling for Compass, a Sars-Cov-2 Prevalence Study in Fifteen Diverse Communities in the United States [Internet]. medRxiv; 2023 [cited 2023 May 1]. p. 2023.01.10.23284400. Available from: https://www.medrxiv.org/content/10.1101/2023.01.10.23284400v1 pmid:36711739
  40. 40. Gelberg L, Andersen RM, Leake BD. The Behavioral Model for Vulnerable Populations: application to medical care use and outcomes for homeless people. Health Serv Res. 2000 Feb;34(6):1273–302. pmid:10654830
  41. 41. Andersen R, Newman JF. Societal and Individual Determinants of Medical Care Utilization in the United States. Milbank Q. 2005;83(4):Online-only-Online-only.
  42. 42. U.S. Census Bureau QuickFacts: Baltimore city, Maryland (County) [Internet]. [cited 2021 Jul 22]. Available from: https://www.census.gov/quickfacts/fact/table/baltimorecitymarylandcounty/AGE295219
  43. 43. Health Care for the Homeless—Baltimore and Maryland [Internet]. 2015 [cited 2022 Jun 6]. Homelessness in Baltimore. Available from: https://www.hchmd.org/homelessness-baltimore
  44. 44. Wen LS, Warren KE. Combatting the opioid epidemic: Baltimore’s experience and lessons learned. J Public Health Oxf Engl. 2018 Jun;40(2):e107–11. pmid:28985362
  45. 45. Fenton Justin, Costello Darcy, Prudente Tim. Baltimore Sun. 2022 [cited 2022 Jun 6]. Baltimore tried new ways to stop the violence in 2021, but homicides and shootings remain frustratingly high and consistent. Available from: https://www.baltimoresun.com/news/crime/bs-md-ci-cr-year-end-violence-20211231-nhiw6lykgzbofginwehe57cohq-story.html
  46. 46. Baltimore City Health Department [Internet]. 2014 [cited 2022 May 27]. Neighborhood Health Profiles—Frequently Asked Questions. Available from: https://health.baltimorecity.gov/node/231
  47. 47. Frequently Asked Questions (FAQs) | BNIA–Baltimore Neighborhood Indicators Alliance [Internet]. [cited 2022 Sep 11]. Available from: https://bniajfi.org/faqs/
  48. 48. BNIA–Baltimore Neighborhood Indicators Alliance | Providing reliable, actionable quality of life indicators for Baltimore’s neighborhoods [Internet]. [cited 2024 Feb 19]. Available from: https://bniajfi.org/
  49. 49. Kind AJH, Buckingham WR. Making Neighborhood-Disadvantage Metrics Accessible—The Neighborhood Atlas. N Engl J Med. 2018 Jun 28;378(26):2456–8. pmid:29949490
  50. 50. Kind AJ, Jencks S, Brock J, Yu M, Bartels C, Ehlenbach W, et al. Neighborhood Socioeconomic Disadvantage and 30 Day Rehospitalizations: An Analysis of Medicare Data. Ann Intern Med. 2014 Dec 2;161(11):765–74.
  51. 51. Schafer JL. Multiple imputation: a primer. Stat Methods Med Res. 1999 Mar;8(1):3–15. pmid:10347857
  52. 52. Mueller M, Purnell TS, Mensah GA, Cooper LA. Reducing racial and ethnic disparities in hypertension prevention and control: what will it take to translate research into practice and policy? Am J Hypertens. 2015 Jun;28(6):699–716. pmid:25498998
  53. 53. Perman J. The Greatest Gap: Health Inequity in Baltimore [Internet]. University of Maryland, Baltimore; 2016 Jun. Available from: https://www.umaryland.edu/media/media-archive/Health-Equity-June-2016.pdf
  54. 54. Clark CR, Ommerborn MJ, A Coull B, Pham DQ, Haas JS. Income Inequities and Medicaid Expansion are Related to Racial and Ethnic Disparities in Delayed or Forgone Care Due to Cost. Med Care. 2016 Jun;54(6):555–61. pmid:26974677
  55. 55. Parker K, Minkin R, Bennett . Pew Research Center’s Social & Demographic Trends Project. 2020 [cited 2023 Apr 19]. Economic Fallout From COVID-19 Continues To Hit Lower-Income Americans the Hardest. Available from: https://www.pewresearch.org/social-trends/2020/09/24/economic-fallout-from-covid-19-continues-to-hit-lower-income-americans-the-hardest/
  56. 56. Tolbert J, Ammula M. 10 Things to Know About the Unwinding of the Medicaid Continuous Enrollment Provision [Internet]. KFF. 2023 [cited 2023 Apr 10]. Available from: https://www.kff.org/medicaid/issue-brief/10-things-to-know-about-the-unwinding-of-the-medicaid-continuous-enrollment-provision/
  57. 57. U.S. Department of the Treasury [Internet]. 2023 [cited 2023 Jul 31]. Assistance for American Families and Workers. Available from: https://home.treasury.gov/policy-issues/coronavirus/assistance-for-American-families-and-workers
  58. 58. Saloner B, Gollust SE, Planalp C, Blewett LA. Access and enrollment in safety net programs in the wake of COVID-19: A national cross-sectional survey. PLoS ONE. 2020 Oct 6;15(10):e0240080. pmid:33022013
  59. 59. Barroso A, Horowitz JM. Pew Research Center. [cited 2023 Jun 14]. The pandemic has highlighted many challenges for mothers, but they aren’t necessarily new. Available from: https://www.pewresearch.org/short-reads/2021/03/17/the-pandemic-has-highlighted-many-challenges-for-mothers-but-they-arent-necessarily-new/
  60. 60. Vaidya V, Partha G, Karmakar M. Gender differences in utilization of preventive care services in the United States. J Womens Health 2002. 2012 Feb;21(2):140–5. pmid:22081983
  61. 61. Jones EB. Unmet Need for Medical Care: The Role of Mental Health Status. Community Ment Health J. 2021 Jan;57(1):121–7. pmid:32303934
  62. 62. Yu SWY, Hill C, Ricks ML, Bennet J, Oriol NE. The scope and impact of mobile health clinics in the United States: a literature review. Int J Equity Health. 2017 Oct 5;16:178. pmid:28982362
  63. 63. Heidari OO’Conor KJ, Meyer D, Cargill V, Lowensen K, Farley JE. High Primary COVID-19 Vaccine Series Completion by People Who Inject Drugs When Colocating Services at a Syringe Services Van. J Addict Med. 2023 Mar 10;
  64. 64. Augenstein J. Opportunities To Expand Telehealth Use Amid The Coronavirus Pandemic. Health Aff Forefr [Internet]. [cited 2023 Apr 10]; Available from: https://www.healthaffairs.org/do/10.1377/forefront.20200315.319008/full/
  65. 65. Auchus IC, Jaradeh K, Tang A, Marzan J, Boslett B. Transitioning to Telehealth During the COVID-19 Pandemic: Patient Perspectives and Attendance at an HIV Clinic in San Francisco. AIDS Patient Care STDs. 2021 Jul;35(7):249–54. pmid:34242090
  66. 66. North S. These Four Telehealth Changes Should Stay, Even After the Pandemic. Fam Pract Manag. 2021 May;28(3):9–11. pmid:33973756
  67. 67. Vital Signs Indicator: Percent Population 16–64 Not in Labor Force [Internet]. [cited 2024 Feb 19]. Available from: https://bniajfi.org/indicators/Workforce%20And%20Economic%20Development/nilf/2021