Portions of this manuscript have been previously presented at the American Congress of Rehabilitation Medicine:

Carlozzi, N.E., Kallen, M.A., Moldovan, I., Bragg, A., Howard, J.M., Miner, J.A., Graves, C., & Mitchell, S.M. (2021, September). The Re-Engineered Discharge for Diabetes Computer Adaptive Test (REDD-CAT): New patient-reported outcome measures that evaluate social determinants of health for persons with type 2 diabetes mellitus. Invited oral poster presentation accepted for presentation at the American Congress of Rehabilitation Medicine (ACRM); virtual meeting (due to COVID-19 pandemic).

The relationship between housing and health has been well-established in public health research with housing conditions, stability, affordability, and location as key areas for intervention.1 The Shaw conceptual model of housing and health further illustrates that hard factors (physical or material) and soft factors (social or meaningful) can both directly and indirectly impact health.2 Specifically, housing conditions, housing structure, and homelessness (hard factors), and individual perceptions, impact of poor housing on mental health, worries about affordable housing, and feelings of security (soft factors) directly impact health.2 Additionally, income, wealth, and proximity to services and facilities (hard factors), as well as household culture (soft factors), can indirectly impact health. At the neighborhood level, availability of local services/facilities and features of the environment (hard factors), as well as community culture and cohesion (soft factors), also indirectly impact health.2

While previous research has established that lack of or insufficient housing can have a deleterious impact on health, our ability to quantify this impact is limited by the lack of a consensus definition for housing insecurity. Definitions of housing insecurity can include any or all of the following components: housing stability, housing affordability, housing quality, housing safety, and/or homelessness.3 As such, the measures that have been used to assess housing security are equally varied. In fact, a recent systematic review identified 14 different screening tools that varied in terms of concepts measured, number of items, and psychometric rigor (for a detailed review of these measures, see 3). No measures were identified that exhibited comprehensive reliability and validity data, and most studies only employed a single item to assess housing, which, by design, only focuses on a single aspect of housing and has limited response option variability.

Despite lacking a consensus definition and a consensus measure, housing insecurity remains a key social determinant of health.1,2 Census-based estimates indicate that more than 25% of households are cost burdened (i.e., at least 30% of their household income goes toward rent),4 with even higher rates of individuals indicating concerns about housing affordability (rates range between 59.9 and 72.8% by state).5 Housing insecurity is also associated with increased risk for physical and mental health problems6,7,8,9,10,11 and with poor access to ambulatory care and high rates of acute care.12 People with type 2 diabetes mellitus (T2DM) are among those populations with high rates of housing unaffordability and housing insecurity.12,13,14,15,16 In T2DM, these high rates of housing insecurity are disruptive to the self-management routines that are critical to the successful treatment and maintenance of the disease,17,18 and they increase the risk for poor health outcomes (including elevated blood glucose levels, high low-density lipoprotein cholesterol levels, high blood pressure, overall worse health-related quality of life).14,19,20,21,22,23 Not surprisingly, housing insecurity in T2DM is also costly, with associated elevations in diabetes-related emergency department use, hospitalization, and outpatient visits15,23,24,25,26.

We aimed to develop and validate a new comprehensive patient-reported outcome measure (PRO) of housing insecurity that could be used to better understand the impact that housing has on health outcomes at the point of discharge from the hospital. We operationalized housing insecurity as a multidimensional concept that includes housing instability, housing affordability, housing safety, housing quality, neighborhood quality, and homelessness, all of which disproportionally impact low-income families in the USA.12 Specifically, this new measure was developed as part of a larger study focused on developing and validating measures that capture important social determinants of behavior.27,28,29 We tested this measure in people with T2DM, although we would expect this measure to have broad applicability across a number of diverse clinical populations This report describes the development of this new PRO, the Re-Engineered Discharge for Diabetes Computer Adaptive Test (REDD-CAT) Housing Security measure, and provides preliminary score reliability and validity data.

METHODS

Study Participants

A total of 225 individuals with T2DM participated in this study. Study eligibility included a clinical diagnosis of T2DM, > 18 years of age, ability to communicate in English, and ability to provide informed consent. In order to complete the PROs independently, participants had to have at least a 5th grade reading level, which was confirmed by correctly pronouncing the first 10 words on the Wide Range Achievement Test 4th Edition (WRAT4) Reading Subtest;30 participants with errors on the first 10 words were provided assistance with reading survey items. Study participants were recruited from the Boston Medical Center (BMC) healthcare system using three different screening approaches: (1) weekly lists were generated of diabetes patients with upcoming outpatient appointments using BMC’s Clinical Data Warehouse; (2) the electronic health record was queried to generate inpatient census reports of inpatients with T2DM; and (3) participants that had agreed to be contacted for future research studies (as part of the consent process for a previous T2DM research study) were contacted. Data were collected in accordance with the local institutional review board, and study participants provided informed consent prior to their participation in this study.

Measures

The REDD-CAT Housing Security PRO

This new PRO was developed using published measurement development standards31 that include both qualitative and quantitative methodology (e.g., classical test theory and item response theory).32,33,34 Literature reviews and semi-structured interviews informed the development of the Housing Security items. Specifically, semi-structured interviews were conducted with 37 patients admitted to a general medical hospital service to identify patient-reported aspects of readmission risk (https://www.pcori.org/research-results/2012/pilot-project-figuring-out-which-patients-are-likely-return-hospital and Cancino et al., 201435). This work identified 15 themes related to readmission risk including housing insecurity (housing instability, housing affordability, housing safety, housing quality, neighborhood quality, and homelessness) that served as the basis for Housing Security items. Item writing relied considerably on the content from the semi-structured interviews; verbatim quotations from the patient interviews were used to develop this content when available. Item writing was iterative and included cognitive interviews with patients with a recent hospital readmission, expert review (i.e., professional inpatient providers; PRO development experts), reading level review (reading level does not exceed the 6th grade), and translatability review (to inform future adaptation into languages other than English). See Fig. 1. The final Housing Security PRO has 22 items and includes three broad subdomains (housing safety [3-item index], housing affordability [6-item scale], and housing features [8-item index]). There are also five stand-alone single items (post-discharge housing status; anxiety about affording housing; safety: domestic violence; anxiety about where will sleep next; and reason(s) staying with family/friends/at a shelter). Scores include a Housing Affordability T-score (M = 50, SD = 10; higher scores indicate less worry about affordability) and Housing Safety and Housing Features index scores (summed scores divided by the number of completed items in the index; higher scores indicate more safety and more available housing features, respectively). The “anxiety about affording housing,” “domestic violence,” and “anxiety about where will sleep next” single items provide scores ranging from 1 to 5, with higher scores indicating less anxiety about affording housing, more personal safety, and less anxiety about where will sleep next.

Figure 1
figure 1

Iterative process for item pool development.

Full size image

Validity Measures

The 8-item Economic Quality of Life (Econ-QOL)36 was used to evaluate perceived economic and financial security. This measure is scored on a T-score metric (M = 50; SD = 10); higher scores indicate worse perceived economic quality of life. We examined those with “better” (scores ≤ 40) versus “worse” (scores ≥ 60) economic quality of life.

PROMIS Severity of Substance Use37,38 provided a measure of self-reported severity of substance use in the last 30 days, Neuro-QoL Anxiety39,40 provided a measure of perceived worry, fear, and hyperarousal, and PROMIS Pain Interference provided a measure of the effects of pain on a range of daily life activities. These measures are scored on a T-score metric (M = 50; SD = 10); higher scores indicate worse severity, anxiety, and pain interference, respectively.

We also collected demographic data and several medical record variables including questions about health insurance and finances.

Data Capture

The study PROs were administered using a HIPPA-compliant electronic data capture system; study participants used either a personal internet-enabled device or a study-specific tablet or laptop to provide item responses.

Sample Size Justification

Sample size requirements were based on the planned graded response model (GRM) analysis and differential item functioning (DIF) analyses that were conducted as a part of the PRO development (specific analyses described below). The proposed GRM analysis required a minimum sample size of N = 200 to generate stable calibration estimates for the PRO.41,42 In addition, Wald-2-based DIF analyses required at least n = 100 participants per subgroup for the proposed demographic comparisons.43

Statistical Analyses

Housing Affordability Scale Development

The Housing Affordability scale was developed using an iterative process that included classical test theory and item response theory approaches, as well as clinical input.32,33,34 Exploratory and confirmatory factor analyses (EFA, CFA; using Mplus software version 7.444) were used to examine scale dimensionality. EFA was used to help identify the number of potential scale subdomains. Items with sparse cells (response categories with n < 5 respondents), low item-adjusted total score correlations (r < 0.40), or items that were non-monotonic (Testgraf Software45) were candidates for exclusion. In CFA analyses, items were candidates for exclusion if they exhibited: low factor loadings (lx < 0.50), or local dependence (residual correlations > 0.20 or a correlated error modification index ≥ 100).32,33,34 CFA was used to confirm the factor structure of potential item sets. Fit criteria were as follows: comparative fit index (CFI) ≥ 0.90; Tucker-Lewis index (TLI) ≥ 0.90; and root mean square error of approximation (RMSEA) < 0.15.46,47,48,49

Next, a constrained, common-slope graded response model (GRM)50 was used to identify items with significant misfit (S-X2 /df effect size > 3)51. This was followed by an investigation for potential differential item functioning (DIF). Items exhibiting meaningful DIF (p < 0.01 group-specific item parameter difference, plus > 2% of DIF-corrected vs. uncorrected score differences exceeding individual case uncorrected score standard errors) were candidates for exclusion. DIF was examined for age (< 60 vs. ≥ 60 years), sex (male vs. female), education (≤ high school vs. > high school), and socioeconomic status (“have enough income to pay rent/mortgage” and “can afford to pay bills on time,” both categorized as never/rarely/sometimes vs. usually/always). Analyses were conducted using IRTPRO software version 3.1.2.52 CFA was also used to confirm the factor structure of the final item set.

Housing Safety and Housing Features Index Development

The Housing Safety and Housing Features indices were developed using an iterative process focused primarily on classical test theory. As with the Housing Affordability scale, EFA and CFA were used to examine dimensionality, following the analytic process outlined above. The multi-dimensional nature of item content precluded a typical unidimensional scale development strategy; the indices, instead, were designed to provide a “count” and an “extent to which” different safety elements and housing features are present.

Preliminary Reliability and Validity Analyses: Housing Affordability

Given that the item response data were normally distributed, we proceeded with parametric analyses. Scores on the new Housing Affordability PRO were examined. Cronbach’s alpha was used to estimate internal consistency reliability (a priori criterion specified as ≥ 0.70).53 Floor and ceiling effects were also examined (a priori criterion specified as ≤ 20%).54,55

Convergent and discriminant validities were examined using Pearson correlations. Convergent validity would be supported by moderate correlations (r’s ≥ 0.36–0.67) between Housing Affordability and Economic Quality of Life.56 Discriminant validity would be supported by less robust or “low” correlations (r’s ≤ 0.35) between Housing Affordability and Neuro-QoL Anxiety, and negligible correlations between Housing Affordability and PROMIS Pain Interference.56

Independent sample t-tests were used to examine known-groups validity. Known-groups categories included those indicating they have enough money to make ends meet versus those who do not have enough money to make ends meet. We expected those who report that they do not have enough money to make ends meet would report more housing insecurity than those with enough money to make ends meet.

In addition, logistic regression models were conducted to determine if Housing Security was related to readmission risk in our sample (high risk was defined as ≥ 2 inpatient admissions in the past 6 months; low risk was defined as ≤ 1 inpatient admission in the past 6 months). These models included patient demographic variables, objective medical record data, existing hospital risk assessment data, and PRO measure scores.

Preliminary Descriptive and Validity Analyses: Housing Safety and Housing Features

Given that Housing Safety and Housing Features scores are count- and extent-based in nature and thus non-normally distributed, we proceeded with nonparametric analyses. Scores on these two new Housing Security indices were examined, with minimum, maximum, median, and modal observed scores reported. The percent of cases with minimum and maximum possible scores was also reported.

Convergent and discriminant validities were examined using Spearman correlations. Convergent validity would be supported by moderate correlations (r’s ≥ 0.36–0.67) between Housing Safety and Housing Features vs. Severity of Substance Use.56 Discriminant validity would be supported by low correlations (r’s ≤ 0.35) between Housing Safety and Housing Features vs. Neuro-QoL Anxiety, as well as negligible correlations between Housing Safety and Housing Features vs. PROMIS Pain Interference.

Mann–Whitney U tests were used to examine known-groups validity. Known-groups categories included (1) those with enough money to make ends meet versus those without enough money to make ends meet; and (2) “worse” vs. “better” economic quality of life. We expected those without enough money to make ends meet to report less housing safety and less housing features than those with enough money to make ends meet; we also expected those with “worse” economic quality of life to report less housing safety and less housing features versus those with “better” economic quality of life.57

Preliminary Descriptive Analyses: Anxiety About Affording Housing, Domestic Violence, and Anxiety About Where Will Sleep Next

Given that there were only single items each to assess “anxiety about affording housing,” “domestic violence,” and “anxiety about where will sleep next,” we reported each item’s minimum, maximum, median, and modal observed scores, as well as the percent of cases with minimum and maximum possible item scores.

RESULTS

Study Participants

Table 1 provides detailed descriptive data for both the sample and the different PROs that were administered as a part of this study. Figure 2 includes a final.pdf version as well as scoring criteria for the REDD-CAT Housing Security.

Table 1 Descriptive Data for Study Participants and PRO Data
Full size table
Figure 2
figure 2figure 2figure 2figure 2figure 2

The final.pdf version as well as scoring criteria for the REDD-CAT Housing Security.

Full size image

Housing Affordability Scale

Table 2 provides a summary of the analyses that supported the retention of the six final items in the Housing Affordability scale. Thresholds ranged from –1.73 to + 1.78 for the subdomain. There was good information (i.e., test information ≥ 5.0) for T-scores between 31 and 66 and acceptable information (test information ≥ 3.3) for T-scores between 26 and 71; marginal reliability was 0.84 for the 6-item measure. Table 3 provides the final characteristics for the new subdomain.

Table 2 Unidimensional Modeling and Analyses for the REDD-CAT Housing Affordability Subdomain
Full size table
Table 3 Final Overall Model Fit for the REDD-CAT Housing Affordability Subdomain
Full size table

Housing Safety and Housing Features Indices Development

The Housing Safety subdomain is evaluated by a 3-item index, while the Housing Features subdomain is assessed by an 8-item index. The Housing Features index is generalized across differing discharge housing locations by referring respondents to, for example, “When I am discharged, the place I will go to.” Housing Safety and Housing Features scores are average item scores, which facilitate scoring and score interpretation when missing items occur.

Reliability and Validity Analyses

Housing Affordability

Internal consistency reliability of the REDD-CAT Housing Affordability subdomain was good, and it was devoid of floor and ceiling effects (Table 4). Convergent validity was supported (Table 5). Those persons with enough money to make ends meet (versus not enough money) reported less housing insecurity, supporting known-groups validity (Table 6). Logistic regression models indicated that Housing Affordability was one of seven variables that was related to readmission risk (i.e., the observed correlation of ≥  ± 0.15 with our risk level-dependent variable).

Table 4 Descriptive Data for the new REDD-CAT Housing Security PROs
Full size table
Table 5 Convergent and Discriminant Validity for the new REDD-CAT Housing Security PROs
Full size table
Table 6 Known-Groups Validity for New REDD-CAT Housing Security PROs
Full size table

Housing Safety and Housing Features

Score distributions and the percentages of individuals with the highest and lowest scores are provided in Table 4. Convergent validity and discriminant validity was supported (Table 5). There was a trend (p < 0.07) for a significant difference between those with enough money to make ends meet to report less concern with housing safety relative to those without enough money to make ends meets (Table 6). All other known-groups analyses had results as were hypothesized (Table 6).

Anxiety About Affording Housing, Domestic Violence, and Anxiety About Where Will Sleep Next

Score distributions and the percentages of individuals with the highest and lowest scores are provided in Table 4 for these three single-item assessments.

DISCUSSION

This report describes the development of a new measure designed to capture important components of perceived Housing Security. This new measure, the REDD-CAT Housing Security PRO, captures three critical components of housing insecurity: Housing Affordability (6-item scale), Housing Safety (3-item index), and Housing Features (8-item index). To our knowledge, this PRO represents the first comprehensive measurement system of housing insecurity; it is both reliable and has good scale characteristics. There was also support for convergent and discriminant validity, as well as known-groups validity. Taken together, this new PRO has demonstrated that it is psychometrically sound; that is, the majority of our psychometric findings were in accordance with a priori hypotheses.58

The new Housing Security PRO includes two discharge housing location items, three distinct scale and index scores, and three single-item assessments. The first discharge housing location item, “When I am discharged, I will go to” has five response options, with respondent instructions to choose the best option: “my own home/family residence,” “a friend’s home/residence,” “a shelter,” “I do not have any options for housing or shelter,” and “I don’t know.” The second discharge housing location item, “With regard to my discharge” has nine response options, with respondent instructions to choose all options that apply; options are worded to elicit reasons for the anticipated discharge housing location (e.g., “because I need someone to take care of me” and “because I cannot afford independent housing”). Individuals that indicate a shelter, no options, or “I don’t know” should be flagged for additional follow-up to help ensure a safe housing placement at discharge.

Housing Affordability raw scores are converted to T-scores (using Table 5), with lower scores indicating more significant concerns with housing affordability. Interpretation of obtained Housing Affordability scores is as follows: (1) scores equal to or below 40 (i.e., ≤ 1 SD below the normative mean) indicate Housing Affordability concerns that exceed 83.9% of persons with T2DM; scores less than or equal to 30 (i.e., ≤ 2 SDs below the normative mean) indicate Housing Affordability concerns that are worse than 97.9% of persons with T2DM.

Housing Safety scores (i.e., the average score across completed items) range from 1 to 5, with higher scores indicating greater perceived safety. Additional follow-up is warranted for any average score that is less than 4. Housing Features scores (the average score across completed items) also range from 1 to 5, with higher scores indicating more/more consistently present features of housing. Additional follow-up is warranted for any average score that is less than 4. Finally, any response other than “never” on the Anxiety About Affording Housing, Domestic Violence, or Anxiety About Where Will Sleep Next single-item assessments warrants clinical follow-up.

While this is the first measure designed to capture multiple domains of housing security, there are also several important limitations to our study. First, there are several limitations with regard to generalizability. For example, the new PRO was developed and tested in patients with T2DM; therefore, future testing is needed to determine if this measure will exhibit clinical utility in other populations. In addition, the study sample was recruited through an urban safety-net health system; findings may not be generalizable to other types of health systems. Furthermore, there were high rates of Black/African American patients and low rates of Hispanic/Latino patients in this study; more work is needed to ensure generalizability to other racial/ethnic groups. Second, while these data provide a preliminary look at reliability and validity, future work is needed in independent samples to confirm these findings and to examine the responsiveness of scores on this measure in the context of clinical follow-up. Third, while this tool is designed to screen for individuals that have housing insecurity, it is not yet designed to generate follow-up and appropriate referrals, the value of which warrants additional study. In addition, there is ongoing work examining the feasibility and acceptability of using the comprehensive REDD-CAT measurement system as part of the discharge planning process, which should be available in late 2023.

In sum, the new REDD-CAT Housing Security PROs is a psychometrically sound measure that can be used to identify individuals that are at high risk for negative outcomes due to housing insecurity. This measure, in combination with the broader REDD-CAT measurement system, provides a valuable tool for identifying unmet needs among persons with T2DM. It is our hope that this comprehensive measurement system for capturing important social determinants of health can provide an easy way to identify individuals at high risk for negative outcomes and provide them with appropriate referrals to help mitigate these unmet needs and ultimately maximize patient health-related quality of life.