Validation of the HeLD-14 functional oral health literacy instrument in a general population

Introduction

Oral health literacy (OHL) is defined as, “the degree to which individuals have the capacity to obtain, process, and understand basic oral health information and services needed to make appropriate health decisions and act on them” (US Department of Health Human Services, 2000). Estimates of OHL prevalence are equated with general health literacy, estimating that only one in ten adults fully understand written health material (Institute of Medicine Committee on Health Literacy, 2004). Individuals with low general health literacy are more likely to have low OHL and are less likely to utilize preventive care, have poorer overall and oral health, and higher mortality rates compared to those with adequate health literacy (Baskaradoss, 2018; Kleinman, Horowitz & Atchison, 2021; National Institutes of Health, 2021). The recognition of the impact of OHL on health outcomes is reflected in recent national initiatives to measure oral health literacy. Health literacy is a central focus of Healthy People 2030 (Office of Disease Prevention and Health Promotion, 2021). One of the initiative’s overarching goals is to, “Eliminate health disparities, achieve health equity, and attain health literacy to improve the health and well-being of all” (Office of Disease Prevention and Health Promotion, 2021). Similarly, the Strategic Research Plan of the National Institute of Dental and Craniofacial Research includes the objective to, “Accelerate research on best practices of …health literacy that overcome obstacles and improve behaviors and practices” (National Center for Dental Craniofacial Research, 2021). Evaluating these proposed improvements in OHL among the American population requires designing and testing valid measurement instruments.

Researchers have developed several OHL instruments, the majority of which are based on two proposed constructs: reading and numeracy (Dickson-Swift et al., 2014). Subsequent instruments proposed additional constructs hypothesized to better characterize functional OHL, consisting of four to seven constructs. One such instrument is the 29-item Health Literacy in Dentistry (HeLD-29) scale, initially designed to measure OHL in an Australian Aboriginal population (Jones et al., 2014). Adapted from the Health Literacy Measurement Scale (HeLMS) (Jordan, Buchbinder & Osborne, 2009), seven constructs were hypothesized described as representing receptivity, understanding, support, economic barriers, access, communication, and utilization (Jones et al., 2014). Results of factor analysis found a unidimensional solution following exploratory factor analysis (EFA). The authors reported low, yet statistically significant, correlations between the seven latent variables, leading to confusion about the dimensionality of OHL (Jones et al., 2014). A shortened version, the HeLD-14 consisted of two items for each of the seven constructs, and was designed for settings where only a limited number of items can be administered (Jones et al., 2015). Factor analysis of the shortened version found six rather than seven of the original principal components as “utilization” dispersed onto the “communication” domain (Jones et al., 2015). A Chinese version (HeLD-C) found a unidimensional solution and validated the instrument in an adult Chinese population (Lui et al., 2021). Two subsequent studies both found six dimensions in a small sample of Thai adults with special needs, and a large population of Brazilian non-institutionalized older adults (Sermsuti-Anuwat & Piyakhunakorn, 2021; Soares et al., 2022). While the dimensionality of the HeLD remains undetermined, its’ comprehensive foundation makes it an instrument of interest in measuring functional oral health literacy. Discrepancies in dimensionality are not unusual when analyzing newer instruments. Previous analysis of another functional OHL instrument, the Oral Health Literacy-Adult Questionnaire (OHL-AQ), was found to be unidimensional despite prior publications suggesting multiple dimensions (Flynn, John & Sistani, 2018; Sabbahi, Limback & Rootman, 2009; Sistani et al., 2013). In addition to identifying construct dimensionality, studies intended to reduce the number of items in a validated instrument is standard practice to limit survey burden for participants, particularly when administering a survey in a clinical or community setting. One instrument of note is the Oral Health Impact Profile (OHIP) designed to measure Oral Health-Related Quality of Life (OHRQoL). The original version consisted of 49 items and was subsequently reduced to 14 items that increased its utility (Slade & Spender, 1994). The most recent version consists of five items and has been shown to be valid and reliable in multiple languages (John et al., 2014; John et al., 2006; Simancas-Pallares et al., 2020). In order to assure the validity and reliability of the HeLD-14, the authors intend to follow the research processes used for the OHIP-5 and other similar instruments.

The purpose of this study, therefore, was to investigate the dimensionality of and validate the HeLD-14 in a general U.S. population. The study’s research hypothesis was that the HeLD-14 is unidimensional.

Materials & Methods

Ethical approval. The study was approved by the University of Minnesota Institutional Review Board (STUDY00016028).

Results

Surveys were initiated by 651 participants at two Minnesota County (n = 63) and the State Fair (n = 588) with 631 completed surveys available for analysis. Characteristics of the study population are reported in Table 1. The majority of participants were female, White, non-Hispanic, reported completing more than four years of college, and the average age was 43.6 years.

The mean response and interquartile range (IQR) of each HeLD-14 item are reported in Table 2. All items exhibited a mean score close to the maximum score of five and a small IQR of zero or one, reflecting that most members of the surveyed population indicate high oral health literacy.

EFA results

In the exploratory dataset, we used a scree plot, parallel analysis, and the ratio of the first to second eigenvalue to determine the number of factors. In Fig. 1, the scree plot indicates that one factor accounts for most of the variance in the exploratory dataset. Parallel analysis also indicated a one factor solution, and the ratio of first to second observed eigenvalue was 12.43, a value much larger than 4 which has been proposed as a minimum indicator of unidimensionality (Reeve et al., 2007). All of these measures strongly suggest that a single factor is sufficient to account for the data on this scale. The factor loadings for a one-factor solution are shown in Table 2. All loadings are equal to or larger than 0.7 indicating a very strong relationship between these items and the latent construct of OHL. In summary, these results consistently suggest that OHL as measured by the HeLD-14 is a unidimensional and well-defined construct.

Discussion

This is the first study to evaluate the psychometric properties of the HeLD-14 in a general US population. The EFA results suggest a one-factor solution and indicate that the latent construct of OHL is a unidimensional construct. Our study is in line with Jones et al. (2014) who found a single factor solution for the HeLD-29 in an Australian indigenous population when the eigenvalue greater than one criterion was used to determine the number of factors (Jones et al., 2014). No model fit results or further explanation was provided (Jones et al., 2014). Lui et al. (2021) also reported a single-factor structure of the HeLD-C in the adult Chinese population. A better fit was found in our study compared to the HeLD-C for RMSEA (0.067 vs 0.08, respectively) as adequate fit models less than 0.07 are desirable. Similar results indicating adequate model fit were found in both our study and the HeLD-C for SRMR (0.07 vs 0.05, respectively), CFI (0.992 vs 0.95, respectively), and TLI (0.991 vs 0.94, respectively) (Lui et al., 2021).

In contrast, the validation of the HeLD-14 in English among an Australian Aboriginal English-speaking population did not report model fit analysis (Jones et al., 2015), thus compromising comparisons with our results. The authors conceptualized a seven-dimensional structure, and found that six factors loaded onto the model as expected, but found that two items dispersing onto another single factor (Jones et al., 2015). The Brazilian version of the HeLD-14 administered to 535 elderly adults found the same six dimensions as (Soares et al., 2022) using psychometric network analysis, but only after modification of the original hypothesized model (Jones et al., 2015). However, adequate fit results were reported (χ² = 241.01, p < 0.001; CFI =0.97; RMSEA =0.064) (Soares et al., 2022). An exploratory factor analysis of the Thai version (HeLD-Th) among a small group of adults with physical disabilities (n = 160) extracted six intercorrelated factors with eigenvalues greater than one. As all communalities were more than 0.4, all seven dimensions were accepted (Sermsuti-Anuwat & Pongpanich, 2019).

Discrepancies in findings may be due, in part, to various analytic approaches used to procure results, or lack thereof. Soares et al. (2022) used a novel network analysis approach and confirmed the six-factor result reported by Jones et al. (2015). However, it is not clear how this approach identified the number of factors and how it differs from traditional EFA and CFA. Further, the initial analysis did not yield the hypothesized structure, and modifications were made to yield the directed results  (Soares et al., 2022). Sermsuti-Anuwat & Pongpanich (2019) reported six dimensions for the HeLD-Th version, although discrepancies were found in factor loadings. The HeLD-Th reported that the constructs of “Understanding” and “Access” loaded onto the same factor unlike “Communication” and “Utilization” loading onto the same factor in the Brazilian study (Soares et al., 2022; Sermsuti-Anuwat & Pongpanich, 2019).

While Lui et al. (2021) used exploratory structural equation modeling, they limited their analysis to 13 items as one payment-related item was not relevant to the Chinese population (Nunnally, 1978). Regardless, their results were similar to this study’s and used similar analytic methods. The elimination of one item would not affect the outcome when loading onto a unidimensional model (Hu & Bentler, 1999).

The previous validation studies were conducted in diverse populations: lower income Australian Aboriginals, wealthy Brazilians, Thai adults with developmental disabilities, and dental outpatients in a Chinese hospital (Jones et al., 2014; Jones et al., 2015; Lui et al., 2021; Soares et al., 2022; Sermsuti-Anuwat & Pongpanich, 2019). The majority of participants in this study were White non-Hispanic Americans with high levels of educational attainment. While it is beneficial to test an instrument in diverse cultures and populations, more studies are required to further explore the dimensionality of the HeLD. It is not uncommon for researchers to report different dimensional structures of new instruments exploring latent variables, as with the related but distinct concept of OHRQoL.

Findings from studies of OHRQoL dimensionality using the OHIP instrument shed some light on the interpretation of oral health literacy’s dimensionality. The OHIP, like the original HeLD instrument, started with a seven-dimensional structure (Slade & Spender, 1994). Later a short instrument, the OHIP-14 followed (Slade, 1997). OHIP-14 and HeLD-14 have the same number of initially proposed domains/dimensions as well as items for each dimension. When OHRQoL was initially researched, several dimensional studies were performed, in line with our current work with the HeLD (Mumcu, Hayran & Ozalp, 2007; Segu et al., 2005). These early studies all consistently rejected the seven OHRQoL domain structure until finally a four-dimensional structure was identified and confirmed (John, 2021). The four-dimensional structure, a concept with world-wide applicability (Ingleshwar & John, 2023), has also, as well as OHL, a strong general factor (John et al., 2014). This could also be shown for the shortest oral health impact profile, the OHIP-5 (John et al., 2006) an instrument that performs consistently across different settings (Simancas-Pallares et al., 2020; Larsson et al., 2014), and is recommended for OHRQoL assessment across multiple settings and oral conditions (John, 2022; John et al., 2021; John et al., 2022).

Based on the similarities between OHRQoL and OHL, it can be expected that a simple unidimensional structure is a reasonable fit for OHL and based on such findings further reduction in the number of items can be expected to increase practicability of all OHL instruments applications across settings.

Results of this study confirmed the positive relationship between OHL and OHRQoL such that greater oral health literacy is correlated with better OHRQoL.

Association analyses of demographic characteristics with both HeLD-14 and OHIP-5 sum scores found that HeLD-14 scores increased with age. This result is consistent with one previous study reporting similar results (Lui et al., 2021), but inconsistent with previous HeLD-29 and HeLD-14 results where sum scores decreased with increasing age (Jones et al., 2015; John et al., 2022). Women in the current study had significantly greater HeLD-14 scores compared to men which is consistent with two other studies although differences did not reach statistical significance or comparisons were not reported (Lui et al., 2021; Sermsuti-Anuwat & Pongpanich, 2019). However, Jones et al. (2015) reported that males had higher HeLD-29 and HeLD-14 sum scores compared to women (p < 0.05) (Jones et al., 2015). This study found that those graduating from high school/GED had lower HeLD-14 scores than those with more than a four-year college degree (p < 0.05). These results are consistent with those found for both the HeLD-29 and HeLD-14 (Jones et al., 2015). However, both Bado et al. (2020) and Sermsuti-Anuwat & Pongpanich (2019) found no differences in HeLD-14 scores by educational level. White participants had higher HeLD-14 scores than non-Whites in this study. Of published studies using the HeLD-14, only one reported race (Bado et al., 2020) and found no statistically significant differences. The only statistically significant finding between OHIP-5 sum scores and demographic characteristics in the current study was between participants with educational attainment of high school graduation/GED and a four-year college degree (p < 0.05) indicating improved OHRQoL with increasing education. This is consistent with similar studies where more years of education were also associated with improved OHRQoL (Husain & Tatengkeng, 2017; Wide & Hakeberg, 2018). However, other studies have found no statistical significance association between education and OHIP scores (Bado et al., 2020; Ikebe et al., 2012), or decreased OHRQoL with increasing educational attainment (Collins et al., 2019; Fernanes et al., 2006). These differences may reflect variation in educational systems throughout the world, or differences in survey categories used to measure educational attainment.

Conclusions

Our results indicate that the construct of OHL is likely unidimensional and that the HeLD-14 is an appropriate instrument to measure OHL in a general US English speaking population. Additional studies are needed in diverse populations to further explore the dimensionality of OHL.

Supplemental Information