Assessing the validity of digital health literacy instrument for secondary school students in Ghana: The polychoric factor analytic approach

The emergence of the coronavirus pandemic resulted in the heightened need for digital health literacy among the youth of school-going age. Despite the relevance of digital health literacy among the general public (including students), it appears the measurement of digital health literacy is still a challenge among researchers. Recently, Dadackinski and colleagues adapted existing digital health literacy measures to fit the COVID-19 situation. Since this development, the instrument has been widely used with few validation studies with none in Africa and specifically, in Ghana. The purpose of the study was to assess the validity of the digital health literacy instrument (DHLI) for secondary school students in Ghana using the polychoric factor analysis. We sampled 1,392 students from secondary schools in Ghana. The digital health literacy instrument was administered to the respondents, thereof. The study confirmed the four latent structure of the DHLI. Further, sufficient validity evidence was found regarding the construct validity of the DHLI. The findings from the study support the validity of the DHLI and its utility within the Ghanaian context. With the growing need for digital health literacy among younger people globally, the DHLI provides sufficient grounds for scaling them based on their level of literacy. There is a need for the instrument to be adapted and re-validated in Ghana and among different populations to widen its reproducibility.

Introduction

The COVID-19 pandemic has affected several lives and sectors of the economy, including education. During the period, students experienced huge psychological consequences threatening their health and well-being (1–7). The sudden onset of COVID-19 was accompanied by an “infodemic”, overabundance of valid and invalid health-related information on COVID-19 (8–12). A recent study revealed that the “infodemic” is so prevalent that 82% of the text ratings examined (i.e., 1,856 of the 2,276 reports) were classified as being “false” (13). During the pandemic, the internet and social media platforms have become important sources of health-related information on the disease and its protective behaviours (11, 14, 15). Conversely, this has led to an explosion of unchecked information and the spread of misinformation (8, 16–19). Recent studies have found that adolescents and young adults (including students) are frequent users of these types of digital media (20, 21), however, they found it challenging dealing with the vast amount of COVID-19 related information as a result of difficulties in seeking, discovering, understanding, judging, and utilising reliable COVID-19 related online information (11, 22–24).

The “infodemic” on the internet and social media could affect students' protective behaviours and mental health. It could also imperil the government and health authorities' efforts to manage the pandemic-related setbacks. Accordingly, for students to effectively navigate the complex information to remain healthy and take relevant precautions using the information available, they would need a high level of digital health literacy (DHL). DHL is the ability to seek, discover, understand, critically appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem (25–27). DHL reflects the specific degree of skills and abilities necessary to use digital health technology and services (28, 29). DHL incorporates interactivity across web-based platforms including social media (27). Amid the COVID-19 pandemic, DHL has become an indispensable resource in promoting mental health and psychological well-being among students (27, 30). DHL is a key competence to navigating web-based COVID-19–related information and service environments and addressing the challenges of online health information and services (11, 22).

Despite the relevance of DHL among the general public (including students), it appears the measurement of DHL is still a challenge among researchers. In recent years, several instruments have been developed to measure DHL (31). One of the prominent inventories is the eHealth Literacy Scale (eHEALS) (26, 32). However, the eHEALS has several limitations. First, it focuses on measuring the search for and evaluation of online information, but it does not address critical and interactive health literacy. Second, its validity is unclear and does not consider the new tools provided by the internet and technologies (27, 32, 33). Due to continuous changes in media and technologies, extant researchers advocated for a new instrument for DHL that considers a broad spectrum of applications (27, 34). Subsequently, van der Vaart and Drossaert (27) developed the Digital Health Literacy Instrument (DHLI). DHLI aims to incorporate the skills necessary to use the broad spectrum of applications offered by the internet and communication technologies and give valid information about people's actual competence level. The DHLI includes interactivities on the web, so-called “e-Health 2.0 Skills”. DHLI is composed of seven subscales (i.e., operational skills, navigation skills, information searching, evaluating reliability, determining relevance, adding self-generated context, and protecting privacy), each including three items to be answered on a 4-point Likert scale ranging from 1 (very difficult) to 4 (very easy).

Based on this questionnaire, Dadackinski and colleagues proposed the DHLI in relation to COVID-19 (i.e., COVID-HL) (14, 22, 35). Although DHLI was developed during the COVID-19 period, it can be implemented or used in other conditions. Dadackinski and colleagues adapted DHLI in the context of the COVID-19 pandemic from van der Vaart and Drossaert (27). To generate the COVID-19 DHLI, five out of the seven original subscales were included and modified with respect to COVID-19: searching the web for information on COVID-19 (DHLIsearch); adding self-generated content on COVID-19 (DHLIcont); evaluating the reliability of COVID-19-related information (DHLIrely); determining personal relevance of COVID-19-related information (DHLIrelev); and protecting privacy on the internet (DHLIpriv) (14, 22, 35). Each of these dimensions contains three questions. The internal consistencies (Cronbach α) of the first four subscales were acceptable to good (0.70 < α < 0.83). Due to low reliability, the protecting privacy subscale (α = 0.46) was omitted from COVID-HL.

Since the adaptation of COVID-HL from DHLI, researchers have adapted and validated COVID-HL to ascertain its psychometric properties, factor structure and functionality in different jurisdictions such as Portugal (36), Spain (37), Italy (38) and Korea (39). These validation studies confirmed the four-factor structure and demonstrated good validity and reliability of the scale. For example, in Italy, the Italian DHLI showed good psychometric characteristics. However, the protecting privacy subscale was excluded given the criticalities presented in the validation process. CFA confirmed the four-factor structure (i.e., information searching, evaluating reliability, personal relevance, and self-generated content, (38). Also, in Portugal, Martins et al. (36) discovered that a four-factor structure of the instrument (i.e., information searching, self-generated content, evaluating reliability, and personal relevance) was supported by confirmatory factor analysis and had good internal consistencies. Similar findings have been reported in Korea (COVID-HL-K; 39) and in Spanish-speaking countries like Spain, Puerto Rico, and Ecuador (COVID-HLI-S; 37). The authors of the validation studies recommended representative studies to shed light on different target groups and their COVID-19–related DHLI. Aside these validation studies, the scale (COVID-HL) has been used by other researchers to assess the level of DHL among university students without examining its psychometric properties, factor structure and functionality in other geographical locations such as the UK (40), US (41), Germany (22, 24), Portugal (42), Slovenia (11), China (43), Korea (44), Pakistan (45) and Vietnam (15).

Despite several validation studies on DHLI across several western societies, there is no documented study evaluating the reliability, validity and applicability of the instrument in assessing COVID-19 health-related information during the pandemic period in the sub-Saharan African region. Due to the vulnerability of the youth and their widespread use of the internet through diverse web-based tools (e.g., Facebook, Twitter, Youtube, WhatsApp), secondary school students were targeted as the normative reference group for the current study. In Ghana, the educational system is grouped into three parts: 9 years of Basic education (i.e., early childhood education [kindergarten], primary and junior high school), 3 years of secondary education (i.e., senior high school/vocational and technical schools) and 3–4 years tertiary education (i.e., academic university, technical university, colleges of education and nursing training colleges). Besides, most of the previous validation studies used university students (36–39). This makes it difficult to ascertain the utility and applicability of the DHLI among secondary school students. Additionally, the only DHLI validation study which investigated gender differences, revealed that males had higher levels of DHL as compared to their female counterparts (37). Although this study in question was conducted among university students, it provides a prompt for recent studies to investigate the issue of gender invariance which this present study did.

The issue of health literacy (HL) has been a serious concern for stakeholders in Ghana and has recently been shown to be important for improving universal health coverage (UHC) in Ghana (46). HL is associated with health, well-being and quality of life (47, 48). Conversely, there is no nationwide assessment of health literacy. However, some researchers have examined aspects of health literacy among specific groups. For example, using the adapted Health Literacy Knowledge and Experience Survey Instrument (HLKES), Koduah et al. (49) established that health literacy (HL) knowledge was generally low among student nurses and practicing nurses, with student nurses having significantly lower scores than practising nurses. Similarly, low or limited HL was found among university students (50), street youth (i.e., people between ages 12 and 24 years) (51, 52) and the general population (including students) (46, 47) and women living with breast cancer (48). These studies on HL in Ghana mostly adapted European Consortium for Health Literacy Questionnaire (HLQ-EU-16) (47, 50) and HLKES (49). These instruments (i.e., HLQ-EU-16 and HLKES) are widely used to measure HL, however, they present some limitations such as lack of clear psychometric properties, interactive health literacy and digital health information. Accordingly, this calls for a new scale or instrument to measure Ghanaian's DHL since we are in the digital world. Hence, the overall purpose of the study was to assess the validity of the DHLI for secondary school students in Ghana, using the polychoric factor analysis. Three objectives guided the conduct of the study: (i) to identify the factor solution of the DHLI, (ii) to assess the construct validity of the DHLI, and (iii) to evaluate the measurement invariance of the DHLI based on gender. The outcome of this investigation would be valuable in the development of related policies aiming to increase DHL and compliance with the policies meant to control COVID-19. It would also help to plan and prepare effective communication interventions for this sub-population.

Materials and methods

Study setting and participants

This study forms part of the multi-national study by the COVID-19 Digital Health Literacy Network (https://covid-hl.orghttps://covid-hl.eu/). The study setting covered secondary school (senior high school) students within the northern zone of Ghana. This area was the primary focus of this research because of the distinct characteristics possessed by the inhabitants. Particularly, students who had not schooled and/or stay within the region for 10 years or over were excluded. Several reports emerging from the northern belt of Ghana have indicated that more than 40 percent of the inhabitants are living in poverty with 8–9 people out of every 10 persons are living below the poverty line according to the Ghana poverty reduction strategy document (53). Additionally, Ofori-Boateng and Bab (53) indicated that the study setting is characterized by low school completion rates, school drop-out, late start of school and other challenges like working and schooling at the same time. It is not surprising that this present study found young adults between 18 and 25 years who were still in secondary school. A report by the World Bank (54) also mentioned that the populace in the northern zone of Ghana has little chance of breaking out of poverty regardless of the kind of employment they engage in.

This study adopted the descriptive cross-sectional survey design to sample one thousand, three hundred and ninety-two (1,392) secondary school students from the Northern regions of Ghana using a multi-stage sampling technique. The simple random technique was first used to select two regions in the northern part of Ghana. Secondly, five schools from each region were sampled using cluster sampling. Then, the individual students in the schools were purposefully recruited based on whether they had resided in the region for more than 10 years. Seven hundred and two (n = 702, 50.4%) of the participants were males while six hundred and fifty-four (n = 654, 47.0%) were females and thirty-six (n = 36, 2.6%) having diverse sexes. The ages of participants ranged from 14 years to 25 years (Mean age = 18.90; SD = 1.95). Through a random procedure, 500 cases out of the 1,392 were used for the first part of the analyses (exploratory factor analysis, EFA) based on the recommendations of Dimitrov (55) who established that a sample of 400 is appropriate, even in instances where the correlation among the items are low. To improve a more accurate estimate, we added 100 cases for the EFA making a total of 500 as earlier indicated (56). The confirmatory factor analysis (CFA) was conducted with 792 cases which were deemed appropriate for four factors considering the low correlations and high power (57, 58).

Measures

Digital health literacy instrument (DHLI)

The DHLI which was originally developed by van der Vaart and Drossaert (27) with 7-subscales and 21-items and later adapted by Dadaczynski et al. (14, 22, 35) to the context of COVID-19 was assessed to determine its factor solution, construct validity, and measurement invariance for gender among secondary school students in Ghana. The study made use of the most current validated form of DHLI (38) with 4-subscales with each having 3-items measured on a 4-point Likert type scale where 1 (Very easy), 2 (Easy), 3 (Difficult), and 4 (Very difficult). The 4-subscales include; Information Searching (e.g., “when you search the Internet for information on coronavirus or related topics, how easy or difficult is it for you to find the exact information you are looking for”?), Self-generated Content (e.g., “when typing a message (on a forum or social network) about coronavirus or related topics, how do you express your opinion, thoughts or feelings in writing”?), Reliability (e.g., “when you search the internet for information on coronavirus or related topics, how easy or difficult is it for you to check different websites to see whether they provide the same”?), and Determining Relevance (when you search the Internet for information on the coronavirus or related topics, how easy or difficult is it applicable to you?). The reliability coefficient values of the four subscales were acceptable to good (0.70 < α < 0.83) (59).

Procedure

This survey procedure was officially endorsed by the University of Education, Winneba's Ethical Review Board (ERB) with a reference number DAA/P.1/Vol.1/39. Further approval was sought from headmasters of all secondary schools in the Northern region who took part in the study. Regardless of one's tribe, every secondary school student who had attended school and lived in any part of the Northern regions for more than 10 years and could read fluently, comprehend and write in the English language was eligible to be involved in the study. Twelve (12) research assistants were trained and taken through the survey instrument from beginning to end with each instruction and item thoroughly explained to them to help with data collection after seeking their voluntary consent to assist in the data collection process. The researchers then began the recruitment process by visiting the secondary schools involved with the research assistants to establish a good rapport and discuss the rationale of the study with both teachers and students. Additionally, each item on the survey instrument was discussed and explained in detail to all students during which there was an opportunity for further clarifications on any item if need be.

Prior to collecting the data, all the participants were asked to sign written informed consent forms to declare their readiness and willingness to be involved in the study. Participants were also told that involvement in the study was purely voluntary and that they had the liberty to continue responding to the items or withdraw at will. Moreover, they were assured of the confidentiality of the responses they would provide. Anonymity was also ensured by asking participants not to provide their names on the survey instruments. All COVID-19 safety protocols were adhered to by providing nose masks and hand sanitizers to each participant. Water, liquid soap and tissue papers were also provided to ensure that the process did not expose any participant to the risk of COVID-19 infection.

Following adherence to all ethical considerations, the DHL survey instruments were distributed to the participants in their various classrooms during their free periods with the help of the research assistants to respond to the survey items. Translation of items on the survey instrument was deemed unnecessary because participants could all read fluently and comprehend the English language. The content of the instrument was explained to the study participants. Responding to the survey items took about 15–20 minutes after which all answered questionnaires were retrieved and sealed in brown envelopes for safe keeping. The entire data collection process lasted for approximately two (2) months.

Statistical analyses

The data were first screened for missing data; however, none was found. Primarily, the data were analyzed using EFA and CFA to address the objectives of the research. Both the EFA and CFA were performed based on the polychoric approach to factor analysis. Before the major analyses, the data were screened for data entry errors, outliers and any abnormalities. Descriptive statistics and item analyses were performed by exploring the association between the items (polychoric correlation), median, skewness and kurtosis of the items. Statistics on item location and adequacy indicators were also assessed to decide whether some items were adequate to be used in the EFA. Three indices were considered (i.e., Quartile of Ipsative Means (QIM), Relative Difficulty Index (RDI), and Measure of Sampling Adequacy (MSA) based on the recommendations of Lorenzo-Seva and Ferrando (60). Regarding QIM, in a normal range test, a few of the item estimates should be found in the extreme positions in the quartiles and the majority should be found in the middle quartile. For RDI, which evaluates the position of the items, nearly 75% of item values should fall between 0.40 and 0.60, and for MSA, items with estimates below 0.50 should be removed as they measure the same domain as the rest of the items.

The polychoric correlation matrix was performed based on the Bayes Modal Estimation using Monte Carlo simulation (61). The EFA was, therefore, conducted using optimal parallel analysis based on minimum rank factor analysis (62). The use of optimal parallel analysis suggests that the focus of the EFA is to identify major factors. The Promin approach (i.e., oblique rotation method) was used for the factor rotation. The EFA was performed using the FACTOR computer programming software (Version 12.1) (63). This research proposed 4-factor model which was strictly first-order and this is because there was no evidence from previous studies that the DHLI lends itself to the second-order model (36–39). The CFA was further conducted in the R-studio environment using the lavaan package (64) with the diagonally-weighted least squares (DWLS) approach to estimation. With the sample size of more than 400 cases, the DWLS was found appropriate although the weighted least squares- mean and variance (WLSMV) could also equally provided better fit indices (65). The following indices were used to judge the model fit of the specified models: Standardized Root Mean Square Residual (SRMR < 0.08), Goodness-of-Fit Index (GFI,  > 0.90), Root Mean Square Error of Approximation (RMSEA < 0.10), Comparative Fit Index (CFI > 0.90) and Tucker-Lewis Index (TLI > 0.90) (66). As a preliminary analysis, the covariance error matrix was inspected and it came out that there is no covariance error structure.

The ordinal alpha reliability estimate was used for judging the reliability of the DHL dimensions. This reliability estimation approach which was proposed by Zumbo et al. (67) has been found to accurately estimate the reliability coefficient compared to Cronbach's alpha. The ordinal alpha uses the polychoric correlation matrix for estimating the reliability coefficient involving ordinal data (68). The confidence intervals for the reliability estimates were also computed to guide the interpretations. A measurement invariance test was also conducted using the multiple indicators of the distinct models based on gender. The diverse gender group was removed from the measurement invariance analysis because they were quite small and this might have affected the results of the invariance test.

Results

Item analysis statistics

The descriptive statistics for the items, which include the item-correlation, median, skewness and kurtosis among others, are presented in Table 1.

TABLE 1

Table 1. Polychoric matrix, median, skewness and kurtosis of the items.

The associations among items under the same factor were found to be fairly moderate (69). For example, the “search” construct domain yielded correlation coefficients between 0.40 to 0.57. Similarly, the “express” sub-scale had coefficients ranging between 0.40 to 0.62. The median values ranged from 1.99 to 2.59 (see Table 1). Whereas the skewness values ranged from 0.06 to 0.36, the values for the kurtosis estimate were between −1.48 to −1.20. Both the skewness and kurtosis values were within acceptable limits (70). Furthermore, the QIM, RDI and MSA values showed that all the items were appropriate and sufficient for the EFA (60). Although the correlation coefficients for some of the items were low ( < 0.20), these items were maintained for two reasons. First, the sample size for the analyses was selected to take into consideration the low correlation coefficients. Secondly, simulation studies (71) have advised against removing items based on low correlations when those items have sufficient factor loadings in a CFA analysis.

Adequacy of polychoric correlation matrix and model fit

The determinant of the polychoric correlation matrix was found to be appropriate, with a value of 0.008. The Kaiser-Meyer-Olkin (KMO) test also showed a good estimate of 0.898 which was greater than 0.60 (72). Further, Bartlett's statistic showed a significant test result, χ2 (66) = 6741.0, p < 0.001, reflecting that the EFA was appropriate (69). The RMSR was less than 0.08 (SRMR = 0.019) showing model adequacy.

Factor solution of DHLI

With the data satisfying the adequacy assumptions, the EFA was performed to assess the number of factors of the DHLI. The outcome of the optimal parallel analysis is presented in Table 2.

TABLE 2

Table 2. Output from optimal parallel analysis based on Minimum rank factor analysis.

The results from the optimal parallel analysis revealed a four-factor structure of the DHLI (see Table 2). The analysis further showed that the four factors accounted for about 70.13% of the variances in DHL of secondary school students. The factor determinacy index was also found to be adequate with all the factors having an estimate greater than 0.80.

Confirmatory factor analysis

The construct validity of the DHLI was assessed using the following indicators: factor loadings, average variance extracted (AVE), and reliability coefficient from the CFA model.

Model fit

The model fit of the specified model (4-factor structure, 12-items) was acceptable. The results showed the following model fit for the CFA: χ2 (48) = 375.168 (p < 0.05), GFI = 0.986, CFI = 0.982, TLI = 0.976, SRMR = 0.057, and RMSEA = 0.070. Except for the chi-square indicator, all the others had adequate model fit indices.

Construct validity

The details of the analysis are presented in Table 3.

TABLE 3

Table 3. Factor loading, AVE, and reliability coefficient.

The outcome of the analysis in Table 3 showed that, for the information sharing domain, the factor loadings ranged from 0.643 to 0.711, with an AVE of 0.470 and ordinal alpha reliability of 0.743, CI(0.712, 0.774). The AVE value for the information searching domain failed to meet the recommended cut-off of > 0.5; this could be due to low item variances in explaining the construct as well as the number of items. For the self-generated content sub-scale, the loadings were from 0.624 to 0.778, with an AVE of 0.518 and ordinal alpha reliability of 0.906, CI(0.879, 0.934). Similarly, the reliability domain, as well as the determining relevance dimension, also had items with factor loadings of 0.702 to 0.741 and 0.718 to 0.824, respectively. The reliability estimate based on the omega ω reliability procedure showed coefficients ranging between 0.720 to 0.799. These two dimensions also had AVE values greater than 0.50 and reliability estimates higher than 0.70 (73). For all the items, the loadings were sufficient (74). Except for the “information searching” dimension, the other dimensions showed a sufficient level of AVE indicating an adequate level of construct validity (75).

Inter-factorial correlations

The inter-factorial correlation is presented in Table 4.

TABLE 4

Table 4. Inter-factorial correlations.

As presented in Table 4, the correlation coefficients ranged between 0.68 and 0.42. For instance, the relationship between self-generated content and information searching was 0.68 whereas the relation between reliability and self-generated content is 0.42 (also see Figure 1).

FIGURE 1

Figure 1. First order CFA model with 4-factor structure and 12 items.

Measurement invariance for gender

The study tested for measurement invariance for gender (see Table 5).

TABLE 5

Table 5. Multiple indicators for measurement invariance for gender.

Two distinct CFA models were fitted for male (χ2 = 9711.13, p < 0.001; CFI = 0.982; TLI = 0.974; GFI = 0.989; RMSEA = 0.049; SRMR = 0.054; MFI = 0.980) and female (χ2 = 9716.13, p < 0.001; CFI = 0.985; TLI = 0.975; GFI = 0.984; RMSEA = 0.043; SRMR = 0.056; MFI = 0.981). The measurement invariance was tested based on the recommendations of Chen (76) (i.e., achieving -0.01 change in CFI,  ≤ 0.015 change in RMSEA and SRMR change cut-off of 0.015 (residual/scalar invariance) or 0.030 (metric invariance). Following the suggestion of Chen (76), the measurement invariance of the DHLI was established with ΔCFI of 0.003, ΔRMSEA of 0.006 and ΔSRMR of 0.002. The results further showed that scalar/residual and metric invariance were satisfied with the use of the instrument.

Discussion

In this study, the validity of the DHLI was assessed among secondary school students in Ghana, specifically, by examining the factor solution and construct validity through the polychoric factor analytic procedure. The study revealed that, within each dimension, the items were moderately related confirming observations from previous studies (36, 38). However, the inter-dimension item relationships were fairly low signifying homogeneity among the items based on the sub-constructs they measure. There was enough evidence of model adequacy for the polychoric analysis. A 4-factor solution was derived based on the optimal parallel analysis. The 4-factor solution explained more than two-thirds (70.13%) of DHL of secondary school students. Further, a very strong relation was found between the true DHL of secondary school students and that which was estimated. This finding confirms that of previous studies (14, 22, 27, 35–37). Martins et al. (36), for example, found that the 4-factor solution explained 59.5% of the variance in the DHL inventory among university students in Portugal. The study of Rivadeneira et al. (37), though confirming the current study, it rather identified a 5-five factor solution. This reflects all the four dimensions in the current validation, and in addition, the privacy protection dimension which was not considered in the current study. Additionally, the four dimensions are reflected in the seven dimensions originally identified by van der Vaart and Drossaert (27).

The 4-factor solution was confirmed in a CFA with acceptable model fit indices. Additionally, all the item loadings for the various sub-dimensions were greater than 0.70, except items SE1 (…make a choice from all the information you find?) and EX1 (…clearly formulate your question or health-related worry?) which had 0.643 and 0.624 loadings, respectively. Even with these items, they explained about 40% of the variance of their latent traits. Overall, the item loadings were adequate in the measurement of the sub-dimensions of DHL among secondary school students in Ghana. Except for the information searching dimension, which had an AVE of 0.47, all the other dimensions had AVEs greater than 0.50, and these met the minimum recommended threshold (74, 75). Notably, in the case of the information searching dimension, even though the AVE was not up to the recommended level, it was very close to 0.50 (i.e., an AVE value of 0.47), and for that matter considered adequate considering the number of items under the dimension (75). The low AVE of the information searching dimension also might be due to the notion that the errors of measurement are larger than the variances explained by the information searching construct and the accuracy of the item contributions as well as the information searching construct. Nevertheless, the reliability coefficient of 0.743 for the information searching dimension provided evidence of convergent validity and thus, the low AVE value for the dimension might not be a concern as suggested by Fornell and Larcker (75). Furthermore, the other dimensions of the DHLI also had sufficient internal consistencies. The outcome of this study is in line with previous studies in other countries that have reported high internal consistencies and AVEs (36–39). It must be noted that not all consistencies functioned the same for secondary school students, as different dimensions have varying levels of precision in measuring the various aspects of DHL.

The study further established measurement invariance of the DHLI based on gender. This result implies that the DHL construct had a similar meaning and structure for both male and female students. Consequently, the DHL construct can be meaningfully construed across both male and female students, justifying the utilisation of the instrument for scaling students into their DHL levels irrespective of their gender.

The DHLI is functionally applicable in other contexts such as in Ghana. More importantly, the current study provides diverse applicability in terms of the cohort of students. While the previous studies used university students who are early adults aged between 20 and 24 years, participants in the current study used secondary school students who were averagely aged 18 years, thus, late adolescents. Generally, it can be said that the DHLI is ecologically robust and age-wise reproducible, though more studies are needed to further validate it. Considering the outbreak of diseases and the proliferation of technology and other ICT devices, the development and re-validation of DHLI is timely as it provides useful information on the easiness or otherwise difficulty in accessing, evaluating, and use of health-related information through the blend of technology and competence among students.

Limitations and future directions

Although this research provides direction to future studies on the psychometrics of the DHLI, it has some limitations. The study involved secondary school students clustered in the Northern zone of Ghana. These students may possess some characteristics which are different from students from other regions. Furthermore, the findings of this study on gender measurement invariance differed from what exists in literature and this presents mixed research findings. Further studies should consider pursuing the issue of gender measurement invariance since gender is likely to be a key variable in the measurement of DHL.

Implications for educational and health practice

The study offers much insight into the adaptation and utility of the DHLI among the youth within the Ghanaian setting. Educational and public health practitioners could make use of the DHLI to identify students with inadequate DHL for appropriate interventions to be rolled out to them. This is particularly important since recent studies have established a relationship between DHL and protective health behaviours (11, 14, 15). With the acceptable applicability of the DHLI, educators could adapt the instrument to study the sense of digital literacy in a general search for educational materials for learning by students. The DHLI can offer a platform where the efficacy of intervention digital health programs can be tested.

Conclusion

The findings from the study support the validity of the DHLI and consequently, its utility within the Ghanaian context. With the growing need for digital health literacy among younger people globally, the DHLI provides sufficient grounds for scaling them based on their level of literacy. There is a need for the instrument to be adapted and re-validated in Ghana and among the different samples to widen its reproducibility. The study establishes that secondary school students digital health literacy can be understood from four perspectives (i.e., searching for information, self-generated context, assessing the reliability of the information and determining the relevance of the information). Although the findings of the study are useful to start the discussions on the utility of DHLI, a number of challenges were identified and other areas of validation could not be covered. For instance, the information searching dimension needs further investigation on why the AVE estimate failed to reach the recommended level. Also, it appears the inter-factorial correlation between the “information searching” and “self-generated content” was relatively high and this could have implications on the further investigation of the latent structure of the instrument. The study recommends that future studies should conduct discriminant analyses and differential item analyses to understand the study's result and other features of the instrument.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by The study was approved by the University of Education, Winneba's Ethical Review Board (ERB) with a reference number: DAA/P.1/Vol.1/39. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

JEH and FQ conceived the idea. FQ performed the analysis. EKA, FQ, FA, JEH, MSS, RSKA, JBF, and TS prepared the initial draft of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

The authors sincerely thank Bielefeld University, Germany for providing financial support through the Open Access Publication Fund for the article processing charge.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Agormedah EK, Henaku EA, Ayite DMK, Ansah EA. Online learning in higher education during COVID-19 pandemic: a case of Ghana. J Educ Technol and Online Learning. (2020) 3(3):183–210. doi: 10.31681/jetol.726441

CrossRef Full Text | Google Scholar

2. Frimpong JB, Agormedah EK, Srem-Sai M, Quansah F, Hagan JE Jr. Examining risk perception and coping strategies of senior high school teachers in Ghana: does COVID-19-related knowledge matter? COVID. (2022) 2:660–73. doi: 10.3390/covid2050050

CrossRef Full Text | Google Scholar

3. Hagan JE Jr, Quansah F, Ankomah F, Agormedah EK, Srem-Sai M, Frimpong JB, et al. Linking COVID-19-related awareness and anxiety as determinants of coping Strategies’ utilization among senior high school teachers in cape coast metropolis, Ghana. Soc Sci. (2022a) 11(3):137. doi: 10.3390/socsci11030137

CrossRef Full Text | Google Scholar

4. Hagan JE Jr, Quansah F, Frimpong JB, Ankomah F, Srem-Sai M, Schack T. Gender risk perception and coping mechanisms among Ghanaian university students during the COVID-19 pandemic. Healthcare. (2022b) 10(4):687. doi: 10.3390/healthcare10040687

CrossRef Full Text | Google Scholar

5. Quansah F, Hagan JE Jr, Sambah F, Frimpong JB, Ankomah F, Srem-Sai M, et al. Perceived safety of learning environment and associated anxiety factors during COVID-19 in Ghana: evidence from physical education practical-oriented program. European journal of investigation in health. Psychol Educ. (2022a) 12(1):28–41. doi: 10.3390/ejihpe12010003/

CrossRef Full Text | Google Scholar

6. Quansah F, Ankomah F, Hagan JE Jr, Srem-Sai M, Frimpong JB, Sambah F, et al. Development and validation of an inventory for stressful situations in university students involving coping mechanisms: an interesting cultural mix in Ghana. Psych. (2022b) 4(2):173–86. doi: 10.3390/psych4020015

CrossRef Full Text | Google Scholar

7. Quansah F, Hagan JE Jr, Ankomah F, Srem-Sai M, Frimpong JB, Sambah F, et al. Relationship between COVID-19 related knowledge and anxiety among university students: exploring the moderating roles of school climate and coping strategies. Front Psychol. (2022c) 13:820288. doi: 10.3389/fpsyg.2022.820288

CrossRef Full Text | Google Scholar

8. Ahinkorah BO, Ameyaw EK, Hagan JE, Seidu A-A, Schack T. Rising above misinformation or fake news in Africa: another strategy to control COVID-19 spread. Front Commun. (2020) 5:45. doi: 10.3389/fcomm.2020.00045

CrossRef Full Text | Google Scholar

9. Chong YY, Cheng HY, Chan HYL, Chien WT, Wong SYS. COVID-19 pandemic, infodemic and the role of eHealth literacy. Int J Nurs Stud. (2020) 108:103644. doi: 10.1016/j.ijnurstu.2020.103644

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Hua J, Shaw R. Corona virus (COVID-19) “infodemic” and emerging issues through a data Lens: the case of China. Int J Environ Res Public Health. (2020) 17(7):2309. doi: 10.3390/ijerph17072309

CrossRef Full Text | Google Scholar

11. Vrdelja M, Vrbovšek S, Klopčič V, Dadaczynski K, Okan O. Facing the growing COVID-19 infodemic: digital health literacy and information-seeking behaviour of university students in Slovenia. Int J Environ Res Public Health. (2021) 18:8507. doi: 10.3390/ijerph18168507

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Zarocostas J. How to fight an infodemic. Lancet. (2020) 395:676. doi: 10.1016/S0140-6736(20)30461-X

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Islam MS, Sarkar T, Khan SH, Mostofa Kamal AH, Hasan SMM, Kabir A, et al. COVID-19-Related infodemic and its impact on public health: a global social Media analysis. Am J Trop Med Hyg. (2020a) 103:1621–9. doi: 10.4269/ajtmh.20-0812

CrossRef Full Text | Google Scholar

14. Dadaczynski K, Okan O, Messer M, Rathmann K. University students’ sense of coherence, future worries and mental health: findings from the German COVID-HL-survey. Health Promot Int. (2022) 37:daab070. doi: 10.1093/heapro/daab070

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Nguyen LHT, Vo MTH, Tran LTM, Dadaczynski K, Okan O, Murray L, et al. Digital health literacy about COVID-19 as a factor mediating the association between the importance of online information search and subjective well-being among university students in Vietnam. Front Digit Health. (2021) 3:1–10. doi: 10.3389/fdgth.2021.739476

CrossRef Full Text | Google Scholar

16. Bin-Naeem S, Bhatti R, Khan A. An exploration of how fake news is taking over social media and putting public health at risk. Health Info Libr J. (2021) 38(2):143–9. doi: 10.1111/hir.12320

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bin-Naeem S, Kamel-Boulos MN. COVID-19 misinformation online and health literacy: a brief overview. Int J Environ Res Public Health. (2021) 18(15):8091. doi: 10.3390/ijerph18158091

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gerts D, Shelley CD, Parikh N, Pitts T, Ross CW, Fairchild G, et al. “Thought I”d share first”: an analysis of COVID-19 conspiracy theories and misinformation spread on twitter. JMIR Public Health Surveill. (2021) 7:e26527. doi: 10.2196/26527

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Grimes DR. Medical disinformation and the unviable nature of COVID-19 conspiracy theories. PLoS ONE. (2021) 16:e0245900. doi: 10.1371/journal.pone.0245900

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Luo T, Chen W, Liao Y. Social media use in China before and during COVID-19: preliminary results from an online retrospective survey. J Psychiatr Res. (2021) 140:35–8. doi: 10.1016/j.jpsychires.2021.05.057

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Islam MS, Sujan MSH, Tasnim R, Ferdous MZ, Masud JHB, Kundu S, et al. Problematic internet use among young and adult population in Bangladesh: correlates with lifestyle and online activities during the COVID-19 pandemic. Addict Behav Rep. (2020b) 12:100311. doi: 10.1016/j.abrep.2020.100311

CrossRef Full Text | Google Scholar

22. Dadaczynski K, Okan O, Messer M, Leung AYM, Rosário R, Darlington E, et al. Digital health literacy and web-based information-seeking behaviors of university students in Germany during the COVID-19 pandemic: cross-sectional survey study. J Med Internet Res. (2021) 23(1):e24097. doi: 10.2196/24097

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Dodd RH, Dadaczynski K, Okan O, McCaffery KJ, Pickles K. Psychological wellbeing and academic experience of university students in Australia during COVID-19. Int J Environ Res Public Health. (2021) 18:866. doi: 10.3390/ijerph18030866

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Okan O, Bollweg TM, Berens E-M, Hurrelmann K, Bauer U, Schaeffer D. Coronavirus-Related health literacy: a cross-sectional study in adults during the COVID-19 infodemic in Germany. Int J Environ Res Public Health. (2020) 17(15):5503. doi: 10.3390/ijerph17155503

CrossRef Full Text | Google Scholar

25. Dunn P, Hazzard E. Technology approaches to digital health literacy. Int J Cardiol. (2019) 293:294–6. doi: 10.1016/j.ijcard.2019.06.039

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Norman CD, Skinner HA. eHEALS: the eHealth literacy scale. J Med Internet Res. (2006) 8:e27. doi: 10.2196/jmir.8.4.e27

PubMed Abstract | CrossRef Full Text | Google Scholar

27. van der Vaart R, Drossaert C. Development of the digital health literacy instrument: measuring a broad Spectrum of health 1.0 and health 2.0 skills. J Med Internet Res. (2017) 19(1):e27. doi: 10.2196/jmir.6709

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Harris K, Jacobs G, Reeder J. Health systems and adult basic education: a critical partnership in supporting digital health literacy. Health Lit Res Pract. (2019) 3(3 Suppl):S33–s36. doi: 10.3928/24748307-20190325-02

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Robbins D, Dunn P. Digital health literacy in a person-centric world. Int J Cardiol. (2019) 290:154–5. doi: 10.1016/j.ijcard.2019.05.033

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Paakkari L, Okan O. COVID-19: health literacy is an underestimated problem. Lancet Public Health. (2020) 5(5):e249–e50. doi: 10.1016/S2468-2667(20)30086-4

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. (HLS-EU) consortium health literacy project European. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. (2012) 12(80):1–13. doi: 10.1186/1471-2458-12-80

CrossRef Full Text | Google Scholar

32. van der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van de Laar MA. Does the eHealth literacy scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res. (2011) 13:e86. doi: 10.2196/jmir.1840

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Quinn S, Bond R, Nugent C. Quantifying health literacy and eHealth literacy using existing instruments and browser-based software for tracking online health information seeking behavior. Comput Hum Behav. (2017) 69:256–67. doi: 10.1016/j.chb.2016.12.032

CrossRef Full Text | Google Scholar

34. Norman CD. Ehealth literacy 2.0: problems and opportunities with an evolving concept. J Med Internet Res. (2011) 13:e125. doi: 10.2196/jmir.2035

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Dadaczynski K, Okan O, Rathmann K. COVID-19 Health Literacy Survey: University Students (COVID-HL-Survey). Questionnaire and Scale Documentation. Public Health Centre Fulda (PHZF) at the Fulda University of Applied Sciences / Interdisciplinary Centre for Health Literacy Research at Bielefeld University. 2020. Available at: https://pub.uni-bielefeld.de/record/2942920

36. Martins S, Augusto C, Martins MR, Silva MJ, Okan O, Dadaczynsky K, et al. Adaptation and validation and of the digital health literacy instrument for Portuguese university students. Health Promot J Aust. (2022) 1–9. doi: 10.1002/hpja.580

CrossRef Full Text | Google Scholar

37. Rivadeneira MF, Miranda-Velasco MJ, Arroyo HV, Caicedo-Gallardo JD, Salvador-Pinos C. Digital health literacy related to COVID-19: validation and implementation of a questionnaire in hispanic university students. Int J Environ Res Public Health. (2022) 19:4092. doi: 10.3390/ijerph19074092

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Lorini C, Velasco V, Bonaccorsi G, Dadaczynski K, Okan O, Zanobini P, et al. Validation of the COVID-19 digital health literacy instrument in the Italian language: a cross-sectional study of Italian university students. Int J Environ Res Public Health. (2022) 19:6247. doi: 10.3390/ijerph19106247

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Chun H, Park EJ, Choi SK, Yoon H, Okan O, Dadaczynski K. Validating the digital health literacy instrument in relation to COVID-19 information (COVID-DHL-K) among south Korean undergraduates. Int J Environ Res Public Health. (2022) 19:3437. doi: 10.3390/ijerph19063437

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Frings D, Sykes S, Ojo A, Rowlands G, Trasolini A, Dadaczynski K, et al. Differences in digital health literacy and future anxiety between health care and other university students in England during the COVID-19 pandemic. BMC Public Health. (2022) 22(1):1–9. doi: 10.1186/s12889-022-13087-y

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Patil U, Kostareva U, Hadley M, Manganello JA, Okan O, Dadaczynski K, et al. Health literacy, digital health literacy, and COVID-19 pandemic attitudes and behaviours in U.S. College students: implications for interventions. Int J Environ Res Public Health. (2021) 18:3301. doi: 10.3390/ijerph18063301

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Rosário R, Martins MRO, Augusto C, Silva MJ, Martins S, Duarte A, et al. Associations between COVID-19-related digital health literacy and online information-seeking behavior among Portuguese university students. Int J Environ Res Public Health. (2020) 17(23):8987. doi: 10.3390/ijerph17238987

CrossRef Full Text | Google Scholar

43. Amoah PA, Leung AY, Parial LL, Poon ACY, Tong HHY, Ng WI, et al. Digital health literacy and health-related well-being amid the COVID-19 pandemic: the role of socioeconomic status among university students in Hong Kong and macao. Asia Pac J Public Health. (2021) 33(5):613–6. doi: 10.1177/10105395211012230

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Chun H, Yoon H, Choi SK, Park EJ. COVID-19 Related digital health literacy and preventive health behaviors among college students: intention to vaccinate and adherence to preventive measures. Korea J Popul Stud. (2021) 44:121–41. doi: 10.3390/ijerph19063437

CrossRef Full Text | Google Scholar

45. Zakar R, Iqbal S, Zakar MZ, Fischer F. COVID-19 and health information seeking behavior: digital health literacy survey amongst university students in Pakistan. Int J Environ Res Public Health. (2021) 18:4009. doi: 10.3390/ijerph18084009

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Amoah PA, Phillips DR. Health literacy and health: rethinking the strategies for universal health coverage in Ghana. Public Health. (2018) 159:40–9. doi: 10.1016/j.puhe.2018.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Amoah PA. Social participation, health literacy, and health and well-being: a cross-sectional study in Ghana. SSM-Popul Health. (2018) 4:263–70. doi: 10.1016/j.ssmph.2018.02.005

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Kugbey N, Meyer-Weitz A, Asante KO. Access to health information, health literacy and health-related quality of life among women living with breast cancer: depression and anxiety as mediators. Patient Educ Couns. (2019) 102(7):1357–63. doi: 10.1016/j.pec.2019.02.014

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Koduah AO, Amoah PA, Nkansah JO, Leung AY. A comparative analysis of student and practising nurses’ health literacy knowledge in Ghana. Healthcare. (2021) 9(1):38. doi: 10.3390/healthcare9010038

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Ansu-Yeboah E, Edusei A, Gulis G. Comprehensive health literacy among undergraduates: a Ghanaian university-based cross-sectional study. HLRP: Health lit res pract. (2019) 3(4):e227–37. doi: 10.3928/24748307-20190903-01

CrossRef Full Text | Google Scholar

51. Amoah PA, Phillips DR, Gyasi RM, Koduah AO, Edusei J. Health literacy and self-perceived health status among street youth in kumasi. Ghana. Cogent Med. (2017) 4(1):1275091. doi: 10.1080/2331205X.2016.1275091

CrossRef Full Text | Google Scholar

52. Amoah PA, Koduah AO, Gyasi RM, Gwenzi GD, Anaduaka US. The relationship between functional health literacy, health-related behaviours, and sociodemographic characteristics of street-involved youth in Ghana. Int J Health Promot Educ. (2018) 57(3):116–32. doi: 10.1080/14635240.2018.1552835

CrossRef Full Text | Google Scholar

53. Ofori-Boateng K, Bab I. Assessment of poverty levels in selected districts of rural Ghana. Assessment. (2015) 8(2422-845X):130–55. Retrieved from https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.676.5883/rep=rep1/type=pdf on January 20, 2022

Google Scholar

54. World Bank. Ghana: poverty assessment. Washington, DC, USA: World Bank (2020).

55. Dimitrov DM. Statistical methods for validation of assessment scale data in counseling and related fields. Alexandria, VA: American Counseling Association (2012).

56. Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. (2005) 10:1–9. doi: 10.7275/jyj1-4868

CrossRef Full Text | Google Scholar

57. Boomsma A, Hoogland JJ. The robustness of LISREL modeling revisited. In: Cudeck R, du Toit S, Sörbom D, editors. Structural equation models: present and future. A festschrift in honor of karl jöreskog. Lincolnwood, IL: Scientific Software International (2001). p. 139–68.

58. Kyriazos TA. Applied psychometrics: sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology. (2018) 9:2207–30. doi: 10.4236/psych.2018.98126

CrossRef Full Text | Google Scholar

59. Quansah F. The use of cronbach alpha reliability estimate in research among students in public universities in Ghana. Afr J Teach Educ. (2017) 6:56–64. doi: 10.21083/ajote.v6i1.3970

CrossRef Full Text | Google Scholar

60. Lorenzo-Seva U, Ferrando PJ. MSA: the forgotten index for identifying inappropriate items before computing exploratory item factor analysis. Methodol (Gott). (2021) 17(4):296–306. doi: 10.5964/meth.7185

CrossRef Full Text | Google Scholar

61. Choi J, Kim S, Chen J, Dannels S. A comparison of maximum likelihood and bayesian estimation for polychoric correlation using monte carlo simulation. J Educ Behav Stat. (2011) 36(4):523–49. doi: 10.3102/1076998610381398

CrossRef Full Text | Google Scholar

62. Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods. (2011) 16:209–20. doi: 10.1037/a0023353

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Lorenzo-Seva U, Ferrando PJ. FACTOR 9.2 A comprehensive program for fitting exploratory and semiconfirmatory factor analysis and IRT models. Appl Psychol Meas. (2013) 37(6):497–8. doi: 10.1177/0146621613487794

CrossRef Full Text | Google Scholar

64. Rosseel Y. Lavaan: an R package for structural equation modeling. J Stat Softw. (2012) 48:1–36. doi: 10.18637/jss.v048.i02

CrossRef Full Text | Google Scholar

65. DiStefano C, Morgan GB. A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Struct Equ Model: Multidiscip Res J. (2014) 21(3):425–38. doi: 10.1080/10705511.2014.915373

CrossRef Full Text | Google Scholar

66. Kline RB. Principles and practice of structural equation modeling. New York, NY, USA: Guilford Publications (2015).

67. Zumbo BD, Gadermann AM, Zeisser C. Ordinal versions of coefficients alpha and theta for Likert rating scales. J Mod Appl Stat Methods. (2007) 6:21–9. doi: 10.22237/jmasm/1177992180

CrossRef Full Text | Google Scholar

68. Gadermann AM, Guhn M, Zumbo BD. Estimating ordinal reliability for Likert-type and ordinal item response data: a conceptual, empirical, and practical guide. Pract Assess Res Eval. (2012) 17:3. doi: 10.7275/n560-j767

CrossRef Full Text | Google Scholar

69. Tabachnick BG, Fidell LS. Using multivariate statistics. 7th ed. Boston: Pearson Education (2019).

70. Bryne B. Structure equation modeling with AMOS: basic concepts applications and programs. New York: Routledge Taylor and Francis Group (2010).

71. Bergqvist E, Tossavainen T, Johansson M. An analysis of high and low intercorrelations between mathematics self-efficacy. Anxiety, and Achievement Variables: a Prerequisite for a Reliable Factor Analysis. Educ Res Int. (2020) 2020:1–11. doi: 10.1155/2020/8878607

CrossRef Full Text | Google Scholar

72. Kaiser H. An index of factorial simplicity. Psychometrika. (1974) 39:31–6. doi: 10.1007/BF02291575

CrossRef Full Text | Google Scholar

73. DeVellis RF. Scale development: theory and applications, 4th ed. London, UK: Sage Publications (2017).

74. Civelek ME. Essentials of structural equation modelling. Lincoln, NE, USA: University of Nebreaska (2018).

75. Fornell C, Larcker D. Evaluating structural equation models with unobserved variables and measurement error. J Mark Res. (1981) 18:39–50. doi: 10.1177/002224378101800104

CrossRef Full Text | Google Scholar

76. Chen FF. Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equ Modeling. (2007) 14:464–504. doi: 10.1080/10705510701301834

CrossRef Full Text | Google Scholar