Introduction
The worldwide spread of the coronavirus disease 2019 (Covid-19) pandemic has resulted in several psychosocial consequences, including stigmatisation and discriminatory behaviours against people who have, or might have, the disease (Bagcchi, Reference Bagcchi2020; He et al., Reference He, He, Zhou, Nie and He2020; Kousoulis et al., Reference Kousoulis, Van Bortel, Hernandez and John2020; WHO, 2020).
Literature on previous viral outbreaks and epidemics reports that infected patients have often been labelled, stereotyped, discriminated against, treated separately and experienced loss of status because of a perceived link with the disease (Van Bortel et al., Reference Van Bortel, Basnayake, Wurie, Jambai, Koroma, Muana, Hann, Eaton, Martin and Nellums2016; Baldassarre et al., Reference Baldassarre, Giorgi, Alessio, Lulli, Arcangeli and Mucci2020; Lasalvia, Reference Lasalvia2020; Gronholm et al., Reference Gronholm, Nosé, Van Brakel, Eaton, Ebenso, Fiekert, Hanna, Milenova, Sunkel and Barbui2021). Worryingly, it has been found that social stigma often persists even after outbreaks have ended (Overholt et al., Reference Overholt, Wohl, Fischer, Westreich, Tozay, Reeves, Pewu, Adjasso, Hoover and Merenbloom2018; James et al., Reference James, Wardle, Steel and Adams2020).
Social stigma may be experienced by an individual in three forms: enacted stigma (overt behaviours), perceived stigma (awareness of stereotype) and internalised stigma (personal value) (Pescosolido and Martin, Reference Pescosolido and Martin2015). Enacted stigma refers to overt acts of discrimination and humiliation directed at a person because of his or her stigmatised status, which captures the interpersonal aspect of stigma. The process of rejection and unfair treatment experienced by the stigmatised person overlaps with the concept of ‘experienced discrimination’ (Thornicroft et al., Reference Thornicroft, Rose, Kassam and Sartorius2007). By contrast, perceived stigma and internalised stigma capture the intrapersonal aspect of stigma, i.e., perceived stigma refers to the subjective awareness of social stigma, whereas internalised stigma (also known as self-stigma) describes the process of an individual accepting society's negative evaluation and incorporating this into personal value and sense of self (in this process, perceived stigma represents a precondition for the development of internalised stigma).
In the context of the Covid-19 pandemic, experienced discrimination refers to instances during which a given person with suspected or confirmed Covid-19 experiences devaluation, unfair treatment, or exclusion from others due to a perceived link with the disease, whereas internalised stigma refers to the awareness of devaluation or a stereotype of oneself because of a perceived linkage with Covid-19. Due to internalised stigma, people with Covid-19 may discredit themselves and accept that they deserve to be treated unequally and expect to be stigmatised further (Ransing et al., Reference Ransing, Ramalho, de Filippis, Ojeahere, Karaliuniene, Orsolini, da Costa, Ullah, Grandinetti and Bytyçi2020). Both experienced discrimination and internalised stigma are interrelated and play a crucial role in the personal and psychological adjustment to the disease. The way stigmatised people respond to stigma - by either conforming to it (self-stigmatisation) or resisting it - can affect the impact of stigma in a community, irrespective of the actual level of enacted stigma or discrimination (Deacon, Reference Deacon2005). High levels of internalised stigma reduce the incentives to challenge stigmatisation, which has negative consequences for both the individual and public health programmes. In fact, in order to avoid discrimination, people infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) may be reluctant to seek healthcare or may try to hide the disease or misreport symptoms, thus reducing early detection and treatment (Des Jarlais et al., Reference Des Jarlais, Galea, Tracy, Tross and Vlahov2006; Stangl et al., Reference Stangl, Earnshaw, Logie, van Brakel, Simbayi L, Barré and Dovidio2019) and posing difficulties in controlling the spread of the infection (Van Bortel et al., Reference Van Bortel, Basnayake, Wurie, Jambai, Koroma, Muana, Hann, Eaton, Martin and Nellums2016). Social stigma may also affect the mental health of stigmatised people. Initial evidence suggests that stigma is associated with PTSD symptoms, depression, and anxiety among patients hospitalised with Covid-19 (Liu et al., Reference Liu, Chen, Liu, Nie and Lu2020). The experience of being treated differently from others due to Covid-19 may be also indirectly associated with anxiety, depression and insomnia through the mediating effect of shame and internalised stigma (Li et al., Reference Li, Zheng, Le, Zhuo, Wu, Ma and Tao2020a).
However, the issue of Covid-19 related stigma and its relationships with mental health outcomes in people infected with SARS-CoV-2 has not been sufficiently addressed in the literature and empirical evidence on the extent and severity of stigmatisation among persons surviving Covid-19 is still lacking. This research gap is substantially due to the lack of standardised measures specifically designed to measure interpersonal and intrapersonal aspects of stigma in people with Covid-19. Recently, a measure adapted for Covid-19 patients and based on a previous HIV/AIDS stigma scale has been published. However, this scale does not specifically address the different components of interpersonal (experienced discrimination) and intrapersonal stigma (internalised) (Huang et al., Reference Huang, Sun, Zhang, Lu and Chen2022) and it seems to lack a sound theoretical basis.
The present paper aims to fill this research gap by reporting on the development and psychometric properties testing of two standardised multidimensional measures assessing, respectively, experienced discrimination and internalised stigma in persons infected with SARS-CoV-2 or who have had full blown Covid-19 disease.
Methods
Item generation and pre-testing
A two-phase process was carried out for the development of the scales. In the first phase (February–March 2021), candidate items were identified through a comprehensive literature review of relevant sources addressing personal stigma in patients with infectious diseases (MERS, SARS, Ebola virus and HIV/AIDS). We did not find psychometrically validated scales addressing personal stigma among Covid-19 patients. On the other hand, a series of psychometrically tested scales designed to assess stigma in people with HIV/AIDS were found which represented the basis for item generation in the present study. Specifically, items for the experienced discrimination scale were drawn from Berger et al. (Reference Berger, Ferrans and Lashley2001), whereas for the self-stigma scale were drawn from Sayles et al. (Reference Sayles, Hays, Sarkisian, Mahajan, Spritzer and Cunningham2008), which represent the most frequently used measures in the literature for assessing, respectively, HIV related experienced stigma and self-stigma (Earnshaw and Chaudoir, Reference Earnshaw and Chaudoir2009; McAteer et al., Reference McAteer, Truong, Aluoch, Deathe, Nyandiko, Marete and Vreeman2016; Wanjala et al., Reference Wanjala, Too, Luchters and Abubakar2021). Moreover, for both experienced discrimination and self-stigma, items were also drawn from Visser et al. (Reference Visser, Kershaw, Makin and Forsyth2008), who developed three parallel stigma scales to assess personal views of HIV related stigma. With regard the item selection process, for the experienced discrimination scale items were chosen in order to cover all the potential everyday life domains where respondents might have experienced negative reactions from others due to their Covid-19 infection; for the internalised stigma scale, items were chosen in order to cover all the possible negative feelings that respondents might have experienced in relation to their Covid-19 infection or all the public stereotypes related to Covid-19 that respondents might have internalised. In the item generating process, we did not translate any specific item from the original English version of the scales taken into account, but we only extrapolated their thematic elements; therefore, items composing both the newly developed scales were directly produced in Italian. This phase led to the development of two drafted versions of the scales, with one addressing experienced discrimination and the other exploring internalised stigma. Both scales, developed in Italian, were designed as self-administered scales. The two drafted versions contained, respectively, 24 items and 19 items. Respondents indicate their degree of agreement on a 5-point Likert scale ranging from ‘strongly agree’ to ‘strongly disagree’. In the second phase (April 2021), face and content validity of the new scales developed in the previous phase were tested in a focus group session with subjects recovered from Covid-19. This phase aimed to use results obtained from the focus group to decide on final, definitive versions for validation work. Eligible participants for the focus group were identified by the research team. The focus group comprised a purposefully selected sample (Palinkas et al., Reference Palinkas, Horwitz, Green, Wisdom, Duan and Hoagwood2015) of nine participants (four women and five men), aged between 25 and 55 years (average age 35,3). They all had developed full blown Covid-19, with two of them having been hospitalised and seven having been treated at home. Participants were asked to discuss aspects of the scales related to ease of use, comprehension, acceptability, relevance of items and response options, time taken to complete and recommendations for improvement. The focus group session took place using an online video conferencing tool and was led by an experienced moderator (A.L.). A previously prepared semi-structured topic guide was followed during the session. The focus group was video recorded (with participant consent) and transcribed for analysis. Based on the learnings and insights from the focus group, minor changes were made to the wording of the items of both questionnaires. Furthermore, in both questionnaires, the various item statements were turned into questions (e.g., from ‘My friends and family members were ashamed of me because I have had Covid-19;’ to ‘How much were your friends and family members ashamed of you because you have had Covid-19?’), responses were categorised using a 4-point Likert scale from ‘not at all’ to ‘a lot’, a ‘Not applicable’ option was added; finally, regarding the experienced discrimination scale items were reduced to 14 and for the internalised stigma scale items were reduced to 17.
Piloting the questionnaires
The two scales developed according to the steps mentioned above were tested in an online cross-sectional survey conducted between 10 September and 10 November 2021. The newly developed stigma scales were hosted on the web-based survey platform SurveyMonkey and could be self-rated by participants through their PCs, smartphones, or other mobile devices. Participants were adults (⩾18 years) who had been infected with SARS-CoV-2 or had developed Covid-19. The online survey was conducted within a closed Facebook discussion group comprising people who had been infected with SARS-CoV-2 or had developed Covid-19, which at time of the study had more than 10 000 members across Italy. The study description and invitation to participate, as well as the link to the online questionnaires, were posted to all members by the group administrator (M.C.). Participants were aware of the purpose of the study and gave informed consent before participating. They completed the survey anonymously, voluntarily and without any remuneration. To assess test–retest reliability upon completion of all questionnaires, participants were asked to leave their email address if they wished to do so. This subsample was invited to complete the questionnaires again after two weeks.
The questionnaires
The Covid-19 Experienced DISCrimination scale (CEDISC) and COvid-19 INternalised Stigma scale (COINS), both developed in Italian, were designed to measure, respectively, experienced discrimination and internalised stigma among people infected with SARS-CoV-2 or who had survived Covid-19. The CEDISC starts with an opening question asking participants whether they might have been treated unfairly (e.g., with hostility, avoidance, rejection, prejudice) by other people as they were infected with SARS-CoV-2 or diagnosed with Covid-19 in a series of everyday situations. In order to avoid any possible misunderstanding with the specific content that the scale was aiming to address, a guiding note was given explaining that the situations listed in the questionnaire referred to the period following recovery from Covid-19 (or following isolation due to a positive test), once the respondent had returned to her/his usual social life. The COINS also starts with an opening question asking participants if they might have felt uncomfortable (e.g., embarrassed, ashamed, guilty) as they were infected with SARS-CoV-2 or diagnosed with Covid-19 across a series of possible situations. For both scales, all items are scored on a 4-point Likert scale: 0 = not at all, 1 = a little, 2 = moderately and 3 = a lot. A ‘Not applicable’ option is available. The CEDISC version that was completed in the online survey included 14 questions; however, two items were removed after data analysis as one item had a high number of ‘Not applicable’ responses (36.3%) and the other one had a factor loading <0.40. Thus, the final version of the questionnaire for perceived stigma presented here constitutes 12 items (an English translation of the original Italian scale is shown in Appendix A). The COINS version completed in the online survey included 17 questions, but four items were removed after factor analysis as they had factor loadings <0.40. The final version of the internalised stigma questionnaire therefore comprises 13 items (an English translation of the original Italian scale is shown in Appendix B).
For both questionnaires, a mean score is calculated for the global scale and each subscale. No items are reverse coded. A higher score indicates a higher experienced discrimination and, respectively, internalised stigma level. A strategy for the interpretation of scores may be addressed by applying the midpoint of 1.5, thus identifying four stigma categories: <1 minimal; 1–1.5 low; 1.5–2 moderate; and 2+ high (Brohan et al., Reference Brohan, Clement, Rose, Sartorius, Slade and Thornicroft2013). In addition, a total score may be calculated by counting the number of items in which individuals scored 1, 2 or 3.
Statistical analysis
Descriptives were given by absolute frequencies and percentages for categorical variables and means and standard deviations for continuous variables. All analyses were performed with SPSS 26 and Stata 17.
Construct validity
Construct validity was established by conducting an exploratory factor analysis (EFA) based on the principal component factoring with Promax rotations. Kaiser-Meyer-Olkin's measure of sampling adequacy (KMO) and Bartlett's test of sphericity were estimated to explore the model's adequacy. Factors with eigenvalues greater than one were retained. Only items with factor loadings >0.4 were considered in the final model. No correlation with other scales measuring similar constructs was performed as no validated instruments assessing experienced discrimination and internalised stigma for Covid-19 were available.
Reliability
The reliability was assessed by considering: (1) consistency over subscales (internal consistency) and (2) consistency over time (test–retest reliability). The internal consistency was assessed using Cronbach's α with a criterion of 0.70 for a good internal consistency (Cronbach, Reference Cronbach1951). To assess test–retest reliability of items, weighted kappa coefficients were calculated with values >0.41 indicating a moderate agreement (Landis and Koch, Reference Landis and Koch1977). Two-way mixed effect intraclass correlation coefficient (ICC) was used to calculate the test–retest reliability for the global mean score and the subscales. A criterion of 0.75 was used to indicate acceptable reliability (Koo and Li, Reference Koo and Li2016).
Precision
The precision (i.e., how well each item fits within the scale) was examined by Kendall's tau-b coefficient. A correlation <0.30 was indicative of an unacceptable fit (Furr, Reference Furr2021).
Acceptability
In order to establish the extent to which the scale was acceptable for the target population, the following aspects were examined: (1) maximum endorsement frequencies (MEF) and (2) aggregate adjacent endorsement frequencies (AEF) (Furr, Reference Furr2021). In considering MEF, the n (%) of respondents endorsing each response category was established. MEF >80% in any category indicates that the item may need further consideration. AEF assesses the proportion of responses in two or more adjacent scale points of an item, where the criterion of >10% was considered acceptable.
Feasibility
The feasibility was assessed by registering the time taken to complete each online questionnaire. More than 20 min was considered indicative of an unbearable participant burden. Finally, the percentage of participants who completed each questionnaire was calculated.
Results
Participants' characteristics
The online survey involved 579 participants who completed the CEDISC questionnaire, 519 (89.6%) of whom also completed the COINS questionnaire. In terms of socio-demographic and SARS-CoV-2 characteristics, the 60 participants who refused to complete the COINS did not differ from the 519 who completed it, with the only exception of Covid-19 symptoms, which occurred more frequently in people who completed the assessment (93.1% v 81.7%, p = 0.005 Fisher's exact test). One hundred and fifty-five participants completed the retest after two weeks. No differences were found with respect to socio-demographics and SARS-CoV-2 infection characteristics between the test and the retest samples (Table 1).
Table 1. Socio-demographics and clinical characteristics for the test sample (n = 579) and the retest sample (n = 155)
Scoring
The distribution of items pertaining to the final version of both CEDISC and COINS scales are given in Table 2.
Table 2. Response frequencies and percentages of the CEDISC (n = 579) (top part) and the COINS (n = 519) (bottom part)
Construct validity
By considering the CEDISC scale, 12 items converged over a 2-factor solution accounting for 49.2% of the variance (KMO 0.894; Bartlett's test p < 0.001). The first factor, namely ‘Social life’, accounted for 39.7% of the variance and constituted 7 items, while the second factor, namely ‘Close relations’, accounted for 9.5% of the variance and constituted 5 items. Regarding the COINS questionnaire, 13 items converged over a 3-factor solution accounting for a variance of 67.7% (KMO 0.827; Bartlett's test p < 0.001). The first factor, namely ‘Self-perception’, accounted for 34.5% of the variance and comprised 7 items. The second factor, namely ‘Close relations’, accounted for 22.7% of the variance and included 4 items. Finally, the third factor, namely ‘Social life’, accounted for 10.5% of the variance and consisted of 3 items (Table 3).
Table 3. Factor loadings from the exploratory factor analysis [principal component extraction; Promax rotations; factor loadings >0.4 (in bold) were retained] for the CEDISC (n = 579) (top part) and the COINS (n = 519) (bottom part)
Reliability
By considering the CEDISC questionnaire, the Cronbach's α value for the global score was 0.848 indicating a good internal consistency. The alpha value for the items ranged from 0.831 to 0.850. By considering the two subscales, the Cronbach's alpha was 0.770 and 0.777, respectively. Regarding the COINS questionnaire, the Cronbach's α value for the global score was 0.837 indicating a good internal consistency. The alpha value for the items ranged from 0.815 to 0.832. By considering the three subscales, the Cronbach's α was 0.855, 0.924 and 0.868, respectively (Table 4).
Table 4. Internal consistency for the global score and the subscales (Cronbach's α) for the CEDISC (n = 579) (top part) and the COINS (n = 519) (bottom part)
The COINS subscale ‘Close relations’ showed a value exceeding 0.90 due to the presence of one item (‘I would understand if my neighbours avoided me because I have had Covid-19’) with a Cronbach's α of 0.927. This item was retained, despite the possible redundancy, because it addresses a relevant aspect of participants' close relations.
By considering test–retest reliability, three items (25%) in the CEDISC questionnaire had kappa values between 0.61 and 0.80 (substantial agreement) and seven (58.4%) between 0.41 and 0.60 (moderate agreement). Only two items showed a fair agreement (values between 0.21 and 0.40). By considering the COINS questionnaire, ten items (76.9%) had a weighted Cohen's kappa value indicating a moderate agreement (0.41–0.60). The three items in the ‘Social life’ subscale showed 86.6%, 75.6% and 83.0% of agreement on the category ‘not at all’, thus generating a test–retest cross-tabulation which is a sparse matrix (online Supplementary Table S1).
Finally, ICC calculated for the CEDISC questionnaire showed that the global mean score and the ‘Close relationships’ subscale had an excellent test–retest reliability, while the ‘Social life’ had a good reliability. Regarding the COINS questionnaire, ICC indicated a good test–retest reliability for the global score and the two subscales ‘Self-perception’ and ‘Close relations’. For the subscale ‘Social life’ the ICC value was low due to the very high agreement on the same category (‘Not at all’) for all the pertaining items (Table 5).
Table 5. Test–retest reliability for the global score and the subscales (intraclass correlation coefficient ICC) for the CEDISC (top part) and the COINS (bottom part) (n = 155)
a 0.51–0.75 moderate; 0.76–0.90 good; 0.91–1 excellent (Koo and Li, Reference Koo and Li2016).
b The items pertaining to this subscale agree on the category ‘Not at all’ (86.6%, 75.6% and 83.0%, respectively).
Precision
By considering the CEDISC questionnaire, the Kendall's tau-b coefficients for the global scale ranged from 0.360 to 0.556 (p < 0.001), thus indicating that all items fit well with the score of the scale. Moreover, the two subscales showed values ranging from 0.506 to 0.567 and from 0.469 to 0.705, respectively. The COINS questionnaire showed Kendall's tau-b coefficients ranging from 0.290 to 0.606 (p < 0.001) for the global scale score (the lowest value 0.290 pertain to one of the three items which constitute the subscale ‘Social life’). The correlations within each subscale were very good (online Supplementary Table S2).
Acceptability
Regarding the CEDISC questionnaire, the MEF criterion was slightly violated only by one item, with the ‘not at all’ category showing a frequency of 80.7%. The AEF criterion was violated when considering the adjacent categories of ‘moderately’ and ‘a lot’ for five items, ranging from 6.5% to 9.1%. By considering the COINS questionnaire, the MEF criterion was violated by one item on the category ‘not at all’ (83.8%). The AEF criterion was violated when considering the adjacent categories of ‘a little’ and ‘moderately’ for one item (6.3%) and the categories ‘moderately’ and ‘a lot’ for three items (5.7%, 6.4% and 7.3%).
Feasibility
The mean completion times for the CEDISC and COINS were, respectively, 4.5 min (SD 2; range 2–16) and 3.5 min (SD 2.5; range 1.5–18). All participants completed the CEDISC scale, while 10.4% refused to fill in the COINS scale.
Discussion
To our knowledge, this is the first study to develop and validate two questionnaires assessing, respectively, experienced discrimination (CEDISC) and internalised stigma (COINS) among people who had been infected by SARS-CoV-2 or who survived Covid-19.
Research so far has developed specific scales measuring public stigma (stereotypes and misconceptions endorsed by the public in relation to Covid-19) (Kantor and Kantor, Reference Kantor and Kantor2021; Nochaiwong et al., Reference Nochaiwong, Ruengorn, Awiphan, Kanjanarat, Ruanta, Phosuya, Boonchieng, Nanta, Chongruksut and Thavorn2021) and perceived social stigma (awareness of public stigma or belief that others hold stigmatising thoughts or stereotypes about Covid-19) (Huang et al., Reference Huang, Sun, Zhang, Lu and Chen2022). As far as we know, specific standardised scales addressing experienced discrimination (actual experiences of being treated unfairly by others) and internalised stigma (internalisation of the negative stereotypes about Covid-19 endorsed by the general population) are still lacking. With specific regard to internalised stigma, the few research published on this topic suggests that this represents a crucial dimension in predicting adverse mental health outcomes (specifically, PTSD, anxiety, depression, demoralisation, low self-esteem) in patients surviving Covid-19 (Li et al., Reference Li, Zheng, Le, Zhuo, Wu, Ma and Tao2020a, Reference Li, Liang, Yuan and Zeng2020b; Mahmoudi et al., Reference Mahmoudi, Saffari, Movahedi, Sanaeinasab, Rashidi-Jahan, Pourgholami, Poorebrahim, Barshan, Ghiami and Khoshmanesh2021), thus deserving special attention. Since social stigma predicts long-term adverse mental health outcomes in people infected with Covid-19, it is critical for Covid-19 interventions to target stigma in order to both reduce its psychosocial impact on people infected with SARS-CoV-2 or who developed the disease and to remove a key factor that may potentially hamper full recovery in those surviving Covid-19 (Ransing et al., Reference Ransing, Ramalho, de Filippis, Ojeahere, Karaliuniene, Orsolini, da Costa, Ullah, Grandinetti and Bytyçi2020). Yet, stigma-reduction interventions tailored around people infected with Covid-19 are non-existent. Thus, there is a need for research to generate knowledge to address Covid-19 related stigma and discrimination.
The psychometric evaluation conducted in this study shows that the 12-item CEDISC, with its 2-factor structure, is a reliable and valid self-report measure for assessing experienced discrimination among people who tested positive for SARS-CoV-2 or survived Covid-19. The factor analysis showed that the CEDISC can adequately measure experienced discrimination as a whole and through the two dimensions of ‘Social life’ and ‘Close relations’.
Similarly, the psychometric properties of the 13-item COINS, with a 3-factor structure, reveal that this questionnaire is a reliable and valid self-report measure for assessing internalised stigma in the same population. In detail, the factor analysis showed that the COINS measures internalised stigma as a whole and through the three dimensions ‘Self-perception’, ‘Close relations’ and ‘Social life’. The reliability analysis revealed a good internal consistency and most of the items showed at least moderate agreement in the test–retest comparison for both scales. The precision demonstrates that all items fit well with the scores in both questionnaires. The acceptability, assessed by MEF and AEF, was slightly violated in a minimal number of items.
Finally, both scales were completed within five minutes by most participants, thus proving to be feasible instruments.
These standardised measures, focusing on interpersonal and intrapersonal aspects of social stigma, will allow to gain a more in-depth knowledge on the psychosocial consequences of the Covid-19 pandemic and to promote more research in this field.
Strengths and limitations
A major strength of this study lies in the sample size of nearly 600 participants, which allowed us to validate the two scales relying on a statistically robust sample. The second strength relates to the inclusion of people who became positive or were diagnosed with Covid-19 in different pandemic waves, which could have reflected a different pattern of stigmatisation. The third strength pertains to the pre-testing phase, which was conducted by engaging a group of participants representing a wide range of characteristics (age, gender and working status). This study has also some limitations. First, the sample used in developing and validating the two scales may not be representative of the overall population of interest, both in terms of socio-demographic composition and in terms of Covid-19 severity: in fact, female gender in our sample is overrepresented, while elderly people are under-represented. Moreover, symptomatic Covid-19 patients or people admitted to hospital for Covid-19 are probably overrepresented (whereas, e.g., those asymptomatic tested positives are underrepresented). Second, as the study was conducted among members of a social media community, selection bias might have been occurred, thus limiting participation of those who do not regularly access to social media and, in general, of all people affected by ‘digital poverty’ and ‘digital inequality’. Third, both scales were developed in Italian and validated within an Italian sample, thus limiting their use in other cultural and geographical contexts. Fourth, both scales did not include open-ended questions, so the study did not collect qualitative data that might have provided more insights into participants' experiences. Fifth, as with any study that relies on online data collection, biases such as response bias and social desirability bias might have affected the results. Finally, due to the time of data collection, participants tested positive or diagnosed in the early pandemic stages were being asked to recall a period that occurred months earlier; thus, their recollections might not be accurate.
Conclusions and future research
The present study indicates that both the CEDISC and the COINS represent two valid and reliable measures that may be used in studies examining the role of stigma and discrimination in Covid-19 patients, and in research evaluating interventions designed to mitigate stigma and discrimination in this population. Both scales, therefore, could be incorporated into public health surveys as a part of clinical and intervention research. These newly developed scales were specifically designed for Covid-19, but they might be also used in relation to other (similar) types of infections/pandemics in the future.
The factorial structure of both scales should also be replicated by a Confirmative Factor Analysis in a different sample of people infected with SARS-CoV-2 or with Covid-19. Moreover, future studies are needed on the changing dynamics of stigma in different stages of the pandemic. Finally, further research will be necessary to assess the psychometric properties of both scales in different populations and among people from different cultural backgrounds, as there are relevant differences in attitudes towards illness and experiences of illness, health and stigma across cultures. With this latter regard, a cultural adaptation of the scales will be needed as they were developed within a specific geographical context.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S204579602200021X
Data
Data will be available upon reasonable request.
Acknowledgement
The authors thank all participants for their time and excellent cooperation in the survey and focus group.
Author contributions
A. L., C. B. conceived and designed the study. C. B. designed the statistical analysis plan. A. L. and C. B. analysed the data and developed the figures and tables. A. L., C. B., L. B., M. C., T. V. B. reviewed the findings and contributed to the interpretation. All authors contributed intellectual content during the drafting and revision of the work and approved the final version of the article.
Financial support
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Conflict of interest
The authors declare that they have no competing interests.
Ethical standard
The authors assert that all procedures contributing to this study comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The study received ethical approval from the Verona University Committee on Research (CARP) (n.32.R1_2021).
Appendix A
Covid-19 experienced discrimination scale – CEDISC
This questionnaire asks about situations where you might have been treated unfairly (e.g., with hostility, avoidance, rejection, prejudice) by other people because you were infected by SARS-CoV-2 or diagnosed with Covid-19. NOTE: The situations listed below refer to the period following recovery from illness (or following isolation due to positive test) once you have returned to your usual social life.
Each item has 4 response options. Please read each question carefully and choose one answer for each question. If the answer does not apply to you, please answer ‘Not applicable’.
(1) How much people think you must have done something wrong to deserve getting Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(2) How much do you think it was a mistake to share with others that you have had Covid-19 as you have been treated unfairly?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(3) How much your friends and family members were ashamed of you because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(4) How much have you been treated unfairly by your family members because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(5) How much have you been treated unfairly by your friends because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(6) How much have you been treated unfairly in areas of public life (e.g., sporting centres, clubs, cultural events, voluntary groups) because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(7) How much have you been treated unfairly at work/at school because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(8) How much have you been treated unfairly by healthcare professionals (GP, nurses, treating clinicians) because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(9) How much have you been treated unfairly on your social media because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(10) Do you think that unfair treatment towards you might have been also determined by mass media information on Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(11) How much difficulty have you had returning back to work or other public activities after being tested negative for Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(12) How much have you been avoided while showing mild upper respiratory symptoms (e.g., runny nose, coughing, sneezing, etc.) even after being tested negative for Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
Appendix B
Covid-19 internalised stigma scale – COINS
This questionnaire asks about times when you might have felt uncomfortable (e.g., embarrassed, ashamed, guilty) because you were infected by SARS-CoV-2 or diagnosed with Covid-19.
Each item has 4 response options. Please read each question carefully and choose one answer for each question. If the answer does not apply to you, please answer ‘not applicable’.
(1) How much do you feel you are not as good a person as others because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(2) How much do you feel ashamed that you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(3) How much do you feel that it is your fault that you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(4) How much do you feel embarrassed that you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(5) How much have you avoided telling others that you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(6) How much have you stopped socialising with people because of their negative reactions to your having Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(7) How much do you feel uncomfortable to go anywhere outside of my house because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(8) How much would you understand your family members if they avoided you because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(9) How much would you understand friends if they avoided you because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(10) How much would you understand your neighbours if they avoided you because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(11) How much would you agree with employers if they would not employ you because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(12) How much would you understand people if they excluded you from areas of public life (e.g. sporting centres, clubs, cultural events, voluntary groups) because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
(13) How much would you understand doctor if he/she was not available to see you because you have had Covid-19?
-
☐ Not at all ☐ A little ☐ Moderately ☐ A lot ☐ Not applicable
-
Open access
