http://orcid.org/0000-0003-2227-8233
Figures
Abstract
Objectives
Stigma toward people with mental health problems (MHP) in primary health care (PHC) settings is an important public health challenge. Research on stigma toward MHP is relatively scarce in Chile and Latin America, as are instruments to measure stigma that are validated for use there. The present study aims to validate the Opening Minds Scale for Health Care Professionals (OMS-HC) among staff and providers in public Chilean PHC clinics, and examine differences in stigma by sociodemographic characteristics.
Methods
803 participants from 34 PHC clinics answered a self-administered questionnaire. Confirmatory factor analysis was completed. Average 15-item OMS-HC scores were calculated, and means were compared via t-test or ANOVA to identify group differences. Correlations of OMS-HC scores with other commonly used stigma scores were calculated to evaluate construct validity.
Citation: Sapag JC, Klabunde R, Villarroel L, Velasco PR, Álvarez C, Parra C, et al. (2019) Validation of the Opening Minds Scale and patterns of stigma in Chilean primary health care. PLoS ONE 14(9): e0221825. https://doi.org/10.1371/journal.pone.0221825
Editor: Jorge Guillen, Universidad ESAN, PERU
Received: May 23, 2019; Accepted: August 15, 2019; Published: September 5, 2019
Copyright: © 2019 Sapag et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper. There are ethical restrictions on publicly sharing the data which are imposed by the Research Ethics Committee at the Pontificia Universidad Católica de Chile, approval number 16-090, as well as by other local Ethics Committees. Participants provided informed consent on the understanding that their data would 1) be kept private and confidential; 2) would be stored securely and only accessed by authorised members of the research team; 3) be reported in de-identified aggregate format. Requests for access to a deidentified copy of the data can, however, be sent to the University of Newcastle Human Research Ethics Committee (email: etica.investigacion@med.puc.cl) for their consideration. In addition, it may be that other local Ethics Committees have to review the request too.
Funding: This research was funded by a Chilean government Regular FONDECYT grant, #1160099 to JCS, entitled “Estigma hacia la Enfermedad Mental entre Profesionales de la Atención Primaria en Chile (Stigma toward mental illness among primary care professionals in Chile)", Fondo Nacional de Desarrollo Científico y Tecnológico (FONDECYT), Comisión Nacional de Investigación Científica y Tecnológica (CONICYT), Chile. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Stigma toward people with mental health problems (MHP) is a significant public health challenge [1,2]. Stigma is a phenomenon comprising negative thoughts and actions toward a certain individual or group, and according to Link & Phelan, “exists when elements of labeling, stereotyping, separation, status loss, and discrimination occur together in a power situation that allows them”([3], p. 377). There is ample evidence of stigma about people with MHP in health care settings and by health care professionals [4–6]. Stigmatization is a barrier to adequate care for those with MHP [7,8], and it can lead to them receiving lower-quality physical health care services than others [9,10]. Experiencing stigma in health care can also result in reduced MHP treatment adherence and engagement [11] and a decrease in quality of life [12,13] for those with MHP.
Primary health care (PHC) provides an opportunity to strengthen care for MHP. The high prevalence of mental disorders among PHC patients [14] and the fact that most patients with MHP will access the health care system through PHC providers [15,16] make this an ideal setting to implement early screening and treatment strategies for MHP. Additionally, services to treat MHP at the PHC level could potentially reduce stigma and discrimination, as people with these conditions are generally treated by the same providers and in the same, community-based location as people with other health conditions [17]. However, studies in multiple locations have confirmed the presence of stigma in PHC settings [4,18,19].
In Latin America, mental health care services are increasingly being integrated into the PHC system [20], and within the region, Chile has been a leader of this model, with interdisciplinary family health teams in public PHC centers providing mental health services [21]. The system of public PHC centers (Centros de Salud Familiar, CESFAM), as well as other public mental health centers, specialty care establishments, and hospitals, serves more than 75% of the Chilean population (>13.6 million people) [22,23]. Residents covered by public insurance register at the CESFAM that is located closest to their home and receive PHC services there, including the mental health services mentioned above [22].
There is limited research on stigma toward MHP in PHC settings in this region [24,25], despite the significant treatment gap for MHP and the reported stigmatizing attitudes among providers [25,26]. In Chile, specifically, only 38.5% of patients with a psychiatric diagnosis receive any kind of mental health care treatment [19]. One challenge for understanding and addressing stigma toward MHP in Chile is the lack of validated, culturally relevant instruments to measure the phenomenon [27].
To address this gap and enable further study of stigma in this context, the present study seeks to examine the results of the cross-culturally adapted Spanish Opening Minds Stigma Scale for Health Care Providers (OMS-HC) in Chilean PHC settings, exploring differences in levels of stigma by sociodemographic groups, and validate the instrument through analysis of its internal consistency, factor structure, and correlation with other scales commonly used to measure stigma.
Materials and methods
The current study is part of a cross-sectional project designed to investigate the presence of stigma toward MHP and substance use issues in Chilean PHC (“Stigma toward Mental Illness in Primary Health Care Professionals in Chile”, FONDECYT Regular #1160099).
Measures
PHC staff and providers completed the self-administered questionnaire in Spanish, which included a battery of scales to measure various aspects of stigma toward MHP and substance use issues. The OMS-HC was developed in Canada to measure attitudes and behavioral intentions toward MHP in health care providers, and one of the highest-quality instruments to measure stigma [28]. Created with a total of 20 items, initial factor analysis resulted in a two-factor, 12-item version of the scale [28]. However, a subsequent validation study resulted in a three-factor, 15-item version [29], which has been used in Canada [30] as well as internationally, including in Italy [31], Pakistan [32], and Singapore [33]. This version of the OMS-HC was used in the present study. The OMS-HC was chosen for this study because of its strong psychometric properties from prior validations [29–30], particularly in the PHC setting [30], and wide international use [31–33], as well as because research team members had been working on stigma research using this instrument since 2009. The cross-culturally adapted, 15-item OMS-HC score was analyzed as the primary outcome of interest.
A multi-phase cross-cultural adaptation process was completed, including nine phases (preparation; three independent translations; expert committee synthesis; researcher and PHC professional focus groups and PHC user cognitive interviews; expert committee synthesis 2; two independent back translations; expert committee synthesis 3; pilot testing; and final revisions). These phases were developed based on recommendations from Beaton et al. [34] and the WHO [35] and incorporated an array of diverse voices from PHC contexts in Chile to meet three Adaptation Objectives defined prior to beginning the adaptation process (Understandability, Relevance, and Acceptability and Answer Options) [35]. The original, English version and the cross-culturally adapted Spanish version can be found as Supplemental Information for this study (see S1 and S2 Files).
Multiple international validations of the OMS-HC [29–31], have found a three-factor structure of the 15-item scale, and we sought to examine this structure in the present study to facilitate international comparison. This instrument can result in a possible score of 15 to 75, with a higher score indicating more stigmatizing attitudes and behavioral intentions [28]. Questions in the scale address attitudes regarding disclosure of MHP, attitudes toward individuals with MHP, and the social distance professionals would like to have with people who have MHP. Response options are presented in a 5-item Likert scale (Strongly Disagree = 1, Disagree = 2, Neither Agree nor Disagree = 3, Agree = 4, and Strongly Agree = 5). Seven items in the scale require reverse coding [29].
The questionnaire also included cross-culturally adapted versions of other scales used to measure different aspects of stigma toward MHP or substance use issues. The Mental Illness: Clinician’s Attitudes (MICA) scale was designed to measure attitudes toward people with MHP in students and health professionals, and contains 16 items, with a theoretical score range form 16 (least stigmatizing) to 96 (most stigmatizing) [36]. The Modified Bogardus Social Distance (MBSD) scale measures attitudes toward certain populations by focusing on the types of relationships the respondent would be willing to have with members of that population [37]; in this case, two versions of the scale were used, referring to people with schizophrenia and cocaine paste dependence. The scale has a theoretical range of 6 to 24, with higher scores indicating greater social distance. The Recovery Scale (RS) measures beliefs regarding the potential of people with severe mental illness to recover (in this case, a severe mental illness or a severe addiction to a psychoactive substance) [38]. The RS has 13-items, its theoretical range is from 13 to 117, with higher scores demonstrating negative views about recovery potential. Finally, the Attribution Questionnaire (AQ-27) evaluates nine stereotypes about people with mental illness (in this case, schizophrenia), with a range of 3 to 27 to for each stereotype and higher scores indicating greater endorsement of the stereotype [38].
Demographic information was also collected from PHC staff and providers, including variables such as age, gender identity, marital status, nationality, indigenous identity, religious beliefs, role within the CESFAM, experience working in PHC, educational background, among other. There were also questions specifically regarding the participant’s experience with mental health care in the CESFAM, how important they believed mental health/substance use services are within the CESFAM, and personal experience (whether in themselves or close family/friends) with MHP, as these variables have been related to stigma among health care providers in the literature [32,39–41].
Sample
To sample CESFAMs, the research team first contacted all public Health Districts in the country. Out of 29 Health Districts, 14 expressed interest in participating in the study and eight were selected, representing seven out of 16 regions in Chile, from Atacama in the North to the Lakes region in the South. Municipalities within participating health districts and Directors of each CESFAM were contacted to receive approval.
Once CESFAM participation was confirmed, data collection was scheduled between March and October 2017, with special meeting times set aside by CESFAM Directors to encourage participation. Approximately 50% of providers and staff responded to the survey in these meeting times. All PHC staff and providers attending these meetings filled out the self-administered questionnaires after providing informed consent. In the Coquimbo region, because of a scheduling conflict, none of the physicians from the five CESFAMs there were able to participate, so the research team scheduled a second data collection date there, with successful participation of physicians at that time. Inclusion criteria for PHC staff included being an active, hired, and full-time member of the CESFAM staff at the time of data collection.
Analysis
The present study analyzed the sample of PHC staff and providers that responded to all OMS-HC items. To assess the factor structure suggested by Modgill et al. for the 15-item OMS-HC scale [29], confirmatory factor analysis was performed using structural equation modeling (SEM). The Chi-squared test was used to analyze the discrepancy between data and the theoretical model implied; although this test is sensitive to sample size, in this case it is valid, as the sample size was large (n = 803). To evaluate model fit, the root mean square error of approximation, comparative fit index, Tucker-Lewis Index, standardized root mean square residual, and coefficient of determination were calculated. Internal consistency of the OMS-HC scale was evaluated using Chronbach’s α. To evaluate construct validity of the scales, Pearson correlations were calculated between the OMS-HC scores and other stigma scale scores (MICA, MBSD, RS, AQ-27). Individual and average OMS-HC scale scores were computed, and mean scores between sociodemographic groups were compared using Student’s t-test and ANOVA with Bonferroni adjustment for multiple comparisons. Findings from these comparisons were confirmed using non-parametric tests (Wilcoxon ranks and Kruskal-Wallis). Statistical analysis was performed using Stata 14 and SPSS Version 17.
This study was approved by the Research Ethics Committee at the Pontificia Universidad Católica de Chile, approval number 16–090.
Results
Sociodemographic characteristics of CESFAM staff and providers are provided in Table 1.
A total of 803 professionals responded to all OMS-HC items and comprised the sample for this analysis, with a response rate of nearly 100%. The sample comprised mostly women (76.4%), the mean age of CESFAM staff and providers was 37.9 years, and nearly all participants were Chilean. A total of 19 different roles in the CESFAM, from cleaning personnel to nurses to physicians, were represented among survey respondents; more than 75% of participants belong to the health teams in the centers. On average, respondents have 10 years of experience working in PHC. Approximately 40% of the surveyed population had personal experience with MHP or substance use issues in either a close family member or friend, while 22% reported receiving treatment for these problems themselves.
Confirmatory factor analysis was performed to test whether the factor structure of the three subscales was appropriate in this sample. Item loadings from this analysis, as well as factor covariances, are presented in Table 2.
Multiple indicators of model fit were calculated for this analysis. The root mean square error of approximation was 0.052 (considered excellent if <0.06 [42,43]), the standardized root mean square residual was 0.048 (considered excellent fit if <0.08 [42,43]), and the coefficient of determination R2 was 89%, all indicating excellent model fit. The Tucker Lewis Index (0.798) and Comparative Fit Index (0.832) were lower than the ideal of ≥0.95 [42,43].
Analysis of the internal consistency of the complete, 15-item OMS-HC scale showed an acceptable, though slightly low, internal consistency with a Cronbach’s α of 0.69. The three subscales identified by Modgill et al. [29], which correspond to attitudes toward people with MHP, attitudes toward disclosure of MHP, and social distance from those with MHP, had lower Cronbach’s α in this sample (0.53, 0.48, and 0.60, respectively).
To analyze construct validity of the OMS-HC, Pearson correlations were calculated for OMS-HC scores and other scale scores, and results are presented in Table 3. The OMS-HC scale had relatively strong positive correlations with the MICA (0.683), AQ-27 (0.572), and MBSD schizophrenia scores (0.504). Weak negative correlations were found with the Recovery Scales (-0.101 for severe mental illness, and -0.109 for severe psychoactive substance addiction). All correlations were statistically significant.
In the overall sample, the mean OMS-HC score was 34.55, with a range of 15 to 58 and a standard deviation of 7.02. Exploration and comparison of means by sociodemographic characteristics (see Table 4) revealed statistically significantly lower levels of stigma (lower OMS-HC score) for the Health Team compared with the Administrative team in the CESFAM. Additionally, those with additional training in mental health had a significantly lower average OMS-HC score than those without training. Similarly, CESFAM staff or professionals who had a family member, friend, or their own personal experience with MHP or substance use issues had lower average OMS-HC scores than those who did now know someone who has received treatment for these issues.
Finally, there were some differences in average OMS-HC score by CESFAM role, as demonstrated in Fig 1. Psychologists had the lowest mean score (29.77), followed by occupational therapists (29.88) and social workers (31.08). Security personnel, cleaning personnel, and medical technicians had the highest averages of the sample, with mean OMS-HC scores of 40.00, 39.86, and 38.50, respectively. No differences in OMS-HC score were observed between men and women or between age groups.
Note: CESFAM = Centro de Salud Familiar (Chilean Public Primary Health Care Clinic). Bars represent 95% Confidence Interval. Security Personnel and Nursing Technician do not have Confidence Intervals represented due to small sample size.
Discussion
The main goal of this study was to validate and examine the factor structure of the cross-culturally adapted version of the OMS-HC in the Chilean context. Confirmatory factor analysis was completed, with multiple measures of model fit found to be acceptable and confirming the validity of the three-factor structure from Modgill et al [29] among Chilean PHC staff and providers. This three-factor structure was also validated in additional studies in Canada [30], with a similar three-factor structure containing 14 items found to be valid in Singapore [33]. In this validation, the Cronbach’s α for the three subscales were low, indicating that it is more appropriate, in the Chilean context, to utilize the complete, 15-item OMS-HC scale rather than individual subscales.
We sought to assess construct validity of the OMS-HC scale in the Chilean PHC context by measuring the extent to which the scale scores correlated with other, commonly used measurements of aspects of stigma. The strong positive correlation between the OMS-HC scale and the MICA scale was encouraging, indicating appropriate measurement of stigma toward MHP in PHC staff and providers. Moderate correlations with measures of social distance (MBSD) and stereotypes (AQ-27) toward those with schizophrenia suggests that some aspects of these phenomenon are being captured by the OMS-HC, however there are others that the scale does not address. The slight negative correlations with both versions of the RS were somewhat surprising, as a larger total RS score indicates more negative feelings regarding the potential for recovery. However, the OMS-HC does not explicitly address the potential for recovery in those with MHP, and thus these negative correlations are not of concern.
Additionally, the present study also sought to examine the results of the application of the OMS-HC in PHC centers throughout Chile, and identified the presence of stigma toward those with MHP among PHC staff and providers. The average OMS-HC score of 34.55 among the 803 study participants indicates that there is stigmatization of people with MHP in this setting. When considering average scores from other international studies of stigma in health care providers, this finding is fairly consistent. In a study of stigma in Canadian community health centres, the average score for the 15-item OMS-HC among providers was 30.38 [30], while the average 14-item OMS-HC score among varying health care providers in Singapore was 35.7 [33] (indicating greater stigma, as the summation of scores for 15 items would be higher than this). Due to the importance of culture and social context in shaping stigma toward MHP [27,44], as well as the different versions of OMS-HC used in these studies, cross-national comparisons are imprecise. However, we conclude that Chilean PHC workers and professionals have levels of stigma that are similar to those in other countries.
Finally, comparison of mean OMS-HC scores in sociodemographic groups revealed that people with greater experience or contact with people with lived experience of mental illness or substance abuse had lower levels of stigma. Those with personal experience of mental illness or substance abuse, whether in themselves, a close family member, or friend, as well as those with additional training in mental health had lower levels of stigma toward MHP and substance abuse. Similarly, in the CESFAM, psychologists and social workers, who are typically involved in the mental health programs provided in the PHC centers, demonstrated less-stigmatizing attitudes and behavioral intentions than their peers. This suggests that including training regarding stigma and interactions with those with MHP could be a viable strategy for intervention to reduce stigma, which is consistent with findings from intervention studies that recognize increased social contact with people in recovery and training regarding stigma as important elements for intervention success [45–47]. Targeting interventions toward those that do not typically interact with patients with MHP and substance use issues in the PHC setting may also be a valuable strategy for future work. In addition, the Spanish language OMS-HC could be used in healthcare settings in Chile and elsewhere to identify staff and providers that have greater stigma toward MHP and refer them to stigma reduction interventions or programs.
There are some limitations of this study. The self-report nature of the OMS-HC creates the potential for social desirability bias in the results, which could lead to artificially low OMS-HC scores. However, this may have been mitigated by the anonymous nature of the survey. Additionally, as this was a cross-sectional sampling of Chilean PHC professionals, differences between OMS-HC scores in subgroups of staff and providers should not be considered causal. Finally, the internal consistency found for the 15-item OMS-HC scale is low, just below the generally accepted α = 0.7, and some indicators of model fit in the confirmatory factor analysis were lower than ideal. It is possible that, in the Chilean context, the cross-culturally adapted version of the OMS-HC does not fully measure the concept of stigma, despite favorable pilot and cognitive testing during the adaptation process. Further research is required using this cross-culturally adapted scale to examine ways to improve reliability. Furthermore, future research incorporating the patient perspective and experience of stigma in PHC could add an important dimension to the understanding of stigma toward MHP in Chile, as PHC staff and providers may have bias when reporting their own ideas regarding MHP and stigma. While some research has been conducted to incorporate and measure the patient perspective on MHP stigma in Chile and the Latin America region [25,48], more should be done.
This, the present study represents the first large scale examination of stigma toward MHP in PHC providers in Chile, and is an important step forward for stigma studies in South America.
Conclusion
This study contributes to knowledge regarding stigma toward MHP in Chile, and validates the OMS-HC for future use in stigma studies in Chile and Latin America as a whole. The stigma results presented here, showing the presence of stigma toward MHP in Chilean CESFAM, demonstrate the need for future interventions to address and reduce this stigma in the region. Additionally, future use of the cross-culturally adapted Spanish language OMS-HC will be important for regional comparisons and further study. To address stigma in this context, this research team has applied for funding for an intervention study in CESFAMs in Chile to seek to reduce stigma toward MHP in PHC. Additionally, in July 2018, the Consortium for Stigma Research in Mental Health and Substance Use Issues in the Americas was created by fourteen practitioners, policy makers, and researchers representing six countries in the region to advance local research and interventions to reduce stigma toward MHP.
Supporting information
S1 File. Opening Minds Survey Spanish.
Opening Minds Scale for Health Care Professionals (OMS-HC) in Spanish. This is the cross-culturally adapted Spanish version of the Opening Minds Scale for Health Care Professionals (OMS-HC) used in the present study.
https://doi.org/10.1371/journal.pone.0221825.s001
(PDF)
S2 File. Opening Minds Survey English.
Opening Minds Scale for Health Care Professionals (OMC-HC) in English. This is the original version of the Opening Minds Scale for Health Care Professionals (OMS-HC), which was cross-culturally adapted for use in the present study in Spanish (for Spanish version, see S1 File).
https://doi.org/10.1371/journal.pone.0221825.s002
(PDF)
Acknowledgments
We thank all collaborators from Primary Health Care Centers and Servicios de Salud for their participation, partnership and dedication to this initiative.
References
- 1. Vistorte AOR, Ribeiro WS, Jaen D, Jorge MR, Evans-Lacko S, Mari JdJ. Stigmatizing attitudes of primary care professionals towards people with mental disorders: A systematic review. Int J Psychiatry Med 2018;53(4):317–338. pmid:29807502
- 2.
World Health Organization. Mental Health Action Plan: 2013–2020. Geneva2013.
- 3. Link BG, Phelan JC. Conceptualizing stigma. Annu Rev Sociol. 2001;17:363–85.
- 4. Lawrence D, Kisely S. Inequalities in healthcare provision for people with severe mental illness. J Psychopharmacol 2010;24(4_supplement):61–68.
- 5. Pellegrini C. Mental illness stigma in health care settings a barrier to care. Cmaj 2014;186(1):E17. pmid:24295864
- 6. Schulze B. Stigma and mental health professionals: a review of the evidence on an intricate relationship. Int Rev Psychiatry 2007;19(2):137–55. pmid:17464792
- 7. Clement S, Schauman O, Graham T, Maggioni F, Evans-Lacko S, Bezborodovs N, et al. What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychol Med 2015;45(1):11–27. pmid:24569086
- 8. Thornicroft G, Rose D, Kassam A. Discrimination in health care against people with mental illness. Int Rev Psychiatry 2007;19(2):113–22. pmid:17464789
- 9. Koroukian SM, Bakaki PM, Golchin N, Tyler C, Loue S. Mental illness and use of screening mammography among Medicaid beneficiaries. Am J Prev Med 2012;42(6):606–9. pmid:22608377
- 10. Thornicroft G. Physical health disparities and mental illness: the scandal of premature mortality. Br J Psychiatry 2011;199(6):441–2. pmid:22130744
- 11. Sirey JA, Bruce ML, Alexopoulos GS, Perlick DA, Friedman SJ, Meyers BS. Stigma as a barrier to recovery: Perceived stigma and patient-rated severity of illness as predictors of antidepressant drug adherence. Psychiatr Serv 2001;52(12):1615–20. pmid:11726752
- 12.
Stuart H, Arboleda-Florez JH., Sartorius N. Paradigms Lost: Fighting Stigma and the Lessons Learned. Oxford: Oxford University Press; 2012.
- 13. Verhaeghe M, Bracke P. Organizational and Individual Level Determinants of Stigmatization in Mental Health Services. Community Ment Health J 2007;43(4):375–400. pmid:17380385
- 14. Ansseau M, Dierick M, Buntinkx F, Cnockaert P, De Smedt J, Van Den Haute M, et al. High prevalence of mental disorders in primary care. J Affect Disord 2004;78(1):49–55. pmid:14672796
- 15. Kates N, Arroll B, Currie E, Hanlon C, Gask L, Klasen H,et al. Improving collaboration between primary care and mental health services. World J Biol Psychiatry 2018:1–18.
- 16.
Sapag JC, Mohamoud S., Khenti A. Addressing stigma and discrimination toward people with mental illness and/or substance use problems in primary care settings. In: Khenti A, Sapag J.C., Mohamoud S., Ravindran A., editor. Collaborative Mental Health: An Advanced Manual for Primary Care Professionals. Ontario, Canada: Centre for Addiction and Mental Health (CAMH); 2012.
- 17.
World Health Organization & World Organization of Family Doctors (WONCA). Integrating mental health into primary care: a global perspective. Geneva: World Health Organization; 2008.
- 18. Mascayano Tapia F, Lips Castro W, Mena Poblete C, Manchego Soza C. Estigma hacia los trastornos mentales: características e intervenciones. Salud mental 2015;38:53–58.
- 19. Minoletti A, Rojas G, Horvitz-Lennon M. Salud mental en atención primaria en Chile: aprendizajes para Latinoamérica. Cad Saúde Colet 2012;20:440–447.
- 20. Sapag JC, Rush B, Ferris LE. Collaborative mental health services in primary care systems in Latin America: contextualized evaluation needs and opportunities. Health Expect 2016;19(1):152–169. pmid:25655020
- 21.
Ministerio de Salud de Chile. Plan Nacional de Salud Mental y Psiquiatría. 2000.
- 22.
Fondo Nacional de Salud. 2018 11 Dec. 2018. Red Pública Preferente. <https://www.fonasa.cl/sites/fonasa/beneficiarios/informacion-general/red-preferente>. 11 Dec. 2018.
- 23. Castillo-Laborde CA-S X, Hirmas-Adauy M, Matute I, Delgado-Becerra I, Nájera-De Ferrari M, Olea-Normandin A, et al. Health Insurance Scheme Performance and Effects on Health and Health Inequalities in Chile. MEDICC Review 2017;19(2–3):57–64.
- 24. Sapag JC, Sena BF, Bustamante IV, Bobbili SJ, Velasco PR, Mascayano F, et al. Stigma towards mental illness and substance use issues in primary health care: Challenges and opportunities for Latin America. Glob Public Health 2018;13(10):1468–1480. pmid:28766377
- 25. Mascayano F, Tapia T, Schilling S, Alvarado R, Tapia E, Lips W, et al. Stigma toward mental illness in Latin America and the Caribbean: a systematic review. Braz J Psychiatr 2016;38(1):73–85.
- 26.
Pan American Health Organization. Plan of Action on Mental Health: 2015–2020. Washington, DC, USA2014.
- 27. Yang LH, Valencia E, Alvarado R, Link B, Huynh N, Nguyen K, et al. A theoretical and empirical framework for constructing culture-specific stigma instruments for Chile. Cad Saude Colet 2013;21(1):71–79. pmid:23990755
- 28. Kassam A, Papish A, Modgill G, Patten S. The development and psychometric properties of a new scale to measure mental illness related stigma by health care providers: The opening minds scale for Health Care Providers (OMS-HC). BMC Psychiatry 2012;12(1):62.
- 29. Modgill G, Patten SB, Knaak S, Kassam A, Szeto AC. Opening Minds Stigma Scale for Health Care Providers (OMS-HC): Examination of psychometric properties and responsiveness. BMC Psychiatry 2014;14(1):120.
- 30. van der Maas M, Stuart H, Patten SB, Lentinello EK, Bobbili SJ, Mann RE, et al. Examining the Application of the Opening Minds Survey in the Community Health Centre Setting. Can J Psychiatry 2018;63(1):30–36. pmid:28665144
- 31. Destrebecq A, Ferrara P, Frattini L, Pittella F, Rossano G, Striano G, et al. The Italian Version of the Opening Minds Stigma Scale for Healthcare Providers: Validation and Study on a Sample of Bachelor Students. Community Ment Health J 2018;54(1):66–72. pmid:28647819
- 32. Laraib A, Sajjad A, Sardar A, Wazir MS, Nazneen Z. Perspective About Mental Illnesses: A Survey Of Health Care Providers Of Abbottabad. J Ayub Med Coll Abbottabad 2018;30(1):97–102. pmid:29504341
- 33. Chang S, Ong HL, Seow E, Chua BY, Abdin E, Samari E, et al. Stigma towards mental illness among medical and nursing students in Singapore: a cross-sectional study. BMJ Open 2017;7(12).
- 34. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000;25(24):3186–3191. pmid:11124735
- 35.
World Health Organization (WHO). Management of substance abuse: Process of translation and adaptation of instruments [Internet]. https://www.who.int/substance_abuse/research_tools/translation/en/.
- 36. Gabbidon J, Clement S, van Nieuwenhuizen A, Kassam A, Brohan E, Norman I, et al. Mental Illness: Clinicians’ Attitudes (MICA) scale-psychometric properties of a version for healthcare students and professionals. Psychiatry Res 2013;206(1):81–7. pmid:23084597
- 37.
Link B, T. Cullen F, Frank J, F. Wozniak J. The Social Rejection of Former Mental Patients: Understanding Why Labels Matter. 1987.
- 38.
Corrigan P. A Toolkit for Evaluating Programs Meant to Erase the Stigma of Mental Illness. Illinois Institute of Technology; 2012.
- 39. Evans-Lacko S, Henderson C, Thornicroft G. Public knowledge, attitudes and behaviour regarding people with mental illness in England 2009–2012. Br J Psychiatry Suppl 2013;55:s51–7. pmid:23553695
- 40. Evans-Lacko S, Malcolm E, West K, Rose D, London J, Rusch N, et al. Influence of Time to Change’s social marketing interventions on stigma in England 2009–2011. Br J Psychiatry Suppl 2013;55:s77–88. pmid:23553699
- 41. Henderson C, Noblett J, Parke H, Clement S, Caffrey A, Gale-Grant O, et al. Mental health-related stigma in health care and mental health-care settings. Lancet Psychiatry 2014;1(6):467–482. pmid:26361202
- 42. Lt Hu, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling 1999;6(1):1–55.
- 43. Schreiber JB, Nora A, Stage FK, Barlow EA, King J. Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review. J Educ Res 2006;99(6):323–338.
- 44. Abdullah T, Brown TL. Mental illness stigma and ethnocultural beliefs, values, and norms: an integrative review. Clin Psychol Rev 2011;31(6):934–48. pmid:21683671
- 45. Gronholm PC, Henderson C, Deb T, Thornicroft G. Interventions to reduce discrimination and stigma: the state of the art. Soc Psychiatry Psychiatr Epidemiol 2017;52(3):249–258. pmid:28144713
- 46. Knaak S, Modgill G, Patten SB. Key ingredients of anti-stigma programs for health care providers: a data synthesis of evaluative studies. Can J Psychiatry 2014;59(10 Suppl 1):S19–26. pmid:25565698
- 47. Corrigan PW, Morris SB, Michaels PJ, Rafacz JD, Rusch N. Challenging the public stigma of mental illness: a meta-analysis of outcome studies. Psychiatr Serv 2012;63(10):963–73. pmid:23032675
- 48. Alonso J, Buron A, Bruffaerts R, He Y, Posada‐Villa J, Lepine JP, et al. Association of perceived stigma and mood and anxiety disorders: results from the World Mental Health Surveys. Acta Psychiatrica Scandinavica 2008;118(4):305–314. pmid:18754833