Internalized stigma and its correlates among patients with severe mental illness

Introduction

Stigma is often quoted according to Goffman’s definition as an “attribute that is deeply discrediting” and that reduces the bearer “from a whole and usual person to a tainted, discounted one.” A person marked by stigma struggles with foreclosure, diminished self-esteem, discrimination, and isolation.¹

However, the process of stigmatization must occur before the consequences of stigma are visible in the experiences of a given person. According to Link et al, this commences when individual differences are distinguished. Then, dominant cultural beliefs link labeled persons to negative characteristics – to stereotypes. In consequence, categories of “us” and “them” are developed, and labeled persons experience status loss and discrimination that lead to multiple negative emotional consequences.²

Mental health research has differentiated three different interrelated levels of stigma: the social level, the structural level, and the internalized level.³ Social stigma, which may be seen in different fields of life, classifies mentally ill people by society because of their disease. Condemnation of mentally ill persons is usually associated with belief in stereotypes and discrimination. Structural stigma is directly connected with social stigma, but it is related to institutional practices, cultural norms, and conditions that are proper to the society.^3,4 Research on a social and structural level is equally important. This study focuses on internalized stigma in severe mental disorders, which corresponds to the individual stigma level emphasizing the importance of protection and strengthening factors linked with the internalization stigma process. According to the stigmatization process described by Link et al, internalized stigma is its last stage.² Understanding how internalized stigma develops and how it interacts with other intrapersonal and health-related factors seems to be crucial for research and clinical practice. According to Livingston and Boyd, internalized stigma is

a subjective process, embedded in the sociocultural context, characterized by negative feelings (about the self), maladapted behavior, the transformation of identity or the application of stereotypes resulting from an individual’s experiences, perceptions or anticipation of negative social reactions on the basis of their mental illness.

Psychiatric patients consider themselves as devalued participants of social life, and they believe that stereotypes about mental illness are genuine.⁵ Research shows that mental illness is one of the most socially exclusive stigmata. The problem of stigma is widely discussed in research and in the field of shaping social attitudes. However, it is often affected by definitional chaos and misuse of the term. Currently, due to undergoing changes in the psychiatric and clinical treatment of patients in European countries such as Poland, there is increasing focus on the idea of an “open door” and environmental psychiatry, causing detachment of the stigma phenomenon.⁶ Many studies have confirmed a negative correlation between internalized stigma and self-esteem, the quality of life, the meaning in life, hope, and susceptibility to treatment.^7–9 To date, the correlation between internalized stigma and clinical variables remains unclear, and there are few studies exploring such correlations, especially in developing countries. Some studies confirm an association between the number of hospital admissions and internalized stigma,¹⁰ whereas others contradict them and show no such relation.¹¹ The duration of the disease has not been confirmed to be linked with internalized stigma.^8,10 The influence of the type of clinical diagnosis is also unestablished.^11,12 On the contrary, the results of Holubova et al showed that the degree of internalized stigma is lower in patients with unipolar affective disorder compared with patients with psychotic disorders (PDs)¹³

Stigmatization and discrimination against people with mental illness are universal problems. These issues continue to be of particular importance, in light of the difficulties associated with implementing the social integration of psychiatric patients, which is a basic postulate of modern psychiatry and clinical psychology. Taking into account the individual perception of the social context of each patient, it is necessary to propose an individualized and integrated treatment strategy. Nowadays, psychiatrists should put a lot of emphasis on activities directed against stigma, including internalized stigma because there is no doubt that stigma in mental illness is one of the most important obstacles to the improvement and progress of mental health care.¹⁴ The purpose of this study was to measure the level of internalized stigma in patients with severe mental illnesses [PDs, unipolar and bipolar affective disorders (BDs)] and to analyze the relationship between demographic and clinical variables and internalized stigma in those diseases. In this study, it was assumed that health-related factors such as the type of diagnosis, the duration of the disease, the type of treatment (outpatient vs inpatient), and the number of hospitalizations could correlate with the level of internalized stigma in patients with severe mental illnesses as well as demographic variables. This assumption is the development of the theoretical socio-cognitive-behavioral model of internalized stigmatization proposed by Muñoz et al, which does not take into account the clinical factors that constitute a significant context of the reality of every patient suffering from severe mental illness.¹⁵

Patients and methods

Study design

A two-phase cross-sectional study design was used. Phase I was a pilot study designed to check the reliability (test–retest reliability and internal consistency reliability) of the Internalized Stigma of Mental Illness (ISMI) scale by Ritsher et al, a psychometric instrument measuring the internalized stigma – in the Polish sociocultural context.¹⁶ Permission to use the tool was obtained directly from the author, who provided us with the original tool and its Polish version. The translation, due to lack of the Polish validation, was prepared by two independent translators, at the I Psychiatric Clinc in the Institute of Psychiatry and Neurology in Warsaw. Phase II was designed to evaluate the level of internalized stigma and its clinical correlates among patients with severe mental disorders. The project was conducted within the framework of research aimed at promoting young scientist funded by the Ministry of Science and Higher Education (registration number Pbmn 165). The Ethics Committees of the Wroclaw Medical University, Poland, approved the study protocol (no 404/2014). All participants provided informed consent and signed the agreement to participate in the study.

Study sample and setting

The study took place in three psychiatric hospitals in Lower Silesia in Poland between 2015 and 2017. The clinical groups consisted of patients with severe mental illnesses defined as nonorganic PD, unipolar affective disorder, depression (D), and BD. The inclusion criteria were participants of both sexes, between 18 and 70 years old, who were hospitalized on stationary or daily units for at least 4 weeks and whose mental status improved according to the clinical assessment. For this study, severe mental illness was defined as the presence of the following diagnoses in the past month: major depressive disorder; bipolar disorder I and II; schizoaffective disorder; and persistent delusional disorder according to the Tenth Revision of the International Classification of Diseases (ICD-10 criteria). The inclusion of participants in the study at this stage of treatment process is associated with an increased likelihood that they will have preserved insight into their illness. The exclusion criteria were severe acute symptomatology, comorbid and substance abuse disorders, organic brain disease, severe somatic disease, and intellectual disability. Participation in the study was voluntary, without any incentives and after confirming the participant’s understanding of the purpose of the study, and a written informed consent was obtained. The questionnaires were filled in a separate room at the hospital where the patient was hospitalized. During the study, participants were guaranteed privacy, the possibility to ask questions at all times and access to the help of the researcher. The time was not limited, although it usually took from 30 to 60 minutes to fill in the questionnaires. Participants returned the completed questionnaires to the researcher. Phase I included 34 patients who filled in the self-reported questionnaire on internalized stigma (ISMI) twice with a 1-week gap (test–retest reliability), which minimized the occurrence of any random factors, developmental aspects, and the effects of remembering answers. One hundred twenty participants took part in Phase II. Finally, data from 114 patients were collected for statistical analyses due to missing data. Four patients resigned to participate in the study after they started filling in the questionnaires.

Measuring instruments

The following instruments were used:

1. The demographic questionnaire: the individual profile sheet that contained information about patient sex, age, employment status, education, and marital status.
2. The clinical questionnaire: filled in by the researcher after the analysis of the patient’s medical records, who conducted a structured interview including information on the type of diagnosis, duration of illness, number of hospitalizations, the type of treatment (inpatient vs outpatient), current pharmacological treatment, and the presence of suicide attempts.
3. The ISMI scale: one of the most commonly used questionnaires (ISMI), originally developed by Ritsher et al and translated into Polish, was used to measure the level of internalized stigma among participants.¹⁶ It consisted of 29 statements, allowing calculation of the results in five domains: 1) alienation, 2) stereotype endorsement, 3) experience of discrimination, 4) social withdrawal, and 5) stigma resistance. The score in each domain was calculated by adding the item scores together and then dividing them by the total number of answered items. If anything was not answered, the result was divided by the total number of answered items. The resulting score ranged from 1 to 4. In the study, a four-category method was used to interpret each domain.¹⁷ The interpretations of the results were as follows: 1.00–2.00, minimal/no internalized stigma; 2.01–2.50, mild internalized stigma; 2.51–3.00, moderate internalized stigma; 3.01–4.00, severe internalized stigma. The ISMI was established as reliable and valid for diverse cultures, disorders, and conditions.¹⁸ Cronbach’s α coefficient of reliability of the original version was 0.85.

Statistical analysis

The reliability of the test was verified using Cronbach’s α coefficient. The test–retest reliability was evaluated using the intraclass correlation coefficient. Normality of data was assessed using the Shapiro–Wilk test and visual examination using a histogram and a Q-Q plot. Because of lack of normality in our dataset, we chose to use nonparametric statistics (Mann–Whitney, Kruskal–Wallis, Spearman tests). Demographic variables, clinical variables, and stigma scores were calculated using descriptive statistics (mean and SD). The difference in quantitative variables between the diagnoses was assessed using the Kruskal–Wallis test with a post hoc analysis of multiple comparisons using the Mann–Whitney test with the Holm correction. Differences in the qualitative variables between diagnoses were calculated using the chi-squared test with multiple comparisons applying the Holm correction. The correlation between stigma scores and quantitative variables was assessed using the Spearman correlation coefficient, while the Kruskal–Wallis test with a post hoc analysis of multiple comparisons using the Mann–Whitney test with the Holm correction was performed in case of qualitative variables. The statistical significance level was set at 5% for all statistical tests. The analysis was carried out in R for Windows, version 3.4.3.¹⁹

Ethical considerations

This study touches very sensitive areas of the human psyche. Hence, some negative psychological results of participating in the study were possible, such as sadness, fear, or anger. To minimize those effects, the following points were ensured:

1. all patients were informed about the possibility of having strong negative or positive emotions during and after filling in the questionnaires;
2. all patients were informed about the possibility of quitting the study at any time without any consequences;
3. patients began to fill in the questionnaires after making sure they were ready and prepared to take part in the study;
4. during and after taking part in the study, the participants were able to converse about any reflections and emotions with the researcher, who was obliged to provide psychological support.

Results

Reliability of the IMSI scale

Cronbach’s α reliability coefficient for the IMSI scale was 0.85 in the first time point and 0.87 in the second time point, indicating a good internal consistency for IMSI. The test–retest reliability was evaluated using the intraclass correlation coefficient and amounted to 0.83 for the total IMSI score. Stigma resistance (0.44) was the only domain that showed a fairly poor repeatability of the scores.

Study sample characteristics

The demographic and clinical characteristics of the study sample (n=114) are presented in Table 1. There were slightly more women than men in the study (55%). The majority of the respondents were patients with PD (n=51), followed by patients with unipolar affective disorder (n=44), and patients with BD (n=19). The mean age of the participants was 42.46±14.1 years. Patients with PDs were significantly younger compared with patients with D (P=0.00043). In the clinical groups, there were no statistically significant differences in the education level, employment status, or marital status. The majority of the participants had vocational training (38%), received annuity (41%), and were single (46%). The mean duration of the disease was 9.65±8.7 years among all participants, and it did not differ significantly between clinical groups. Fifty-eight percent (n=67) of the study participants were outpatients. The mean number of hospital admissions was 5.95±6.9. Patients with PD were hospitalized significantly more often than patients with D (P=0.011). However, the highest mean number of hospitalizations was observed in patients with BD (7.68±9.2), but due to the noticeably lower number of patients (n=19) in this group, no statistical significance was observed. Forty-six patients (40%) reported that they attempted suicide. There was no significant difference in the number of suicide attempts between clinical groups. Seventy-nine participants (69%) were characterized by preserved insight. A lack of insight was more frequently observed (P=0.047) in patients diagnosed with psychosis (70% of patients with no insight were PD patients).

Table 1 Demographic and clinical characteristics of the study sample Notes: Data are presented as n (%) or mean±SD. *Statistically significant P-value. Abbreviations: D, depression; PD, psychotic disorder; F, female; M, male; ns, not significant.

Pharmacological treatment

According to the medical history, 79 patients (86.9%) used antipsychotic medications. Sixty-four patients (70.4%) used antidepressant medications and 40.7% of the patients used both antipsychotic and antidepressant drugs. Thirty-eight patients (40.8%) were treated with mood stabilizers – antiepileptic drugs and lithium. Twenty-one patients (23.1%) were treated with anxiolytic drugs.

Level of internalized stigma in severe mental disorders

As a whole, the study sample presented mild internalized stigma (2.23±0.5). However, a statistical difference was observed between patients with PDs and those with unipolar affective disorder and D (P=0.039; see Table 2). Depressed participants were characterized by higher stigma perception that was close to a moderate level of stigmatization based on the total ISMI score (2.46±0.6) and had a moderate level of stigmatization in the alienation domain (2.85±0.8). Moreover, the highest result was scored in the alienation domain (2.63±0.8) compared with the other stigma domains in all clinical groups.

Table 2 Internalized stigma in mental disorders in the study sample Notes: Data are presented as n (%) or mean±SD. aKruskal–Wallis rank sum test; in the case of a significant difference, a pairwise comparison using Wilcoxon rank sum test was used. Abbreviations: D, depression; PD, psychotic disorder; ISMI, Internalized Stigma of Mental Illness; ns, not significant.

Internalized stigma measured using the ISMI total score did not correlate with age (Spearman’s r=0.14; P=0.12). However, statistical analysis showed a positive correlation between age and the stereotype endorsement domain (Spearman’s r=0.21; P=0.026; see Figure 1).

Figure 1 Correlation between age and stereotype endorsement in the study sample.

Taking into account different demographic characteristics, there was no significant difference in the internalized stigma perception, either in the total ISMI score or in all ISMI subdomains between groups formulated according to gender, marital, as well as employment status (Table 3). However, an association with the education level was observed. Patients with vocational training were characterized by higher internalized stigma than those who graduated from universities (P=0.02). Moreover, the same effect was found in three of five subdomains: stereotype endorsement (2.30 vs 1.80, P=0.03), perceived discrimination (2.52 vs 1.90, P=0.02), and social withdrawal (2.52 vs 1.93, P=0.03). Higher stigma results refer to participants with vocational education.

Table 3 The level of internalized stigma perception as measured by ISMI in relation to demographic and clinical variables in the whole study sample Notes: aKruskal–Wallis rank sum test; in the case of a significant difference, a pairwise comparison using Wilcoxon rank sum test was used. *Statistically significant P-value. Abbreviations: ISMI, Internalized Stigma of Mental Illness; ns, not significant.

There was no difference between inpatient and outpatient treatment in terms of the total ISMI score (P=0.21) and all ISMI subdomains, including alienation (P=0.12), stereotype endorsement (P=0.34), perceived discrimination (P=0.18), social withdrawal (P=0.27), and stigma resistance (P=0.51). Moreover, there was no difference in the ISMI total score between patients with and without a history of suicide attempts (P=0.33) as well as between those with insight into the illness and characterized by a lack of such insight (P=0.078).

The duration of the disease significantly correlated with internalized stigma measured according to the ISMI total score (Spearman’s r=0.23; P=0.01) as well as the ISMI subdomains such as alienation (Spearmen’s r=0.20; P=0.03), stereotype endorsement (Spearman’s r=0.24; P=0.01), and perceived discrimination (Spearman’s r=0.21; P=0.02). The social withdrawal subdomain (P=0.06) and stigma resistance (P=0.57) were not related to the duration of the disease. Moreover, internalized stigma did not correlate with the number of hospitalizations in the study sample (P=0.38).

Discussion

The aim of this study was to measure the level of internalized stigma in patients with severe mental illnesses (defined as nonorganic PDs, unipolar affective disorder depression, D, BD) and to investigate its correlation with demographic and clinical variables in the Polish cultural context. Prior to discussing internalized stigmatization, it is essential to portray the level of social stigma of mental illness in Poland. According to research, the awareness of mental health disorders is considered low in Poland. Mental illness is considered shameful, and people believe that psychiatric patients should be kept out of sight.¹⁴

The study was based on the theoretical model of stigmatization in mental illnesses, taking into account the phenomenon of internalized stigma. The operationalization of internalized stigma was carried out using the IMSI questionnaire.¹⁶ The evaluation of the psychometric properties of the Polish version of the questionnaire in the Polish cultural context was accurate and enabled the observational phase of the study.

The obtained results indicated that the mean level of internalized stigma in the study sample was 2.23±0.5, which indicates mild internalized stigma. Compared with studies using the ISMI scale in other countries, this score is very similar. In the studies realized by Picco et al in Singapore (2.37),²⁰ Girma et al in Ethiopia (2.32),²¹ and Brohan et al in Lithuania (2.34),²² the mean level of internalized stigma was higher but was still considered mild. The results from Greece presented by Brohan et al (2.97)²² indicated moderate-to-severe internalized stigma. On the contrary, internalized stigma was lower in the study by Lv et al in China (2.17).²³ It was even lower in the results collected by Brohan et al from Slovenia (2.00)²² and very low in the study results by West et al from the USA (1.29)²⁴ compared with the present research. Results concerning Lithuania, Greece, and Slovenia were obtained from the pan European study.²² In most of the aforementioned studies, the mean level of internalized stigma is mild, with the exception of two studies – by West et al and concerning Slovenia from the pan European study by Brohan et al, where there was minimal to no internalized stigma.^22,24 Due to sociogeographic reasons, it may be assumed that the results from Slovenia should be comparable with those from Poland. However, Svab and Tomori compared the development of the Slovenian mental health service with services in Slovakia, UK, and the Netherlands. In Slovenia, mental health care is included in a nongovernmental sector, which primarily provides social support for patients with severe mental illnesses.²⁵ This socially oriented and integrated service may reduce the level of stigma among its users. However, such hypotheses exceed beyond the scope of this study.

In the present study, the significant difference between the levels of internalized stigma in different clinical groups is shown only between PDs and unipolar affective disorder and D. Depressed patients presented a higher total ISMI score (2.46±0.6) than patients with PDs (2.19±0.5). This is contradictory to the findings by Grover et al and Holubova et al, which showed that higher internalized stigma was observed in patients with a schizophrenia spectrum disorder.^13,26 However, there were also studies presenting no significant differences in the internalized stigma level between the clinical groups.^11,12 In previous studies, those discrepancies were explained by methodological differences. In the study by Holubova et al, interstudy differences were explained by different methods of data collection and differing numbers of outpatient facilities collecting the data. However, in the present study, data were also collected in various psychiatric hospitals. In addition, the type of treatment (inpatient vs outpatient) was taken into account in the analysis. The obtained results showed that there was no significant difference in stigma perception between hospitalized patients and those from day-care clinics. In this study, the significant difference between depressed and psychotic patients was visible especially in the alienation domain, which measures the subjective experience of being less than a full member of society, or having a spoiled identity.¹ This result can be understood as an effect of one of the axial symptoms of D associated with an excessive inadequate depreciation of self-value as a human being. Consequently, it may manifest itself in a greater sense of alienation. We did not see this effect in psychotic patients, who often have a certain lack of insight into their emotional states (which differs from a lack of insight into the illness). Therefore, this subject possibly does not concern them.

The analysis of the effect of the demographic factors on the level of internalized stigma showed that there were no significant differences between gender, employment, and marital status on the total ISMI score. However, participants with a higher education presented significantly lower levels of internalized stigma compared with those with vocational training. It may be assumed that higher education makes people more conscious of their disease, which may be related to their perception of the attitude society has toward mental disorders. This finding coincides with studies by Lv et al and Girma et al^21,23 and another pan European stigma study by Krajewski et al,²⁷ which provided similar results. Furthermore, there are studies confirming a relationship between internalized stigma and education in the domain of stereotype endorsement.^10,24 Taking into account this specific domain, in the present study, there was a positive correlation between age and the stereotype endorsement, which indicated that the older the patient was, the more he/she agreed with the common stereotypes about people with a mental illness. This relationship may be seen as a result of perceptual changes that also affect the area of modern psychiatry. They also suggest a greater openness and tolerance of mental disease in younger generations, which could be the result of social campaigns and psychoeducation that increase general social awareness in the field of mental illness.

Based on the analysis of all the clinical variables in the study, only the duration of the disease measured in years showed a significant relationship with the perceived level of internalized stigmatization, which is in correspondence with the results of Lv et al.²³ Hence, people tend to face increasing consequences of mental illness and are exposed to the increasing stigmatization components such as alienation, stereotype endorsement, and discrimination the longer they live with the disease. The symptoms and consequences of mental illness may be identified based on the long-lasting experience of negative events that affects a person’s daily living. Nevertheless, this result is not consistent with the studies by Vrbova et al and Shin et al, which showed no significant association between the level of internalized stigma and the duration of the disease.^10,28

Limitations of the study

This study has some limitations. In particular, the cross-sectional design does not provide a good basis for the analysis of the factors that can be assessed as influencing the internalized stigma perception, for example, a type of diagnosis; nevertheless, the important clinical and individual variables could be evaluated as stigma correlates. Furthermore, the group of patients with BD was small compared with the group of patients with PDs and unipolar affective disorder. This may explain the lack of significant associations between patients with BD and those from the remaining two clinical groups. Moreover, the small study sample may limit the generalization of the results. In this study, the severity of the symptoms was not taken into consideration, which may have affected the questionnaire answers. The duration of disease was significant, although the onset of the disease should have been analyzed as another factor.

Patient-filled questionnaires depend on the patient’s introspective ability and his/her willingness to share information and may also be considered a limitation. Hence, factors such as motivation and fatigue may have influenced the results.

Conclusion

To the authors’ best knowledge, this is the first study to investigate the level of internalized stigma and it correlates among people with severe mental illness in Poland. We took into consideration the health-related factors as well as demographic variables that could be important in the perception of internalized stigma. The study findings contribute to a better understanding of the theoretical background of internalized stigma model. It demonstrates that in Poland mild internalized stigma appears to be common among people with severe mental illnesses. Its level is comparable with that in other countries. The obtained results showed that the degree of perception of internalized stigma may be linked to the type of diagnosis, the duration of the disease, and the level of education. Moreover, age is identified as an associated factor of one of the components of internalized stigma, the stereotype endorsement. These observations seem to be important for future education, where emphasis should be placed on promoting knowledge and awareness about mental illnesses and its negative stereotypes, even starting from a primary education level. Furthermore, knowledge about mental illnesses in Poland is insufficient.

The results indicate an urgent need to introduce stigma assessment in psychiatric patients as a mandatory procedure in daily practice. It seems that this is of particular importance in the high-risk group – which involves long-term psychiatric patients, older people with lower education, and those diagnosed with D. In addition, according to the observation that the highest scores were obtained in the alienation domain, there is a need to develop internalized stigma interventions preventing people with severe mental disorders from experiencing negative social reactions. This finding is most important for policy makers and may have practical implications. Based on the perception of people with severe mental illness, who have emphasized their great sense of social alienation, the introduction of systemic programs increasing the sense of social belonging is mandatory.

Despite the fact that the concept of stigma in mental illness in social contexts is increasingly popular, stigma research as well as stigma assessment and reduction programs are still in their infancy. It seems that the in-depth analysis of individual factors, such as personality traits, should be a further implication of the present study. In addition, it would be important to analyze the relationship between social stigma and internalized stigma in further research as well as to establish psychological, clinical, and social predictors of the stigma phenomenon and the impact of the level of stigmatization on the treatment and recovery process.

On the individual level, these recommendations will promote the autonomy and better quality of life of many persons living with severe mental illness.

Acknowledgments

The project was conducted within the framework of research aimed at promoting young scientists, funded by the Ministry of Science and Higher Education (registration number Pbmn 165).

Author contributions

All authors contributed toward data analysis, drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Goffman E. Stigma and Social Identity. 1963. Englewood Cliffs, N.J., Prentice-Hall.
2.		Link BG, Phelan JC, Stigma C. Conceptualizing stigma. Annu Rev Sociol. 2001;27(1):363–385.
3.		Corrigan PW, Kerr A, Knudsen L. The stigma of mental illness: Explanatory models and methods for change. Appl Prev Psychol. 2005;11(3):179–190.
4.		Hatzenbuehler ML, Link BG. Introduction to the special issue on structural stigma and health. Soc Sci Med. 2014;103:1–6.
5.		Livingston JD, Boyd JE. Correlates and consequences of internalized stigma for people living with mental illness: a systematic review and meta-analysis. Soc Sci Med. 2010;71(12):2150–2161.
6.		Załuska M, Kobrzyńska-Zochowska E, Dyduch A. Psychiatryczne leczenie stacjonarne – zadania i koszty, ze szczególnym uwzgle{ogonek}dnieniem pacjentów chorych somatycznie i psychicznie [Psychiatric inpatient treatment – tasks and costs, in particular regarding patients with comorbid somatic and mental disorders]. Postep Psychiatr i Neurol. 2012;21(4):251–257.
7.		Oliveira SE, Carvalho H, Esteves F. Internalized stigma and quality of life domains among people with mental illness: the mediating role of self-esteem. J Ment Health. 2016;25(1):55–61.
8.		Mashiach-Eizenberg M, Hasson-Ohayon I, Yanos PT, Lysaker PH, Roe D. Internalized stigma and quality of life among persons with severe mental illness: the mediating roles of self-esteem and hope. Psychiatry Res. 2013;208(1):15–20.
9.		Ehrlich-Ben Or S, Hasson-Ohayon I, Feingold D, et al. Meaning in life, insight and self-stigma among people with severe mental illness. Compr Psychiatry. 2013;54(2):195–200.
10.		Vrbova K, Prasko J, Holubova M, et al. Self-stigma and schizophrenia: a cross-sectional study. Neuropsychiatr Dis Treat. 2016;12:3011–3020.
11.		Holubova M, Prasko J, Latalova K, et al. Are self-stigma, quality of life, and clinical data interrelated in schizophrenia spectrum patients? A cross-sectional outpatient study. Patient Prefer Adherence. 2016;10:265–274.
12.		Kamaradova D, Latalova K, Prasko J, et al. Connection between self-stigma, adherence to treatment, and discontinuation of medication. Patient Prefer Adherence. 2016;10:1289–1298.
13.		Holubova M, Prasko J, Matousek S, et al. Comparison of self-stigma and quality of life in patients with depressive disorders and schizophrenia spectrum disorders – a cross-sectional study. Neuropsychiatr Dis Treat. 2016;12:3021–3030.
14.		Beldie A, den Boer JA, Brain C, et al. Fighting stigma of mental illness in midsize European countries. Soc Psychiatry Psychiatr Epidemiol. 2012;47(Suppl 1):1–38.
15.		Muñoz M, Sanz M, Pérez-Santos E, et al. The state of knowledge about internalized stigma on the basis of the socio-cognitive-behavioral mode. Psychiatry Res. 2012;196(1):164.
16.		Ritsher JB, Otilingam PG, Grajales M. Internalized stigma of mental illness: psychometric properties of a new measure. Psychiatry Res. 2003;121(1):31–49.
17.		Lysaker PH, Roe D, Yanos PT. Toward understanding the insight paradox: internalized stigma moderates the association between insight and social functioning, hope, and self-esteem among people with schizophrenia spectrum disorders. Schizophr Bull. 2007;33(1):192–199.
18.		Boyd JE, Adler EP, Otilingam PG, Peters T. Internalized Stigma of Mental Illness (ISMI) scale: a multinational review. Compr Psychiatry. 2014;55(1):221–231.
19.		Core Team R. (2017). R: A language and environment for statistical computing. R Found Stat Comput Vienna, Austria URL; 2017 R Foundation for Statistical Computing. Available from: https://www.r-project.org/. Accessed August 8, 2018.
20.		Picco L, Lau YW, Pang S, et al. Mediating effects of self-stigma on the relationship between perceived stigma and psychosocial outcomes among psychiatric outpatients: findings from a cross-sectional survey in Singapore. BMJ Open. 2017;7(8):e018228.
21.		Girma E, Tesfaye M, Froeschl G, Möller-Leimkühler AM, Dehning S, Müller N. Facility based cross-sectional study of self stigma among people with mental illness: towards patient empowerment approach. Int J Ment Health Syst. 2013;7(1):21.
22.		Brohan E, Elgie R, Sartorius N, Thornicroft G; GAMIAN-Europe Study Group. Self-stigma, empowerment and perceived discrimination among people with schizophrenia in 14 European countries: the GAMIAN-Europe study. Schizophr Res. 2010;122(1–3):232–238.
23.		Lv Y, Wolf A, Wang X. Experienced stigma and self-stigma in Chinese patients with schizophrenia. Gen Hosp Psychiatry. 2013;35(1):83–88.
24.		West ML, Yanos PT, Smith SM, Roe D, Lysaker PH. Prevalence of internalized stigma among persons with severe mental illness. Stigma Res Action. 2011;1(1):3–10.
25.		Svab V, Tomori M. Mental health services in Slovenia. Int J Soc Psychiatry. 2002;48(3):177–188.
26.		Grover S, Avasthi A, Singh A, et al. Stigma experienced by caregivers of patients with severe mental disorders: A nationwide multicentric study. Int J Soc Psychiatry. 2017;63(5):407–417.
27.		Krajewski C, Burazeri G, Brand H. Self-stigma, perceived discrimination and empowerment among people with a mental illness in six countries: Pan European stigma study. Psychiatry Res. 2013;210(3):1136–1146.
28.		Shin YJ, Joo YH, Kim JH. Self-perceived cognitive deficits and their relationship with internalized stigma and quality of life in patients with schizophrenia. Neuropsychiatr Dis Treat. 2016;12:1411–1417.