Validity and Reliability Study of the Turkish Version of the Orbach Mikulincer Mental Pain Scale-8 in a Clinical Population

Introduction

Scientific advances have improved treatment options for many leading causes of death, but suicide and suicide-related mortality rates have remained stable for decades.¹ Suicide is often the result of the interaction of biological and social risk factors such as sociodemographic characteristics, personality traits, lack of social support, genetic predisposition, physical illnesses, and mental disorders.² The importance of suicide, which is among the most urgent public health concerns facing society, is highlighted in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) by recommending Suicidal Behavior Disorder as “Conditions for Further Study”.³

Suicide has preoccupied researchers for many years, and several theories have been proposed to explain suicidal behavior. Durkheim⁴ suggested that suicide is caused by social factors such as lack of belonging, while Freud⁵ suggested that suicide is caused by anger directed inward at a loved one. In the following years, Neuringer reported that cognitive rigidity⁶ and Beck reported that hopelessness were causal factors for suicide.⁷ Since the 1990s, researchers have proposed theories focusing on the idea that suicide is a way of escaping unbearable circumstances such as negative self-awareness and aversive emotions,^8,9 Williams suggested that suicide is a remedy to escape feelings of defeat and entrapment.¹⁰

Psychological pain is another unbearable condition in which suicide is accepted as a way of escape.¹ Although attempts at conceptualization go back much further, Shneidman stands as the pioneer who systematically defined psychological pain and put forward the psychological pain-suicide hypothesis. With the term psychache, Shneidman defined psychological pain as an undesirable condition encompassing emotions such as shame, guilt, humiliation, grief, despair, and anger. Shneidman’s psychache hypothesis posits that suicide is a way of escape from intense, extreme, and unbearable psychological pain, and there is no suicide without psychache.¹¹

Following Shneidman’s pioneering work, new definitions of psychological pain have been suggested, incorporating terms such as mental, psychic, and emotional pain.¹² According to Orbach et al, psychological pain, as an irreversible form of distress, is characterized by profound negative emotions and cognitions originating from adverse alterations in the self.¹³ Common definitions of psychological pain often include loss of personal values, thwarted psychological needs, and persistence of pain.^14,15 Many studies have confirmed the clinical relationship between psychological pain and suicide in mental disorders, especially major depression, and have demonstrated that psychological pain serves as a mediating factor in the manifestation of suicide risk.^16–18

The need to develop psychometric scales has emerged to make psychological pain, which is a subjective experience, clinically functional and adequately assessed. One of the scales assessing psychological pain is the Orbach and Mikulincer Mental Pain Scale (OMMP), with 44 items and nine factors.¹³ Orbach et al¹³ developed the original OMMP scale by asking a sample of inpatients and healthy individuals to answer various questions about psychological pain. The scale items were administered to two more samples of volunteers, and a final version of 44 items was created. The OMMP includes detailed questions about the cognitive and emotional components of psychological pain and assesses both the intensity and sub-dimensions of psychological pain.¹⁹ Studies conducted in clinical and non-clinical samples and different cultures revealed varying results about the psychometric properties and scale structure of the OMMP, and the internal consistency of the OMMP subscales did not meet the recommended values. For more consistent results, it has been suggested to reduce items and subscales to create a more psychometrically valid scale. Therefore, Casanova et al evaluated the psychometric properties of the 44-item OMMP scale in a large sample of individuals from different ethnic backgrounds, with different activity levels and reduced the number of items and factors; developed the Likert-type OMMP-8 scale with eight items and three factors. Casanova et al reported Cronbach’s alpha coefficients of 0.835 for the “Experience of Irreversibility” factor, 0.856 for “Emotional Flooding”, and 0.767 for “Narcissistic Wounds” during the internal consistency analysis of the OMMP-8 scale.¹²

The lack of progress in suicide prediction and prevention requires developing tools to assess suicide risk. Although mild psychological pain is common in the general population, severe psychological pain is associated with psychopathology and suicide risk.²⁰ The OMMP-8 scale, which assesses the intensity of psychological pain quickly, has not yet been adopted in Turkish. We aim to demonstrate the validity and reliability of the Turkish version of the OMMP-8 scale in a sample of healthy individuals and individuals with major depressive disorder. Thus, we believe that we will provide researchers and clinicians with a tool to assess psychological pain in clinical and non-clinical samples and to use it as a possible risk indicator for suicide.

Materials and Methods

Permission and Translation

The first author obtained permission from Russell Baker, the corresponding author of the OMMP-8 scale study, via e-mail before the study. Two bilingual psychiatrists translated the original version of the OMMP-8 scale into Turkish. The Turkish version was translated back into English and compared by linguists who had no previous access to the original scale. Both researchers and linguists approved the Turkish version before the study.

Sample and Procedure

The sample consisted of individuals with major depressive disorder and healthy volunteers. We included 166 patients diagnosed with major depressive disorder according to the DSM-5, who applied to Çukurova University School of Medicine, Department of Psychiatry outpatient clinics. Inclusion criteria were to be between the ages of 18–65, to know Turkish and to be literate, and to have no cognitive deficits and psychotic symptoms. In the control group, we included 144 healthy individuals who were not diagnosed with any psychiatric disorders, were similar to the depression group in terms of sociodemographic variables such as age, gender, and educational status, and lived in the same neighborhood. The control group consisted of hospital staff relatives who willingly expressed interest in participating in scientific research. No financial incentives were provided to the participants.

After psychiatric interviews according to DSM-5 criteria, we excluded six participants with generalized anxiety disorder and seven with obsessive-compulsive disorder from the depression group to eliminate the confounding effects of comorbid mental disorders. Five participants were also excluded from the depression group due to not completing the scales. We included only patients utilizing either Selective Serotonin Reuptake Inhibitors or Serotonin-Norepinephrine Reuptake Inhibitors in monotherapy for the study. The mean Hamilton Depression Rating Scale (HDRS) score for the patient group is 24.9, with a standard deviation of 10.1. The severity of depressive disorder in the patient group has been categorized as moderate to severe.²¹ From the control group, we excluded two participants who filled out the scales incompletely, four participants with generalized anxiety disorder, and eight with panic disorder. We continued the study with 148 participants with major depressive disorder and 130 healthy controls. We re-administered the OMMP-8 scale to 52 participants two weeks apart to assess test-retest reliability. The Ethics Committee of Çukurova University Faculty of Medicine for Non-Interventional Clinical Research approved the study on September 01, 2023 (decision no. 33). All participants completed an informed consent form. The study was conducted in accordance with Declaration of Helsinki.

Sample Size

It is recommended to have participants at least 5 to 10 times greater than the number of scale items in validity and reliability studies.²² We also conducted a power analysis with G Power v3.1 to calculate the minimum required sample size. The study required at least 105 participants in each group with a moderate effect size (Cohen’s d = 0.50), a power of 0.95, and a margin of error of 0.05 (p = 0.05).²³ As a result, we concluded that the sample of 278 participants was sufficient.

Measures

Sociodemographic and Clinical Data Form

This form inquires about sociodemographic features such as age, gender, marital status, educational status, and place of residence. This form also includes clinical data, such as a history of suicide attempts. Any action to end life at any period of life was considered a suicide attempt. The presence of suicide attempts was confirmed through a comprehensive review of patient history, interviews with family members, and examination of health system records.

Orbach and Mikulincer Mental Pain Scale-8 (OMMP-8 Scale)

Casanova et al developed the OMMP-8 scale by reducing the number of items and factors of the 44-item OMMP. It is an 8-item and 3-factor scale assessing the intensity of psychological pain. It includes three factors: Experience of irreversibility, Emotional flooding, and Narcissistic wounds. The Cronbach’s alpha coefficients are 0.835, 0.856, and 0.767, respectively. It is a self-report scale with a 5-point Likert-type (0=strongly disagree, 4=strongly agree). Higher scores indicate higher levels of psychological pain.^12,19

Mee-Bunney Psychological Pain Assessment Scale (MBPPAS)

This self-report, 10-item scale evaluates the frequency and intensity of current and past three-month psychological pain in a 5-point Likert-type. The responses on the scale are evaluated between “strongly disagree” and “strongly agree”. Higher scores reflect more frequent and intense psychological pain.¹⁶ Demirkol et al demonstrated that Cronbach’s alpha value was 0.95 in the Turkish adaptation study.²⁴

Hamilton Depression Rating Scale (HDRS)

The HDRS is a clinician-administered scale consisting of 17 questions that measures the severity of depressive symptoms. A cut-off score of 17 was determined to distinguish mild depression from moderate depression and 24 to distinguish moderate depression from severe depression.²¹ Higher scores on the scale indicate an increase in the severity of depressive symptoms. Akdemir et al found Cronbach’s alpha coefficient as 0.75 in the Turkish adaptation study.²⁵

Beck Scale for Suicidal Ideation (BSIS)

The BSIS, which assesses suicidal ideation and designed attempts, consists of 19 items and five sections and is administered by a clinician. The scale’s sections are named ‘Wish to be Dead, Passive Suicidal Ideation, Active Suicidal Ideation, Specific Plans, and Suicide Attempts’. The scale score is calculated as the arithmetic sum of the item scores. High scores reflect the seriousness of suicidal ideation.²⁶ In the Turkish validity and reliability study, Cronbach’s alpha was 0.84.²⁷

Beck Hopelessness Scale (BHS)

The BHS, which assesses the level of hopelessness, is a 20-question self-report scale. Participants answer the questions as yes or no.²⁸ No definitive cutoff score has been designated for the scale. An increase in the scale score supports that the rate of hopelessness is exacerbated. In the Turkish validity and reliability study, Cronbach’s alpha was 0.86.²⁹

Statistical Analysis

Exploratory factor analysis (EFA) was performed to determine the structure of the OMMP-8 scale. The principal axis factoring (components) method was chosen for factor extraction, and the varimax method was chosen for rotation. The factors with eigenvalues higher than one were extracted. Confirmatory factor analysis (CFA) was performed to verify the structure defined by EFA for the OMMP-8 scale. The Mardia test revealed that multivariate normal distribution was not met before CFA. Therefore, robust maximum likelihood estimation based on the covariance matrix was used. The Comparative Fit Index (CFI), Normed Fit Index (NFI), Goodness of Fit Index (GFI), and Root Mean Square Error of Approximation (RMSEA) were assessed. Cronbach’s α, McDonald’s ω coefficients, and corrected item-total score correlations were calculated for the internal consistency and reliability of the OMMP-8 scale. The intraclass correlation coefficient was used for test-retest reliability. Correlation analysis was performed with the scores of the HDRS, BSIS, BHS, and MBPPAS to determine the convergent validity of the OMMP-8 scale. Pearson correlation analysis was used because the scale scores were normally distributed. In addition, we performed discriminant analysis to determine whether the OMMP-8 scale scores differentiate depression and control groups and the participants with and without suicide attempts in the depression group. Statistical analyses were performed with Jamovi (Version 2.3.28), JASP (Version 0.17.2.1), and LISREL (version 8.50) programs. The significance level was considered as 0.05 (p-value).

Results

The mean age of the participants in the depression group was 36.72 ± 11.24 years and 37.09 ± 10.29 years in the control group. There were 102 women (68.9%) in the depression group and 82 women (63.1%) in the control group. One hundred five participants (70.9%) lived in city centers in the depression group and 102 (78.5%) in the control group. The mean duration of education in the depression group was 10.98 ± 3.13 years, and 10.95 ± 3.06 years in the control group. Seventy-nine participants (53.4%) in the depression group and 59 (45.4%) in the control group were married (p>0.05, for each). In summary, the participants in the depression and the control groups were statistically similar and comparable in terms of sociodemographic variables.

Table 1 presents the items’ mean scores and internal consistency coefficients of the OMMP-8 scale. The corrected item-total score correlation values ranged between 0.64 and 0.79. Item-total score correlation values higher than 0.30 indicated that the items were discriminative and valid.

Table 1 Item Analysis of the OMMP-8 Scale

Kaiser-Meyer-Olkin (KMO=0.825) and Bartlett’s statistic values (3200.136/sd=28/p<0.001) showed that the sample was suitable for EFA. Table 2 presents the factors’ eigenvalues and explained variance ratios obtained from EFA. In the OMMP-8 scale, which had a three-factor structure with eigenvalues greater than 1, 93.18% of the total variance was explained.

Table 2 Exploratory Factor Analysis of the OMMP-8 Scale

Table 3 presents the factor loadings obtained from EFA. Factor 1 included two items (item 1.4), Factor 2 included three items (item 2, 5, 7), and Factor 3 included three items (item 3, 6, 8). Since the factor loadings of each item were higher than 0.30, it was concluded that the items measured the construct in their respective factors.

Table 3 Item-Factor Loadings of the OMMP-8 Scale

A confirmatory factor analysis (CFA) was conducted to determine whether the OMMP-8 scale’s Turkish version has a three-factor structure. Model fit was evaluated based on GFI values, factor loading values, and error variances in the context of CFA, and a measurement model was prepared (Table 4 and Figure 1). Fit index values, factor loading values (Max-Min), and error variance (Max-Min) values were calculated.

Table 4 Confirmatory Factor Analysis of the OMMP-8 Scale

Figure 1 Confirmatory factor analysis of the OMMP-8 scale.

The X²/sd value was less than 3, indicating that the model fit the data well. The CFI value was 1.00, NFI was 1.00, and GFI was 0.98. The values above 0.90 revealed that the model fit the data well. The RMSEA index was 0.000 for the model, which was less than 0.080, indicating that the model provided an excellent fit to the data. The factor loading values of all items were higher than 0.30. The overall evaluation of the fit indices revealed that the three-factor model fit the data well (Table 4).

Table 5 presents the reliability analyses of the OMMP-8 scale. Cronbach’s alpha and McDonald’s omega values were 0.96 for Factor 1, 0.97 for Factor 2, and 0.98 for Factor 3. Cronbach’s alpha and McDonald’s omega values were 0.92 for the whole scale. These results suggest that the OMMP-8 scale is reliable.

Table 5 Reliability Analysis of the OMMP-8 Scale

Table 6 presents the convergent and concurrent validity results of the OMMP-8 scale. There were statistically significant positive correlations (p<0.05 for each) between OMMP-8 scale scores and HDRS (r = 0.851), BSIS (r = 0.836), BHS (r = 0.825) and MBPPAS (r = 0.881) scores. These findings suggested that the OMMP-8 scale provided convergent and concurrent validity. There was no statistically significant correlation between age and HDRS, BHS, and OMMP-8 scale scores and (p>0.05, for each).

Table 6 Convergent and Concurrent Validity of the OMMP-8 Scale

The OMMP-8 scale was administered to 52 participants two weeks apart to evaluate the test-retest reliability, and the intraclass correlation between the two measurements was calculated. The test-retest reliability was 0.979 for Factor 1, 0.985 for Factor 2, 0.986 for Factor 3 and 0.981 for the overall OMMP-8 scale (Table 7). These findings suggest that the test-retest results of the OMMP-8 scale are reliable.

Table 7 Test-Retest Analyses and Confidence Intervals of the OMMP-8 Scale

Whether the OMMP-8 scale correctly classified the participants diagnostically (depression vs control group) was examined by discriminant analysis. The mean OMMP-8 score was 20.02 ± 7.09 in the depression group and 6.13 ± 3.75 in the control group. The chi-square value (χ ((1))^2=246, 842; p<0.001) for the Wilks Lambda was statistically significant. These findings indicated that the discriminant capacity of the function was significant, and groups could be separated through this function. The eigenvalue of the function was 1.450, and the canonical correlation was 0.769. These results revealed that the function was effective in separating the groups. 81.76% (n = 121) of the depression group and 98.46% (n = 128) of the control group were correctly classified. The overall correct classification rate of the discriminant function was 89.6%. Table 8 shows the results of discriminant analysis and classification percentages.

Table 8 Discriminant Analysis of the OMMP-8 Scale for Differentiating Diagnostic Groups

Whether the OMMP-8 scale correctly classified those with and without suicide attempts in the depression group was also examined by discriminant analysis. The mean OMMP-8 score of those with suicide attempts was 25.04 ± 5.05 and 15.13 ± 5.10 for those without suicide attempts. The chi-square value (χ ((1))^2=98,352; p<0.001) for the Wilks Lambda of the function was statistically significant. The eigenvalue of the function was 0.97, and the canonical correlation value was 0.70. These findings indicated that the function was effective in separating the groups. Of those with suicide attempts, 82.19% (n = 60) and 82.67% (n = 62) of those without suicide attempts were correctly classified. The overall correct classification rate of the discriminant function was 82.4%. Table 9 shows the results of discriminant analysis and classification percentages.

Table 9 Discriminant Analysis of the OMMP-8 Scale for Differentiating Participants with and without Suicide Attempts in the Depression Group

Discussion

The psychometric examinations of the OMMP scale in various languages and cultures did not confirm the consistency of the nine-factor structure,^29,30 so Casanova et al tested the eight-item OMMP-8 scale, which had appropriate internal consistency values with the three-factor structure. Our study demonstrated that the Turkish version of the OMMP-8 scale fits the three-factor model well. The factors to which the items belong (Figure 1) in our study are the same as Casanova et al. Therefore, we named the factors in this study as Experience of Irreversibility (factor 1), Emotional Flooding (factor 2), and Narcissistic Wounds (factor 3) like Orbach et al and Casanova et al^12,13 Casanova et al found Cronbach’s alpha coefficient as 0.835 for “Experience of Irreversibility” subdimension, 0.856 for “Emotional Flooding”, and 0.767 for “Narcissistic Wounds” in the internal consistency analysis of the OMMP-8 scale. However, they reported that follow-up studies should support these data.¹² We found that the Turkish version of OMMP-8 scale had a Cronbach’s alpha coefficient of 0.92 for the total score, 0.96 for the “Experience of Irreversibility” subdimension, 0.97 for the “Emotional Flooding”, and 0.98 for the “Narcissistic Wounds”. According to these results, the OMMP-8 scale Turkish version has excellent internal consistency and is reliable.³¹ The most important outcome of this study is that the OMMP-8 Turkish version is valid and reliable among patients with depressive disorder and healthy controls.

Psychological pain often accompanies psychiatric disorders. Orbach et al reported that psychological pain, depression, and anxiety are closely related; however, these concepts are not exactly the same and have different aspects.¹⁹ Mee et al stated that the relationship between depression and psychological pain is bidirectional and that the exacerbation of one exacerbates the other.³² Psychological pain, which is not classified as a diagnostic criterion or disorder in DSM-5, can be a distinguishing feature of psychiatric disorders.^3,33 For example, psychological pain was associated with the sub-dimensions of the disorders in individuals diagnosed with borderline personality disorder and obsessive-compulsive disorder^17,34 and was distinctive from healthy controls.³³ In our study, the fact that the total score of the OMMP-8 scale successfully distinguished between patients with depressive disorder and healthy controls revealed that the diagnostic discrimination of the scale was successful. Future studies evaluating the total and sub-dimension scores of the OMMP-8 scale among various psychiatric disorders and non-clinical samples may explain how the sub-dimensions of psychological pain differ according to psychiatric diagnosis. Previous studies demonstrated that patients with schizophrenia³⁵ and bipolar disorder²³ with suicide attempts have higher levels of psychological pain than those without suicide attempts. Casanova et al did not evaluate OMMP-8 scale scores in predicting suicide risk and pointed to future studies.¹² In our study, the total score of the OMMP-8 scale successfully differentiated participants with and without suicide attempts in the depression group. The results suggest that the scale successfully predicts suicide risk, and its discrimination function is adequate.

Although psychological pain is frequently associated with mental disorders and suicidality, it can also occur in healthy individuals without psychiatric diagnoses. Previous studies have shown that psychological pain is closely associated with depression, anxiety, hopelessness, suicidal thoughts, and suicide attempts in clinical and nonclinical samples of adults and adolescents.^14,36 Correlation analyses in our study revealed that OMMP-8 total score was correlated with HDRS, BSIS, BHS, and MBPPAS in both depression group and health controls. These results indicate that the convergent and concurrent validity of the scale is appropriate. To the best of our knowledge, this is the first study to determine the concurrent validity of OMMP-8 with another psychological pain scale (MBPPAS). Test-retest reliability assesses the consistency of test scores over time and is essential for monitoring changes in the related variable.³⁷ Our test-retest scores revealed that the OMMP-8 scale is reliable for tracking changes in psychological pain.

Casanova et al demonstrated that younger individuals experience higher levels of psychological pain than older individuals.¹² Similarly, Tossani et al revealed that young people had more severe psychological pain in the validation study of the Italian version of the OMMP.³⁸ We did not find a significant relationship between age and the severity of psychological pain, hopelessness and depressive symptoms, which may be explained by the hypothesis that there is an indirect relationship mediated by coping skills and personality traits rather than a direct relationship between psychological distress and age.³⁹ We did not evaluate coping skills; future studies may help to clarify this issue by investigating the relationship between age, psychological pain, and coping skills with various mediation models.

This study has some limitations and strengths. First, we evaluated whether the participants had lifetime suicide attempts, so we could not evaluate proximal and distal risk factors for suicide attempts. Future studies may help to better understand this issue by evaluating patients with more recent suicide attempts. We did not evaluate the number of suicide attempts in participants with suicide attempts, so we could not demonstrate a relationship between recurrent attempts and psychological pain. We recommend that future studies evaluate the number and severity of suicide attempts. Our study evaluated patients with depression admitted to a university hospital and healthy controls. Evaluating patients with different psychiatric diagnoses would be helpful to reach more generalizable results. Our sample consisted of adult participants; adaptation studies of the OMMP-8 scale in adolescent and geriatric populations will provide a better understanding of the relationship between age and psychological pain. The two-week period for test-retest reliability is relatively short; we recommend future studies to evaluate scale scores’ stability for a longer interval. Finally, we evaluated the psychometric properties of the OMMP-8 scale, which offers the chance to assess psychological pain quickly. Examining the psychometric properties and factor structure of the original 44-item OMMP scale in the Turkish population will reveal whether the scale differs in various cultures. The strengths of our study include the evaluation of the concurrent validity of the OMMP-8 with another mental scale, the MBPPAS, the demonstration of the discrimination function of the scale in two different ways according to the presence of psychiatric diagnosis and suicide attempts, and the adaptation of the scale in a different culture compared to the original version.

Conclusion

In conclusion, the fact that suicide rates have not decreased despite improvements in treatment options for mental disorders suggests that this issue should be further investigated. OMMP-8 is a scale that all healthcare professionals can complete in a short time. Our study showed that the Turkish version of the scale can be used in both depressed patients and healthy controls and is a valid and reliable scale. Further studies in different psychiatric diagnoses, cultures, and languages are needed to determine the risk of suicide, prevention, and treatment approaches.