Validation of a Turkish Translation of the Perceived Occupational Stress Scale and Measurement Invariance Across Turkish and Italian Workers

Introduction

Work-related stress represents a significant concern that impacts both employees and employers. It is a pervasive and complex phenomenon that has garnered increasing attention due to its profound impact on individuals, organizations, and society as a whole.¹ Work-related stress arises when employees encounter adverse or negative emotional states within their work environment. The inability of employees to effectively deal with the demands of their daily job responsibilities can result in the emergence of work-related stress. Factors such as inadequate management of work organizations, insufficient focus on work design, ineffective management and leadership practices, unfavourable working conditions, and the presence of a highly competitive work environment are recognized as primary contributors to the experience of stress within the modern business landscape.²

Employees in various industries encounter high levels of stress in their work environments. Extensive research has shown that the experience of work-related stress can substantially impact the mental well-being of these workers. For example, in a recent scoping review conducted by Nikunlaakso et al,³ work-related stress has been identified as a contributing factor to an elevated risk of diverse mental health problems, encompassing increased levels of anxiety, depression, sleep disorders or insomnia, psychological distress, burnout, and stress. Also, a study by Acquadro, Maran and Begotti⁴ investigated the occurrence and characteristics of workplace violence directed at employees, revealing its consequences in terms of risks of exhaustion, disengagement, poor workplace satisfaction, and diminished emotional self-efficacy. Furthermore, a systematic review and meta-analysis study demonstrated that interventions emphasizing strengths, such as mindfulness practices, can equip workers with essential competencies and skills, leading to improved health, well-being, and overall quality of life.⁵

Based on the Job Demands-Resources model,^6–8 the working environment and conditions can be categorized into two distinct dimensions: job demands and job resources. Job demands encompass the various physical, psychological, social, and organizational requirements inherent to a specific job role, exerting both physical and psychological consequences on individuals. Conversely, job resources encompass the array of material, psychological, social, and organizational resources made available by the organization to enhance personal capabilities, facilitate learning, stimulate work enthusiasm, and foster active engagement.⁹ These two fundamental components possess the capacity to exert either positive or negative effects on employees’ overall well-being. When there exists an imbalance between the levels of job demands and job resources, individuals may encounter detrimental strain. Achieving a harmonious equilibrium between these two constructs is crucial for fostering the holistic well-being of individuals within the workplace setting. Empirical research grounded in the JD-R model has substantiated the proposition that various job resources play a moderating role in attenuating the association between distinct job demands and the manifestation of burnout. These research findings underscore the significance of demands and resources as significant predictors of both the well-being and performance of workers.⁸

Therefore, it is imperative to undertake a better investigations and assessments to thoroughly explore and quantify the phenomenon of work-related stress, alongside the potential job resources that exert a notable influence on the overall well-being of individuals within the workplace. Such rigorous examination assumes paramount significance in advancing the quality of working environments and promoting the holistic welfare of employees, particularly in the face of demanding circumstances that significantly impact occupational settings. In light of this, the Perceived Occupational Stress (POS) scale, recently introduced by Marcatto et al,¹⁰ has emerged as a novel concise tool designed to assess an individual’s subjective perception of experiencing stress in the workplace. However, since the measure is relatively new in the literature, there remains a dearth of cross-cultural investigations pertaining to the applicability and validity of this measure. Therefore, the present study aims to contribute empirical evidence supporting the cross-cultural suitability and utility of the POS scale in research endeavours.

Overview of the POS

The developers of the POS argued that the scale was designed to serve as a brief measurement tool specifically developed to assess occupational stress.¹⁰ The POS scale, consisting of representative items that specifically assess work-related stress, offers a distinct advantage over lengthier measures that typically impose additional demands on workers by necessitating administration during their working hours. By utilizing a multi-item approach, the POS was found to meet fundamental psychometric standards compared to single-item measures. Furthermore, the careful selection of items that specifically focus on subjective perceptions of work-related stress helps workers avoid confusion with stress unrelated to their occupation, which can be evaluated using additional instruments measuring perceived general stress.

A pragmatic approach was employed to construct a concise item set that encompassed the widely shared and commonly accepted concepts pertaining to work-related stress.¹¹ Although there is no universally agreed-upon definition of work-related stress, a broad consensus exists regarding three fundamental components: (i) it arises as a reaction to excessive pressures encountered in the workplace, (ii) workers experience this stress when they encounter difficulties in effectively coping with the demands of their job, and (iii) it can exert detrimental impacts on both mental and physical health.^12,13 Consequently, three items were formulated to align with these elements. Furthermore, Marcatto et al¹⁰ introduced an additional item to the scale, which explicitly focuses on workers’ subjective perception of the overall “stressfulness” of their work. Thus, the POS scale with four items is considered a practical solution for efficiently assessing occupational stress while reducing the burden on workers and enhancing the accuracy of work-related stress measurements.

Present Study

The POS measure was originally developed within the Italian context. There is a need to assess its cross-cultural applicability, particularly in the non-western setting of Turkey. This represents a significant research gap about whether an instrument initially validated in one cultural context can be effectively applied to another. Conducting validation and cross-cultural studies has the potential to contribute to the broader understanding of the cross-cultural utility and performance of the scale, ultimately enhancing its relevance and applicability for diverse populations. It is important to note that the mere development or adaptation of a scale, in and of itself, does not inherently signify a groundbreaking contribution to research. However, in cases where specific languages lack such scales, creating or adapting one without immediate associations to other variables can significantly enhance the existing literature. Upon review of the Turkish academic literature, a short scale (eg, POS scale) measuring the occupational stress levels of employees was absent. Thus, the validation of POS can be perceived as bridging this particular gap in the literature. Also, it is important to recognise that in adaptation studies, different research outcomes may arise, mainly due to linguistic and cultural distinctions. The presence or absence of dissimilar findings in adaptation studies can also serve as a significant contribution to the existing literature. Therefore, this research holds an important merit as it contributes to the existing body of knowledge by broadening the cross-cultural validation for the POS scale. The evaluation of POS is particularly important due to the potential influence of cultural variations on workers’ interpretation of the self-report items aimed at measuring work-related stress. Also, conducting measurement invariance of POS assessment in Turkey and Italy offers the opportunity to enhance the cross-cultural applicability of the POS and broaden the limited range of measures available for cross-cultural comparisons of work-related stress across diverse occupational contexts and settings. In this study, our objective was to establish the validity of the POS measure in the Turkish context and examine its measurement invariance across both Turkish and Italian samples, thereby comparing and ensuring its reliability and cross-cultural applicability for assessing work-related stress. In this regard, it is expected that the Turkish version of the POS would provide a convenient, valid, and reliable instrument for assessing occupational stress among workers. We also expect that the POS measure would offer evidence supporting measurement invariance across Turkish and Italian samples, strengthening its cross-cultural applicability.

Method

Sample and Procedure

This study used a quantitative approach followed by descriptive research based on exploratory and confirmatory analysis. The current research used a diverse group of adult participants, ranging in age between 18 and 65 years. These individuals were drawn from two different countries, with sample sizes varying from 160 (Italian) and 350 (Turkish) resulting in a total sample size of 510. The sample characteristics and the procedure employed in each country are provided below. Informed consent was obtained from all participants before conducting the study. The anonymity and responses of participants were assured.

Turkish Sample

Using a convenience sampling approach, data were collected from 350 teachers at all school levels, including preschool, primary school, middle school, and high school. The participants consisted of 55.7% male and 44.3% female. The age range of the participants varied between 25 and 65 years, with an average age of 36.93 years (SD = 7.65). Data were collected online using a secure online data collection platform. The inclusion criteria for this study included individuals employed as teachers across all school levels who could provide responses via online data collection platforms. Conversely, non-teachers who lacked access to online data collection platforms were excluded from participation.

Italian Sample

The Italian sample (described in Marcatto et al¹⁰) comprised 160 participants who represented a diverse range of workers from various sectors such as social services, human resources, and education. Among the participants, the majority were females (60.6%). The age range of the participants varied from 18 years old and above, with a notable proportion falling within the 50–59 years age group (36.3%). The data collection process involved the administration of a paper-pencil version of the measures by a research assistant using a convenience sampling approach.

Measures

Perceived Occupational Stress (POS) Scale.¹⁰ The POS scale includes four items that specifically measure workers’ occupational stress over the past six months. The scale requires participants to self-report their perceived level of stress in the workplace using a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The scores for each participant are averaged across the four items to derive the POS score, which ranges from 1 (indicating the lowest perceived stress) to 5 (indicating the highest perceived stress). Cronbach’s alpha was found to be 0.85 in the Turkish sample and 0.82 in the Italian sample.

Translation of the POS in Turkish

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This study was carried out following the acquisition of ethical approval from the Ethics Committee of Batman University under reference number E.131160. For the Turkish translation, the research process commenced with obtaining permission to translate the POS scale, which involved following a standard back-translation procedure. Three field experts proficient in both English and Turkish translated the scale from English to Turkish. Subsequently, an independent bilingual field expert performed the back translations. All translations were thoroughly assessed by a team of two researchers and underwent finalization. The scale was then carefully evaluated for comprehensiveness, and necessary adjustments were made and finalized prior to its actual implementation.

Data Analysis

Data collected from respondents were entered into SPSS to identify any outliers. All responses were found to be genuine, and no participants were excluded from the subsequent data analysis. Initially, an independent sample t-test and analysis of variance (ANOVA) test were performed to assess potential differences in the identified factor across various groups. Following this, factor analysis using the principal component extraction technique was conducted to identify the factor structure for the POS. Cronbach’s alpha was used to estimate the internal consistency reliability. Multigroup confirmatory factor analysis was used to test the measurement invariance. The statistical analyses in this study were performed using SPSS 23 for descriptive analysis and exploratory factor analysis (EFA), while multi-group confirmatory factor analysis (CFA) was conducted to test measurement invariance using AMOS 23.

Results

The distribution of the POS items and a composite score is reported in Table 1. Since all values of skewness and kurtosis fall within the +2/-2 range, a normal univariate distribution can be reasonably assumed.

Table 1 Item Analysis of the POS Items

Differences Between Groups

The results of the independent samples t-test indicated that there was no significant difference between males (M = 2.98, SD = 0.92) and females (M = 3.05, SD = 1.07) in terms of their scores on the POS, t(348) = 0.623, p = 0.53. Similarly, the results of ANOVA revealed no statistically significant differences in age among the groups, F(34, 315) = 1.26, p = 0.16, suggesting no notable discrepancies in age among the different groups.

Exploratory Factor Analysis

Since the POS was not yet validated in Turkish, a Principal Component Analysis (PCA) was performed to investigate the factor structure of the POS. Kaiser-Meyer-Olkin (KMO) measure yielded a value of 0.81, exceeding the recommended threshold of 0.70, suggesting the suitability of the data for the factor analysis.¹⁴ Bartlett’s test of sphericity confirmed a significant relationship among the variables, χ2(6) = 605.60, p < 0.001, indicating significant inter-item correlations for factor analysis.¹⁵ Furthermore, the analysis extracted a single factor for the POS with an eigenvalue of 2784 accounting for 69.61% of the total variance. The factor loadings and communalities are reported in Table 2.

Table 2 Results of the Principal Component Analysis on the POS Items

Measurement Invariance by Language

In assessing the measurement invariance (reported in Table 3), specific cut-off values for the goodness of fit were applied, including a Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) equal to or greater than 0.95, and a Root Mean Square Error of Approximation (RMSEA) equal to or less than 0.08.¹⁶ These cut-off values were utilized in the first step, known as Configural invariance (M1) where the results indicated that the underlying structure of the POS, including the loadings on the latent factor, was supported in both language groups. In the second step, Metric invariance was examined to determine if each item contributed to the latent factor in a similar manner across the two groups. The model fit of Metric invariance (Model 2) was found to be non-significantly different from Configural invariance (Model 1), indicating support for Metric invariance, also referred to as “weak invariance.” In the third step, Scalar invariance (Model 3), which captures mean differences in the latent factor, was tested. However, the comparison between Model 3 and Model 2 revealed a delta higher than 0.01 in both CFI and TLI, indicating a lack of complete Scalar invariance. As a result, complete Scalar invariance was rejected. In the final step, Partial scalar invariance (Model 4) was tested which involved releasing item 4, an increase in goodness of fit was observed, as indicated by the delta in TLI and RMSEA compared to Model 2. This suggests that Partial scalar invariance has been achieved, allowing for a more refined understanding of the measurement invariance across the two language groups.

Table 3 Results of the Multi-Group CFA Models

Discussion

This study provides empirical evidence regarding the structural and psychometric characteristics of the POS measure, using a sample comprising Turkish teachers and Italian workers in the sectors of social services, human resources, and education. The Turkish adaptation of the POS exhibited a robust internal consistency and yielded a unidimensional factor structure that assesses individuals’ subjective perceptions of work-related stress. Additionally, the measurement model demonstrated metric invariance and partial scalar invariance across the samples, suggesting its suitability for comparing scores among workers in different countries. In particular, our study findings indicate that metric invariance and partial scalar invariance have been established across Turkish and Italian languages. However, the attainment of complete scalar invariance was hindered by the presence of item 4. This suggests that the strict equality of intercepts for all items across the two language groups was not fully supported. It is plausible that this lack of complete scalar invariance is attributed to the comparison of samples that are similar but not entirely overlapping. Specifically, the Italian sample comprised a partial representation of teachers, whereas the Turkish sample exclusively consisted of teachers. This distinction in sample composition may have influenced the observed variation in the scalar invariance across the two language groups. Further research is warranted to explore these nuances and refine our understanding of the underlying factors influencing scalar invariance in cross-linguistic studies.

The present study aligns with Marcatto et al¹⁰ research on Italian workers by confirming the unidimensional factor structure of the POS. Additionally, Marcatto et al¹⁰ demonstrated that the POS accounted for a substantial amount of variance in work-related outcomes, surpassing alternative measures of occupational stress such as the Effort-Reward Questionnaire,¹⁷ Maslach Burnout Inventory,¹⁸ and Health and Safety Management Standards Indicator Tool.¹⁹ Collectively, previous research provided evidence for the external validity of the POS scale, indicating that perceived stress levels are a separate construct from work stressors and are able to explain the relationship between stressors and strain.¹⁰ The findings of the current study contribute further evidence regarding the factor structure of the POS and its measurement invariance across Turkish and Italian samples, facilitating cross-cultural comparisons in research and practical applications.

Implications and Contributions

This study provides support for the unidimensional model of the POS scale in the Turkish context, as well as its measurement invariance across different samples of Turkish teachers and Italian workers in the social services, human resources, and education sectors. For researchers, it is recommended to adopt the one-factor model of the POS and consider the general domain factor in their studies conducted in various contexts. Additionally, researchers can make direct comparisons between individual differences and mean values. This approach enables a better understanding of workers’ occupational stress. A Turkish version of the POS measure could serve as a practical tool for the local evaluation and ongoing monitoring of occupational stress levels within Turkish workplaces, as well as for European researchers and practitioners working with diverse work populations. These findings carry significant implications for assessing occupational stress among different populations. Furthermore, mental health providers should focus on reporting total scores when using the POS to assess occupational stress among workers. This approach ensures a better evaluation and facilitates effective interventions to mitigate work-related stress. These findings contribute to the understanding and measurement of occupational stress and provide important guidance for future research and practical applications in the field of occupational well-being and mental health.

Limitations

The limitations of this study include the homogeneity of the Turkish sample, consisting only of teachers, which may not fully represent the broader workers population in Turkey. Therefore, caution should be made in generalizing the findings to workers from different ethnicities and socioeconomic backgrounds. Another limitation is the variation in participant recruitment procedures, with online surveys used for Turkish teachers and paper-pencil surveys for Italian workers, which may have influenced the emerging results reported in this study. Future research should aim to address these limitations by including larger and more diverse samples collected with the same recruitment procedure to replicate the current findings and explore within-culture variations and cross-cultural dynamics in greater depth. Moreover, criterion variables were not included in this study to examine various forms of validity, such as criterion-related validity and predictive validity. The use of self-reported measures in assessing these variables may introduce common method bias. Therefore, future studies should address potential measurement biases and investigate the criterion-related validity of the unidimensional factor of the POS scale by examining its relationship with relevant criterion variables. Additionally, it is vital to explore the predictive capability of the POS in explaining occupational health outcomes.

Conclusion

In summary, the measurement of POS displayed robust measurement characteristics, including a confirmed one-factor structure, strong internal consistency reliability, and consistent measurement across groups in Turkish and Italian cultures. The POS measure can be suggested as a practical measurement tool for cross-cultural investigations aimed at assessing individuals’ perceptions of occupational stress. However, further research is necessary to establish its measurement invariance across a broader range of countries. The findings about the complete scalar invariance suggest a cautious use of the POS in research that compares correlates and mean levels of occupational stress across different cultures. Our findings indicate that occupational stress exhibits a significant degree of invariance across two languages, suggesting its suitability for cross-cultural studies on work-related stress. While future cross-cultural research is warranted to further validate the utility of the POS, our results provide initial evidence of its potential as an effective measure for evaluating work-related stress among workers from Turkey and Italy.