Validation and Measurement of Psychometric Properties of the “Nursing Work Environment Questionnaire” (NWE-q) in Italian Nurses

Introduction

A Healthy Workplace: Definition

The workplace can be distinguished by many elements that affect each person and contribute to the person’s occupation.¹ A very frequent attitude in several researchers was to recognize healthcare settings by taking into consideration this overview and interviewing their own workers’ beliefs about the particularities of their regular activities.² The best concept of the workplace assured the best circumstances for physical, mental, and social well-being. The International Standard ISO 6385:2016³ identified physical, chemical, biological, organizational, social, and cultural conditions in which each element contributed to the entire employer’s condition. Workplaces in nursing have received significant attention in recent decades.^4,5

The Healthy Nursing Workplace: Definition

The American Association of Critical Care Nurses recognized six standards for healthy workplaces: skilled communication, true collaboration, effective decision-making, appropriate staffing, meaningful recognition, and authentic leadership.³ As the importance given to nursing activities, the related workplace gained more importance in its related motivation and in improving the nursing performance involved. Lake⁶ identified the nursing workplace as a set of work activities that improve or restrict nursing practice. Bae⁷ identified several work conditions and their respective associations with patient outcomes by grouping the work requirements into 10 concepts: autonomy, philosophy emphasizing the quality of clinical care, nurse participation, supportive managers, collaborative relationships with physicians, collaborative relationships with peers, staffing, decentralization, patient-centered climate, and busyness. Workplace assessments are essential elements that address nursing management through planning and establishing requirements in the nursing field.⁶ Globally, numerous tools have been adopted to evaluate the nursing workplace^8–10 and each study has identified their own instrumentation defined the own instrument as the most helpful instrument to evaluate the nursing workplaces, too.¹⁰ Each study suggested its own usage by recommending it, highlighting its efficient psychometric properties, high discriminant skills, and opportunity to compare it with other tools presented in the literature.¹¹ By considering the essential elements assessed in the nursing workplace, the important factors for professional practice were identified as all factors recognized by nurses as essential to ameliorate their continuous improvement in their practical performance,^12–14 by also considering other important elements, such as the leadership of the coordinator, interprofessional relations, and nurses’ empowerment within the organization.^15–18

The Healthy Nursing Workplace: The Italian Condition

The Italian nursing scenario is made up by some specific phases: from the beginning of 2000, the National Health Service dealt with an important economic crisis with the European countries,¹⁹ which highlighted the important presence of the nursing shortage.^20–22 Therefore, public health and welfare services were more inclined to other priorities²³ by leading and increasing in unemployment arrangement in graduate nurses and replacing them with less-trained healthcare workers by decreasing the availability of resources dedicated for nurses.^24,25 Then, after the introduction of the Legislative Decree 81/2008,²⁶ nurses begun to be considered of interest in the healthcare context by also monitoring their nursing work environments and preventing turnover rates and ameliorating well-being conditions among them by considering nurses’ quality of work life or work-related stress, job satisfaction also taking into account at the same time.^27–29 In this context, some dedicated tools were improved to evaluate health in work environments accomplished by several projects improved by hospitals to ameliorate the well-being in the nursing contexts which seemed to be strictly connected to the patient’s quality perceptions’.³⁰

Purpose Statement

The present study aimed to validate an alternative version of the work environment questionnaire, in order to have an easier, brief and more immediate tool to use to assess work environment perceptions among Italian nurses.

Materials and Methods

Study Design

This was a prospective, descriptive, and validation study using the Nursing Work Environment Questionnaire (NWE-q). Validation and developmental studies were conducted in two phases. During the first phase, the NWE-q was developed, and in the second phase, it was administered to nurses. The approach suggested by the European Statistical System was followed.³¹

The NWE-q Development

Initially, NWE-q was created ad hoc by studying available literature. Attention was paid to all possible aspects that could influence perceptions of well-being in nurses’ work environments. Additionally, by considering the “Best Practice Guidelines” of the Registered Nurses’ Association of Ontario (RNAO),¹⁸ which is described as a healthy work environment for nurses, a complex and multidimensional system that includes several elements and relations among all the elements included, in which practice contexts could ameliorate the health and well-being of nurses, quality patient/client outcomes, organizational functioning, and societal-systemic results. The Conceptual Model for Healthy Work Environments for Nurses expressed by the RNAO Association (2013) constituted a healthy work environment as a result of the inter-confidence among person (micro level), institutional (meso level), and external (macro level) network elements.

Therefore, by considering three main dimensions in the nursing work environment, namely the individual, organizational, and system levels (RNO), and after creating a draft in the Italian language, the NWE-q was translated into English for further publication. The questionnaire was forward-translated and adapted from English to Italian and vice versa by two different translators without contact between them. The two translated versions were compared, and the final version was approved after slight modifications to the wording. The NWE-q was first validated using face-to-face administration to a small sample of ten registered nurses at T0 and T1 (Table 1).

Table 1 The Nursing Work Environment Questionnaire (NWE-q)

For each item, a value ranging from one (negative perception) to seven (positive perception) was assigned. Values were thought as for further analysis in the NWE-q content validity assessment, both the “Individual Work Performance Questionnaire”³² and in the “Organizational Nursing Performance”³³ were used and both used in their scoring values ranging from 1 to 7.

The Data Collection Process

The NEW-q questionnaire was administered twice, on-line, in an anonymous form through the Google Modules function through some nursing groups present in the Facebook and Instagram nursing social pages to the same nurses by asking to them to identify their questionnaires by a numerical code at T0, which should be the same at the second data collection (T1), as 10 days after the first data collection.

The two data collections performed allowed to assess the test-retest reliability or reproducibility by comparing the compilation of the questionnaire twice, at T0 and T1, and the degree of correlation between the scores obtained was assessed using the t-test for paired sample statistical correlations (Table 2). Internal consistency was also assessed at:

• T0: α-Dimension1= 0.932, α-Dimension2= 0.955, α-Dimension3= 0.881;
• T1: α-Dimension1= 0.928, α-Dimension2= 0.944, α-Dimension3= 0.888.

Table 2 Paired Samples Statistics

Participants

Nurses who voluntarily agreed to participate were enrolled in the study. A total of 376 Italian nurses were randomly recruited. Sampling was conducted between November 2021 and March 2022. All participants employed in all Italian regions and healthcare settings as registered nurses, nursing managers, or coordinators were considered potential participants. Participation was voluntary therefore, it was asked to give their consents and then, to full fill all the questionnaire.

The Questionnaire Administered

The questionnaire contained 82 items divided into four main sections. In the first part of the questionnaire, the following sociodemographic characteristics were collected:

• Gender, if nurse was female or male;
• Years of work experience were divided into two sub-groups: nurses who worked for 5 years and nurses who worked for over 6 years.
• Shift work: nurses were employed only during the morning and afternoon, or if they also worked during the night.
• Nursing education level, as nurses studied for only three years in the nursing disciplines, or nurses studied for over three years in the nursing disciplines.

The second section of the questionnaire included an individual work performance questionnaire,³² which contained 18 items developed in the Netherlands to assess three main dimensions of work performance: task performance (items no.:1–5), contextual performance (items no.:6–13) and counterproductive work behavior (items no.:14–18). For each item a 5-point Likert scale was used, ranging from 0 (seldom) to 4 (always) for task and contextual performance and from 0 (never) to 4 (often) for counterproductive work behavior. The individual work-performance questionnaire was validated based on previous studies and showed good psychometric properties.³²

The third part of the organizational work performance questionnaire³³ was inserted, which included 38 items divided into eight sub-dimensions: contextual (11 items), professional (4 items), clinical skills (6 items), interpersonal communication (3 items), problem-solving (4 items), professional ethics (3 items), teamwork (4 items), and leadership (4 items). For each item a 7-point Likert scale ranging from 1 (“never required” to 7 as “critical”) was used. All items showed high scale reliability, and tests for normality of the tool showed evidence of a normal distribution of the data.³³

Finally, the NWE-q, which contained 27 items, was administered. For each item a 7-point Likert scale was used, ranging from 1 (negative value significance) to 7 (positive value significance).

Data Analysis

All data were stored in an Excel data sheet and processed using IBM SPSS Statistics for Windows version 20. Sociodemographic data were considered categorical variables, specifically dichotomous variables, and were presented as frequencies and percentages.

All p-values greater 0.05 showed a “weak evidence”, while p-values less than 0.05 showed “strong evidence” and also p-values less than 0.001 suggested “very strong evidence.” Parametric statistics were assessed because all data associated with the three questionnaires were normally distributed. Convergent validity was measured using the Pearson’s correlation test. Internal consistency for each dimension of the NWE-q was assessed using Cronbach’s alpha coefficient (Cronbach’s α). Values with α > 0.70 were considerable acceptable levels of internal consistency. Additionally, to assess construct validity and analyze the factor structure of the data, varimax rotation with the Kaiser–Meyer–Olkin (KMO) index and the Bartlett test of sphericity were also performed. All data referring to KMO higher than 0.40 were considered acceptable. Additionally, strong evidence from Bartlett’s sphericity test was assessed at a level of p<0.001. Then, the main element examination, thanks to direct oblique rotation, was performed for further assessment of the construct validity. This favored oblique rotation since the literature suggested a greater order factor in their confirmatory factor analyses. Loading factors ≥ 0.30 were established as a standard to recognize an important element. Every statement with the highest load on each of the three factors was considered to be a statement related to that factor, which was not different from the original questionnaire. Reliability was evaluated by measuring internal consistency. Therefore, the Cronbach’s alpha coefficient was evaluated for the entire questionnaire, and each of the three dimensions was obtained. To investigate stability, the test–retest method was performed. NWE-q values were analyzed to determine the interclass correlation coefficient (ICC). Moreover, to evaluate convergent validity, the correlations between the NWE-q values for each subdimension and its total value with both the individual and organizational work performance questionnaires were assessed. Finally, group validity was evaluated according to the socio-demographic characteristics collected, including gender, years of work experience, shift, and nursing educational level, based on the independent t-test.

Ethical Approval

A protocol study was conducted during questionnaire presentations. Participation was voluntary, and those interested in participating were presented with the opportunity to express informed consent. The confidentiality and anonymity of the information was guaranteed according to the principles of the Declaration of Helsinki. This study was approved by the ethics committee (protocol number:7077/2021).

Results

A total of 376 nurses were recruited for this validation study. The participants’ characteristics are shown in Table 3.

Table 3 Socio-Demographic Characteristics of Participants (n=376)

The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was 0.964, which is higher than the standard value. In addition, Bartlett’s test of sphericity provided strong evidence. From the data obtained, it emerged that the factorability of the correlation matrix described an evident structure in NWE-q (Table 4).

Table 4 KMO and Bartlett’s Test

The total explained variance is 64.265%. Specifically, the first, second, and third factors influenced 54.923% of the total variance, 5.256%, and 4.085% of the total variance, respectively (Table 5).

Table 5 Total Variance Explained Output

By considering principal component analysis due to oblique rotation, three main sub dimensions were defined with their respective construct validity analyses (Table 6).

Table 6 Factors Loading of the NWE-q Items

Good internal consistency was registered for all items of the NWE-q (27 items) and for each sub dimension. Additionally, very strong interclass correlations were assessed for the NWE-q total scores and for each sub-dimension, with the exception of the system dimensions, in which the interclass correlation showed a weak relationship. Strong correlations were also found between the NWE-q total score and each sub dimension (Table 7).

Table 7 The Internal Consistency, Reliability and Interclass Correlation Coefficients of the NWE-q

Strong positive evidence was assessed between the NWE-q for both the total and each sub-dimension correlated with the items of the individual nursing performance questionnaire. However, weak evidence was found in the individual and system dimensions of NWE-q correlated with counterproductive behaviors (Table 8).

Table 8 Correlations Between the NWE-q and the Individual Nursing Performance Questionnaire

Additionally, very strong evidence was found between the NWE-q total and each sub-dimension, correlated with all the sub-dimensions of the organizational nursing questionnaire, with the exception of the correlations assessed with the clinical skills sub-dimension, in which the NWE-q weakly correlated with its individual, system, and total dimensions (Table 9).

Table 9 Correlations Between the NWE-q and the Organizational Nursing Performance Questionnaire

Finally, considering gender characteristics, NWE-q reported strong evidence in the individual and total scores, as females recorded higher values than males. The same trend with weak evidence was also observed for both the organizational and system sub-dimensions. Weak evidence was also reported by considering shifts, as nurses employed only during the morning and afternoon reported weak evidence in the NWE-q and in each sub-dimension. No further relevant evidence was assessed by considering work experience and nursing education level (Table 10).

Table 10 Differences in the NWE-q Sub Dimensions According to Gender, Work Experience, Shift and Nursing Educational Level

Discussion

The present study aimed to evaluate the psychometric properties of the NWE-q by assessing its internal consistency, construct, and convergent and discriminant validity among Italian nurses. The NWE-q explored the individual, organizational, and system dimensions in the nursing context to better explain nursing work environment perceptions using a validated, easy, and fast instrument to collect data that will help nursing managers to improve their nursing contexts according to their nurses’ perceptions. The data obtained were promising, and the NWE-q seemed to be easier and faster to complete without requesting the presence of an interviewer, so the online mode seemed to be adequate. Overall, the NWE-q recorded good test–retest reliability and internal consistency. Initially, at the conception stage, by considering the items and their related structures, the three related sub dimensions were considered in an equal number of items. However, the three initial sub-dimensions were well reflected, as the first sub-dimension regarded the individual dimension of each participant and related working duties; the second sub-dimension concerned the organizational dimension in which the individual was employed; and the third dimension regarded the system nursing context. The distribution of the number of respective sub dimensions could also be derived from the perceptions that interviewers held on their organization and then on their system dimensions. However, the NWE-q items were strongly correlated with both the individual and organizational nursing performance questionnaires. Several studies have been undertaken on the assessment of the nursing work environment all around the world.^9,34–41

For example, the Nursing Work Index (NWI) and its related modified versions represents the most frequently used tool to measure the work environment of registered nurses in the current literature.⁴² The first version was created in 90s in the US⁴³ and was successively modified during the years by referring to contemporary nursing work settings and by assigning different levels of importance to more contemporary dimensions, such as productive nursing work environment, and later to nursing-centeredness to patient and patient safety issues. Moreover, the individual sub-dimensions validated in the NWE-q could overlap with the autonomy dimensions of NWI. The NWE-q did not include the patient-centered sub-dimension, which could be a consequence of nursing activities.

The Canadian and Australian nursing literature also suggested the Alberta Context Tool (ACT), which was validated in 2006 by a large sample of nurses in several healthcare settings across Canada and Australia and was translated and validated in Germany, Sweden, and France. ACT was developed to evaluate adaptable aspects of organizational context, taking into consideration care providers’ and managers’ application of research suggestions in current practice.⁴⁴

Moreover, the new simplified and revised PES-NWI tool⁴⁵ highlighted three main sub-dimensions to improve both quality of care and working environments by continuing to optimize nursing care. Specifically, the recognized sub-dimensions included participation in management and leadership, a focus on nursing care and interdisciplinary relationships, and adequate responses. These dimensions and their related items, most of them in their significance, could also overlap with the NWE-q items.

Finally, it should be considered that voluntary NWE-q items did not focus on specific relationships with other healthcare professionals to achieve a more generable assessment tool that could better define the general nursing work environment without any particularity. Its easy modality to fully fill it and its generality will be considered to enable more nurses to complete it and to better understand their real working conditions.

Strengths and Limitations

NWE-q can be considered unique in the Italian context for its purpose and simplicity. However, the 376 participants were not representative of all Italian nurses. However, its psychometric properties appear more promising.

Conclusion

The present study provided validated data on the NWE-q. The NWE-q explored the individual, organizational, and system dimensions in the nursing context by better explaining nursing work environment perceptions thanks to a validated, easy, and fast instrument to collect data which will help nursing managers to improve their nursing contexts according to their nurses’ perceptions. Since the NWE-q has demonstrated to be easier to be completed, also in an online mode, and its scoring characteristics seemed to be more accessible and simpler to adopt, it will further assess its psychometric properties in a larger nursing population.