Knowledge and Awareness of Non-Technical Skills Over the Course of an Educational Program in Nursing - A Repeated Cross-Sectional Study

Introduction

Adverse events and errors represent a significant threat to patients undergoing hospital treatment.^1–3 These events and errors are more often linked to failure in non-technical skills (NTS) than lack of technical expertise.^1–5 NTS are defined as cognitive and interpersonal skills that promote safety and complement workers’ technical skills.^2,6 In the hospital, anaesthesia-, emergency-, intensive care-, and perioperative units, represent high-technology wards characterized by complex patient- situations. The coordination and delivery of safe, high-quality care in such wards demands reliable teamwork and collaboration within, as well as across, organizational, disciplinary, technical, and cultural boundaries.^7,8 Hence, patient safety largely depends on the knowledge about and mastery of NTS.^2,5,9–11 Despite this, studies suggest that NTS are unlikely to be adequately introduced in health professional education.^12–15 Thus, knowledge and application of NTS is still a major challenge,^5,16 especially related to systematic and adequate implementation.^13,17,18

In Norway, continuing education of nurse anaesthetists, emergency care nurses, critical care nurses, and operating room nurses consists of a 90 European Credit Transfer and Accumulation System (ECTS) or 120 ECTS program, of which 45 ECTS are completed in clinical practice. Most of the students’ learning processes towards NTS competence take place in clinical practice. Here supervisors are key enablers to succeed with training and implementation of NTS in the continuing nursing programs.^13,19 NTS are difficult to assess because they are part of a holistic behaviour.¹⁶

Studies have shown that a structured assessment of NTS with a valid and reliable criteria-based behavioural marker system is an important contribution to provide specific and formative feedback and to monitor students’ learning progress.^6,12,20,21 Flynn et al²¹ has developed the Nurse Anaesthetist’s Non-Technical Skills (NANTS.no)²¹ based on the Anesthesiologists Non-Technical Skills (ANTS) tool⁶. NANTS.no²¹ is a criteria-based behavioral marker system for systematic training and assessment of NTS, which identifies 4 categories within the NTS; teamwork, situational awareness, task solving and decision making, for each category there are 3–5 elements (Table 1). Students and supervisors experienced NANTS.no contributed to reflection and professionalization of the learning process and the nurse role.²²

Table 1 NANTS.no

Awareness about attitudes and actions is a key factor to preventing adverse events.²³ Knowledge influences our attitudes and actions, which in turn affects clinical practice.²⁴ Understanding the relevance of NTS to patient safety, we addressed this largely ignored issue that was not integrated into our study program. The current status of students and supervisors’ knowledge and awareness about NTS are needed in order to construct an education plan for improvement. The aim of the present study was therefore to examine knowledge and awareness of NTS during an educational program with a focus on NTS.

Methods

Design and Settings

The study had a descriptive, repeated cross-sectional design.²⁵ The study was conducted at a medium-sized university college in Norway that offers continuing education for nurse anaesthetists and emergency care-, critical care-, and operating room nurses.

Learning Activities

An educational program was developed with an emphasis on NTS based on the 4 categories within NANTS.no (Table 1) to fill the void that consisted of lack of concepts and awareness in a field of importance to patient safety. The program comprised various learning activities based on pedagogical principles, and methods described as appropriately related to NTS.^{7,8,12,14,26,27}

The following learning activities were carried out three times per semester; lectures, student group discussions and solving of realistic clinical cases. Skill training, and extended use of simulation scenarios were also carried out 2 times per semester. In addition, NANS.no were used individually during the entire clinical practice period (10 weeks each semester), and in reflection groups together with the teacher and supervisor (three times each semester). The supervisors attended lectures regarding NANS.no and NTS three times at the ward’s educational program days, in addition to regular discussions between supervisors, students, and teachers on how to supervise and evaluate the students’ NTS. NANTS.no has been shown to be suitable for use across all four nursing specialties.²⁸ NANTS.no (Table 1) was prepared as a pocket edition as a reminder of the behavioral marker system. It was given to the students and supervisors easily accessible for use in supervising and evaluating clinical practice.

Participants

The participants were students from a cohort in the continuing education program in nurse anaesthesia, emergency care-, critical care-, and operating room nursing, as well as their supervisors in clinical practice. The participants were recruited into the study at the start of the first semester during a teaching session day for both students and supervisors, at the University College.

Data Collection

A literature search was carried out to find a suitable questionnaire to assess students’ and supervisors’ knowledge and awareness about NTS as NTS appears in NANTS.no, but no suitable questionnaires were found. Hence, a questionnaire was developed by the research team based on what relevant literature described as central to developing NTS competence^1,6,11,16,21 as well as several discussions with eight specialist nurses (two from each unit), and four teachers from the respective educations (Table 2). The questionnaire was then piloted by eight other specialist nurses (not the same as the previous eight specialist nurses), also from the same four units. The pilot participants completed the questionnaire and gave feedback on the face validity and the feasibility of the questionnaire. They all reported the questions to be relevant, understandable, and appropriate, so no changes were made to the questionnaire (Table 2). Students and their supervisors were invited to complete a paper questionnaire at two different times (at the start of education (baseline) and after 12 months of education). The questionnaire was distributed to and answered by the students and supervisors just before the first lecture in NTS (baseline), given on the same day they were recruited. After 12 months, the students answered the questionnaire during a lecture at the University College, while the supervisors were given and answered the questionnaire in clinical practice. Both times they were collected in locked envelopes.

Table 2 Questions Used in the Study

Ethical Considerations

Data were collected in line with ethical guidelines for nursing research in the Nordic countries.^29,30 The respondents received information about the study, that participation was voluntarily, and that data would be treated anonymously and confidentially. The questionnaires were kept locked up and were only available to the researchers. Regional Committees for medical and health research ethics in Norway were not applied because the data were collected anonymously. The Research Board at the University College and Norwegian Centre for Research Data (NSD) approved the project (reference number 43,587).

Statistical Analysis

Descriptive statistics were performed to evaluate knowledge and awareness about NTS among students and their supervisors. Mean scores were calculated for the following variables related to NTS: knowledge, significance in clinical practice, focus in clinical practice, and perceived impact on adverse events. Cross tabulations were used to show the level of self-reported knowledge/importance/focus in clinical practice and the impact on adverse events. Chi-square tests were used to compare students and supervisors as well as different continuing education programs. The use of the pocket edition of NANTS.no was analysed using the mean of a five-point Likert scale with response categories from 1 = never to 5 = very often. Data were analysed using the Statistical Package for the Social Sciences (SPSS), version 26.

Results

In total, 40 students and 51 supervisors responded to the questionnaire (n = 91) at baseline. The students/supervisors participated in the nurse anaesthetist (n = 9/11), emergency care (n = 13/7), critical care (n = 8/8), and operating room nursing (n = 10/25) educational programs (Table 2). The number of respondents who answered the question was somewhat reduced after 12 months to 33 students and 29 supervisors, representing nurse anaesthetist (n = 8/0), emergency care (n = 5/5), critical care (n = 9/4), and operating room nursing (n = 11/20) specialties.

The respondents’ knowledge and awareness of NTS was assessed with the questionnaire (Table 2). Categorical responses to questions about NTS with comparisons between students and supervisors at baseline and at 12 months (Table 3). The baseline results showed that there was a numeric difference between the reported knowledge/focus in clinical practice on one hand and the importance/impact on adverse events on the other, and this gap was reduced after 12 months of educational focus on NTS (Figure 1). Supervisors reported significantly stronger focus on NTS in clinical practice than students, both at baseline and after 12 months (Table 3). In addition, categorical responses to baseline questions about NTS between specialty categories for students and supervisors was measured (Table 4). Comparisons across specialties revealed only limited numerical differences, except one significant difference (NTS impact on adverse events among supervisors, Table 4).

Table 3 Categorical Responses to Questions About Non-Technical Skills with Comparisons Between Students and Supervisors

Table 4 Categorical Responses to Baseline Questions About Non-Technical Skills Between Specialty Categories for Students and Supervisors

Figure 1 Baseline and 12 months follow=up for NTS in students.

We found that for supervisors, focus on NTS in clinical practice was regarded as higher than for students at baseline (p = 0.004) and at 12 months (p = 0.01). When asked to what extent the supervisors supervised and evaluated students in NTS, there was an increase from baseline (mean 3.5, SD 0.8) to 12 months follow-up (mean 3.9, SD 0.8). To what extent they perceived that a focus on NTS was useful in evaluating and supervising students, the supervisors reported a mean score of 4.7 (SD 0.5) at baseline and a mean score of 4.3 (SD 0.8) at the 12 months follow-up. Use of the pocket edition in clinical practice after 12 months was similar but not very high for students (mean 2.2, SD 1.0) and supervisors (mean 2.1, SD 1.2).

Discussion

This study examined students’ and supervisors’ knowledge and awareness about NTS during a continuing education program. NTS was found to be of great importance in clinical practice and of great importance for the prevention of adverse events, although self-assessed knowledge and awareness of NTS in clinical practice was low. Even though the knowledge of NTS was low, may their assessment of NTS importance in the clinic indicate that the respondents had an idea of what NTS was without being able to elaborate or explain it properly.²⁸ There also may be some literacy problems on how to verbalize different characteristics of NTS. A lack of conceptual understanding may limit the use of NTS, and make it difficult to understand when NTS fail, which have been found in previous studies.^14,28,31 These can be useful findings to construct an educational plan for improvement. A conceptual understanding requires a high level of awareness³² and presupposes the ability to translate concepts into relevant contexts.³³ Although many health professionals are familiar with the importance of NTS for patient safety, many practitioners still are uncertain about the concept and significance of NTS,²⁸ which can make NTS challenging to implement in clinical practice.

The distance between the respondents’ self-reported knowledge and focus of NTS in clinical practice on the one hand, and the importance of NTS in clinical practice and in preventing adverse events on the other, was somewhat reduced after 12 months of pedagogical focus on NTS (Table 1, Figure 1). This shows that it is possible to influence students’ and supervisors’ knowledge and awareness of NTS, similar findings are reported in Griffin et al s study.²⁶ Multifaceted interventions are reported to have better outcomes than single interventions.^14,35 In addition, pedagogic principles highlight variation in learning strategies^12,14,27 and are the background for the choice of multifaceted interventions in the present study. There was still some gap after 12 months. One explanation for this finding may be that in general, implementing new knowledge in clinical practice is challenging.^34,36 Another explanation may be that it is known that fatigue and stress are two elements that can affect coping and focus on NTS, regardless of any coping strategies.^12,34,36 In any case, this is a finding that needs to be looked into more closely. It is also important to bear in mind that self-assessment is a subjective measurement and may not give a correct picture of the students’ actual level of competence.³⁷

There was no difference between specialties among students or supervisors (Table 4) at baseline, which suggests that efforts to increase knowledge and awareness about NTS is important both during education and in clinical practice. Another possible explanation for different groups giving comparable answers about NTS may be a similar understanding of NTS in this group of specialist nurses.²⁸ Conversely, supervisors’ focus on NTS in clinical practice was higher than for students at both measurements (Table 3), which probably reflects to a certain extent a natural maturation process that follows from the supervisor’s long experience and insight.²¹ To anticipate and to think ahead and respond to situational changes, are skills that are improved through experience and require a high degree of pattern adaptation.²¹

The supervisors perceived the focus on NTS to be useful in the supervision and evaluation of students, and there was an increase from baseline to the 12 months follow-up measurement. In contrast, there was a noticeably lower number who used NTS at the 12 months follow-up measurement, which highlights an inconsistency between the perceived importance and use of NTS. The fact that more supervisors eventually used NTS in supervision and assessment may be related to the supervisors’ increased knowledge and competence related to NTS and their increased experience with NTS’ usefulness in supervision and their increased familiarity with the tool. This is also confirmed by feedback from supervisors who reported that it was easier to supervise and assess NTS competence when they had concepts for describing NTS skills. The supervisors’ focus on NTS in clinical practice also increased when they gained more knowledge about NTS and were given a tool (the NANTS.no) (Table 1). Nevertheless, fewer benefits were reported when using NTS in student guidance and assessment, and there was a further reduction after 12 months.

However, a smaller number reported positively on the benefit of using NTS in guidance and assessment of students and even a reduction after 12 months. This result may reflect that the professional’s actions in practice will often be governed by established knowledge of routines, even when knowledge of new methods and the duty to put them into practice are present.³⁴ The pocket edition of the NANTS.no was used to a much lesser extent than we expected at the follow-up measurement because the supervisors’ informal statements had indicated that they thought the NTS pocket edition was useful. Some supervisors also attributed to the pocket edition increased understanding of and attention to NTS. These statements are inconsistent with an SD of 1.2, which tells us that some respondents adopted it, while others did not. Several factors can explain this such as the supervisors’ different personalities and differences in how they take on new challenges. In addition, there might be a relationship between seeing the usefulness of the tool and seeing that it is easy to use. The ever-increasing focus on efficiency in practice also leads to a large degree of automation of actions, which makes it more challenging to introduce new tools despite their being considered useful. It can be challenging to maintain innovations over time,³⁴ and our findings may reflect that our follow-up of the use of the pocket edition was not good enough.

Our experience is that attention regarding NTS must be maintained over time. Reviews of implementation research, document that the majority of interventions can achieve moderate improvements,^14,24,38 but fewer studies have reported on the long-term retention of any interventions that are undertaken.¹⁴ Knowledge fades quickly, and courses extending over weeks or months are considered an advantage.¹⁴ The relevance of changes depends on their ability to persist, and the outcome of a study must be seen in connection with the duration of the intervention. More lasting behaviour change is dependent upon continuous education and being given the possibility for training and monitoring NTS.¹⁴ The literature emphasizes that education is an important tool for increasing patient safety,^13,18 and early exposure to NTS can contribute to developing and shaping students’ long-term patient safety knowledge, behaviour, and future mastery.^13,16,18

In the last 20 years, much has been done to develop strategies and solutions to increase NTS competence,¹⁸ but relatively few studies on NTS include both school teaching and clinical practice in their interventions, learning arena, and integration. Supervised clinical training is of great importance for the implementation of NTS competence among students,¹⁹ which this study recognizes and emphasizes. This study included the supervisors in several of the learning activities to promote the supervisors’ NTS competence in order to improve knowledge transfer, and to ensure competent assessment of students’ learning. In training and implementation, emphasis is placed on the importance of giving feedback after the assessment,^9,39 and the importance that training in NTS is assessed in situations with the entire team present.¹² These are considerations that make it absolutely necessary to ensure the NTS competence of those who are to assess NTS in situations where such skills are observed. Along with simulation, clinical practice is an arena where NTS is practiced under realistic conditions.³⁹

Student-active methods such as simulations, encourage deeper learning,¹⁸ and are shown to be an effective and efficient method to learn NTS.^12,21,26,40 We also used facilitated reflection groups and case discussion to engage students in a meaningful manner, and these methods are described as an essential strategy of particular benefit for patient safety education and implementation,¹⁸ and are more effective for long-term learning.⁴¹

Strengths and Limitations

One limitation in the present study is that we used a non-validated questionnaire. However, we conducted a comprehensive review of the literature, in addition to having discussions with various expert members and performing a pilot test, to increase the face and content validity of the questionnaire. It is also a limitation that the anonymous data excluded the opportunity to perform longitudinal analyses at the individual level. Moreover, the sample size was relatively small, and the study was conducted in only one university college. This limits the generalizability of our findings. In addition, there was some attrition from baseline to 12 months which may be related to the way the questionnaire was distributed. Despite these limitations, the findings from this study suggest that education and training in NTS increase knowledge and awareness of NTS in clinical practice both for students and supervisors.

A strength of the study is the inclusion of teaching and training of the supervisors as well as the students in order to identify the need for knowledge and awareness of NTS because the supervisors are of great importance in influencing the students’ professional competence development. However, our study must be validated in a larger setting because the sample size is quite small.

Conclusion

This study suggests that there is an untapped potential for better quality and patient safety due to a lack of knowledge and awareness about NTS. Integration of NTS in various learning activities seems to strengthen students’ competence about NTS and may have important potential for improvement in both education and clinical practice. Supervisors reported a significantly stronger focus on NTS in clinical practice, which highlights the need for NTS training among students. Different groups provide comparable answers about the NTS, which suggests that the understanding is similar across different specialties and roles. The results provide a basis for further work but more research on the link between knowledge and awareness with competence will strengthen the basis for implementation of NTS through an educational program. Integration of NTS must take place both in school and in the clinics, and the supervisors’ NTS competence must be ensured.

Implications

Integrating NTS into health education curricula is essential to increase the conceptual understanding of NTS’s clinical role in patient safety, and quicker development of NTS competence in healthcare professionals may optimize patient care. It may be beneficial to use a variety of learning methods when teaching NTS. Identifying enthusiastic practitioners who are effective communicators and who have the initiative to help develop this is vital.