Impact of Virtual Reality Mental Health Nursing Simulation on Nursing Students’ Competence

Introduction

Background

Currently, one in ten, about 792 million people worldwide is affected by mental health problems, accounting for 10% of the global economic burden.¹ The majority of mental health issues, primarily anxiety and depression, pose a concern as the age of initial diagnosis is progressively decreasing, with cases emerging in individuals under the age of 14.² Nurses are on the front lines of caring for patients suffering from the global burden of mental health concerns.³ More people are experiencing mental health concerns as a result of COVID-19, creating a significant crisis for professionals dedicated to care for them.⁴ The number of mental health professionals is critically low, reaching a ratio of one professional per 10,000 individuals.⁵

Challenges in Mental Health Nursing Education

As a global health issue, mental illness has gained much attention, with nurses playing a crucial role as members of interdisciplinary teams in mental healthcare settings. Therefore, nursing schools offer MH nursing as a mandatory course in their undergraduate curriculum designed to equip students with knowledge and practical skills in MH nursing. During the practicum, nursing students systematically assess the client’s MH status using various communication strategies that help build a therapeutic relationship, develop critical thinking ability (CTA) for health problem identification and diagnosis, and acquire therapeutic communication skills.⁵

However, owing to various factors, such as clients who are reluctant to communicate, psychotic behaviors requiring immediate intervention, and students’ prejudice stigma and fear about mental illnesses, nursing students encounter obstacles in actively engaging with effective mental health nursing education.⁶ MH rotations often generate higher levels of anxiety, fear, and negative attitudes among nursing students compared with other clinical rotations.⁷ Establishing therapeutic relationships and conducting in-depth interviews with mentally ill clients, who may be hesitant to disclose personal information owing to social stigma, can impose a heavy burden on nursing students. Furthermore, the unpredictable nature of psychotic behaviors and the challenging behaviors of aggressive patients can also create barriers to relaxed clinical rotations.⁷

In such a challenging educational environment, nursing students experience anxiety during MH nursing practicum, which influences their knowledge, confidence, and clinical skills. Therefore, innovative strategies that can offer nursing students practice opportunities to develop their competence in caring for individuals with mental disorders are urgently needed.

The Need for a VR-Based Solution

Simulation is an educational training method that employs relevant scenarios to achieve specific learning objectives in unique situations and processes.⁸ It enables students’ competencies to be evaluated without threatening patient safety. Furthermore, the application of simulation in education fosters a secure learning environment that promotes repetitive practice, allowing students to undergo the learning process multiple times. This approach enhances critical thinking, problem-solving skills, and safe practices while affording students the opportunity to make mistakes without adverse consequences. In mental health nursing practicum courses, simulation creates a realistic yet safe setting, alleviating anxiety and fear among students and contributing to an improved overall learning experience.^9,10

In the MH nursing practicum, simulation uses standardized patients (SPs) or high-fidelity simulators. SPs, who are trained to portray patients based on standard processes from a coaching manual, are commonly used as a safe and effective means of practicing MH nursing.¹¹ However, concerns regarding the need for continuous training and management of SPs to maintain their quality are ongoing, as are concerns regarding the varying levels of proficiency and responses given by the SPs during the same scenario depending on the individual serving as the SP.¹² On the other hand, high-fidelity simulators are not as realistic and have limitations in accurately depicting the mental problem or symptoms.¹³

Virtual reality (VR) is an advanced technology that integrates various engineering and information technologies to create computer-generated environments that closely resemble the real-world environment, situation, or technology. The ability of VR to transcend limitations of time and space, allowing multiple students to participate simultaneously, is one of its key advantages.¹⁴ VR is being increasingly applied in therapeutic communication with patients, clinical decision-making processes, and motor control techniques.¹⁵ While VR training programs have been utilized in various areas of nursing, including emergency patient resuscitation, acute diseases, and pediatric care, relatively few cases focus on MH patients.¹⁴

The ultimate goal is to create an appropriate and secure environment for MH nursing practicum by leveraging engineering and information technology for nursing students. Furthermore, this study seeks to validate the program in terms of successfully achieving the MH practicum learning objectives. The findings will contribute to identifying essential educational elements for the safe implementation of such practicum and will serve as a foundation for educating competent psychiatric nurses.

Aim

The aim of this study is to investigate the impact of a VR-based MHNPS program on the following factors: nursing knowledge related to mental disorder (NKMD), attitudes toward mental illness (AMI), communication self-efficacy (CSE), problem-solving process (PSP), CTA, and mental health nursing clinical confidence (MHNCC) in nursing students.

Study Hypotheses

Hypothesis 1 (H1): Nursing students who undergo the VR-based MHNPS program will demonstrate a higher level of NKMD compared to the baseline.

Hypothesis 2 (H2): Nursing students who undergo the VR-based MHNPS program will demonstrate higher CSE compared to the baseline.

Hypothesis 3 (H3): Nursing students who undergo the VR-based MHNPS program will demonstrate enhanced PSP compared to the baseline.

Hypothesis 4 (H4): Nursing students who undergo the VR-based MHNPS program will demonstrate enhanced CTA compared to the baseline.

Hypothesis 5 (H5): Nursing students who undergo the VR-based MHNPS program will demonstrate better AMI compared to the baseline.

Hypothesis 6 (H6): Nursing students who undergo the VR-based MHNPS program will have higher MHNCC compared to the baseline.

Materials and Methods

Study Design

To validate the effects of the aforementioned MHNPS program, we adopted a one-group pretest-posttest repeated measures experimental study.

Study Population

Nursing students who had completed at least one semester of a MH nursing theory course and have yet to begin the practicum were recruited; those who provided an informed consent were enrolled. Sample size was determined using G*Power 3.1.9.7 software. Following previous literature,¹⁶ the sample size was calculated based on a significance level (⍺) of 0.05, power (1-β) of 0.95, and two repeated measures, and the minimum sample size was calculated to be 45. Based on an expected dropout rate of 20%, 54 participants were initially required for the study. However, after excluding 4 participants who did not complete the second post-intervention survey, the final number of participants included in the study was 50.

Study Instruments

This study adopted the following instruments:

Nursing Knowledge Related to Mental Disorder (NKMD)

NKMD was measured using 20 items, with five items for each of the modules (schizophrenia, depressive disorder, bipolar disorder, alcohol use disorder). A higher score indicates a higher level of NKMD. The Cronbach’s α (for reliability) was 0.73.

Communication Self-Efficacy (CSE)

CSE was assessed using the self-efficacy questionnaire developed by Axboe et al,¹⁷ and adapted and modified for use on Korean nursing students by Gil and Sung.¹⁸ This 12-item tool uses a 10-point Likert scale, and a higher score indicates higher CSE. The Cronbach’s α was 0.98 at the time of Korean adaptation and 0.96 in the current study.

Problem-Solving Process (PSP)

PSP was assessed using the PSP tool for adults developed by Lee et al.¹⁹ This 30-item tool comprises five domains: clarification of problem, exploration of solution, decision making, execution of solution, and evaluation and reflection. Each item is rated on a five-point Likert scale, with a higher score indicating better PSP. The Cronbach’s α was 0.93 at the time of development and 0.95 in the current study.

Critical Thinking Ability (CTA)

CTA was assessed using the critical thinking tendency tool developed by Yoon.²⁰ This 27-item tool comprises seven domains: intellectual passion/curiosity, prudence, confidence, systematicity, intellectual fairness, healthy skepticism, and objectivity. Each item is rated on a five-point Likert scale, with a higher score indicating higher CTA. The Cronbach’s α was 0.84 at the time of development and 0.87 in the current study.

Attitudes Toward Mental Illness (AMI)

AMI was assessed using the AMI scale developed by Singh et al.²¹ The scale was first translated by an English-proficient researcher, reviewed by two researchers, and back-translated. Awkward vocabulary and sentence structures were revised without altering the meaning of the source text. This 20-item tool comprises cause, treatment, and outcome of mental illness, as well as attitudes toward the impact of a mental illness on the individual and society. Each item is rated on a five-point Likert scale, with a higher score indicating more negative AMI. The Cronbach’s α was 0.84 at the time of development and 0.91 in the current study.

Mental Health Nursing Clinical Confidence (MHNCC)

MHNCC was assessed using the scale developed by Bell et al²² and modified into a four-point scale by Martinez;²³ the scale was first translated into Korean. This 15-item tool uses a four-point Likert scale, with a higher score indicating greater MHNCC. The Cronbach’s α was 0.84 at the time of development and 0.92 in the current study.

General Characteristics Questionnaire

General characteristics included sex, age, school year, prior clinical practicum experience, prior simulation experience and satisfaction of the VR simulation was expanded to in-school practicum nursing.

Data Collection

Data were collected from third-year nursing students, who had completed an MH nursing theory course but never taken a practicum, in one geographic region (Gangwon State) from October 30, 2022, to January 6, 2023. As the COVID-19 pandemic prevented on-site clinical rotations, the practicum was conducted online. Prior to beginning the practicum, the students were informed about the purpose and method of the study, confidentiality, and freedom to withdraw or not participate without any disadvantages. A total of 54 students who voluntarily consented to participate were enrolled.

After the practicum orientation and before beginning the VR simulation program, participants were provided with a link to the pre-intervention survey. The post-intervention survey was also completed via the same method, immediately and six weeks after the practicum. A total of four students who did not complete the second post-intervention survey were excluded, and the remaining 50 were included in the final analysis.

Application of the VR Simulation Program

The MHNPS program was implemented as a web-based intervention, comprising ten modules: two for schizophrenia (delusion/hallucination), bipolar disorder manic, three for depressive disorder (geriatric, adolescent with suicidal ideation, middle aged), post-traumatic stress disorder (PTSD), alcohol use disorder, dementia and obsessive-compulsive disorder (OCD). Each module consisted of 13 steps, which are detailed in Figure 1.

Figure 1 Flow and contents of the simulation program.

To facilitate accessibility and flexibility, a dedicated website was created for students to access the program at their convenience, without limitations on time or location. Each student was assigned a unique ID and password to securely log into the website. The website allowed students to record their learning activities for each step of the program.

Participants engaged in a one-week (5 days) learning period, during which they completed a total of four modules (schizophrenia with hallucinations, geriatric depression, bipolar disorder manic, and alcoholism) from the available ten modules (Figure 2). The participants were divided into six simulation groups to facilitate the learning experience. Debriefing and question-and-answer sessions were conducted on day 3 (online) and day 5 (in-person) of the learning period.

Figure 2 Screenshot of the VR scenario.

On the first day, participants were given an orientation of the program. They then completed the major depressive disorder module on day 1, substance use disorder module on day 2, schizophrenia module on day 3, and bipolar disorder on day 4, and group debriefing sessions on days 3 and 5. During the group debriefing sessions, the participants were asked the following opening questions: “How was this simulation? (Things you felt, things that threw you off)” and “What do you think you have done well?” These were followed by specific questions about each module: “What was the patient’s situation?” “What were the priority nursing problems?” “What nursing interventions can you provide?” Finally, they were asked closing questions: “How can you apply the learned knowledge and skills in real-world situations?” and “Can you share the most important take-away from this practicum?” Each member of the group shared their experiences to wrap up the simulation. The participants had an opportunity to implement directly “activity therapy” (a type of therapeutic nursing intervention), and each team presented their implementation. Furthermore, to enhance their learning, the participants also had an opportunity to see and match the actual medications used for the symptoms of the disease in each module.

Data Analysis

Data were analyzed using IBM-SPSS 22.0 software. Participants’ general characteristics were analyzed using real numbers, percentage, mean, and standard deviation. Paired t-tests were adopted to investigate changes across the time points (pre-, immediately post-, and six-week post-intervention).

Ethical Considerations

This study was approved by the Institutional Review Board at the Gangneung Wonju National University (approval number: GWNUIRB-2022-23) and was conducted in accordance with the Declaration of Helsinki principles. The information sheet for the participants explained the purpose and method of the study, guarantee of voluntary participation, freedom to withdraw at any time, and management of study data anonymously by coding in accordance with the Bioethics Act. An explanation of the study was provided online before obtaining voluntary informed consent.

Results

Participants’ General Characteristics

Participants’ general characteristics are presented in Table 1. The mean age was 22.24 years (SD=3.48), and female students were predominant (n=43, 86.0%). The most common type of prior clinical practicum courses were both online and offline(hospital) (n=34, 68%), followed by offline only (n=13, 26%) and online only (n=3, 6.0%). Regarding simulation, most students experienced simulation practice online (n=42, 84.0%). The mean satisfaction with major was 7.58/10, indicating a relatively high satisfaction.

Table 1 General Characteristics of Participants (N=50)

Effects of the VR-Based MHNPS Program

NKMD was found to have significantly changed immediately (t=8.17, p<0.001) and six weeks after the program (t=6.10, p<0.001). CSE had significantly changed immediately (t=4.36, p<0.001) and six weeks after the program (t=4.04, p<0.001). PSP had not significantly changed immediately after the MHNPS program (t=1.97, p=0.054), but had significantly changed at six weeks after the program (t=3.87, p<0.001). CTA had significantly changed immediately (t=2.04, p=0.046) and six weeks after the program (t=3.45, p=0.001). AMI had significantly changed immediately after the MHNPS program (t=−2.22, p=0.031), but the change was not significant six weeks after the program (t=0.32, p=0.754). MHNCC was significantly changed immediately (t=4.25, p<0.001) and six weeks after the program (t=4.95, p<0.001). Therefore, hypotheses H1-H6 were supported (Table 2).

Table 2 Effects of Mental Health Nursing Practice Simulation

Discussion

To explore the transfer of learning despite students’ restricted patient experiences during the MH nursing practicum, we examined the potential of our simulation program as an educational medium to bridge the gap between classroom and clinical setting. The goal was to ensure that knowledge, skills, and attitudes pertinent to MH nursing acquired in the classroom are effectively applied in clinical settings. Our results indicated that the VR-based MHNPS program significantly improved NKMD (knowledge), CSE, PSP, and CTA (skills), and AMI and MHNCC (attitude), supporting our hypotheses. Notably, NKMD, CSE, CTA, and MHNCC were significantly improved both immediately and six weeks after the intervention. Facilitating learning transfer is critical for students to effectively apply classroom knowledge, skills, and attitudes to real-life clinical settings. Oh²⁴ discovered that incorporating learning transfer theory into a communication skills training program for nursing students increased learning motivation, transfer motivation, and the perceived importance of communication.

Students were able to acquire more NKMD as they re-do a step if they do not meet pass criteria of the pre-learning, pre-quiz, and post-quiz developed for each scenario. The increase in NKMD was evident not only immediately after but also until six weeks after the intervention. Lee et al¹⁴ stated that repetitive learning sends the learned materials to the long-term memory, thereby expanding the volume of accumulated knowledge. Particularly, our program was designed such that students who fail to achieve the learning objectives at each step must re-do the step and meet the achievement criterion to proceed to the next. Thus, students were able to monitor their own level of learning accomplishments during each step. As demonstrated, designing the program to enable systematic and repetitive learning processes appears to have contributed to the retention of increased knowledge at six weeks after the intervention.

During the simulation, students watched a VR video depicting nurse-patient conversations and analyzed the communication processes. In this process, the student divided the conversation between the nurse and the patient in the VR video into sentences and classified them into therapeutic communication and non-therapeutic communication. Next, if it was determined to be non-therapeutic communication, texting was provided directly to the program on how to change it into correct therapeutic communication. This allowed students to strengthen their counseling skills with psychiatric patients. They also participated in a debriefing session in which they comprehensively analyzed and integrated their opinions through discussions with the instructor. The VR video was based on a real-life case to provide realism akin to an actual encounter in a psychiatric nursing unit. This immersive experience provided students a vicarious experience of assuming the role of the nurse. As such, they were able to learn by identifying things to note in future communications with a patient as well as what went well during their communication with the patient. Donovan and Mullen²⁵ established that engaging in communication simulations enhances students’ confidence in patient communication, as they are able to observe patients’ responses during communication and simultaneously identify key areas of focus.

During simulation, students achieve the given learning objectives as they undergo structured steps of analyzing patient’s case records, performing an MH assessment, analyzing virtual case videos, analyzing therapeutic communication, identifying the nursing problems, implementing nursing interventions, and documenting the activities. The backbone of this process is the nursing process consisting of assessment, diagnosis, planning, implementation, and evaluation, so students are able to analyze a case from various perspectives and gain an expanded understanding of the case. Additionally, by presenting and discussing the analysis with the instructor and peers, the students are guided to decide on the best evidence-based practice and derive plans to implement it. Throughout the simulation intervention, students engaged in the iterative process of analyzing, presenting, and discussing four distinct cases of mental disorders, and such repetitive learning is presumed to have led to improvements in CTA and MHNCC.

PSP was not significantly changed immediately after the intervention but was significantly improved at six weeks after. PSP refers to the ability to navigate through processes of clearly identifying the core problem in a specific situation, formulating a solution based on that problem, and then analyzing and providing feedback on the outcomes.¹⁹ As PSP is a complex process of thinking, the program did not seem to immediately improve students’ PSP but led to improvement after they accumulated repetitive learning experiences for the simulation cases. This is because students repeatedly practice the problem-solving process of assessment, diagnosis, nursing plan, therapeutic nursing intervention and evaluation during the process of learning ten modules. Many previous studies assessing PSP in nursing students have consistently reported high scores in the problem clarification domain but relatively lower scores in the decision-making domain.^19,26 Clarification of the problem can be enhanced through intuitive interpretation of the case analysis. Conversely, decision-making is an outcome of logical reasoning involving integrating the results of case analysis, such as prioritizing alternatives by integrating such results, identifying the pool of feasible alternatives, and setting priorities of the alternatives by comprehensively considering the anticipated benefits and feasibility for the given patient. Thus, the observed trend of improvement in PSP competency through the implementation of this simulation program suggests an enhancement in complex reasoning skills encompassing both problem clarification and decision-making domains. Further research is needed to analyze patterns and develop refresher programs that can continually foster problem-solving competencies.

AMI was significantly improved immediately after the simulation intervention. In the case of students who participated in this simulation, it is believed that the improvement in NKMD and CSE, as well as the improvement in CTA and MHNCC, worked in an integrated manner to show immediate improvement in AMI. Attitude refers to the mental and psychological readiness that affects the individual’s response, encompassing cognitive, emotional, and behavioral components.²⁷ Lehr and Kaplan²⁸ documented that nursing students who participated in MH nursing simulations displayed increased confidence in providing care for patients in the real world, better understanding of patient reactions, and improved assessment and decision-making skills. However, it is important to note that the improvement in attitude was not retained until six weeks after the program, as attitudes are typically formed and changed as a result of diverse experiences encountered over a long period of time. Therefore, in order to ensure changes in AMI, it is necessary to design a step-by-step simulation curriculum rather than a single simulation. This is because the more simulations students experience, the more they can maintain or improve AMI changes. Simulation reduces students’ biases about mental health and improves their AMI.^29,30 Particularly, Attoe et al²⁹ found that MH simulations are effective in improving interprofessional roles and responsibilities and attitudes toward interprofessional collaboration. Attaining a change in attitude entails a complex process, and additional research should explore the theoretical basis of transforming AMI and develop objective instruments to assess the degree of change.

This is a single-group intervention study targeting nursing students at single nursing school, and subject selection bias is possible. Therefore, when generalizing the findings of this study, caution is advised. Nevertheless, the VR-based MHNPS developed in this study showed significant improvement in all six effect measurement variables. In MH nursing, it is difficult to predict patient’s hallucinations, auditory hallucinations, and behavioral outbursts that manifest as a defense mechanism. Thus, clinical simulations, such as our simulation program, have been reported to be effective in equipping nursing students with competence to deal with emergencies that arise while providing care for patients with a mental illness.¹³ Our simulation program was specifically designed to address real clinical cases, preparing nursing students to deal with such severe mental psychiatric symptoms. By exposing students to difficult and severe cases of mental health disorders within the virtual environment, the program helps them gain the necessary skills and confidence for patient care before they enter clinical settings. Donovan and Mullen²⁵ also proposed virtual simulation to be implemented as an active learning archetype in MH nursing. Such an approach provides an opportunity for students to experience difficult or severe cases of MH disorders and prepares them and increases their confidence with patient care before they progress to the real clinical setting. Therefore, we anticipate that this simulation program will enhance nursing professionals’ confidence and foster positive attitudes by bridging the gap between theoretical learning and clinical practice in MH nursing education for nursing students.

At the end of the simulation, a debriefing was held to review each student’s experiences and what helped them learn. During this process, perspectives were offered and shared on what nurse intervention would be undertaken based on the patient’s response and what nursing knowledge and abilities would be necessary. This study has a shortcoming in that it cannot evaluate this process in detail. Therefore, it is vital to comprehend the educational and learning experiences of subjects in the simulation process from multiple viewpoints using a study methodology that integrates qualitative research. In the future, it will be necessary to expand the basis for developing improved simulation modules based on this.

Conclusion

With growing demands for virtual clinical practicum as a result of the spread of COVID-19, the effectiveness of VR-based clinical practicum has been assessed in numerous studies. In line with such attempts, this study investigated the effects of a VR-based MHNPS program consisting of cases of delusion, hallucination, mania, suicidal ideation, and OCD on MH nursing competence in nursing students. Based on the transfer of learning theory, all six study parameters were significantly improved after the intervention, suggesting that our MHNPS program may serve as a useful alternative educational medium in cases in which in-person clinical rotations are restricted. Moreover, these effects spanned all three domains, namely knowledge, skills, and attitude, pertaining to MH nursing. In conclusion, mental health nursing practice faces unique challenges since it serves individuals with mental illnesses who have difficulty forming interpersonal relationships or communicating. This study assessed a VR-based MHNPS program. Our findings showed significant improvements in students’ mental health nursing competencies. This suggests that MHNPS is proven to be very useful as an educational medium for psychiatric nursing practice for nursing students. Future studies should call for strengthening mental nursing capacity to rehabilitate and improve the quality of life of mentally ill people living in the community. Therefore, if we develop community-based mental illness case scenarios based on our successful MHNPS program, we expect to be able to significantly enhance nursing students’ psychiatric nursing practice capabilities.

Limitations of the Study

Since this study focused on nursing students attending a local university in one country, studies in similar situations need to be repeated. Additionally, developing and maintaining VR programs can be costly.