The Use of in situ Simulation in Healthcare Education: Current Perspectives

Introduction

In situ simulation is simulation-based education that takes place in the clinical setting where participants usually work, rather than dedicated training centers. Increasingly in situ simulation is used in medical education to improve professional skills, team functioning and clinical care.¹ In situ simulation can be defined as a team-based simulation strategy involving interdisciplinary healthcare team members working in their own environment on patient care units.² It is based on creating a safe environment to experience and practice managing a scenario which is high risk in real clinical care, such as cardiac arrest, and to consider appropriate actions should the real situation arise. In situ simulation has been designed and adapted by many, with data supporting both high-fidelity and low-fidelity simulations in a number of systematic reviews.^3,4 Fidelity commonly describes to what extent a simulator represents real life.⁵

In situ simulation is used for a number of different purposes, including medical education, assessment, quality improvement purposes and patient safety initiatives.^5,6 The use in clinical practice to improve professional competencies and interdisciplinary practice has seen a significant rise in recent years. Its positive outcomes have been described, such as improved patient morbidity and mortality, improvement in clinical skills of staff and better organizational performance.^7,8 Its use, in particular clinical disciplines, has received attention, such as resuscitation and obstetrics and gynecology. Published reviews emphasize the lack of an evidence base, guidelines or consensus for best practice in simulation, which must be guided by a clear overview of different approaches to in situ simulation.^4,9,10 This review seeks to understand the current approaches to in situ simulation training across medical and healthcare education, specifically identifying the setting and purpose of its use and evaluating its impact.

Methods

We conducted a literature search using the terms: ‘in-situ simulation’ and ‘in situ simulation’. OVID, AMED (Allied and Complementary Medicine), HMIC Health Management Information Consortium, London Health Libraries, Embase, Emcare, MEDLINE, and Social Policy and Practice databases were searched for eligible studies only studied including humans in English were included, 208 articles were initially found and duplicates were removed. During title and abstract screen, editorials, commentaries and conference abstracts were excluded as well as unrelated articles. Fifty-three articles were screened for full-text analysis against the inclusion criteria, 30 articles met the inclusion criteria and were included in this study.

Eligibility criteria included article type, setting and intervention. Included studies had to present primary data, while comment and editorial type articles were excluded. Settings were limited to healthcare, from primary through to hospital and specialist care. The training intervention had to be in situ simulation with an educational component, meaning that simulation to test equipment or facilities were excluded, along with simulation for assessment purposes. The article had to describe the in situ simulation design and evaluate how the simulation has helped with the training of the healthcare staff. Articles where the primary outcome is service improvement and patient outcomes were excluded. A flowsheet of article inclusion is shown in Figure 1.

Figure 1 Flowchart of article inclusion.

Data were extracted and information was inputted into a pre-designed excel spreadsheet. Information extracted included general demographic data as well as: (i) aim and setting, (ii) simulation design, (iii) evaluation methods and (iv) outcomes and impact. After data extraction, subcategories were grouped together and data were analyzed via thematic analysis and a narrative synthesis was carried out.

Results

Thirty articles met the inclusion criteria. The setting of the simulations ranged between specialties and this can be seen in Table 1. Five studies were descriptive, nine studies were pre- and post-simulation studies, nine studies assessed the impact and outcomes of the simulation and seven studies focused on the simulation design itself.

Table 1 Overview of Different Practices Adopted in “in situ Simulation” Today

Studies included a range of healthcare from all professionals including general staff, nurses, pharmacists, junior doctors, trainees, registrars, and consultants.

Data extracted from each section were grouped and synthesized which is reported on in each of the sections below.

In situ Simulation Setting and Purpose

Purpose of Simulation Training

The overarching aim in all included studies was education of healthcare staff, including doctors of all levels, nurses and allied health professionals. Most studies had either their primary or secondary aim to improve teamwork and communication, specifically within an interprofessional team setting.^11–13 Other aims included increasing patient safety or orientating staff to a new clinical environment.^14–16 A few studies evaluated the design of the simulation and what design would be most effective.^17–21 There were also specialty specific aims such as resuscitation skills.^15,22,23

Setting

Simulation has been used in a number of different specialties within medicine, the most common being resuscitation skills.^{11–13,15,22–28} Other specialties included obstetrics and gynecology, midwifery, anesthesiology and mental health.

Simulations on resuscitation were carried out in different settings ranging from the emergency department, ward trauma rooms, outpatient settings, Intensive care units, and operating rooms.^{11,13,18,22,24,25,27} The emergency departments ranged from low, medium to high volume departments.^16,23,25 Some studies scheduled the simulations in the morning, due to the decreased number of patients who attend ED at that time.^11,27 Others opted for a flexible timetabling approach, rescheduling the simulation if there was a demand for the emergency team.¹¹

The educational component either involved managing different emergency scenarios or focused on specific skills. General adult studies focused on cardiac arrests, airway management, interpreting vital signs and hemorrhage control.^11,18,24,29 However, the majority of studies were specific to pediatric emergencies. Some of these focused on skills such as drawing blood, placing nasogastric tubes, urinary catheters and IV lines.^22,23 While others focused on managing acute presentations such as cardiac arrest, neonatal resuscitation, seizures and status epilepticus, trauma, toxicology, sepsis and shock, respiratory distress, fracture management, airway management and diabetic ketoacidosis.^12,23,25,27 These simulations tried to focus on emphasizing the important differences between pediatric versus adult resuscitation, by introducing challenges seen specifically in pediatrics such as parents surrounding a small cot and scattered toys.²⁷

Skills were either chosen by carrying out needs assessment surveys for the staff prior to simulation design or taken with the guidance of resources such as the pediatric resuscitation practical approach or pediatric perioperative cardiac arrest registry.^12,13,23,28

Simulations in the Obstetrics and Gynecology specialty took place in labor rooms, inpatient rooms, postnatal wards, operating theatres or in one case in a separate room.^20,21,26,30 Most studies focused on common obstetric emergencies. The most common cases being post-partum hemorrhage and shoulder dystocia followed by obstetric hypertensive emergency and eclamptic convulsions.^20,26 Most of the simulations used patient actors and occasionally the birthing woman’s partner, with only one study in the operating theatre using a manikin.²¹ The only gynecological study found, focused on skills improvement such as improving dexterity, fine motor and spatial relationships in laparoscopy.³⁰ More recently, in situ simulation training has been used in a community setting for midwives to prepare for home-birth-related emergencies.³¹ The scenarios were developed by the multi-disciplinary team including obstetricians, nurses and midwives.²⁶

One study assessed the effects of designing a simulation lab in the work space with easy access for staff to use both in designated and free time but was limited by the lack of outcome measures.³⁰ Although less commonly, other specialties have also adopted in situ simulations. Simulation in anesthetic skills was directed at both anesthetists but also non-anesthetists.^19,32 In situ simulation for anesthetists was carried out in the operating room to practice tracheal intubation.¹⁹ Non-anesthetists practiced sedation skills in pediatric emergency settings.³² A study in a mental health triage ward focused on medically deteriorating patients in a mental health setting, choosing priority training needs for the staff with scenarios such as respiratory distress, diabetic hypoglycemia, hanging and choking.³³ A simulation on cricothyroidotomy evaluated whether the simulation practice would improve skills after didactic teaching, compared to teaching alone.¹⁷ One study focused on emergency presentations out of the hospital setting and was based in primary care practices. Emergency presentations such as asthma exacerbations, cardiac arrest, a seizing child and severe allergic reactions were simulated for the staff to learn how to handle these in a setting which does not specialize in such presentations.³⁴ Finally, one study set in an intensive care unit focused on pericardial tamponade and impending cardiac arrest management.³⁵

Simulation Design

Simulation design showed a similar gross structure in most studies with a briefing, the scenario and a debrief session. However, how these components were carried out differed across studies.

Briefing

Briefing was adopted by most studies, most commonly in the form of presentations and didactic lectures.¹⁷ A few studies, however, opted for a more hands-on approach by offering training on specific skills before the simulation.^15,17 The content of the briefing most commonly involved orientating participants to the manikin and the simulation practice.^22,29,33,36 In some cases, this was taken a step further by clarifying the objectives the participants have, and highlighting core concepts fundamental for good practice such as teamwork, communication.^15,22 While in other cases, clinical education was the priority, such as one study on pediatric resuscitation, which highlighted the differences in managing pediatric and adult crises and another study on cricothyroidotomy used briefing to train participants, who were able to practice the clinical skills before the simulation took place.^15,17 Finally, one study gave participants a self-assessment on a scenario during the briefing.³⁶

Simulated Scenario

The scenarios themselves included either low-, medium- and high-fidelity manikins. This was mainly based on the resources and finance the specific hospitals had to carry out their simulations. A range of equipment was adopted. In most cases, a manikin was used; however, sometimes simulated patient actors were used, these were volunteers, healthcare staff such as midwives or medical students.^24,31,37 Some studies adopted both actors and manikins, for example, simulating the patient child, and an actor representing the parent.²⁵ One study used virtual reality simulators instead, for participants to be able to practice laparoscopic surgical skills.³⁰

Most simulations were announced, three studies evaluated unannounced simulations. One study told staff a simulation will occur within the next 6-month period and provided useful guidelines online to help participants prepare.²⁰ The duration of the individual simulation training programs ranged from a few days to a year.^23,26 Some centers decided to have simulations once or twice a week, while others had monthly simulations, in some cases, the scenarios became more complex as the months progressed.^12,29,30 Additionally, repetition of scenarios was sometimes adopted for the same participants, some studies did this on the same day after the debrief, others repeated this months later to detect longer term impact.^22,29,34 One study had a one-off simulation followed by a reassessment visit, the reassessment focused on areas of deficiency in the design of the simulation which allowed for an improvement in the simulation before it runs again.²² The shortest simulation lasted 10 minutes, and the longest a full 12-hour shift.^13,37

Staff who were involved as participants were usually a full multi-disciplinary team consisting of different specialists and nurses at different stages of training, mostly acting within the same role as they would in real life. Sometimes, participants who did not have a role acted as observers either by being in the room or watching through a live video stream.³³ A few studies directed nurses specifically, focusing on their training of skills alone rather than the full multidisciplinary team.^36–38

Debrief

All studies had a debrief session which lasted from 10 to 60 minutes. In most studies, debriefs were facilitated by an experienced instructor. Some instructors had completed a debriefing course before the session.^25,26,29 Others used resources that helped guide the session such as scripts, checklists or questionnaires.^13,16,25 Only one study mentioned the use of a specific model, the reflective Diamond debrief model, while most others were guided by general and scenario-specific checklists and guidelines.³³ One study used the pre-recorded videos of the simulation to guide and help participants better reflect on their performance.²²

The most common themes were reflecting on teamwork and communication.^11,14,26,39 There was also a focus on knowledge-based components, such as discussing improvement, identifying deficits in knowledge and reflecting on technical skills and guidelines.^{17,22,25,26,34,39} Some debriefs also focused on organizational feedback and proposals for change in the future.^18,20 As well as reflection, some debriefs provided participants with didactic information on relevant medical topics.^25,26

Organizational Factors

During the simulation itself, some centers ensured participants would be completely free when undergoing the simulation by providing dedicated time to do this during their working hours and replacing the participants by other staff to ensure patient safety is not compromised.^20,25,30 For example, one study had both dedicated simulation time given to the trainees per week but allowed trainees to use the resources during the rest of their working hours if the shift allowed.³⁰ In other cases, such as the full shift simulation, participants had to balance seeing patients at the same time, due to this, some simulations had to be cancelled due to increased influx of patients.³⁷ Only two studies initially had mandatory simulations and a third study converted simulations to become mandatory after detecting its usefulness on participants.^13,16,30

A number of studies adopted a multi-center approach, involving from 3 to 20 centers to carry out the same in situ simulations in their own centers.^{12,22,23,25,34,36,39} Representative from each site attended an orientation lasting 1–2 days where they received training on the project and on using the manikin.^12,23 The representatives then went on to implement the simulations in their site. In other cases, the organizers of the project visited the hospitals to carry out the simulation.^22,39 One group had a flexible approach with scenario design, in which the needs of each center were individually considered.³⁶

Evaluation

Twenty-four studies reported an evaluation of their training. Six evaluation methods were identified in this data: Pre and post questionnaire, instructors, video recording assessments, interviews and focus groups, quality improvement and post-simulation confidence levels. The most common method of evaluating the participants is through pre- and post-simulation self-assessed surveys. Most of these questionnaires were measured on a 5-point Likert scale.^{13,20,22,34,37} Most studies had a one-off questionnaire of confidence levels and evaluated whether those increased post-simulation. Some studies used a longitudinal approach with a baseline survey, a post-survey and a retention survey at either 4, 6 or 12 months after to assess long-term effects of the simulation.^24–26

One study assessed how the learning outcomes of the healthcare professionals changed their practice through an interview process, while another led a post-simulation focus group.^31,33 Questions regarding comfort level with task, the effect on change in their practice and perception on teamwork were included. Other studies had instructors present during the simulation, some were given specific checklists or validated tools such as the Johns Hopkins disaster tool, in order to evaluate the participants.^{11,12,15,25,26,32} In some cases, this was one instructor, while in other cases a number of instructors evaluated independently and discussed the scores until a consensus was reached, or an average score was calculated.²⁵ Studies which had video recorded the simulation used this for evaluation, evaluators who were specialists in the specific setting assessed the videos. In one case this was assessed both before and after simulation by two blinded reviewers and compared if there was any discrepancy.³⁹ In another case, one blinded reviewer evaluated the videos.¹⁶ One study developed an assessment tool with 10 parameters which helped assessors.²⁹

Additionally, the evaluation of the in situ simulation itself was carried out by taking opinions from participants to improve the practice.^13,36 One study designed different surveys for different multidisciplinary team members to increase specificity.¹³ Moreover, outcome measures related to the simulation setting were evaluated such as measuring cardiac arrest, postpartum hemorrhage or incidence reporting rates before and after simulation to look for clinical improvement.^11,26,33

Outcomes

The two most common outcomes explored were the changes in teamwork and confidence levels in the specific specialty. Studies on resuscitation found increased confidence with arrest issues.¹⁵ One study showed increased confidence in nurses in managing emergencies.¹³ The study focusing on pediatric readiness scores found a significant improvement in the score of participants.²⁵ Interestingly, one study had a significant difference in preparedness and confidence of staff in emergency cases in a primary care setting (p<0.05), with the office staff showing better improvement than primary care providers.³⁴ Mental health staff had significant improvement in managing medical deterioration.³³ There were no significant changes after the airplane crash trauma scenario, this was a simulation of 60 simulated patients in the emergency department.²⁴

In situ simulation proved to be effective to orientate new staff to a new medical unit, the study by Lee found significant improvement in the staff’s orientation and confidence in dealing with scenarios in the new setting.³⁶

Results on teamwork and communication also proved positive in most cases with a better understanding of roles, and improvement in general teamwork communication.^15,16,29 The debrief session helped reveal problems within the team which were able to be discussed and improved, such as the lack of challenging each other when something is not done right within the team.¹⁵ One high-fidelity study showed an outstanding improvement in teamwork and communication with a p-value of 0.000005.²⁹ The effects on teamwork of long term repeated simulations showed no improvement over the 4-year period, and the repeated exposure did not necessarily improve performance.¹¹

One study compared video-based training with simulation-based training using a manikin in the surgical ICU setting.³⁵ Findings between the two groups overall scores did not differ, and both groups were found to have a significant improvement after their teaching compared to their baseline. However, when comparing the amount of improvement within the two groups themselves, both the objective and subjective components of the questionnaire were significantly better in the simulation group with p-values of P= 0.03 and P= 0.002, respectively. However, when comparing unannounced and announced simulations in an observational study of two participant groups, no differences were found in participant perception of the simulation.¹⁸

In terms of patient outcomes, there was a reduction in cardiac arrest rates and an increase in incidence reporting rates, meeting the aims of the simulations.^11,33 A low-fidelity simulation assessed nurses who have a longer time based on the acute ward. A decrease in the incidence of unexpected cardiac arrest was observed, due to the nurses’ ability to report signs of deterioration to physicians earlier.¹¹ This study proved that low-fidelity equipment can still be used in institutions with lower financial resources and still get effective results. Additionally, the study by Theilen et al on paediatric resuscitation found that regular in situ simulations of the healthcare team decreased the time to recognize deteriorating patients and escalate to the intensive care leading to better patient outcomes.¹⁴ Simulations were generally seen very positively by participants in post-simulation surveys, where most participants felt they would benefit from continuous simulations in the future.^12,23

Discussion

Our key findings are the increased attention in situ simulation receives from certain disciplines such as pediatric and adult resuscitation and obstetrics. The main educational design usually falls under three main categories of briefing focused on teamwork and orientation, scenario and debriefing facilitated by an instructor to discuss key reflections. Pre and post surveys are commonly used to evaluate outcomes which most commonly included confidence levels in the specific skills, teamwork and patient outcomes.

Clinical Setting and Aims

The most common setting took place in the emergency department, most commonly pediatric resuscitation. Studies in the emergency department are of vital importance due to the fast nature of the practice and the importance of a multidisciplinary approach. Caring for a child has its own challenges such as smaller size, lack of communication and different management protocols. Eighty percent of child mortality is attributed to treatable causes in the first 24–48 hours of admission; therefore, education in an acute pediatric setting is vital in all emergency hospitals.⁴⁰ Obstetrics and gynecology encompasses difficult surgical skills and life-threatening emergency conditions; in situ simulation for such procedures offers an ethical way to allow trainees to develop these skills. The use of in situ simulations in other settings was less common, reflecting the lack of utilization of the practice throughout all disciplines. This finding demonstrates the opportunity for medical specialties and settings to learn from frontrunners such as pediatrics and obstetrics and gynecology.

Simulations were carried out in both high and low volume hospitals. A survey reports that some rural hospitals perform fewer than five common pediatric procedures in a year. Simulations to increase education in low volume hospitals are therefore needed.^12,22 This demonstrates the importance of in situ simulations to develop skills for rare emergencies or procedures, as well as to improve routine practice. Similarly, logistical considerations can be made to deliver simulation regardless of patient volume and size of facilities.

Scenarios were usually chosen based on either needs assessment reports or official registry guidelines. Needs assessment can make simulations more center specific, giving participants the education in skills they feel they are missing. This highlights the importance of aligning in situ simulation training and its objectives to the wider organizational context.

Design

The duration of the scenarios varied from minutes to a full shift. Longer scenarios were perceived as more complicated and increased participants’ anxiety. Longer simulations can be argued to add realism to the practice, this is especially true when participants have to simultaneously tend to patients encouraging participants to balance a busy department which is true of everyday practice. However, this may give less focus to the educational component of the simulation as participants focus on the stress. Studies commonly had to cancel simulations due to staff demand, this is especially true for emergency department simulations, where there is unpredictable patient influx. Some simulations were occasionally scheduled to run in the mornings where there is less patient influx or had an interchangeable schedule to minimise this issue. Having dedicated teaching time for simulations minimises cancellation rates, while releasing staff from their clinical responsibilities is a challenge, with high rates of centers being under-staffed. Taking this into account, designing a flexible approach to simulations such as shorter scenarios which can be rescheduled to different time slots, is essential to minimize cancellation.

Some studies repeated simulations to improve outcomes. The second simulation was carried out after a preassessment period from participants, sharing what changes need to be done to the simulation design.²² This ensures the improvement of the simulation itself but also providing a continuity of education for participants. Some studies included observers as a method of increasing access to simulation practice, where staff who did not have roles within the simulation were able to observe instead. Having repeated practice can improve learning but involving as many staff as possible in simulations is also important; finding a balance between effective simulation outcomes for participants and increasing access to simulation practice should be key.

The low-fidelity studies proved to be effective for participants, these are important as it is the only reasonable way to offer simulation training in low-resource environments.¹¹

There was no significant difference between announced or unannounced simulations, patients possibly go through the same stress response even when they were prepared for the simulation, negating any preparation they may have done. Additionally, having announced simulations does not necessarily lead to staff preparing for the session. Unannounced simulations can be argued to be more realistic of a real situation, but could also negatively affect the educational purpose of the practice because of increased stress. Further comparisons of effectivity and educational outcomes of announced and unannounced simulations can help make more informed decisions when designing simulations.

A number of studies used a multi-center approach, with a number of sites taking place in the same simulation. One study even offered more flexible individualized scenario which took into account the specific centers demographics and needs. Including multi-centers is an efficient way to expand the access to simulation studies, especially to hospitals who may not have the resources to do so themselves.

Debrief was very important in terms of educational outcomes. Facilitators were sometimes trained on carrying out the debrief session; alternatively, guidance can be given by pre-designed checklists or script to bring up specific issues which need to be discussed. Training facilitators can be a challenge due to lack of time or resources of some centers, therefore opting for less resource-demanding ways such as a checklist can be an efficient way to ensure debrief sessions are meeting the correct requirements.

Evaluation

Most studies used self-reported surveys pre- and post-simulation on a Likert scale, such surveys showed a high return rate. Other than surveys, some studies used one on one interviews as the sole measure of outcomes.³¹ Some studies used evaluators who evaluated recordings of the simulation, most of these studies had one evaluator; however, a few studies had more than one blinded evaluator. Objective outcomes were also measured, such as patient outcomes or time taken to complete a skill, which gave a better idea on the effect that in situ simulations have to the clinical practice. Surveys are easy and quick, therefore can give a more complete data set. Additionally, the rating scales give easily measurable outcomes. However, it can be a drawback due to the inflexibility of the replies, participants may understand the scale differently and do not have the opportunity to write their own opinions. The interviews proved to encourage reflection by the participants however are limited by their lack of numerical scale and are a time and resource-consuming process. For decisions from evaluators to be reliable and unbiased, more than one blinded evaluator is needed to assess results fairly and discuss discrepancies. Objective measures give a more valid set of results but can be more time-consuming and difficult to measure. A balance between the ease of the evaluation method for the specific center based on their resources and the quality of data this method gives needs to be analysed prior to deciding on the evaluation methods.

The limitation in the evaluation methods used in most of the studies is the subjectivity of self-reported surveys. Only a small number of studies had assessors rather than the participant themselves to assess the improvement in skill or confidence after the simulation. There is a need to align evaluation approaches within the literature, to allow for better comparability and consistency within studies. Evaluation of specific factors within in situ simulation is also required for a more comprehensive picture of the practice, for example, a focus on changes in staff behavior, clinical outcomes as well as participants’ experience and opinion of the simulations.

In situ simulation has proved to have a positive impact on clinical staff education and has led to better clinical practice and increased patient safety. The variety of approaches demonstrate that there is room for standardizing and aligning existing approaches, while leaving room for local contextual allowances. This may support more areas to engage with this valuable method of training, beyond the few clinical areas that have dominated.

There is a need for better understanding the impact of the educational design of simulations, with a focus on how to best carry out debriefs to ensure participants are benefiting from the practice. Furthermore, there is a need to improve evaluation approaches, with guidance and standards, as well as learning from the literature. Further collation of evidence in the form of systematic reviews and meta-analysis where possible would add a greater understanding to this topic, both in a broad context but also specialty-specific. This should capture both outcomes, but also process measures and the experience of participants to help us to understand the learning mechanisms at play in simulation. It is very important to take into account all aspects of the simulation design before introducing the practice in a center, Figure 2 is a summary of the findings of this review regarding the current practice of in simulations which have had positive outcomes.

Figure 2 Recommendation framework of considerations when planning an in situ simulation.

This review is a current representation of contemporary approaches to simulation. However, there are a few limitations, namely a risk of bias, due to the fact that only successful studies are more likely to be reported in the literature, as well as the possibility of excluding non-English studies. Additionally, due to the varied study designs and settings, it was difficult to make comparisons and therefore concrete conclusions. Due to the nature of this review being narrative and the narrow search strategy we have adopted, some relevant studies were not included according to our search methods and may have added more perspective into the current picture. More rigorous review methods such as systematic reviews would be helpful to ensure no studies are missed and to determine the effectiveness of the simulations.

The lack of routine use of in situ simulation across medical specialties and healthcare settings, coupled with strong evidence of positive impact, presents a case for increasing the availability and application of in situ simulation. Additionally, the coherence of simulation aims with organizational and system goals, such as skilled workforces, patient safety and improved care, further suggest the value of engaging with this method more broadly.

Conclusion

In situ simulation has proved to be useful in a range of different specialties for skills improvement and better team understanding, with the flexibility to meet both local and organizations needs and priorities. Simulation design ranges in terms of fidelity, duration and topic, although there is a clear pattern of considering briefing, simulated scenarios and debriefing. No specific design has proved to be the most efficient and adopting a design which fits into the specific centers resources, local education needs and patient volume, while trying to incorporate the key principles of simulation, are the most important considerations. A recommendation flowsheet has been designed to guide practice for in situ simulations.