In-situ Simulation-Based Module to Train Interns in Resuscitation Skills During Cardiac Arrest

Introduction

Education efficiency is one of the essential factors to improve the outcome of resuscitation during an event of cardiac arrest. Fresh medical graduates enter the “vulnerable period” after their medical internship when they are awarded full licensure to practice independently even while handling critical events like cardiac arrest. They usually have sufficient theoretical knowledge; however, the major challenge is to apply theoretical knowledge in the management of patients. The fresh graduates are not well exposed to managing such cases and not confident enough in resuscitation techniques that need rapid decision making and actions to save lives.¹ The principle of “First do no harm” should be followed during patient care, however, this may be unavoidable when the interns are learning and practicing critical scenarios that are not safe and ethical in a real patient Various standardized courses like Advanced Cardiac Life Support (ACLS) are available to address this gap; however, they are expensive, of short duration and done in an off-site simulation center. Our innovative educational module focuses on workplace-based training of the interns on resuscitation skills during cardiac arrest by utilizing available resources through repeated simulation sessions.

Simulation-based medical education (SBME) is becoming more popular in medical education as it is practical, safe, structured and effective training strategy to train healthcare professionals.² A meta-analysis showed that SBME is superior to traditional clinical medical education in achieving specific clinical skill acquisition goals.³ Experiential learning, contextual learning, deliberate practice, self-directed learning via feedback and debriefing are its primary advantages. However, there is still no standard curriculum for SBME in our context and the medical students are taught almost in the same way as they were decades ago. Other educators have also reported and described the development and implementation of SBME in their institution.⁴ Minha Sa’ar et al implemented a mandatory simulation-based pre-internship workshop which made a significant impact on interns and residents.⁵

In-situ simulation (ISS) training that occurs in real patient care environment,⁶ rather than in a simulation center will allow the interns to experience and examine their native work environment without causing any harm to patients.⁷ This approach will limit the cost of running simulation, increase the fidelity of the simulation and would prevent medical errors by these novice doctors and may play a central role in a student-directed learning model.^7,8 Unlike in real-life clinical encounters, mistakes can be allowed to continue to their ultimate consequences without the need for intervention as there is no safety threat to the patient.⁷ Debriefing done after each exercise will allow real-time reflection, guidance from faculty, and an opportunity to learn from their mistakes and to prevent them during real patient encounter.

The aim of our study was to introduce ISS-based resuscitation skills training for medical interns during the event of cardiac arrest. We hypothesized that the introduction of ISS-based skill training in cardiac arrest module for the medical interns in the Emergency Department will enhance their knowledge/skills and confidence in managing such cases in the future. Our objectives were 1) to sensitize and train the faculties of ED in simulation-based medical education 2) to introduce the ISS module encompassing decision-making scenarios of cardiac arrest to interns 3) to assess the improvement in knowledge, resuscitation skills, confidence, teamwork skills of interns, and 4) to assess the perception of the interns and the ED staff towards implementation of the ISS training.

Materials and Methods

Study Design

Prospective quasi-experimental study with mixed research design (quantitative and qualitative).

Setting

The ISS was conducted in the resuscitation room of the ED of Kathmandu University Dhulikhel Hospital with annual visit of approximately 20,000 patients from all over Nepal. The hospital serves as the university hospital for all the medical programs run by School of Medical Sciences, Kathmandu University (KUSMS). KUSMS launched its own constituent Bachelor of Medicine, Bachelor of Surgery (MBBS) program in August 2001 and till date ten medical colleges offer MBBS program in Nepal with affiliation from Kathmandu University.

Participants

The medical undergraduate students undergo a one-year compulsory rotatory internship after a four and half years of academic study. These interns are posted to various departments as a part of their clinical exposure during internship. During their posting in the ED (one month), they took part in the ISS.

Sampling

All interns posted in the ED during the study period had to participate in the ISS sessions as a part of their internship posting; however, the feedback was voluntary and anonymous.

Study Process and Tools (Figure 1)

The ED faculties (n=3) were sensitized and trained to run a simulation and to debrief by the American Heart Association (AHA) certified Basic Life Support (BLS) and Advanced Cardiac Life Support (ACLS) trainer. A one-day workshop (Figure 2) was also conducted in collaboration with the University of Toledo for the faculties on Simulation technique. The workshop consisted of a didactic lecture and hands-on sessions on formulating the simulation scenario, conducting the actual scenario followed by the debriefing session. The cardiac arrest scenarios, the knowledge questionnaire and the structured skill score sheet based on components of the AHA 2015 cardiac arrest algorithm were designed by the author who was AHA certified instructor for BLS and ACLS. The scenarios consisted of a combination of shockable rhythms (ventricular fibrillation or pulseless VT) and non-shockable rhythms (asystole or pulseless electrical activity). The pre-simulation knowledge test (Supplemental file 1) consisted of demographic details, previous experience (age, gender of the interns, their previous resuscitation training, previous participation in SBME, previous experience in managing cardiac arrest victims and using defibrillation) and 15 items of multiple-choice questions with four options and a “don’t know” option. The structured skill score sheet (Supplemental file 2) consisted of 30 actions with score from 1 to 5 (1-not done, 2-poor, 3-satisfactory, 4-good, 5 best) during each ISS. The content validity was performed by two other experts who were AHA certified BLS/ACLS instructors. The face validity was performed by the Emergency department faculties and the residents. As the final stage, the tools were piloted as a test run on a batch of interns (n=6) not included in the study sample. The trained faculties filled the score sheet for the simulation during the pilot phase separately. An interrater reliability analysis using the Fleiss’ Kappa statistic was performed to determine consistency among the faculties [(n=3), k=0.721 (95% CI, 0.557 to 0.885), p<0.0005]. Elements that were not considered but became apparent in the practice run underwent needful amendments. Two items in the knowledge questionnaire and one item in the skill test sheet were changed. After appropriate remediation, the ISS scenario of cardiac arrest events, the knowledge questionnaire, and the skill score sheet were finalized and introduced.

Figure 1 Flow chart of methodology.

Figure 2 Simulation workshop by Prof Kristopher Brickman in collaboration with the University of Toledo, USA.

Interns are posted to various departments as a part of their clinical exposure during their internship. The group of interns (n=35) who were posted in ED between May–October 2019 took part in the ISS. Each group consists of 5–6 interns. The participation in the ISS was mandatory as part of the academic process. The pre-simulation knowledge questionnaire was distributed before the simulation. The simulation sessions (n=40) were conducted 6–7 times for each group of interns according to the feasibility. The interns participated as a group of 4–6 people (Figure 3A and B); however, a team leader was identified for each simulation session. The simulations were performed using an existing primitive manikin or a volunteer and a cardiac rhythm generator. The scenarios were presented to the interns by one of the 3 trained faculties and the team acted as they would in a real-life scenario. During the simulation, the faculty filled the skill score sheet for the team leader. The time to initiate the chest compression was recorded with stopwatch. The time zero was the time when the patient collapsed and became unconscious in the scenario. The simulation sessions were followed by a period of debriefing session in the group with constructive feedback to the interns for future improvement (Figure 4). The “Pendleton model” was practiced during the debriefing facilitating the self-reflection by the interns in their performance as this was the model used for giving feedback for the other departmental teaching learning purposes. The same 15 knowledge questions were used as the post-simulation knowledge assessment at the end of their posting.

Figure 3 (A) Interns participating in the simulation as a team. (B) Interns participating in the simulation using defibrillator.

Figure 4 Debriefing session after the simulation.

A voluntary written online post-simulation feedback (Supplemental file 3) for interns was collected at the end of their posting. They reflected on their pre- and post-simulation confidence level and their opinion in this method of learning. This was designed by authors and peer validated by the Foundation for Advancement of international medical education and research (FAIMER) fellows. It consisted of six statements to explore the interns’ attitude towards the SBME in a 5-point Likert scale, four items on self-perceived retrospective pre- and post-simulation skills and confidence level in a 5-point Likert scale and two open-ended questions. Written survey (Supplemental file 4) for the ED staff (faculty, resident, and nurses) which was also peer validated by FAIMER fellows was distributed to collect the multisource feedback focusing on their opinion about the ISS training after the completion of the last batch of interns during the study.

Data Analysis

Quantitative

Categorical value is defined as proportion and continuous value as mean with standard deviation (SD) or median with interquartile range (IQR). Comparisons of the proportions with the mean of the pretest knowledge were done with the independent sample t-test. P-value of 0.05 or less was considered statistically significant. The paired t-test was used to compare the pre- and post-simulation knowledge scores. The relationship between the numbers of simulation in each group with the skill score and the time to initiate chest compression was analyzed by the Pearson correlation test. The self-reported pre- and post-simulation confidence level and teamwork skills of the students were compared with the Wilcoxon signed rank test.

Qualitative

The data generated from the open-ended questions in the survey were analyzed by three authors separately for validation and triangulation. The data were repeatedly read and analyzed by them independently and the codes were identified. Thematic analysis was used in identifying themes.

Outcome Measures of the Intervention

• Percentage of the faculties sensitized and trained in ISS
• Number of times ISS module encompassing decision-making scenarios of cardiac arrest conducted for the interns
• Mean difference between the pre- and post-simulation knowledge score of the interns in managing cardiac arrest
• Increase in the skill score in relation to the number of simulations conducted
• Decrease in the time to initiate chest compression in relation to the number of simulations conducted

• The difference in self-reported pre- and post-simulation confidence level in managing cardiac arrest
• The difference in self-reported pre- and post-simulation teamwork skills during resuscitation
• Perception about the ISS from the interns, faculties, nurses and residents of the ED

Results

A total of forty ISS training sessions were conducted during the period of study between May–October 2019. Six groups of interns (n=35) participated in at least one of those ISS sessions. Among the thirty-five interns, 18 (51.5%) were male and 17 (48.6%) were female. Ten (28.6%) of them were trained in resuscitation training. Twelve interns (34.3%) did not know the location of the defibrillator in the ED, and 23 (65.7%) of them were not aware of the type of defibrillator, ie biphasic. Sixteen interns (45.7%) reported that they would use the defibrillator in a patient if needed. Inadequate knowledge and fear of further harm to the patient were the reasons for the reluctance to use the defibrillator as per 17 (48.6%) and 2 (5.7%) interns, respectively. All of them agreed that all departments should be equipped with a defibrillator.

The Knowledge Scores

The pre-simulation knowledge score ranged from 1 to 11 (total score 15) with a median score of 7 (IQR= 3–8) and a mean of 5.89± 2.8 SD (95% CI 4.93–6.85). The difference in pre-simulation knowledge scores among the different gender, previous resuscitation (BLS/ACLS) training, previous involvement in resuscitation, and previous use of a defibrillator is demonstrated in Table 1. Among these factors, previous resuscitation experience was associated with higher pre-simulation knowledge scores (n=23, p=0.015). The post-simulation knowledge score ranged from 6 to 14 (total score 15) with a median score of 12 (IQR= 11–13) and a mean of 11.74± 1.8 SD (95% CI 11.12–12.36). The comparison of the mean of the pre- and post-simulation knowledge score is illustrated in Figure 5. There was a significant improvement in the knowledge score among interns including those who had previous resuscitation experience [t (23) =−14.9, p<0.0005].

Table 1 The Pre-Simulation Knowledge Score in Relation to Gender, Previous Training and Experience (n=35)

Figure 5 Comparison of the pre and the post-simulation knowledge score.

The Skill Scores

Each group of interns practiced in 6–7 ISS training sessions. The skill score among each group and the number of simulations is illustrated in Table 2. The mean skill scores were 63.84, 89.5, 97.5, 106.67, 120, 133.5, and 134.25 for the simulation 1–7, respectively. There was a positive correlation between the number of simulations and the skill scores in each group and it was statistically significant using Pearson correlation test [r (40) =0.985, p=0.01] (Figure 6).

Table 2 Skill Score of the Groups of Interns in Relation to the Number of Simulations (n=40)

Figure 6 Correlation between number of simulations and mean skill score.

The time to initiate the chest compression from the time of cardiac arrest was recorded and is depicted in Table 3. The mean time was 93.2, 44.4, 27.7, 23.4, 19.5, 13.2 and 12.3 seconds for the simulations 1 to 7, respectively. The relation between the time of initiation of the chest compression and the number of simulations in each group is illustrated in Figure 7. The Pearson correlation test showed significant negative correlation [r (40) =−0.867, p=0.05]

Table 3 Time to Initiate Chest Compression in Seconds in Relation to the Number of Simulations (n=40)

Figure 7 Correlation between number of simulations and mean time to chest compression.

Feedback from the Interns

A total of 28 interns filled the online voluntary feedback form with a response rate of 80%. Out of them, thirteen (46.4%) were female and fifteen (53.6%) were male. Thirteen respondents (46.4%) had not participated in simulation-based training previously. The response of the interns to the various statements in the Likert scale of 5 is illustrated in Figure 8. All interns agreed that the simulation provided a safe learning environment for learning, decreased their anxiety while managing such real cases in the future and the ISS should be continued. Majority of them (96.4%) agreed that the debriefing sessions after the simulation provided motivation for self-guided learning and had the potential to identify errors within the clinical environment (89.3%). The thematic analysis of the responses to the open-ended questions from the interns is summarized in Table 4. The difference in the self-reported skills and confidence level pre- and post-simulation is illustrated in Figure 9. The comparison using the Wilcoxon signed-rank test is shown in Table 5. The findings from the multisource feedback from the staff of the ED (faculties, nurses and resident) in regards to the introduction of ISS are depicted in Figure 10.

Table 4 Thematic Analysis of the Post-Simulation Feedback from Interns to Open-Ended Questions

Table 5 The Differences in Self-Reported Pre- and Post-Simulation Skills and Confidence Level of the Interns (n=28)

Figure 8 Feedback from the interns.

Figure 9 The self-reported pre- and post-simulation skills and confidence level of the interns in Likert scale (1=least, 5=most).

Figure 10 Feedback from the ED staff.

Suggestions from Interns

• More devices (manikins)
• More frequently and longer duration per ISS
• More cases with higher complexity
• More opportunity for each intern
• Certification
• Separate location for simulation within the ED
• Video demonstration to orient the participants before the ISS

Verbatim from the ED Staff Feedback on ISS Sessions

• Feasibility: “Dedication and effort needed, depends upon the patient flow in the ED”, “may not be feasible during high caseload”.
• Safety: “Includes most aspects of patient care”.
• Teamwork: “Closed-loop communication, boldness, thoughtfulness, and assessment of team members-all help in building teamwork skills, for effective teamwork”, “multidisciplinary staff and students should be involved”.
• Enhanced technical skills: “Recognizing arrhythmias, defibrillation, cardioversion”.
• Level of confidence: “Increased motivation of interns during the event of resuscitation”.
• Fidelity: “To practice in the place where you work in the real-life gave them more confidence during actual practice”.
• Effect of ISS sessions on your capacity: “Increased confidence in conducting simulation sessions, debriefing technique and own clinical practice”.

Discussion

This study demonstrated significant improvement in the knowledge and skills of the interns for management of the cardiac arrest with the introduction of SBME. The baseline knowledge score about the management of cardiac arrest was overall low (mean 5.89± 2.8 SD) and the difference was not significant among the different gender, previous resuscitation training and previous use of a defibrillator. Those who had participated in resuscitation had significantly higher baseline knowledge. Likewise, the improvement in knowledge score was significant for those who participated in resuscitation as well. The level of clinical exposure and learning curves of the students/interns are variable and dependent on various factors: the quality and attitude of supervisors/coworkers, opportunities to learn, the availability of equipment, workload, personal fear of harm to the patient and medico-legal issues. Unpreparedness for managing emergency care skills was previously reported especially in emergency skills.¹ Almost half of the interns in our study were reluctant to use a defibrillator. The confidence level in managing cardiac arrest was very low. It is critical finding that 12 among the 35 interns did not know the location and 23 were not aware of the type of defibrillator in the ED.

Simulations were designed specifically to improve a thoughtful approach to decision making help acquire an organized approach to resuscitation techniques. The difference between means of the pre- and post-simulation knowledge score in our study was statistically significant. A previous prospective ISS intervention to improve arrhythmia detection within the ED demonstrated that arrhythmia detection rates increased from 5% to 55%.⁹ A systematic review by Rosen et al⁶ found 29 studies of in-situ simulation and suggested positive impacts on learning and organizational performance. Our study also demonstrated a positive correlation of the skill score with the number of simulations performed in each group. Likewise, the time to initiate the chest compression decreased significantly with the number of simulations performed. The mean time was 12.3 seconds for the final simulations. AHA recommends that if a victim is found unresponsive without definite pulse or normal breathing, the high-quality CPR should be initiated immediately within 10 seconds. Although the mean time to compression was still not up to the recommended standard, it had shortened significantly in the course of the training. An observational study¹⁰ found that the survival rates for cardiopulmonary arrest increased to approximately 50% in correlation with an increased number of mock codes. SBME establishes a monitored, safe environment promoting experience and confidence gaining without jeopardizing patients’ safety. We found that all interns and the faculties agreed that the simulation provided a safe learning environment for learning. Our interns reported high satisfaction from these ISS sessions and reported that these simulation sessions should be continued and expanded for other critical conditions as well.

The self-perceived skill of the interns in managing cardiac arrest, using a defibrillator, teamwork skills and the confidence level was low. The mean differences in self-reported pre- and post-simulation skills in managing cardiac arrest/use of defibrillator, teamwork skill and the confidence level of the interns were significant. These critical findings reinforce the need for such training initiatives to the novice doctors to raise their technical and nontechnical skills. All interns agreed that the simulations would decrease their anxiety while managing such real cases in the future. The teamwork and communication in the clinical setting were improved during a simulation program in a trauma scenario.¹¹ Steinemann¹² found that teamwork ratings, task speed, and task completion rates improved within the simulations, and showed that these benefits translated into clinical practice.

In addition to the quantitative results, we also explored the qualitative responses. The thematic analysis of the responses to the open-ended questions uncovered the themes related to the simulation methodology, reflective learning, enhancement of the nontechnical skills and effective feedback. Experiential learning is a process by which learning occurs by having an experience. Learning from direct experience can be more effective if coupled with reflection.^13,14 There were many reflective verbatim in the open-ended responses. One of the written responses was “showed my own abilities in managing stressful clinical condition and I was surprised to know how poor I was in it”. 89.3% of the respondents agreed that simulation had the potential to identify errors within the clinical environment. The respondents quoted “the discussions that followed later showed me how to overcome my limitations”, “I liked how we not only did the simulation but had discussion on what our mistakes were and what we need to work on after the simulations”.

There is an essential role of debriefing in SBME to help transform experience into learning through reflection.¹⁵ Debriefing is a facilitated reflection in the cycle of experiential learning to help identify and close gaps in knowledge and skills.¹⁶ Every simulation session was followed by a debriefing session using the Pendleton model.¹⁷ 96.4% of the interns agreed that the debriefing sessions provided motivation for self-guided learning. Some of the verbatim linked with effective feedback were- “I liked how we not only did the simulation but had discussion on what our mistakes were and what we need to work on after the simulations”, “liked the feedback on how we did and how we can improve on it”. Feedback is the single most important factor that has shown to have the maximum impact on student learning and accomplishment.¹⁸

Educators must be trained in how to use simulation-based teaching/learning strategies for teaching students or training trainees through faculty development programs. A post-simulation semi-structured survey was conducted among the ED staff of various clinical backgrounds including faculty, nurses and residents for multi-source feedback to triangulate the findings and responses. They had a positive attitude towards ISS and reported that it should be disseminated to other critical topics and departments.

ISS training decreases required resources and increases realism and provides a powerful tool for education in resource-limited setting like ours as it takes the training process to a clinical environment.^19,20 The various BLS/ACLS courses including the AHA certified courses are available in Nepal. However, they are costly and are short intensive training. By running the simulation-based training in the work-place, the interns can practice repeatedly in their own native environment without the fear of harming the patients. No additional cost was required for these sessions in our study as the resources available in the ED were utilized. The purpose of medical education at all levels is to prepare physicians with the knowledge, skills and nontechnical skills, required to deliver safe and effective patient care. With the simulation focusing in these domains, the interns, who will be in the frontline for various emergency care will learn and experience the real-life scenarios, review errors using adult learning theory and will also practice the teamwork dynamics among coworkers.

The limitations of this study are that this was not a randomized control study and the knowledge of students with various methods of teaching was not compared (lecture, off-site simulation and in-situ simulation). Only one topic for simulation was introduced and the number of samples was also limited (35). Some planned simulations had to be cancelled due to overcrowding of the ED.

Conclusions

This study demonstrated an increase in the knowledge and skills of the interns after participation in the ISS. The baseline knowledge/skill score and the self-reported confidence level of the interns for managing cardiac arrest were low. The self-reported enhancement in the technical skills, teamwork skills and confidence level was significant. Considering the positive results and feedback, we recommend that the traditional learning model of medical education which puts patients at risk should undergo a shift to a “simulation-based learning model” and should be implemented extensively as a useful educational technique in every hospital. Incorporating the SBME model to existing medical curricula is the need of the current medical education system.

ISS is an effective way to enhance skills, identify knowledge gaps, reduce medical errors, and practice teamwork with the overall goal of improving patient care. Although simulation cannot substitute the real clinical exposure, it promotes experiential learning without jeopardizing patient safety. This new paradigm shift of learning will be appreciated and embraced by other institutions in our context and will revolutionize the medical education in developing countries like ours with this new pedagogical shift to simulation-based learning in the actual working environment.

The next step of this project would be to explore whether the experiential learning afforded by this process provided an opportunity to transfer these skills into the real-life setting by documenting the enhanced patient care and decreased morbidity and mortality of the patients.