Improving Emergency Medicine Residents’ Comfort in Responding to Mass Casualty Incidents: A Cost-Effective Exercise Utilizing Real-Time Feedback

Introduction

Mass casualty incidents (MCIs) are defined as any incident wherein the number of casualties exceeds the capabilities of the local healthcare system¹ and can happen to any community without warning. MCIs encompass both natural disaster events and man-made tragedies, and are increasing in scope and frequency.^1,2 As a member of the first-line response to such events, emergency medicine (EM) physicians must possess both the competence and confidence required to appropriately act during an MCI, and thus mass casualty triage and disaster response has been recognized by the American Board of Emergency Medicine as an essential skillset.³ Unfortunately, EM residents have reported feeling uncomfortable with disaster preparation⁴ and lack of confidence at the prospect of responding to large-scale mass casualty incidents.⁵ Furthermore, studies have shown treatment decisions may be adversely affected when emergency personnel are not adequately trained.⁶ Our academic center, located in the port city of Mobile, Alabama is at risk of both natural disasters (hurricanes, tornadoes) and those man-made due to the region’s multiple industries, including maritime, chemical, and aeronautical. Our residents must be adequately prepared to respond to MCIs both during their training, as well as upon graduation and work force entry.

Accurate and swift patient triage necessarily constitutes the backbone of a successful MCI response. Multiple patient triage systems have been developed, however START (Simple Triage and Rapid Treatment) for adults and JumpSTART for children⁷ remain the most widely utilized in the United States^8. Mass casualty triage training that includes an element of simulation has been described in the literature^9–12 however, to our knowledge this is the first to incorporate immediate in-session feedback. Additionally, many educational interventions rely on standardized patients, full size mannikin equipment, or virtual reality, which may be too cumbersome or costly for many emergency medicine programs to implement.

This novel mass casualty triage exercise utilized QR (quick response) codes, Google Forms,¹³ and moulaged dolls to provide real-time feedback to emergency medicine learners in a small-scale simulation environment. We hypothesized that residents would report higher comfort with responding to mass casualty events after participating in the exercise. Our secondary goal for this study was to pilot a cost-effective and easily replicable mass casualty simulation alternative for emergency medicine residency programs.

Methods

This educational intervention was implemented on two separate occasions, as part of our regularly scheduled academic conference at the University of South Alabama Emergency Medicine Program in May 2023 and December 2024.

Participants consisted of 13 University of South Alabama emergency medicine residents (PGY1=5; PGY2=5; PGY3=3) (May 2023) and 12 University of South Alabama emergency medicine residents (PGY1=6; PGY2=3; PGY3=3) (December 2024). Two of the PGY3 participants in the December 2024 intervention reported having participated in the first exercise during their PGY1 year.

All residents in attendance at regularly scheduled academic conference were included in the intervention and participated in both the pre- and post-exercise surveys. Residents that were unable to attend due to off-service obligations, illness, pre-scheduled vacations/leave of absences, or ACGME hours restrictions were not included in the study. Residents were advised that participation in the surveys was optional, and responses would be kept anonymous. This study was reviewed by the University of South Alabama Institutional Review Board and deemed exempt according to institutional policy (Board reference number: 24–486).

A quasi-experimental pre- and post-survey study design was utilized. The primary outcome measure was participant comfort in responding to an MCI. Prior to beginning the exercise, participants were asked to complete an anonymous pre-exercise survey that asked residents to report their year of training and their level of comfort in responding to MCIs utilizing a 5-point Likert scale. Following the exercise, participants were again asked to complete a survey regarding their overall comfort in responding to MCIs utilizing a 5-point Likert scale, as well as feedback regarding their perception of the quality of teaching methods and educational tools utilized. Both surveys were conducted utilizing Google Forms, accessed by the participants via a QR code embedded in the presentation slideshow. Results of the survey were anonymously stored in Google Sheets. Statistical analysis was then performed after each intervention by calculating the odds ratio.

Upon completion of the pre-exercise survey, participants were given a slideshow presentation of the START and JUMPStart triage algorithms, followed by a briefing of the simulated exercise scenario. The scenarios consisted of a music festival complicated by stage collapse, poor sanitary conditions, and violence (May 2023) and a malfunctioning carnival ride at a traveling circus (December 2024).

Simulated patients in both scenarios consisted of 14 small dolls with injury-specific moulage placed on tables decorated with props (such as a collapsed cardboard “stage”) designed to lend more realism to the scenario. Both adult and pediatric simulated patients were incorporated. Each simulated patient was equipped with an attached QR code linked to a Google Form survey giving pertinent patient information including approximate age and vital signs followed by multiple-choice question asking the participant to select the appropriate triage designation. The surveys were configured to give participants real-time feedback regarding the accuracy of their chosen triage designation, followed by a brief explanation of why their chosen answer was or was not correct. Participants were allowed to reference both the START and JUMPStart algorithms throughout the scenario via QR code links posted on the wall.⁷ A total of 35 minutes was allotted to complete the triage portion of the exercise, and participants were instructed they could triage the simulated patients in any order.

Results

In this study, we sought to determine if participation in a small-scale mass casualty exercise utilizing immediate feedback would result in increased emergency medicine resident comfort in responding to mass casualty incidents (MCIs).

Participants completed both pre- and post-intervention surveys assessing their level of comfort in responding to MCIs. (Table 1 and Table 2). Surveys consisted of 5-point Likert scale with 1=completely uncomfortable and 5=very comfortable.

Table 1 May 2023 Exercise: Residents’ Self-Reported Comfort Level with Responding to an MCIa

Table 2 December 2024 Exercise: Residents’ Self-Reported Comfort Level with Responding to an MCIa

May 2023 Intervention

PGY1 resident participants (n=5) reported a median comfort score of 3 prior to the intervention (range=1–4). Following the intervention, this group reported a median comfort level of 4 (range= 4–5).

PGY2 residents (n=5) reported a median comfort level of 3 prior to the intervention (range=2–3). Following the intervention, this group reported a median comfort level of 4 (range=3–4).

PGY3 residents (n=3) reported a median comfort level of 3 prior to the intervention (range=2-3). Following the intervention, this group reported a median comfort level of 4 (range=4–5).

The median pre-intervention comfort level for all residents participating in the intervention (n=13) was 3 (range=1-4). Following the intervention, the median comfort level was 4 (range=3–5).

Participants (n=13) were separated into groups indicating low to moderate comfort (defined as Likert scale responses of 1–3) and mostly/very comfortable (defined as Likert scale responses of 4–5) and an odds ratio analysis was performed. Comparing reported pre-intervention and post-intervention comfort levels, participants were 66% more likely to respond with either a 4 or 5 on the Likert scale post-intervention, indicating feeling either mostly or very comfortable in responding to MCIs (OR=66; 95% CI=5.22–833.59; p=0.0012).

Following the exercise, participants completed a post-intervention survey to determine their perception of the quality of teaching methods and educational tools utilized (Table 3).

Table 3 May 2023 Exercise: Residents’ Assessment of Session (n=13)

In response to the prompt “This exercise was beneficial to my understanding of the START triage system,” participants (n=13) gave a median Likert score of 5, indicating “completely agree.”

In response to the prompt “Please select the option below that most closely corresponds to your perception of the quality of teaching of today’s exercise,” participants (n=13) gave a median Likert score of 5, indicating “excellent.”

In response to the prompt “Please select the option below that most closely corresponds to your perception of the quality of instructional tools (ie props, QR codes, etc)” participants (n=13) gave a median Likert score of 5, indicating “excellent.”

December 2024 Intervention

PGY1 residents (n=6) reported a median comfort score of 2 prior to the intervention (range=2–5). Following the intervention, this group reported a median comfort level of 4 (range=3–5).

PGY2 residents (n=3) reported a median comfort score of 2 prior to the intervention (range=1–2). Following the intervention, this group reported a median comfort level of 4.

PGY3 residents (n=3) reported a median comfort score of 3 (range=2–5) prior to the intervention. Following the intervention, this group reported a median comfort level of 4 (range=4–5).

The median pre-intervention comfort level for all residents participating in the intervention (n=12) was 2 (range=1–5). Following the intervention, the median comfort level was 4 (range=3–5).

Participants (n=12) were separated into groups indicating low to moderate comfort (defined as Likert scale responses of 1–3) and mostly/very comfortable (defined as Likert scale responses of 4–5) and an odds ratio analysis was performed. Comparing reported pre-intervention and post-intervention comfort levels, participants were 55% more likely to respond with either a 4 or 5 on the Likert scale post-intervention, indicating feeling either mostly or very comfortable in responding to MCIs (OR=55; 95% CI=4.30–703.46; p=0.0021).

Following the exercise, participants completed a post-intervention survey to determine their perception of the quality of teaching methods and educational tools utilized (Table 4).

Table 4 December 2024 Exercise: Residents’ Assessment of Session (N=12)

In response to the prompt “This exercise was beneficial to my understanding of the START triage system,” participants (n=12) gave a median Likert score of 5, indicating “completely agree.”

In response to the prompt “Please select the option below that most closely corresponds to your perception of the quality of teaching of today’s exercise, participants (n=12) gave a median Likert score of 5, indicating “excellent.”

In response to the prompt “Please select the option below that most closely corresponds to your perception of the quality of instructional tools (ie props, QR codes, etc).” Participants (n=12) gave a median Likert score of 5, indicating “excellent.”

Discussion

In this study, we developed a novel training program designed to provide in-session feedback as participants practiced utilizing the START triage system via a simulated mass-casualty exercise. Our residents were 66% and 55% more likely to report feeling mostly or very comfortable with the prospect of responding to MCIs after completing the exercises, as compared to their pre-exercise survey results (May 2023 and December 2024, respectively).

Mass casualty incidents almost universally pose an emotionally charged and stressful situation for the rescuer, thus, it is of paramount importance that medical personnel are comfortable in their ability to apply an established triage algorithm. In this study we chose the START triage algorithm as it is the model utilized by our institution’s disaster protocol. Although not specifically demonstrated in this study, the same principal design could theoretically be adapted for use with other triage systems.

We hoped to demonstrate that residents will be better equipped and more confident to apply the triage algorithm under the stress presented during a real MCI event after the incorporation of real-time feedback. Targeted, immediate feedback has been documented to have positive effects on learning both in the general education sphere^14,15 as well as the medical education literature.¹⁶ To our knowledge, however, there are no other published studies that have utilized immediate in-session feedback, nor QR code technology for mass casualty disaster preparation training.

The secondary goal of our study was to design an intervention that could be easily implemented on a small budget with limited space. It also did not require the participation of faculty members beyond the primary investigator, though it certainly could be adapted to include others as resources and interest allow. Utilizing small dolls and appropriately scaled props, it required minimal space. The type of dolls used were commonly available and thus easily sourced (ours came in bulk from a second-hand store) and did not require specialty mannikin equipment, which is often costly, cumbersome to store, and may be difficult to acquire in the quantity necessary for an adequate scale exercise. Other interventions have incorporated volunteer or paid simulated actors for this purpose. Although human patients do lend an element of realism, it almost always comes at the cost of complex scheduling, potential reimbursement considerations for the simulated actors, acquiring makeup and costuming, and the availability of suitable space. It is possible that emergency medicine resident physicians are not being adequately trained in MCI response simply due to the cost, difficulty, and space constraints that implementing a larger-scale exercise requires. This relative lack of exposure may be contributing to issues of EM residents feeling poorly prepared for disaster scenarios⁴ and uncomfortable with the prospect of acting during large-scale mass casualty events.⁵ Our study demonstrates a simple, cost-effective, and easily adaptable model that can be utilized for mass casualty training in a variety of educational settings worldwide, including those without access to simulation centers, elaborate equipment, or extra faculty facilitators.

In this study, we sought to simplify the exercise design by utilizing a single conference room and small dolls, meanwhile providing valuable educational experience in the form of real-time feedback for our learners. Our results showed that residents reported feeling significantly more comfortable with the prospect of responding to a future MCI after the exercise than they did before. Additionally, participants reported almost unanimous agreement that the exercise was beneficial to their understanding of the START triage system. Participants reported the quality of the teaching and instructional tools as “excellent.”

We acknowledge that the study is small and included a total of 13 and 12 participants, respectively, during each of the exercise implementations. Additionally, given that the exercises took place during regularly scheduled residency conference, two of the participants were included in both iterations which may have skewed results for the December 2024 data. Thus, although our results showed statistical significance, it is unknown whether these results could be generalized to another cohort of emergency medicine residents at a different institution, perhaps with different backgrounds and experience than at ours. Further studies will be required in order to make this determination.

As the post-intervention survey was completed immediately following the intervention, it is also unknown whether these results persist over time. Two residents did participate in both exercises, however, given the principal investigator is a core faculty member and thus knew the applicants personally, participant confidentially in their responses was favored over requesting disclosure of this fact on the survey form. More studies will be necessary to determine whether the positive effects demonstrated in our study are lasting, and to determine optimal timing for resident re-training.

Conclusions

Our study sought to determine if a novel small-scale exercise utilizing QR code technology and immediate feedback resulted in increased EM resident physician comfort in responding to mass casualty incidents (MCIs). Results showed that participants were 66% (May 2023) and 55% (December 2024) more likely to indicate they felt mostly or very comfortable in responding to an MCI after participating in the intervention, successfully achieving our study aim. This study represents a novel low cost and low faculty time commitment alternative for emergency medicine residency programs hoping to provide targeted practice and exposure to mass casualty triage for their resident learners.