Improving Ad Hoc Medical Team Performance with an Innovative “I START-END” Communication Tool

Introduction

Teamwork and communication are essential elements to building and maintaining a strong patient safety culture. Many organizations such as the Anesthesia Patient Safety Foundation, the Institute for Healthcare Improvement, the Joint Commission & Agency for Healthcare Research and Quality (US) have shown that adverse events in hospital are among the leading causes of death and communication errors contribute in at least 2 out of 3 cases.^1,2

Complexity in healthcare can create circumstances that lead to fragmentation of care; considering that during a 4-day hospital stay, a patient may interact with many teams, and approximately 50 different health-care workers.³

Anesthesiology residents are key members of these teams and are often called to provide care in urgent and emergent circumstances.

What makes communication and teamwork hard in urgent medical teams is that the situations faced are uncertain and the composition of individual members is constantly changing.^4–10

Residents are called to urgent scenarios such as cardiac arrests and traumas (Ad Hoc), using pagers/other devices. The resident carrying the pager responds and attends the event. However, the resident carrying the arrest/trauma pager may change from hour to hour throughout the day.^2–7 An Ad Hoc team consists of a group of people who do not work together on a regular basis. They find themselves assembled for a specific purpose, often in an unfamiliar setting, where resources and support personnel are different and unpredictable every time. The inherent nature of these Ad Hoc teams, and the acute medical situations for which they assemble, can be stressful, which in turn can negatively impact team performance and ultimately patient care delivery. Psychological safety has been identified as an important characteristic of high-performing teams yet is often absent under these conditions.^11–17

I START-END (I-Identify; S-Story; T-Task; A-Accomplish/Adjust; R-Resources; T-Timely; E-Exit; N-Next; D-Document and Debrief) is a communication tool and is original work. The tool was formatted into a laminated cognitive aid as shown in (Figure 1A), I START-END tool/cognitive aid FRONT and (Figure 1B) I START-END tool/cognitive aid BACK.

Figure 1 (A) I START-END tool/cognitive aid FRONT. (B) I START-END tool/cognitive aid BACK.

The I START-END tool promotes psychological safety, aids in the reduction of miscommunication and improves situational awareness, maximizing the effectiveness of the Ad Hoc team in unfamiliar and challenging settings.^18,19

A literature review was conducted to determine if the challenges of Ad Hoc teams had been identified and studied per se. A thorough search of existing communication and teamwork tools that specifically targeted episodic or transitory teams (referred to as Ad Hoc in this study) was undertaken (refer to Appendix A).

Most literature, targeting communication and teamwork is geared to a concept of teams that typically do not have the fluidity we see in the healthcare teams of today.²⁰ The latter teams are composed of many different providers per patient and medical care delivery that is episodic, with each discipline acting in silos, addressing a specialized area. Due to this complexity, there is poor overall coordinated collaboration.^21,22

No one person can manage the increasing complexities of patients with multiple medical conditions. This is true in elective situations, and especially so when caring for patients who are acutely ill and present emergently. When the Ad Hoc team arrives, resuscitation must begin immediately. Task delegation by a leader is critical when it comes to Ad Hoc team settings, since many team members on the Ad Hoc team will have overlapping and specialized skill sets. Achieving optimum medical care by an Ad Hoc team requires regular review of the events with closed loop communication. This creates opportunities for others’ input and allows for and anticipates other resources that may be needed, leading to efficient and effective forward planning. The expectation that team members are not just performing a task but are aware of what else is happening (situational awareness), will be fostered by having a standardized framework everyone works from, which is set out by the I START-END tool.

A local needs assessment was administered to members of emergency teams which included emergency physicians, trauma team leaders, internal medicine, anesthesia and surgery residents and fellows until saturation was achieved (n = 35). This assessment identified that there was often unclear leadership in such settings, and other team members’ expertise and roles were not well defined. In addition, patient information was often incomplete, supplies and supports were not readily available, and at the end of the event, it was unclear who would do what and when. Lastly, a team debriefing did not typically occur. Debriefing is an important step in the learning experience of medical personnel and can positively impact safety culture and patient care.^23,24

The needs assessment identified key challenges that a member entering an Ad Hoc situation faces. I START-END was designed using crisis resource management (CRM) principles with the intention of directly addressing these challenges and so enabling the best performance of all members on the Ad Hoc team.^25,26

I START-END was developed to address the challenges of providing care in Ad Hoc settings- it helps a healthcare responder to enter an emergency and respond effectively. I START-END is an additional skill set/tool that gives one agency in situations that feel uncomfortable and might otherwise keep us from speaking up. Starting with introductions, role clarity, creating context, task delegation, and regular status updates – an expectation of speaking up is hardwired into Ad Hoc scenarios. Individuals using the I START-END report that by modeling speaking up, other members of the team become more engaged. It has been used locally and awarded the Innovations in Patient Safety Education “Power of One” from the Canadian Patient Safety Institute in 2016 and recognized by Accreditation Canada as a leading practice in 2017.

Figure 1 outlines the I START-END tool which guides a communication/integration process.

I START-END is a versatile tool, as it is not context specific, and can be used across various disciplines and Ad Hoc teams. The I START-END tool standardizes and sets the expectations for team communication. The goal is to enable shared team understanding and planning. The I START-END tool can be viewed as a non-operating room strategy that promotes engagement and information sharing, similar to the validated Safe Surgery Checklist in the operating room. The Ad Hoc setting presents unfamiliar and evolving challenges with many unknowns which can contribute to a fear of speaking up and being wrong.^27–29

The primary objective of this study was to determine if the standardized I START-END communication tool would improve the performance and communication of anesthesia residents in high-fidelity simulated Ad Hoc emergent scenarios – specifically, being called urgently to care for a patient in an unfamiliar setting and team. The secondary objective was to observe actual in-hospital cardiac arrests (Code Blues) to determine if use of this tool positively impacted teamwork performance.

Methods

Study Population and Setting

This was a pre-, post-observational study using mixed methods to evaluate the efficacy of the I START-END communication tool on team performance. Sunnybrook Health Sciences Centre Institution Research Ethics Board (REB) approval was obtained in May 2014 (Project Identification Number #: 105-2014).

The study was conducted at Sunnybrook Health Sciences Centre in Toronto, ON Canada. All anesthesia residents (PGY1-5) in the University of Toronto program were invited to participate in this simulation study. There were no exclusionary criteria. The participants signed a written consent for video recording and subsequent assessment of their performance by simulation center trained observers. The participants also signed a confidentiality agreement to not disclose any details of the simulation study until after its completion.

A flow chart summarizing the different steps of the study is shown in Figure 2, STUDY FLOW CHART.

Figure 2 Study flow chart.

The total recruitment of participants (n = 47) included PGY1-5 anesthesiology residents(PG1=10, PG2=16, PG3=5, PG4=8, PG5=8).

Participants filled out questionnaires on three occasions: pre- (baseline), post- (1–2 weeks after introduction to I START-END) and late- (3–6 months after introduction to I START-END).

Details are provided in Appendices C–E.

The pre-questionnaire was administered prior to the first simulation to assess residents’ perceptions of the challenges faced when in urgent Ad Hoc scenarios; the second questionnaire followed the second simulation and was designed with the purpose of identifying if the I START-END tool had relevance and was effective in helping a resident be more effective when responding in Ad Hoc emergency situations; the third questionnaire was administered to assess the sustainability of I START-END and to discover if residents found the tool useful in other clinical settings. After the participants completed the pre-questionnaire, participants were randomized to either Scenario A (n = 24) or Scenario B (n = 23). Both Scenarios A and B were high-fidelity simulation scenarios. A participant that was randomized to Scenario A in the pre-phase would be placed in Scenario B in the post-phase and vice versa (1–2 weeks post I START-END exposure). All residents were placed into Scenario C in the late-questionnaire phase (3–6 months post I START-END exposure). Immediately following participation in the pre-phase simulated scenarios, an instructor-led debrief, and a 30-minute didactic session where the I START-END communication tool was introduced.

OUTLINE of 30-Minute Didactic Session Introducing the I START-END Tool

The real conundrum of Ad Hoc is that it is never the same, it is not predictable. How can we prepare?

1. What has been the resident’s experience in Ad Hoc settings?
2. CRM principles are reviewed and their relevance to Ad Hoc scenarios emphasized – specifically the value of each CRM element and the challenge of applying them reliably and comprehensively in stressful situations such as Ad Hoc.
3. The I START-END tool was introduced to the participant and a cognitive aid (a laminated credit card size visual) provided a quick reference to the I START-END tool. See (Figure 1A and B)
4. The participant was provided with 2 case examples of Ad Hoc scenarios by the instructor in which the I START-END tool was applied and each step was discussed.
5. Teach back method; the participant was asked to discuss an Ad Hoc scenario that they had personally experienced. They practiced applying the I START-END tool to that scenario, with the instructor acting as coach.
6. Finally, opportunity for further discussion and questions by the participant were encouraged.

High-Fidelity Simulation Session

High-fidelity simulation sessions were conducted in the Sunnybrook Canadian Simulation Centre at Sunnybrook Health Sciences Centre. Three high-fidelity simulation scenarios were developed by experts in the fields of anesthesia and simulation. Participants would wait in the simulation ante room and be “paged” to the simulation suite (through a door) urgently. They would encounter a sim-patient in crisis, situated in a non-operating room setting with a team of 2 trained confederates who acted as team members within the Ad Hoc setting, also attending the patient. (Details are described in Appendix F). The high-fidelity simulation was trialed for feasibility, consistency, and training of confederates. The two study arms (Scenario A and B) were designed to be of equal difficulty. Participants were exposed to both scenarios at one point during the study. Each of the three scenarios (A, B, C) was beta-tested until mastered by the actors, with the research team participating as the residents.

Evaluation Using Mixed Methods

The questionnaires administered following each simulation scenario were used to assess the residents’ perception of Ad Hoc emergency settings in the context of before and after the introduction of the I START-END tool by using a 5-point Likert scale. Questionnaires were collected and responses were summarized (with anonymity preserved).

Each simulation scenario was recorded. Two anesthesiologists with expertise in emergency management reviewed the videos. The reviewers evaluated 47 residents in the Pre-group, 46 in the Post-group and 25 in the Late-group.

The residents’ technical performance was evaluated using critical action checklists generated for each scenario using a modified Delphi technique (Appendix G).

The technical skills checklist scores in the Pre-, Post-, and Late-groups are shown in Table 1, Technical Skills Checklist Scores.

Table 1 Technical Skills Checklist Scores

The linear mixed model showed a significant change in Technical Skills Checklist scores over time (p < 0.0001). Post scores were found to be significantly lower than pre scores (p = 0.009), whereas the late score was significantly higher (p < 0.0001) than the pre-score.

Similarly, the residents’ non-technical performance was evaluated using Anesthetists Non-Technical Skills (ANTS) tool (Appendix H).^30,31 Video reviewer training was provided to each reviewer prior to evaluating the research study videos. The reviewers practiced with the research team and then calibrated themselves by scoring non-study videos and discussing their ratings until correlation was attained. The raters were blinded to the residents’ training level (PGY 1–5) and independently evaluated the study videos. The video viewing order was randomized to ensure the reviewers were blinded to whether the scenario was pre- or post- I START-END tool training.

The ANTS scores in the Pre-, Post-, and Late-groups are shown in Table 2.

Table 2 Anesthetists’ Non-Technical Skills (Ants) Scores

Residents’ anesthesia non-technical skills performance (ANTS scores) by linear mixed models did not show a statistical change over time (the pre-, post- and late-simulation scenarios) for any of the four domains.

In-Hospital Code Blue Bedside Observations

In addition to Research Ethics Board approval, additional institutional approval was obtained from the hospital Cardiopulmonary Resuscitation committee to allow four trained observers (hospital-based research assistants) to attend actual Code Blue events to assess performance of anesthesiology residents in the context of the entire team. Observer training included video viewing of four Advanced Cardiac Life Support scenarios accompanied by facilitated discussions with experts. The observers rated each scenario until reproducibility of scoring was attained.

The observer rated the Code Blue team using the MAYO High Performance Teamwork Scale³² as outlined in Appendix F. This scale assesses team performance, specifically evaluating Crisis Resource Management skills exhibited, with each item scored whether absent = 0, exhibited inconsistently = 1, or exhibited consistently = 2.

The performance scores of anesthesiology residents who had received I START-END training were compared with those without this training. At the conclusion of the Code Blue, after completing the scoring, the observer would ask the anesthesiology resident whether they had received I START-END training.

Statistical Analysis

Pre-, post-, and late-questionnaire data are presented in graph form in the Supplementary Appendix. Descriptive statistics were calculated for all variables of interest. Tables showing pre, post and late scores on decision-making, task management, situational awareness and teamwork are presented using means and standard deviations. A similar table was created for technical skills checklist scores.

The technical skills checklist scores and ANTS scores were analyzed using linear mixed models to assess changes over time (pre, post, late). Mayo High Performance Teamwork Scale scores at Code Blue events were analyzed using two sample two-sided t-tests comparing differences between trained and untrained residents. Presence or absence of debriefing at the end of the Code Blue event was analyzed using a test of proportions (Fisher’s exact test). All analyses were carried out using SAS Version 9.4 (SAS Institute, Cary, NC, USA).

Results

Between July 10th, 2014, to June 28th, 2016, 47 participants (anesthesia residents) were enrolled in the pre-phase with 46 participants total after the post-phase. One resident completed only the pre-scenario and did not continue in the study.

From the pre-questionnaire responses, 100% identified that teamwork is key to good patient outcomes, 55% felt that emergency situations felt chaotic and 70% stated that working with people you do not know was challenging. Post and late questionnaire responses revealed I START-END tool training was a very positive learning experience. 80–90% of residents reported that the tool gave them an organized approach to ad hoc scenarios, they felt more confident in these settings, they were better able to gain information helpful to care of the patient, their communication with the team improved, as well as, resource planning. Residents stated they would continue to use the tool and apply it to other clinical settings. We assessed the proportion with a positive response by combining the two categories always and often. This proportion was then compared between the post and late assessment. For the 11 questions, the majority had positive responses that only differed by an absolute value of 5% or less post to late, consistent with good sustainability. Residents’ anesthesia non-technical skills performance (ANTS scores) by linear mixed models did not show a statistical change over time (the pre-, post- and late-simulation scenarios) for any of the four domains.

The linear mixed model showed a significant change in Technical Skills Checklist scores over time. Post scores were found to be significantly lower than pre scores (p = 0.009), whereas the late score was significantly higher (p < 0.0001) than the pre-score.

Eighty Code Blue events were captured in total (Jan 29, 2016 – Aug 29, 2016).

Thirty-six events were excluded because the data collected were incomplete, the code was cancelled, or the observer arrived too late. Forty-four Code Blue events were captured and used as scenarios to test the I START-END tool. However, one Code Blue event, initially included, was later excluded, as that Code Blue event was aborted shortly after initiation of resuscitation (<5 mins) when it was realized that the patient’s code status was DNR. In this case, the resident had I START-END training and debrief followed at the bedside, discussing how Code Blue activation could have been prevented, given that the DNR status was clearly documented on the patient’s chart.

Therefore, a total of 43 Code Blue events were captured. There were 32 anesthesiology residents who had not received I START-END tool training and 11 anesthesiology residents who had received I START-END training.

Mayo High Performance Teamwork Scale scores were compared between residents with training [mean of 3.64 (SD 4.13)] and those without training [mean of 3.06 (SD 3.75)].

A two-sample two-sided t-test was run to compare the difference in mean scores between groups and was not significant (p = 0.67).

Whether debriefing occurred between the trained and untrained groups, was analyzed with a test of proportions, using Fisher's exact test with p = 0.03.

Residents with I START-END training debriefed more often (7 out of the 11), compared to those without training (9 out of the 32). The data was retrospectively analyzed and there was no difference in the level of training between these two resident groups.

Discussion

The objective of this project was to improve resident communication and performance on Ad Hoc teams.

This study was inspired by the challenges reported by anesthesiology residents when responding to urgent and emergent situations outside the operating room, such as Code Blues and traumas.

As this study was underway, the elective practice of anesthesia began to change, in that more procedures were being performed in non-operating room settings, such as code stroke in the interventional radiology suite, MRI cardiology procedures, endoscopy suites, and more.

Several residents in the study, after learning the I START-END tool, spontaneously mentioned that they found the tool very useful in navigating these unfamiliar (non-urgent) Ad Hoc events in non-operating room settings.

This has led to more broad application of this communication tool in our institution.

From the post- and late-questionnaire responses, most residents gave very positive feedback about the I START-END tool. Residents reported that the tool was practical and would help them better perform on Ad Hoc teams. There was consensus that more simulation practice and debriefing would be useful. Residents also suggested that residents in other specialties could benefit from this training and its uptake would be better if the tool was more broadly disseminated. Despite the favourable feedback, we were not able to detect a difference in anesthetist non-technical skills scores related to exposure to the I START-END tool. This could be related to the limited practice opportunities between simulations, limited system support and spread of the I START-END tool concept outside the study experience.

The decrease in technical skill checklist scores at the post-simulation session was unexpected. Possible explanations could include lack of resident buy-in (motivation), and perhaps a one to two week time period may be too short a time period to expect integration and consolidation of the tool into practice. After several months, at the late simulation session, there was significant improvement in performance of technical skill checklist scores compared to the pre-session. However, the sample size was small and confounded by the fact that the residents had had several more months of general clinical experience.

The impact of the I START-END tool did not translate into improved teamwork performance at Code Blue events as measured by the Mayo High Performance Teamwork scale. However, debriefing at the end of Code Blue events did occur significantly more often in residents who had had I START-END training. This may be explained by the fact that, after each study simulation session, debriefings focused on the importance of communication and speaking up in teams, which I START-END facilitates. Debriefing is an important learning tool as it allows for reflection of events with others, and helps identify opportunities for improvement, in this way promoting a stronger patient safety culture.

The difficulty in recruiting, scheduling, and following up with residents for this study (after they signed up and agreed to participate in all three phases) was considerable.

Understanding this lack of engagement in an area they themselves recognized as important can be explained by examining the motivation, opportunity, and capability of the resident group.³³ Motivation was suboptimal in this study. Residents’ workloads are high and additional impositions are onerous. This study required one hour on three separate occasions (pre-, post-, and late-) and 40+ minute travel time to and from the study site. Although approval to conduct this research was obtained from the Postgraduate committee, it was up to each resident to organize timely release from their clinical duties. The study was not instead of, but in addition to the residents’ other clinical responsibilities. Furthermore, given that these simulation sessions were not part of the official resident training curriculum and did not impact on their evaluations, residents’ commitment to the project was de-prioritized (eg, many did not remember to bring the cognitive aid cards with them). As the study period extended beyond a year and residents’ clinical rotations changed (off service duties in locations more remote from the study location) – it became increasingly difficult to schedule and facilitate their release from clinical work.

Opportunity was limited by access to the simulation centre. Scheduling was tight and flexibility was limited because of unpredictable clinical responsibilities of both staff and residents. As a result, there was no opportunity for re-booking a missed session. This proved particularly difficult for the late phase of the study.

Capability or the ability to adapt, synthesize and apply new knowledge (such as I START-END) requires practice and support in various contexts. Providing a cognitive aid is a first step, but with inadequate training and practice, the provision of such aids alone has failed to show improvements.^34,35 Due to limited resources, learning was limited to one discrete event in time (simulation, debriefing, and tool training on one occasion) and was not supported by other activities (local and system supports) that might consolidate this learning.

Structured handovers and checklists are strategies used by many high reliability organizations. Standardization of processes, precise delegation of responsibility and anticipated plan going forward are key elements that hardwire safety into such organizations.³⁶

This applies to the complexity in health-care systems today- care is delivered episodically by teams of experts that are not necessarily expert teams.

New strategies are required to ensure our patients receive safe, high-quality care at every encounter and to ensure seamless care delivery and coordination.³⁷ Lorelai Lingard has studied the changing healthcare environment and champions the idea that moving from a purely individual competence model to a more collective competence approach is necessary.³⁸

I START-END focuses on training individuals “how to team”. The hallmarks of healthcare today are increasing complex patients, receiving care in unique non-operating room settings, with multiple professionals sharing care responsibilities episodically. Inserting oneself into such teams requires a plan and voice. The I START-END tool is such a strategy. The framework is a common/standardized script that gives individuals’ agency to engage with others and share vital information more effectively and reliably Ad Hoc. Furthermore, when the I START-END framework is used – it can have a ripple effect – when effective engagement is modeled in front of others it may positively influence others to speak up and also become more engaged.

The I START-END tool is unique in parsing out the steps of effective engagement in a generic way.

In any group, charged with an agenda, the following elements are essential and are set out in the I START-END tool: leadership and role clarity (identify the leader, introduce self), context (story, circumstances), what is the game plan (task delegation and accomplishment/adjustment), what else do we need to do this (resources), checking out with others – permission to speak – as things are changing (timely updates).

The hyphen is significant – a pause – do we need to start again? – Perhaps the story has changed, therefore new tasks and focus need to be adjusted or priorities changed. Complexity requires that a situation is continuously reviewed and action plan realigned to address the changing scenario.

E for exit – letting the leader know you are leaving is more than a formality – the leader may ask you to stay on – also this is an opportunity for the exiting member to summarize their input so far and state aloud any concerns they foresee. Debriefing is key to ensure team members can share what went well and what could be improved. The importance of documentation is self-evident.

The actual uptake of such initiatives requires strategies at the macro-, meso- and micro-levels. Dixon-Woods, McNichol and Martin detail the steps to demystify such improvement work, using a logic model at each of these levels.³⁹ The latter was not considered when designing this study.

A study by Google – Project Aristotle (“the whole is greater than the sum of the parts”) – studied teams to determine the key attributes of high performing teams.⁴⁰ They found that the most important attribute was not related to gender, personality, IQ or EQ. The outstanding feature of high performing teams was that each team member spoke and listened for approximately the same amount of time. This was true REGARDLESS of the task assigned (from the simplest to the most complex). The opportunity and openness for input from all team members was consistently present on all high performing teams. The latter behavior is a feature of psychological safety.⁴¹

In healthcare, an individual's area of expertise creates a comfort zone (I know what I know). Moving beyond this zone means taking risks – admitting you do not know or that a mistake occurred. To voice the latter aloud to others requires a psychologically “safe space” i.e. feeling that your concerns will be heard without judgment.

The safe surgery checklist has had some success toward this goal.⁴² Initiating the checklist by team members, introducing themselves and sharing their understanding of the planned procedure is a good start.

Moving beyond the operating room into the “Ad Hoc world” creates greater challenges for promotion of psychological safety. The Ad Hoc team is a group of individuals, usually meeting for the first time, to perform a given task. The group disbands afterwards and is likely not to work together again. The term “stranger danger” is applicable, meaning it is counterintuitive for humans to be open and engaging when working with others for the first time. Such circumstances do not foster psychological safety.

Therefore, it is unlikely that this group of experts can just become expert collaborators in the moment. I START-END was developed to address such challenging Ad Hoc scenarios. I START-END is an additional skill set/tool that gives one agency in situations that feel uncomfortable and might otherwise keep us from speaking up. Starting with introductions, role clarity, creating context, task delegation, and regular status updates – an expectation of speaking up is hardwired into the Ad Hoc scenario.

Individuals using the I START-END tool report that by modeling speaking up, other members on the team become more engaged, in this way enhancing psychological safety on the team. The concept of tools and checklists has received widespread pushback. Reasons for resistance are many and beyond the scope of this paper.^43–46 A summary position might be – it is not the checklist, it is the culture. Given that the paradigm in healthcare today is based upon multiple, discrete inter-professional care encounters, it is necessary to make the right thing – effective communication – the easy thing to do.

No one would disagree that open and respectful communication is essential in healthcare. In this complex system with many moving parts, it is the interaction among all these parts – the people piece – that needs to be continuously supported and enabled. This is achieved by ensuring hierarchies are flattened and empowering all voices to speak freely without fear of reprisal. Establishing system-wide protocols/tools/frameworks/checklists have been shown to aid in developing a culture of psychological safety and are key to helping a group of individuals work together, as an effective team.^47–49

Conclusions

I START-END was developed to assist residents to perform more effectively on Ad Hoc teams in non-operating room settings. The majority (80–90%) of residents reported that I START-END tool training was helpful, that they would continue to use the tool and that it was applicable to other areas of clinical practice. Simulation experience and debriefing were identified as key to their learning. Debriefing at the end of Code Blue events was twice as likely to occur after training.

The residents also suggested more practice in implementing the tool and a broader spread of this initiative would aid in its uptake.

In conclusion, I START-END is a standardized communication tool that provides residents with a framework that encourages speaking up and engagement in Ad Hoc team situations.

Successful uptake and dissemination of this communication tool requires robust system-wide processes and engagement of stakeholders across the healthcare organization.