Back to Journals » Advances in Medical Education and Practice » Volume 13

An Evaluation of Pre-Hospital Emergency Care Personnel Knowledge About Crisis Resource Management and Perspectives of Educators About Inclusion of Crisis Resource Management in the Pre-Hospital Emergency Care Curriculum

Authors Rowland M, Adefuye AO 

Received 8 March 2022

Accepted for publication 20 June 2022

Published 10 August 2022 Volume 2022:13 Pages 849—864

DOI https://doi.org/10.2147/AMEP.S365436

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Md Anwarul Azim Majumder



Mugsien Rowland,1 Anthonio Oladele Adefuye2,3

1Department of Emergency Medical Care, Boitekanelo College, Gaborone, Botswana; 2Division of Health Sciences Education, Faculty of Health Sciences, University of the Free State, Bloemfontein, Free State, 9301, South Africa; 3Western Atlantic University School of Medicine, Freeport, Bahamas

Correspondence: Anthonio Oladele Adefuye, Division of Health Sciences Education, Faculty of Health Sciences, University of the Free State, Bloemfontein, Free State, 9301, South Africa, Email [email protected]

Purpose: To evaluate the knowledge of pre-hospital emergency care personnel (PECP) in South Africa on the principles, practice of crisis resource management (CRM), and obtain emergency medical care (EMC) educators’ views on the teaching and learning of CRM skills in the pre-hospital EMC curriculum.
Methods: This research was designed as an exploratory study that used a semi-structured questionnaire administered to 2000 PECP and focus group discussion (FGD) with 19 emergency care educators. Chi-squared test and Cramér’s V were used to examine the existence and the strength of an association between cross-tabulated variables. Responses to open-ended questions, as well as the data generated by the FGD, were analysed qualitatively using iterative inductive coding to identify themes.
Results: A response rate of 76% was obtained for the survey. Findings are that the majority (64.5%) of the PECP were not familiar with CRM, though familiarity varied significantly across cadres of PECP (p < 0.001). EMC educators reported that the concept of CRM must be thoroughly researched and developed in the emergency medical service (EMS) context before it is included in the EMC curriculum. The educators reported that early introduction of CRM in the EMC curriculum will have a positive effect on students’ professional development. Difficulties with assessment, knowing what to teach, and lack of universally accepted guidelines or teaching modalities are some of the challenges identified by EMC educators in relation to teaching CRM in the EMC curriculum.
Conclusion: The findings of this study provide new insights into PECP’s knowledge and EMC educators’ views on the teaching and learning of CRM in the EMC curriculum. This study highlights that more research is needed to develop an EMS CRM curriculum. Investigation into the development of a teaching and learning framework for CRM in EMC education could be the focus of future studies.

Keywords: crisis resource management, pre-hospital emergency care personnel, EMC educators

Introduction

Effective team management is recognised as a core element of expert practice in emergency medicine. Pre-hospital emergency care personnel (PECP) constantly need to collaborate with other healthcare professionals, as well as with different cadres of emergency care providers, when delivering pre-hospital care. Despite this, the teaching and training of most undergraduate pre-hospital emergency medical care (EMC) curricula focus mainly on developing graduate PECP who are proficient in core clinical skills, and who can perform complex procedures in the act of administering safe and effective pre-hospital emergency care.1 An obvious shortcoming of this training model is the limited attention given to the non-technical skills practitioners needed to work efficiently in a team-based environment. Failures related to the team, system, and/or process, rather than individual mistakes, have been reported as the most common cause of clinical errors.2 High-risk organisations use crisis resource management (CRM) to enhance teamwork performance, and as a countermeasure to avoid human error.3

CRM is a structured, evidence-based approach to training that is designed to enhance the non-technical skills required for effective teamwork performance in critical circumstances, where the absence of coordinated teamwork could have undesired outcomes.4,5 CRM originated as “cockpit resource management” and was later adapted to “crew resource management” in aviation.6–8 The principles of CRM have been adapted for training in various fields of medical practice, such as in-hospital emergency medicine,1,9 obstetrics10 and anaesthesiology.11 However, very little research has been done on CRM in the pre-hospital setting.12 It has been suggested that CRM can reduce the incidence of clinical error and contribute to effective teamwork and the success of a pre-hospital emergency care plan when applied in combination with good clinical practice.13 Rall et al report that an emergency care practitioner’s knowledge of and adherence to the principles of CRM can enhance patient safety during pre-hospital care.4 Despite these benefits, very few undergraduate pre-hospital EMC training programs purposefully teach and assess CRM principles and skills in their curricula. Rowland et al advocated for the purposeful teaching, learning and assessment of CRM principles and practices in all pre-hospital emergency care curricula.13

A thorough search of the relevant literature reveal that no study has investigated the knowledge of PECP in South Africa on CRM or obtain EMC educators’ views on the inclusion of CRM in the pre-hospital EMC curriculum. This present study evaluated the knowledge of pre-hospital emergency care personnel in South Africa on CRM and obtained emergency medical care educators’ views on the inclusion of CRM in the pre-hospital EMC curriculum. The findings of this study will provide new insights into PECP’s knowledge of the principles, and practice of CRM as well as EMC educators’ views of teaching, learning and assessment of CRM in the EMC curriculum in the South African context.

Materials and Methods

This research was designed as an exploratory study that utilised a semi-structured questionnaire and focus group discussions to obtain data.14,15

Study Population

The study population of the survey comprised PECP in South Africa (employed in either the state or private sectors) who were, at the time of the study, registered with the Health Professions Council of South Africa (HPCSA). The full list and the email addresses of registered PECP were obtained, with permission, from the HPCSA. At the time of this study, the total number of registered PECP for whom the HPCSA could provide email addresses was 24,970 (sample frame). The estimated minimum number of participants required for a statistically significant sample size at 99% confidence level (CL) and 4% margin of error (MOE) was calculated as N = 1000.16 Simple random sampling was used, and the target population comprised N = 2000 PECP. The study sample consisted of respondents who had consented to complete the questionnaire and, thereby, participate in the study.

EMC educators from the four South African universities (Durban University of Technology, Cape Peninsula University of Technology, University of Johannesburg and Nelson Mandela University) that offer a Bachelor in Emergency Care (BEMC) in were invited to participate in the focus group discussions. A purposive non-probability sampling technique was used to select participants (ie a certain group of people were deliberately chosen as participants due to the qualities they possessed).17 A letter inviting EMC educators to participate in the focus group discussions was sent to them via email, through the heads of departments. This invitation letter detailed the aim of the study and what would be required of participants during the focus group discussions. Those interested in participating were required to complete a participant sign-up form – in total 17 EMC educators completed the participant sign-up form. These 17 EMC educators were contacted via email and telephonically to schedule the focus groups, and to request them to complete and sign informed consent forms. The population was, thus, 17 EMC educators who had voluntarily agreed to participate in the study after reading and signing the informed consent form. Follow-up phone calls reminded participants of the focus group discussions. An explorative interview was held with 2 EMC educators, and the information they provided was also included in the data analysis, taking the total participants in the focus group discussions to 19.18

Questionnaire Design and Administration

After a thorough literature review and content analysis had been done, questionnaire items were compiled from factors that had been identified by prior studies.3,9,19,20 A literature search was conducted on databases such as Google Scholar, MEDLINE PubMed, CINAHL, SABINET, Science Direct, and Directory of Open Access Journals, using broad, simple search terms, such as paramedics, non-technical skills, crew resource management, and crisis resource management. Identified concepts were formulated into open- and closed-ended questions. The closed-ended questions were answered using an adapted Likert-scale ranking. Answers to the open-ended questions provided contextual depth to the quantitative data. The questionnaire consisted of two sections. The first section, Section A, elicited participants’ demographic information (ie, data on participants’ ages, genders, registration cadres, employment profiles and provinces of employment). Section B comprised both open- and closed-ended questions and obtained information about PECP’s knowledge of CRM – data collected in this section included PECP’s familiarity with the term CRM, PECP’s knowledge of the principles and practices of CRM, and their definitions of CRM. The questionnaire was administered electronically via the QuestionPro survey management system.21 All potential participants received an email along with a cover letter that elucidated the objectives of the study, as well as the conditions for study participation. A waiting period of one week from date of delivery was given for participants to complete the survey, after which repeated electronic reminders to complete the questionnaire – one per week – were sent for three consecutive weeks. The survey was conducted from 26 January 2021 to 26 February 2021.

The Focus Group Discussion Question

A focus group discussion question developed for the purpose of this study comprised of one open-ended question: What is your view on the inclusion of teaching and learning of CRM skills in the EMC curriculum? This question was used to initiate a robust discussion on the research topic of interest

Focus Group Discussions

For the purpose of this study, five separate discussions were conducted. A skilled facilitator with excellent communication skill and experienced in facilitating focus group discussions led the FGD. The facilitator is an experienced researcher with a PhD in nursing.

Focus group 1 had six participants and lasted 88 minutes; focus group 2 had five participants and lasted 77 minutes; focus group 3 had two participants and lasted 78 minutes; focus group 4 had four participants and lasted 77 minutes and focus group 5 (explorative interview) had two participants and lasted 91 minutes. These focus group discussions were conducted on 28 February, 6 September 2020, 15 September 2020; and 6 October 2020 (FGD 3 and 4), respectively. At the start of each focus group discussion, participants received a short introduction on the purpose of the study and were assured that there were no right or wrong answers, and that all contributions would be valued and would remain confidential. Three of the focus group discussions were conducted virtually using Microsoft Teams due to the Covid-19 restrictions at the time of study, and two were held face-to-face in a well-ventilated boardroom, and observing all Covid-19 safety protocol. All activities during the discussions were audio recorded, to which participants consented. The audio recordings were downloaded to a password-protected computer for safekeeping of the data. Transcripts of the discussions were made available to all the participants, to confirm their accuracy.

Pilot Study and Exploratory Interview

A pilot study was conducted prior to the official start of data collection, to test the suitability of the study design, the chosen data collection methods, and the overall structure of the questionnaire. The pilot study involved 10 PECP. An email with an information sheet and a request to participate in the pilot study, as well as an electronic links to the questionnaire, was sent to each participant. The findings of the pilot study confirmed the adequacy of the chosen data collection method and the overall structure of the questionnaire. The participants in the pilot study did not recommend changes to the semi-structured questionnaire, which took about 15–30 minutes to complete.

An exploratory interview was conducted with two participants who were nominated clinical mentors and educators on the clinical training platform for Nelson Mandela University, to establish the feasibility of the chosen data collection method and to estimate the duration of the focus group discussions. Participants were informed of the date, time, and location of the focus group discussions via email and text messages. The exploratory interview took place on 28 February 2020 and lasted 91 minutes. No adjustments were made to the interview guide, as the participants of the exploratory interview did not recommend any changes; these participants were excluded from the main study, though the data generated by the exploratory interview was included in the study and data analysis.18

Validity

Face validity was achieved by subjecting the questionnaire to a review and approval by a faculty evaluation committee.22 In addition, questionnaire validity was achieved by comparing the questionnaire elements to those of similar studies and by conducting a pilot study.

Reliability

The closed-ended questions of the questionnaire were grouped into subsets and analysed for reliability by using Cronbach’s alpha.23–25 Cronbach’s analysis of the subset of questions in the questionnaire produced values ranging from 0.5 to 0.8, thus, suggesting that the items within the subsets were interrelated and would provide reliable answers to the questions they were designed to investigate.

Data Analysis

Statistical analysis of all numerical data was performed with the statistical software package TIBCO StatisticaTM version 13.5.0.17 and GraphPad Prism 5.01. Results are presented in contingency tables as frequencies and percentages. Chi-squared test and Cramér’s V were used to examine the existence and the strength of an association between cross-tabulated variables. Responses to open-ended questions, as well as the data generated by the focus group discussions, were analysed qualitatively using Atlas.ti 8.3 software (Scientific Software Development, Germany). An independent transcriber transcribed the focus group recordings and an iterative inductive coding was done with the assistance of a co-coder with background in qualitative data analysis. Briefly, all transcribed data were organised into audiotaped documents, the complete transcript was read and re-read by the researchers and the co-coder to familiarise themselves with the content, unit of meaning (codes) were then generated from the transcribed data, generated units of codes were grouped into categories, and major categories grouped to form themes.26,27 The views of the participants of each focus group were considered separately and then compared to those of the other groups, to identify similar themes and to ensure that all codes were accounted for. All the investigators confirmed the identified codes and themes.

Ethical Considerations

Approval to conduct this study was obtained from the Health Sciences Research Ethics Committee of the Faculty of Health Sciences, University of the Free State (Ref. No. UFS-HSD2020/0462/2807). Permission was also obtained from the HPCSA, as well as from the four universities involved, to conduct the research with the educators in their BEMC programmes. Number coding was used, and no names or personal identifiers of participants appear on the data sheet that was sent for transcription or used for analysis. Participants were informed that the research is part of a PhD study and that their anonymised responses will be included in a publication in peer-reviewed academic journal.

Results

Participant Demographics

A response rate of 76% (1510/2000) was obtained in this study. Of the participants, men constituted 62.6% (n = 909) and women 37.4% (n = 542), giving a ratio of 2:1. The median age was 37.4 years (±SD = 1.14 years), with the majority (40.3%; n = 592) within the age bracket 25–34 years; participants had worked for an average of 11.2 years (±SD = 8.4). Participants’ registration cadres were as follows: basic ambulance assistant (BAA) (38.6%; n = 566); ambulance emergency assistant (AEA) (29.4%; n = 432); emergency care practitioner (ECP) (9.4%; n = 138); emergency care technician (ECT) (9.1%; n = 134); critical care assistant (CCA) (8.4%; n =124); emergency care assistant (ECA) (0.2%; n = 3); and other (4.8%; n = 71). The majority of the participants (83.4%; n = 1186) reported that they worked as operational emergency care practitioners, 11.3% (n = 160) worked as emergency care support staff, and only 5.3% (n = 5) worked in EMS communications. Regarding the sector in which they were employed, just over half (53.1%; n = 736) the participants were employed in the private sector, while 46.9% (n = 649) worked in the public sector. The EMS base of 896 (62.9%) participants were in urban areas, 20.1% (n = 286) indicated that they worked in rural areas, and 17.1% (n = 243) worked in semirural areas.

PECP Knowledge of CRM

Familiarity with CRM

A survey was conducted to assess the knowledge of PECPs on CRM. PECPs were requested to indicate either “Yes” or “No” to the question “Are you familiar with the terminology ‘crisis resource management’ (CRM)?”. The findings reveal that only 35.5% (n = 394) of the respondents were familiar with the term CRM, though familiarity differed significantly across the different cadres: More than half (55.8%) the participants registered as ECP were familiar with CRM ; Cramer’s V = 0.2; p < 0.001) (Table 1).

Table 1 PECP Knowledge of CRM

Knowledge of the Principles and Practices of CRM

Having obtained data on respondents’ familiarity with the term CRM, it was determined whether PECP were knowledgeable about the principles and practices of CRM. PECP who had indicated that they were familiar with CRM were asked to indicate either “Yes” or “No” to the question, “Are you knowledgeable of the principles and practice of CRM?”. Results obtained and presented in Table 2 reveal that the majority (54.8%; n = 205) of the respondents indicated that they were knowledgeable about the principles and practices of CRM, though knowledge differed significantly across the cadres (; Cramer’s V = 0.6; p < 0.001). Twenty participants who had indicated that they were familiar with term CRM did not answer this question.

Table 2 PECP Knowledge of the Principles and Practices of CRM

Definition of CRM

PECP who indicated that they were familiar with CRM and were knowledgeable about the principles and practice of CRM were asked to describe the concept CRM in EMS in their own words. Analysis of participants’ description of CRM in EMS yielded five themes, namely resource management at the workplace (110); non-technical skills for effective teamwork (50); principle of team management (50); principle of ensuring safety at the workplace (20); and principle of ensuring effective communication (17) (Figure 1).

Figure 1 PECP descriptions of crisis resource management.

EMC Educators’ Views on Teaching and Learning of CRM Skills in EMC Curricula

Analysis of participants’ responses yielded five themes, namely terminology; developing CRM for the EMC curriculum; benefit to students; teaching approach; challenges of teaching and learning of CRM skills in the EMC curriculum.

Terminology

Some EMC educators reported that the term crisis resource management was a new term to them; they suggested that “crew resource management” should be used in the EMC curriculum instead (cf. quotes #1–#3).

#1

OK, so Crisis Resource Management was new word to me. When I started teaching before I used the word Crisis Resource Management, I only used Crew Resource Management or CRM and that’s what I was exposed to a lot in my work environment, some of my studies that I did and then also, a lot from the aviation industry or the HEMS industry

#2

I must admit that I do not understand why it is being called Crisis Resource Management. For me, crew resource management fits better, because it is not just something that you do in a crisis, it is a means of interacting with each other and communicating with one another in all medical situations. Medical or rescue situations, not just when there is a crisis. Routine operations or routine case management should occur using these principles. In fact, that if you do not use them then you could precipitate a crisis. That is what it means for me, but I just put the little caveat is I really do not get the crisis word for me; it should be crew resource management. That is, it thanks

#3

Yeah, I think I agree with participant 2. As I said before, the first time I was exposed to Crisis Resource Management was when I had to start teaching and one of the lecturers came across and said we should be teaching the works crisis resource management. I might be biased because all I really have known or being exposed to has been crew resource management and I understand that and understand those principles. But I do think they end up being the same thing or just different wording

Furthermore, some participants suggested that, if the term crisis resource management had to be included in the EMC curriculum, it should be defined and characterised in the EMS context. This, they suggested, could be done with a Delphi study involving EMC educators (cf. quotes #4–#8).

#4

I agree with what participant 1 was saying that in EMS generally we work in a perpetual state of crisis. I think when we talk about Crisis Resource Management, we need to define A: what a crisis is, and B: what we mean when we talk about resources, are we talking about people? Are we talking about equipment? I think to come out with the definition of what we are actually talking about is probably quite important because it is not a cockpit. It’s a very fluid thing that is made up of many, many different things

#5

Crisis Resource Management, what do we actually mean? A: What is a crisis? What are the types of crisis that we face? I think they are core principles, but you know sometimes those are very much environment specific so, it would really depend on what we defining as Crisis Resource Management. Then I think one would have to investigate where does that then fit in? Although it is not officially part of the curriculum, but would it? Which is the best subject to teach that in it, depending on what we define it as

#6

I think again it depends on the definition because the crux of all of this is when we talk about this thing, what do we talk about? What do we mean, and I think once we figure out what we mean then we can only really start mining down to decide what do we need to teach people? I think until we know what this concept means and when we talk about, we put 3 words together into, what is in my mind a very complicated concept, what do we mean. Until we can say what we mean I think it is very difficult to try to decide what we need to teach because that definition would need to translate into an outcome somewhere. In addition, you know we are kind of floundering in the dark until we really know what that outcome needs to talk to. That that would probably be the first step, to try to figure out exactly what it is. Then if we are teaching it, where are we teaching it, and if we are teaching, how are we teaching? So, I think it’s a very complex concept that first needs to be defined before we can actually decide what we’re going to do with it

#7

I know I have hopped on about this a lot, but I think for me, it is the fact that it has not been adequately defined. We can talk around the garden as much as we like, but until we have a working definition of what we mean, I am still very unclear about what we are talking about when we talk about Crisis Resource Management in EMS. So, until we have managed to provide a reasonably well formulated definition, I’m not really sure

#8

It would be nice to see some sort of Delphi study within how groups may define what a crisis means because I still don’t know, like [P6] was saying, whether it’s a crisis and a disaster

Developing CRM Content for the EMC Curriculum

Participants were of the view that CRM is a concept borrowed from other professions and that it should by researched properly and developed in the EMS context before it can be taught in the EMC curriculum. These views are expressed in quotes #9–#13.

#9

The CRM that we are using and teaching we have sort of taken it from medicine and we’ve taken it from Aviation medicine and we’re trying to plug it in without really having gone to develop it further for what it really is needed for our students. I think the risks are that we are implementing underdeveloped principles. The principles that I’m using to teach are straight from Medicine so, like participant 2 said as well, we are not considering the Rescue curriculum, we are not considering the communication and the different things that we need to include

#30

I think as EMS, definitely as academics and educators is we do need to spend a bit more time on this that we think we need to spend on it. The reason I say this is because CRM is a natural fit for EMS and because it is a natural fit for EMS people think that we can just implement it. I think just implementing it is not going to be successful. We need to take the time to think about how we are going to implement it and develop it properly. It is a bit of a paradox there in that it being such a natural fit might actually be one of the barriers to implementing it

#10

There is a whole lot of other development factors that need to be considered when developing CRM and teaching CRM to the students. It is important to try to integrate and try to get a good generic mix for all the different types of services and situations that these students might experience in order to help them. I do agree it is important to broaden our viewpoint of it and to include different things

#12

As I said before, I think it is important that we develop it a bit more. So, whatever I’m saying now is just what I think should be included but I do feel there is space for this to be developed taking into consideration the various services and things that students would do

#13

I think there is quite a lot of things that you would be able to add within CRM. But I think what we both agreed on in the beginning is that it would need quite significant development and perhaps some additional research to see what exactly is it that you would have to included

Benefit to Students

Participants of this study agreed that teaching and learning of CRM skills in the EMC curriculum is essential for student training. Participants reported that early introduction of CRM in the EMC curriculum would benefit students by enhancing their clinical management skills, communication skills, teamwork skills, and overall professional development (cf. quotes #14–#19)

#14

Absolutely, I agree with it and from my experience while working on the road, implementing it early on creates a good platform for a practitioner to develop their skills and teamwork and patient safety further so I do think it’s very important

#15

For a start I think it’s essential. But I think it should be included, if you’re going to get lectures on CRM specifically, it is something that should occur right at the beginning, in first year as part of working in teams

#16

Yes, it should be a drill. It should be second nature. By the time they exit, it must be second nature and it should be almost a drill

#17

Yeah, I think, like I said before CRM is vital for professional to develop and if we were to include CRM from an early stage within our program and then foster its development throughout the four years. Four years is quite a lot of time to allow for development, by the time they get to 4th year or even qualify, those principles are like participant 2 said are embedded and are well developed so, it would be harder to breakdown while practicing as a as a new practitioner when they leave

#18

So, look at there is multiple benefits and if we divide them up into let’s say clinical and educational benefits. The clinical benefits obviously are improved efficiency, of management, improved efficiency of communication. The big one which is what its intention is in the beginning is improved patient safety and hopefully fewer adverse events

#19

CRM will allow the students to have a sort of guideline to deal with the teams that they are in and deal with the preceptors that they are with and to help build that professional relationship with their preceptors. Which in turn will allow for better teaching and learning while they are working with those preceptors

In addition, participants reported that teaching CRM skills should be included in the emergency rescue curriculum, as knowledge of the principles and practices of CRM are vital for emergency scene management (cf. quotes #20 and #21).

#20

You might be involved with a very challenging medical case; a premature neonatal etc. so, it’s a generic skill that needs to be cross cutting. It needs to be reinforced not just in the emergency medical care curriculum but in the rescue curriculum as well. I would class it as a fundamental skill

#21

It is important from a scene management point of view, not just a patient safety or clinical management point of view. I think that possibly in that respect it still needs a little bit of development because as far as I know it is currently aimed really at facilitating patient safety and facilitating clinical management. However, acting as a nexus of communication with all these different role players, that aspect of it, I think it could be very valuable to expand it and develop it in that direction as well. To bring it all under one umbrella

Teaching Approach

Teaching approach was the third dominant theme that featured in the participants’ responses. Having reported that teaching CRM skills in the EMC curriculum would be beneficial to student professional development, participants made suggestions related to pedagogical approaches that can be used to teach CRM skills in the EMC curriculum. Some of the teaching approaches used and suggested by the participants are lectures, videos, case study, and role-play or simulation-based education (cf. quotes #22–#29).

#22

I do not really teach it; I explain what it is and then I have them implement it within a very difficult case scenario in a practical level

#23

I use the Claire Brumley case to just explain and these are all things that they’ve been exposed to in the previous years, but then I use those videos to just get them to understand it again and put it within the context of a difficult, critical care retrieval case. They might have had exposure to it while doing the airway management and things like that but yeah, make use of it as a critical care retrieval case to help them with that

#24

I use the case video of the big airline disaster that started on an island, I cannot remember the island in 1960 something, where the two planes crashed into each other and that is where the CRM sort of originated, and I would then develop from there

#25

So, I go into a bit of that and then I go over the Claire Brumley case again with them, and then I give them the principles for CRM itself

#26

I give them a really difficult case with a large team and allow them to go through that and after the case we debrief, and we see how we could have improved our CRM by using various principles. That’s basically what I do

#27

I think other content to include various systems or approaching systems that could be implemented in various different cases. I am just thinking of when I teach Critical Care Transport to give the students a sort of generic system to approach these difficult cases and what things to consider and what things to remember and assess that gives them some structure when it comes to difficult cases or various issues

#28

You cannot leave it as one lecture it must be role-played and simulated throughout. That is a more teaching approach rather than content, but I think interpersonal relations, body language, the emotional side of things, I think participant 1 mentioned giving people some cognitive strategies to control their own emotions

#29

When it comes to the teaching of it, it needs to be powerful and impacting teaching when it comes to doing this because you need to develop these effective domains essentially so role playing would be essential, getting outside or external actors to come in and help students understanding that. I think including that in our teaching is quite important because it allows that development for the student to get all of these skills sorted to implement CRM as well

In addition to the suggested pedagogical approaches, some participants were of the view that teaching of CRM skills can be presented as a continuing professional development course, as explained in quote #30.

#30

I think couching it as a continuing professional development course so that it gets wider access. Having it done in for instance conference presentations, conference workshops, including it in non-discipline related workshops as well

Challenges of Teaching and Learning of CRM Skills in the EMC Curriculum

Participants reported on the challenges that could be encountered when including teaching and learning of CRM in the EMC curriculum. Some of the possible challenges stated by the participants were an overloaded curriculum, difficulties with assessment, difficulty in knowing what to teach, lack of universally accepted guideline or modality of teaching, and students’ poor conceptualisation of concepts (cf. quotes #31–#36).

#31

There is no more space in the Bachelor’s degree in the undergraduate programme for curriculum growth

#32

I think another challenge for us at the university is we’ve already got an overloaded programme so, how do you fit this into the curriculum

#33

Crisis Resource Management is something that’s quite difficult to assess

#34

Yes, as [mentions P1’s name] said, it’s very difficult to teach. I think on a degree level course, a lot of what you’re teaching is thought process and how to think and how to access information and how to interpret and make a decision, you know, those higher-level cognition sort of things [P1 murmurs in agreement], and that’s very important for sure

#35

Using it from different places, not everyone is using the same principles or not everyone is as well trained with it, as we would want them to be. Coming back to the preceptors they might not be completely aware of the CRM principles that we are teaching our students, they might be aware of different things. So, the CRM course that the guys in HEMS [Helicopter emergency medical service] use might be completely different from the things that we have our students, so all of a sudden you might have this clash and it might cause some issues

#36

Whenever you implement it needs to be outlined in the right way, it really needs to be thought out well before implementing it within students fresh from high school with their limited life experience. They might not understand it completely and might misinterpret what you are trying to achieve. I think those are similar risks that could be considered

Discussion

The findings of this study that showed a male to female ratio of 2:1 are consistent with findings of other studies,28,29 and suggest that the EMS profession in South Africa is a male-dominated field. This is not peculiar to South Africa. as Crowe et al report that women are underrepresented in the EMS workforce in the United State of America.30 This underrepresentation could have implications for practice in conservative communities or cultures, such as the Muslim community of South Africa, where some women may feel embarrassed about being attended to by a male PECP.31 Hence, there have been calls for more female PECP to be trained and employed, so that women can be attended to by female PECP.32 The median age of 37.4 years for PECP is consistent with findings by Mothibi et al, and suggest that the EMS workforce in South Africa is dominated by a relatively young population.28

Historically, EMC education and training in South Africa was implemented using short vocational courses, such as the BAA, AEA, and CCA. An increase in the number of accredited training colleges and institutions in the early 90s increased the numbers of short-course trained PECP. Thus, since 2018, the majority of pre-hospital PECP registered with the HPCSA are BAA and AEA practitioners.33 This is reflected in the findings of this study, which are that BAA and AEA practitioners form the majority (68%) of the PECP population. Over the past two decades, EMC education and training in South Africa have undergone continuous review and reform, leading to the emergence of the recently approved three-tier Emergency Care Qualification Framework (ECQF), which was adopted by the national Department of Health and which is aligned with the South African National Qualifications Framework (NQF).34–36

This study found that more than half (53.1%) of PECP were employed in the private sector. This is consistent with findings other studies, which report that healthcare professionals in South Africa prefer to work in the private sector, rather than in the public sector.37,38 Poor workplace conditions and remuneration packages are some of the reasons cited by healthcare professionals for avoiding employment in the public health system.38,39 It is, therefore, plausible that this might also apply to the PECP who participated in this study.

Patient safety is a vital component of professional practice by healthcare practitioners.40 However, achieving patient safety can be affected by various factors in the pre-hospital emergency care environment. Factors such as clinical decision-making by practitioners, adverse events and medication errors, communication within the organisation, emergency driving during patient transportation, and extreme weather conditions have been reported to significantly influence patient safety in the pre-hospital emergency care environment.12 Pre-hospital emergency care personnel are expected to possess knowledge, experience, and excellent judgement skills, so that they can make sound clinical decisions and function optimally and independently in an unstructured and constantly changing environment, such as the pre-hospital emergency care setting.12

An emergency care practitioner’s familiarity with, knowledge of and adherence to the principles of CRM has been reported to enhance patient safety during pre-hospital care.4,13 This study found that the majority (64.5%) of the participants are unfamiliar with CRM, though familiarity varied significantly across the different cadres: More than half (55.8%) the participants who were registered as ECP were familiar with CRM, in contrast to BAA (32.4%), AEA (30.2%), ECA (33.3%), CCA (40.2%), and ECT (30.9%) (p < 0.001) (Table 1). This is not surprising, as ECP-registered PECP might have been exposed to some principles of CRM while undergoing structured, formal training as PECP (eg, 4 year-Bachelor of Emergency Medical Care) – though, not necessarily because it was included in a formal module – as indicated by the educators during the focus group discussions. The majority (54.8%; p < 0.001) of the respondents who reported familiarity with CRM indicated that they were knowledgeable about the principles and practices of CRM (Table 2). This is exemplified by their definitions of CRM, which are consistent with reports by prior studies on the definition of CRM,12,40 and articulate the principles of non-technical skills, interpersonal behaviour, team management, and resource allocation to promote patient and crew safety in the pre-hospital emergency care setting.40,41 These findings suggest that, while some PECP can attain and maintain patient and crew safety consequent of their knowledge of CRM, others may have poor compliance to patient safety due to their lack of familiarity with and poor knowledge of the principles and practice of CRM. This conclusion highlights the need for purposeful teaching, learning and assessment of CRM principles and practices in the undergraduate pre-hospital emergency care curriculum, as proposed by Rowland et al 13 However, it is worth noting that improving patient safety in the pre-hospital setting requires a multifaceted approach. Training must be accompanied by improvements to clinical systems, and maintenance of a true culture of safety in the workplace. CRM or teamwork training can be a lever for the desired culture change.11

Analysis of EMC educators’ responses on the inclusion of purposeful teaching and learning of CRM in the undergraduate EMC curriculum reveals that there is much confusion about which terms for CRM are the most suitable in the EMS context. Terms and concepts are part of university undergraduate and postgraduate courses globally.42 Precise definition of a term and its appropriate use in a curriculum is important for providing clarity and avoiding confusion by educators and students.43 EMC educators in this study were of the view that the term CRM must be properly defined and characterised within the EMS context, before it can be taught in the EMC curriculum. It is understood that concepts, and thus definitions, are often formed by the perceptions of the audience, and these perceptions might differ as a result of language, education, cultural and professional differences.44 Hence, EMC educators in this study suggested that the term “crew resource management” is more suitable for the EMS milieu, since PECP work as a teams or crews. We believe that expert agreement on the proper definition and meaning of CRM in the EMS context is the first step to its inclusion in the EMC curriculum. It is likely that a clear focus for teaching and learning will be provided once the exact meaning is agreed upon by EMS experts through a Delphi study, as suggested by some of the educators.

Over the years, many different articulations of CRM or teamwork training have been promulgated and adapted for training in various fields of medical practice, including in-hospital emergency medicine,1,9 obstetrics10 and anaesthesiology.11 Some of these curricula are academic courses developed specifically for the domain of practice. An example is the Anaesthesia Crisis Resource Management course that was developed by Howard et al 45 Similarly, EMC educators were of the view that the concept of CRM needs to be properly researched and developed in the EMS context, before it can be taught in the EMC curriculum. Curriculum development is described as a collection of plans about teaching and learning; the ability to plan an effective curriculum is a crucial skill for all educators.46 According to Stenhouse (2012), curriculum development is a process that is socially constructed, and it involves educators taking into account contextual factors, rather than pre-specifying objectives when designing a curriculum.47 Curriculum design research involves consulting curriculum statements issued by the government and regulatory bodies, defining objectives, finding information about topics, deciding on suitable teaching methods and choosing ways to assess learning.48 Such research can strengthen the knowledge base in the form of design principles that offer heuristic advice to curriculum development teams and enhance the development of effective curricula.49 It is, thus, plausible that conducting detailed curriculum design research on CRM in the EMS context, as proposed by educators, will promote the development of an effective CRM curriculum for EMC training programmes.

Participants in this study reported that early introduction of CRM skills in the EMC curriculum will have a positive effect on students’ clinical management skills, communication skills, teamwork skills, and overall professional development. This finding is consistent with that of studies that report that education programmes that teach CRM skills have a positive effect on students’ technical skills and their competence in handling crisis events.50 In a study aimed at investigating the effectiveness of teaching CRM for self-efficacy, team efficacy and technical skills of nursing students, Coppens et al report that CRM increases students’ self-efficacy and team efficacy.51 Similarly, Harwayne-Gidansky et al report that teaching CRM in a third-year medical curriculum led to a significant improvement in students’ communication skills, attitudes to and confidence for clinical practice.52 It is, therefore, conceivable that introducing an effective EMC-CRM curriculum will have a similar effect on student skills and professional development. Participant suggestions on the use of simulation-based pedagogy to teach CRM in the EMC curriculum are consistent with findings of prior studies that found that high fidelity simulation is one of the most effective methods of teaching CRM skills.50,53–56 A major advantage of using simulation-based pedagogy is that it can reproduce critical situations, for which students can have a proper debriefing that is aimed at fostering metacognition on technical, cognitive, and interpersonal skills.57 In addition, the use of simulation-based pedagogy has been shown to promote acquisition and performance of CRM skills.58,59 The use of case study as teaching tool as suggested by some of the educators is in accordance with report by Lalonde and Roux-Duffort (2013).60 However, it is worth noting that, pedagogically, we do not yet know what mix of the many different modalities of teaching should be applied, or to what target populations, and at what frequency.11 It has been reported that continuous, intensive, realistic training over an entire career is necessary to achieve a sustained, cumulative effect of teaching CRM skills.11 Similarly, some educators were of the view that CRM can be presented as a continuing professional development course as a teaching modality.

Despite the numerous benefits of teaching and learning CRM skills, it is reported that multiple challenges remain for optimally implementing training on CRM.11 Similarly, EMC educators acknowledged that numerous challenges face the teaching of CRM skills in the EMC curriculum. Identified barriers such as lack of knowledge in the field, educational (teaching and learning) challenges, poor conceptualisation of CRM, and lack of universally accepted modality of teaching are consistent with findings by prior studies.11,60,61 Thus, suggesting that more research is needed to address these challenges, if CRM is to be taught in EMC education.

Limitations

A limitation of this study was that some participants did not answer all the questions in the questionnaire. However, we do not believe that this limitation compromised the validity and reliability of the findings of this study, as responses obtained for each subset of questions surpassed the estimated minimum sample size calculated for this study.

Conclusion

In conclusion, this study investigated the knowledge of PECP and EMC educators on CRM and its inclusion in the pre-hospital emergency care curriculum. The study found that the majority of the PECP were not familiar with CRM or knowledgeable about the principles and practices of CRM. This knowledge was found to vary significantly across the different cadres, with most of the short-course trained personnel reporting limited knowledge of CRM. EMC educators reported that early teaching of CRM could be beneficial to students and could enhance their overall professional development. In addition, they suggested various teaching modalities that can be used to teach CRM to students and stated that more research is needed to develop a CRM curriculum that is suitable for the EMS context. Challenges of teaching CRM in the EMC curriculum, as reported by educators, include an overloaded curriculum, being difficult to assess, not knowing what to teach, the lack of a universally accepted guideline or modality of teaching, and students’ poor understanding. Investigation into the development EMS CRM curriculum and a teaching framework for CRM in EMC education could be the focus of future studies.

Acknowledgments

The authors would like to thank all PECP and EMC educators who participated in this study.

Author Contributions

Both authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Funding

There is no funding to report.

Disclosure

The authors report no conflicts of interest in relation to this work.

References

1. Carne B, Kennedy M, Gray T. Crisis resource management in emergency medicine. Emerg Med Australas. 2012;24(1):7–13. doi:10.1111/j.1742-6723.2011.01495.x

2. Donaldson MS, Corrigan JM, Kohn LT. To Err is Human: Building a Safer Health System. Vol. 6. National Academies Press; 2000.

3. Gross B, Rusin L, Kiesewetter J, et al. Crew resource management training in healthcare: a systematic review of intervention design, training conditions and evaluation. BMJ. 2019;9(2). doi:10.1136/bmjopen-2018-025247

4. Rall M, Gaba D, Howard SK, Dieckmann P. Human performance and patient safety. In: Miller RD, editor. Miller’s Anaesthesia. Elsevier; 2009.

5. Muñoz-Marrón D. Human factors in aviation: CRM (crew resource management). Psychologist Papers. 2018;39(3):191–199.

6. Flin R, Maran N. Identifying and training non-technical skills for teams in acute medicine. BMJ Qual Saf. 2004;13(suppl 1):i80–i84. doi:10.1136/qshc.2004.009993

7. St Pierre M, Hofinger G, Simon R. Crisis Management in Acute Care Settings: Human Factors and Team Psychology in a High-Stakes Environment. Springer International Publishing; 2016.

8. Gaba DM, Fish KJ, Howard SK, Burden A. Crisis Management in Anesthesiology E-Book. Elsevier Health Sciences; 2014.

9. Royal College of Physicians and Surgeons Canada. Optimizing Crisis Resource Management to Improve Patient Safety and Team Performance: A Handbook for All Acute Care Health Professionals. 1st ed. The Practice, Performance and Innovation Unit of the Royal College of Physicians and Surgeons of Canada; 2017.

10. Bracco F, De Tonetti G, Masini M, Passarelli M, Geretto F, Celleno D. Crisis resource management in the delivery room: development of behavioral markers for team performance in emergency simulation. Int J Environ Res Public Health. 2018;15(3):439. doi:10.3390/ijerph15030439

11. Gaba D. Crisis resource management and teamwork training in anaesthesia. Br J Anaesth. 2010;105(1):3–6. doi:10.1093/bja/aeq124

12. Gangaram P, Alinier G, Menacho AM. Crisis resource management in emergency medical settings in Qatar. Int Paramed Pract. 2017;7(2):18–23. doi:10.12968/ippr.2017.7.2.18

13. Rowland M, Adefuye AO, Vincent-Lambert C. The need for purposeful teaching, learning and assessment of crisis resource management principles and practices in the undergraduate pre-hospital emergency care curriculum: a narrative literature review. Australas J Paramedicine. 2021;18. doi:10.33151/ajp.18.820

14. Jayasekara RS. Focus groups in nursing research: methodological perspectives. Nurs Outlook. 2012;60(6):411–416. doi:10.1016/j.outlook.2012.02.001

15. Stalmeijer RE, McNaughton N, Van Mook WN. Using focus groups in medical education research: AMEE Guide No. 91. Med Teach. 2014;36(11):923–939. doi:10.3109/0142159X.2014.917165

16. SurveyMonkey. Sample size calculator. Available from: https://www.surveymonkey.com/mp/sample-size-calculator/. Accessed February 22, 2022.

17. Etikan I, Musa SA, Alkassim RS. Comparison of convenience sampling and purposive sampling. AJTAS. 2016;5(1):1–4. doi:10.11648/j.ajtas.20160501.11

18. Breen RL. A practical guide to focus-group research. J Geogr High Educ. 2006;30(3):463–475. doi:10.1080/03098260600927575

19. Cooper S, Endacott R, Cant R. Measuring non-technical skills in medical emergency care: a review of assessment measures. OAEM. 2010;2:7. doi:10.2147/OAEM.S6693

20. Porter JE, Cant RP, Cooper SJ. Rating teams’ non-technical skills in the emergency department: a qualitative study of nurses’ experience. Int Emerg Nurs. 2018;38:15–20. doi:10.1016/j.ienj.2017.12.006

21. QuestionPro. QuestionPro survey software. Available from: https://www.questionpro.com/. Accessed August 3, 2022.

22. Ringsted C, Hodges B, Scherpbier A. The research compass’: an introduction to research in medical education: AMEE Guide No. 56. Med Teach. 2011;33(9):695–709. doi:10.3109/0142159X.2011.595436

23. Bland JM, Altman DG. Statistics notes: cronbach’s alpha. BMJ. 1997;314(7080):572. doi:10.1136/bmj.314.7080.572

24. DeVellis RF. Scale Development: Theory and Applications. 4th ed. SAGE Publications; 2016:280.

25. Taber KS. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Res Sci Educ. 2018;48(6):1273–1296. doi:10.1007/s11165-016-9602-2

26. Kekeya J. Analysing qualitative data using an iterative process. Contemp PNG Stud. 2016;24:86–94.

27. Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval. 2006;27(2):237–246. doi:10.1177/1098214005283748

28. Mothibi JD, Jama M, Adefuye AO. Assessing the knowledge of emergency medical care practitioners in the Free State, South Africa, on aspects of pre-hospital management of psychiatric emergencies. Pan Afr Med J. 2019;33:132.

29. Butler MW, Adefuye AO. Assessing the knowledge of emergency medical care personnel in the Free State, South Africa, on aspects of paediatric pre-hospital emergency care. Pan Afr Med J. 2019;32. doi:10.11604/pamj.2019.32.98.17718

30. Crowe RP, Krebs W, Cash RE, Rivard MK, Lincoln EW, Panchal AR. Females and minority racial/ethnic groups remain underrepresented in emergency medical services: a ten-year assessment, 2008–2017. Prehosp Emerg Care. 2020;24(2):180–187. doi:10.1080/10903127.2019.1634167

31. Carrim NM, Paruk Z, Do we or don’t we? Muslim women wearing the hijab in the South African Work environment. J Relig Afr. 2021;50(1–2):11–31. doi:10.1163/15700666-12340177

32. Alharthy N, Alswaes S, Almaziad A, Alenazi N, Abdallah M, Alshehry M. Public perception of female paramedics at King Abdulaziz Medical City, Saudi Arabia. Int J Emerg Med. 2018;11(1):1–5. doi:10.1186/s12245-018-0217-4

33. Sobuwa S, Christopher LD. Emergency care education in South Africa: past, present and future. Australas J Paramedicine. 2019;16. doi:10.33151/ajp.16.647

34. National Department of Health: Republic of South Africa. National Emergency Care Education and Training Policy; 2017.

35. Vincent-Lambert C. International perspectives: South African ambulance services in 2020. In: Ambulance Services. Springer; 2015:175–183.

36. Vincent-Lambert C. A Framework for Articulation Between the Emergency Care Technician Certificate and the Emergency Medical Care Professional Degree. University of the Free State; 2011.

37. Econex N. Note 4 NHI Note 4: supply constraints. Econex: Stellenbosch; 2009. Available from: https://econex.co.za/wp-content/uploads/2015/07/ECONEX_NHInote_4.pdf. Accessed August 3, 2022.

38. Mumbauer A, Strauss M, George G, et al. Employment preferences of healthcare workers in South Africa: findings from a discrete choice experiment. PLoS One. 2021;16(4):e0250652. doi:10.1371/journal.pone.0250652

39. George G, Reardon C, Preparing for export? Medical and nursing student migration intentions post-qualification in South Africa. Afr J Prim Health Care Fam Med. 2013;5(1):1–9. doi:10.4102/phcfm.v5i1.483

40. Fanning R, Goldhaber-Fiebert S, Udani A, Gaba D. Crisis resource management. In: Levine AI, DeMaria S Jr, Sim AJ, editors. The Comprehensive Textbook of Healthcare Simulation. Springer; 2013:95–109.

41. Miller RD, Eriksson LI, Fleisher LA, Wiener-Kronish JP, Cohen NH, Young WL. Miller’s Anesthesia e-Book. 8th ed. Elsevier; 2014.

42. Alcina A. Teaching and learning terminology: new strategies and methods. In: Teaching and Learning Terminology. John Benjamins; 2011.

43. Slisko J, Dykstra JDI. The role of scientific terminology in research and teaching: is something important missing? J Res Sci Teach. 1997;34(6):655–660.

44. Van Mil J, Henman M. Terminology, the importance of defining. Int J Clin Pharm. 2016;38(3):709–713. doi:10.1007/s11096-016-0294-5

45. Howard SK, Gaba DM, Fish KJ, Yang G, Sarnquist FH. Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents. ASEMCG. 1992;63(9):763–770.

46. Du Preez P, Simmonds S. Curriculum, curriculum development, curriculum studies? Problematising theoretical ambiguities in doctoral theses in the education field. S Afr J Educ. 2014;34(2):1–14. doi:10.15700/201412071140

47. Stenhouse L. A process model of curriculum. In: Hoadley U, Reed Y, editors. Curriculum: Organising Knowledge for the Classroom. Oxford University Press; 2012.

48. Jacobs M, Gawe N, Vakalisa NCG. Teaching-Learning Dynamics. Pearson Education South Africa; 2016.

49. Van den Akker J. Curriculum design research. Intro Edu Design Res. 2007;37:37–50.

50. Lucas A, Edwards M, Harder N, Gillman L. Teaching crisis resource management skills to nurses using simulation. J Contin Educ Nurs. 2020;51(6):257–266. doi:10.3928/00220124-20200514-05

51. Coppens I, Verhaeghe S, Van Hecke A, Beeckman D. The effectiveness of crisis resource management and team debriefing in resuscitation education of nursing students: a randomised controlled trial. J Clin Nurs. 2018;27(1–2):77–85. doi:10.1111/jocn.13846

52. Harwayne-Gidansky I, Leader A, Zackai S, Strother C. Crisis resource management for third-year medical students: a simulation curriculum. MedEdPORTAL. 2014;10:9951. doi:10.15766/mep_2374-8265.9951

53. Clarke S, Horeczko T, Carlisle M, et al. Emergency medicine resident crisis resource management ability: a simulation-based longitudinal study. Med Educ Online. 2014;19(1):25771. doi:10.3402/meo.v19.25771

54. Tremblay M-L. Simulation-based crisis resource management in pharmacy education. Am J Pharm Educ. 2018;82(6):6531. doi:10.5688/ajpe6531

55. Robertson B, Schumacher L, Gosman G, Kanfer R, Kelley M, DeVita M. Simulation-based crisis team training for multidisciplinary obstetric providers. Simul Healthc. 2009;4(2):77–83. doi:10.1097/SIH.0b013e31819171cd

56. Thomas MJ. Training and Assessing Non-Technical Skills: A Practical Guide. CRC Press; 2017.

57. Fung L, Boet S, Bould MD, et al. Impact of crisis resource management simulation-based training for interprofessional and interdisciplinary teams: a systematic review. J Interprof Care. 2015;29(5):433–444. doi:10.3109/13561820.2015.1017555

58. Lee J, Oh P-J. Effects of the use of high-fidelity human simulation in nursing education: a meta-analysis. J Nurs Educ. 2015;54(9):501–507. doi:10.3928/01484834-20150814-04

59. Lucas A, Edwards M. Development of crisis resource management skills: a literature review. Clin Simul Nurs. 2017;13(8):347–358. doi:10.1016/j.ecns.2017.04.006

60. Lalonde C, Roux-Dufort C. Challenges in teaching crisis management: connecting theories, skills, and reflexivity. J Manag Educ. 2013;37(1):21–50. doi:10.1177/1052562912456144

61. Rajendram P, Notario L, Reid C, et al. Crisis resource management and high-performing teams in hyperacute stroke care. Neurocrit Care. 2020;33(2):338–346. doi:10.1007/s12028-020-01057-4

Creative Commons License © 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.