A suggested emergency medicine boot camp curriculum for medical students based on the mapping of Core Entrustable Professional Activities to Emergency Medicine Level 1 milestones

Introduction

Emergency Medicine (EM) has experienced unprecedented growth in graduate medical education with the expansion of residency programs nationwide.¹ An increasing number of US medical students rank EM as a top specialty choice.¹ Undergraduate programs are incorporating EM experiences in their curricula, but the addition of mandatory, structured EM clerkships still remains an area of slow growth.^2,3 Unlike primary-care specialties such as Internal Medicine or Pediatrics where core standardized clerkship curricula ensure that each graduating US medical student has certain knowledge and skills related to that discipline, this is not so for EM. Medical schools may offer elective EM experiences, but these may have wide variability in goals, settings, and focus, as evidenced by differences between community versus tertiary-care center rotations, pediatrics versus adult EM experiences, and exposure to trauma and ultrasound. Therefore, currently, there may be considerable variability from one medical school graduate to another in the knowledge and skills related to EM.

Program directors across specialties are concerned over the variability in skill sets and lack of preparedness of incoming residents.⁴ In response to these concerns, efforts are being made to better define expectations during physician training with the development of the Core Entrustable Professional Activities (EPAs) for Entering Residency by the Association of American Medical Colleges (AAMC).^5,6 The EPAs are a clear, concise list of what graduating medical students should be entrusted to do without direct supervision on DAY ONE of any residency.^7,8 The entrustment in the tasks of professional practice is important to note and means that a student can perform the task unsupervised.^7,8 EPAs therefore define activities of an entrusted learner and offer a clear definition of what is expected and what can be measured and observed, once a competency has been achieved, regardless of specialty. For example, gather a history and perform a medical examination is a discrete measurable EPA. There are 13 EPAs currently defined and each includes a list of expected behaviors and vignettes that describe the entrustable learner. While every EPA applies to a resident entering EM training, EPA 10 states Recognize a patient requiring urgent or emergent care and initiate evaluation and management, which typifies a core competency of EM.

As part of the New Accreditation System, the American Board of Emergency Medicine (ABEM) and the Accreditation Council for Graduate Medical Education (ACGME) established 23 milestones that EM residents are expected to achieve to varying levels throughout their training.⁹ Milestones are knowledge, skills, and attitudes for each of the competencies such as patient care (PC), professionalism (PROF), and interpersonal and communication skills (ICS), organized in a developmental framework from less to more advanced. They describe tasks for evaluating the performance of a resident from entry into residency through graduation. Level 1 is expected of an incoming resident, Level 4 is a target for graduating residents, and Level 5 is aspirational and a target for an independent, practicing, EM physician.⁹

Currently, residency program leadership, in conjunction with the ACGME-mandated Clinical Competency Committee, determines individual resident competence with regard to milestones. However, with incoming residents, there is a significant lag of 6–12 months before they can be adequately assessed on all Level 1 milestones. To address the potential gaps and variations in the medical school training of incoming residents, most residencies employ variations of an intern orientation. While the end point of this approach is to standardize the level of the incoming residents, it is challenging to accurately gauge their beginning level of familiarity and competence and tangible deficits do exist.¹⁰

To develop discipline-specific competencies for graduating medical students, there has been a recent trend to establish specialty-specific boot camps – currently advocated primarily by surgical specialties.^11–14 The goal of these boot camps is to provide individualized learning, prepare students for entering residency, and address the variability of skills of students attending the program from different medical schools. A recent meta-analysis evaluating the literature on 15 such boot camps showed that participants had significant improvements in clinical skills, knowledge, and confidence.¹⁴

An EM boot camp may be necessary to prepare medical students who match into EM.¹⁰ It is, however, not clear as to what gaps currently exist in the undergraduate medical curricula, and therefore, what issues need to be addressed in this boot camp. Our project’s goal was therefore to perform a needs assessment to inform the design of an EM boot camp curriculum. Toward this goal, we 1) mapped the core EPAs for graduating medical students to the EM residency Level 1 milestones in order to identify the possible gaps/needs and 2) conducted a pilot procedure workshop that was designed to address some of the identified gaps/needs in procedural skills.

Since institutional variability exists, we used the hypothesis that the core EPAs outline competencies for every graduating medical student (regardless of medical school) and provide mapping to the EM milestone Level 1 competencies for an entering resident, which will help us identify some of the overall gaps. Based on a literature review, we also identified core procedural skills and areas of critical importance for the clinical practice of EM and finally, described our pilot EM procedure boot camp.^6,9,15,16

Methods

Institutional review board approval and participant consent was not necessary for this study, as the work was part of an elective curriculum. Description of the needs assessment for EM boot camp curriculum, mapping, and procedure workshop is detailed in the following section:

Needs assessment for EM boot camp curriculum

A systematic approach was used in order to identify the gaps in medical education and to identify topics that would be essential for inclusion in an EM boot camp curriculum. This approach was centered on: 1) identifying gaps between the EPAs and EM milestones (Level 1) and 2) determining what essential and supplemental competencies and skills an incoming EM resident should ideally possess. Based on this information, a pilot 1-day EM procedure boot camp was implemented to address procedural skills and to assess interest and effectiveness. A 2-week structured EM boot camp curriculum is now being conceptualized based on this needs assessment and is planned for implementation in the academic year 2016.

Mapping of EPAs to EM Level 1 milestones

The authors, with expertise in both undergraduate and graduate medical education, collectively reviewed the EPAs and EM milestones. The individual milestones were sequentially mapped to EPAs based on a review of milestones under each sub-competency (Table 1). Each EPA is addressed by at least one EM milestone, while some milestones map to multiple EPAs. However, some milestones are not addressed by any of the EPAs. Table S1 describes mapping in further detail.

Table 1 Core Entrustable Professional Activity (EPA) for Entering Residency vs ACGME milestones for emergency medicine residents6,9 Notes: An “X” at the intersection indicates that the milestone is specifically covered by that EPA. *Indicates milestones that did not map to an EPA. Abbreviations: EPA, Entrustable Professional Activity; PC, patient care; MK, medical knowledge; SBP, systems-based practice; PBLI, practice-based performance improvement; PROF, professionalism; ICS, interpersonal and communication skills; ACGME, Accreditation Council for Graduate Medical Education.

Identification of core procedural skills

We reviewed existing procedural EM milestones and identified an additional list of core procedures that all incoming first-year EM residents should have exposure to and some ability to perform with supervision – not competence. This list was further advised by existing literature (Table 2).¹⁵

Table 2 Procedures9,15

Identification of key clinical conditions requiring reinforcement

Clinical conditions in EPA 10 were reviewed and supplemented with ten emergent conditions recommended by the Clerkship Directors in Emergency Medicine (CDEM) Curriculum Revision Group.¹⁶ Author consensus determined the core list to be included in the EM boot camp curriculum (Table 3).

Table 3 Clinical conditions6,16 Abbreviation: EPA, Entrustable Professional Activity.

Pilot 1-day EM procedure boot camp

In this pilot boot camp, we focused on one aspect of the EM boot camp curriculum for graduating medical students: core procedures fundamental to EM practice. The procedure boot camp was a voluntary, noncredit full-day event. The curriculum had goals and objectives centered on the concept of advanced ABCs. A pretest and survey assessing knowledge and preparedness, confidence, and perceived competence were conducted prior to the workshop. We used a flipped classroom approach where students were expected to review procedures using posted text and videos prior to the session. In a traditional lecture method, the material is presented in the classroom and students later apply this knowledge on their own. Flipping the classroom is different since students gain exposure to material or content outside of class, usually via prereading or online videos, and use class time with a facilitator to apply that knowledge, through problem-solving and practice of skills. The hands-on practice was conducted at three stations in our Simulation Center using task trainers and animal models such as porcine tracheas and ribs (Table 4).

Table 4 Objectives and stations for procedure workshop Abbreviation: IV, intravenous.

A posttest and a psychomotor skills test were performed using three simulation cases to assess the competence of students in practicing ABC skills (Table S2 – sample checklist for intubation). A post-workshop survey assessed student perception and satisfaction (Table S3).

Results

Nine of the 15 EM-matched seniors at our institution attended the voluntary, noncredit procedure workshop. The majority of students reported their motivation as “I don’t want to suck come July” when residency starts. They held expectations of becoming more comfortable with the procedures, and did not expect competence to develop in 1 day. A post-workshop survey assessed student perception and satisfaction on six domains: meeting objectives, training environment, simulation equipment/supplies, instructor preparedness/facilitation, knowledge acquisition/application, and overall experience. All (100%) participants reported satisfaction with the overall simulation experience, felt the workshop met goals, and increased their knowledge and confidence in topics covered. Additionally, 100% of the participants felt the content covered was appropriate for their level of training. Students reported an increased confidence post-workshop in the following procedures: intubation (1.5–2.1), thoracostomy (1.1–1.9), and ultrasound-guided central venous catheterization of internal jugular vein (1.3–2) (a three-point Likert-type scale, with 1= not yet confident/able to perform with supervision to 3= confident/able to perform without supervision). Psychomotor skills testing of procedures showed that on average, 2.3 (26%) of students required verbal prompting with performance errors, 4.3 (48%) worked independently with minor performance errors, and 2.3 (26%) worked independently without performance errors. Open-ended comments included: “excellent”, “very valuable”, “just in time”, “great practice”, “timely”, “we need more training like this”, and “best day in medical school yet”. Some did report they could allocate more time for practice.

Discussion

EM-related competencies in medical school curricula may lack uniformity and standardization.¹⁶ As such, it may not be surprising to see that significant variability exists in the basic knowledge and skills of incoming EM residents. The resulting gap in expected versus actual skill sets is of concern to educators, supervising clinicians, and the public. A standardized EM boot camp may help mitigate these deficiencies by developing specialty-specific competencies. Currently, there is no existing framework for the setting up of an EM boot camp. We looked to the EPAs, EM resident milestones, and competencies for guidance to inform the design of a proposed EM boot camp curriculum.^6,9

Mapping the core EPAs to the EM milestone Level 1 competencies for the entering residents helped to: 1) determine that there likely exists a need for an EM-specific boot camp and 2) identify some of the specific gaps/needs to be addressed by the boot camp curriculum. Seven of the 23 EM-specific milestones did not map to the EPAs (Table 1). Five (PC 6, PC 10, PC 11, PC 12, and PC 13) of these seven are uniquely tied to EM practice. Since EPAs outline general competencies and are not discipline-specific, these gaps are not unexpected. Two competencies, PC 7 and PROF 2, are general and apply across disciplines, but did not map to a corresponding EPA. Admittedly, these are included in categories likely to be covered well in medical school, but there are specific issues unique to EM that fall under these domains. For example, patient disposition (PC 7) is integral to daily EM practice and therefore would need to be addressed.

Based on our mapping, it is clear that any proposed EM boot camp should, at a minimum, focus on the key gap areas related to EM, such as airway management, pain management, procedural sedation, ultrasound, and patient observation in the emergency department (ED), reassessment, and disposition. Practice and instruction – in the list of core procedures and clinical competencies – that we have identified here would also be beneficial. Finally, there is a need for reinforcement of general, but essential topics such as patient-centered communication and professionalism. For example, timely chart completion, duty hour reporting, and procedure logging (PROF 2) should be addressed in a proposed EM boot camp. A Professionalism in Residency session to address the development of the professional identity of trainees as EM physicians would be similarly beneficial.

We identified some supplemental topics based on a review of literature on surgery and EM-resident-aimed boot camps.^10–12 Topics include: advanced cardiac life support skills, medicolegal principles of ED patient care, how to place a patient on a cardiac monitor, work code cart monitors/defibrillators, and perform female genitourinary examinations.^10,11 Advanced topics on orthopedics (splinting), ophthalmology (slit-lamp), and otolaryngology are relevant and may be considered, if time permits. Similarly, high-yield topics such as sepsis, stroke, and asthma may be reviewed to help students critically understand the current status and best practices.

It is important to note that boot camps themselves may be subject to variability and inconsistencies that may limit their generalizability across programs. Therefore, we propose a core standardized EM boot camp curriculum across schools to help optimize learning outcomes.

Future perspectives

The focus areas identified based on the mapping of the core EPAs to the EM milestone, the list of conditions and procedures selected based on literature review, and the feedback obtained from the pilot procedure boot camp is guiding the design of a formal, for-credit, 2-week EM boot camp at our institution that will allow for adequate time to teach, practice, and assess skills. Since many schools cap the number of weeks a student can rotate with a specific department, we recommend that the boot camp be approved under the Office of Education or equivalent so that students are not forced to make difficult choices.

We plan a variety of educational modalities to deliver this proposed boot camp: lectures, podcasts, and videos for flipped classroom e-learning; small group discussions; procedure workshops; and simulations (manikin and standardized patients). Expected resources will include faculty/residents to facilitate, equipment, space, and a budget for supplies. Pre- and post-assessments of knowledge and skills, as well as direct observation, will assess the effectiveness of this curriculum.

Conclusion

Mapping the Core EPAs for Entering Residency to the EM milestones at Level 1 identifies educational gaps for graduating medical students. Gaps include procedural skills such as airway management, procedural sedation, pain management, care of wounds, suturing, and emergency ultrasound, as well as ED-specific skills related to patient reassessment, observation, and disposition. Educators seeking to design EM boot camps for medical students should consider the identified gaps, procedures, and clinical conditions during the development of a core standardized curriculum.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	The National Resident Matching Program. Advanced Data Table for the 2015 Main Residency Match. March 20, 2015. Available from: http://www.nrmp.org/wp-content/uploads/2015/03/ADT2015_final.pdf. Accessed April 30, 2015.
2.	Wald DA, Lin M, Manthey DE, Rogers RL, Zun LS, Christopher T. Emergency medicine in the medical school curriculum. Acad Emerg Med. 2010;17:S26–S30.
3.	Manthey DE, Coates WC, Ander DS, et al. Report of the task force on national fourth year medical student emergency medicine curriculum guide. Ann Emerg Med. 2006;47(3):e1–e7.
4.	Lyss-Lerman PM, Teherani AP, Aagaard EM, Loeser HM, Cooke MM, Harper GMM. What training is needed in the fourth year of medical school? Views of residency program directors. Acad Med. 2009;84(7):823–829.
5.	Englander R, Cameron T, Ballard AJ, Dodge J, Bull J, Aschenbrener CA. Toward a common taxonomy of competency domains for the health professions and competencies for physicians. Acad Med. 2013;88(8):1088–1094.
6.	AAMC Drafting Panel for Core Entrustable Professional Activities for Entering Residency. Core Entrustable Professional Activities for Entering Residency Curriculum Developers’ Guide. 2014. Available from: https://members.aamc.org/eweb/DynamicPage.aspx?Action=Add&ObjectKeyFrom=1A83491A-9853-4C87-86A4-F7D95601C2E2&WebCode=PubDetailAdd&DoNotSave=yes&ParentObject=CentralizedOrderEntry& ParentDataObject=Invoice%20Detail&ivd_formkey=69202792-63d7-4ba2-bf4e-a0da41270555&ivd_prc_prd_key=E3229B10-BFE7-4B35-89E7-512BBB01AE3B. Accessed February 5, 2015.
7.	ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):1176–1177.
8.	ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5(1):157–158.
9.	ACGME and ABEM. The Emergency Medicine Milestone Project. December 2012. Available from: https://www.abem.org/public/docs/default-source/migrated-documents-and-files/em-milestones.pdf?sfvrsn=4. Accessed November 2015.
10.	Santen SA, Rademacher N, Heron SL, Khandelwal S, Hauff S, Hopson L. How competent are emergency medicine interns for Level 1 milestones: who is responsible? Acad Emerg Med. 2013;20(7):736–739.
11.	Ataya R, Dasgupta R, Blanda R, Moftakhar Y, Hughes PG, Ahmed R. Emergency medicine residency boot camp curriculum: a pilot study. West J Emerg Med. 2015;16(2):356–361.
12.	ACS/APDS/ASE Resident Prep Curriculum – Goals and Objectives. American College of Surgeons Website. April 2014. Available from: https://www.facs.org/education/program/resident-prep. Accessed March 30, 2015.
13.	Selden NR, Origitano TC, Burchiel KJ, et al. A national fundamentals curriculum for neurosurgery PGY1 residents: The 2010 Society of Neurological Surgeons boot camp courses. Neurosurgery. 2012;70(4):971–981.
14.	Blackmore C, Austin J, Lopushinsky SR, Donnon T. Effects of postgraduate medical education “boot camps” on clinical skills, knowledge, and confidence: a meta-analysis. J Grad Med Educ. December 2014;6(4):643–652.
15.	Santen SA, Peterson WJ, Khandelwal S, House JB, Manthey DE, Sozener CB. Medical student milestones in emergency medicine. Acad Emerg Med. 2014;21(8):905–911.
16.	Manthey DE, Ander DS, Gordon DC, et al. Emergency medicine clerkship curriculum: an update and revision. Acad Emerg Med. 2010;17(6):638–643.

---

Supplementary materials

Table S1 Detailed mapping and comparison of Core Entrustable Professional Activity for Entering Residency vs ACGME milestones for Emergency Medicine residents6,9 Abbreviations: EPA, Entrustable Professional Activity; PC, patient care; MK, medical knowledge; SBP, systems-based practice; PBLI, practice-based performance improvement; PROF, professionalism; ICS, interpersonal and communication skills; ED, emergency department; BVM, bag-valve mask; ACGME, Accreditation Council for Graduate Medical Education; USMLE, United States Medical Licensing Examination; COMLEX, Comprehensive Osteopathic Medical Licensing Examination.

Table S2 Intubation competency evaluation Abbreviations: CPAP, continuous positive airway pressure; BIPAP, biphasic positive airway pressure; IV, intraveneous; CXR, chest X-ray.

Table S3 The advanced ABCs: a residency preparatory workshop–post-workshop survey________ Please complete this anonymous evaluation by assigning each statement a number corresponding to your opinion and placing a checkmark in the appropriate box. Your opinion matters and will assist us in developing quality instruction to improve educational and patient care outcomes. Thank you for taking the time to complete this form Comments: