The impact on internship of a tailored intern preparation package

Introduction

On commencing internship, new medical graduates (interns) are required to perform core procedural skills, but under variable levels of supervision. Unfortunately, they enter the workplace with a mixed level of knowledge and experience in performing common medical procedures,^1,2 as the opportunity for medical students to perform “hands-on” procedures is limited. There are many reasons for these limitations, including patient safety, legal constraints, senior clinicians requiring their own experience, and the scarcity of some procedural events.³

This variation in practical skills and knowledge during the first month of internship has been identified as one of the contributing factors in increased medical error,^4,5 risks to patient safety, as well as intern well-being.⁶ The major reported knowledge gaps in interns are as follows: prescribing medications;⁷ management of the acutely unwell patient;⁸ non-technical skills (including communication skills, working in a team, hospital policies, politics); and managing certain procedural skills. These are, sadly, topics not usually encountered first hand by medical students.³

Given that interns often work after hours, it is inevitable they encounter clinical situations with which they are unfamiliar and when supervision is less readily available.⁹ Therefore, it is crucial that interns have sufficient awareness of their own abilities (and limitations) to carry out a procedure, or to instead, summon a more senior clinician. Many medical schools worldwide have sought to address these well-known intern knowledge and experience gaps by introducing targeted pre-internship (PrInt) preparation courses.¹⁰

Sydney Medical School implemented a PrInt term in 2000 to facilitate the transition of their graduates into internship. This comprised “shadowing” an intern, performing procedural skills under supervision, and gaining experiential knowledge in non-technical skills, such as hospital protocols and team work.¹¹ While “shadowing” is known to be beneficial to intern preparedness,¹² the student experience is highly variable.¹³ Consequently, we saw the need to introduce an additional tailored 3 day, intensive preparation package, focused on reinforcing their knowledge and competence with core procedural skills. All students at Central Clinical School (CCS) completed this course after their final examinations, and immediately prior to their internship

In this study, we prospectively evaluated the intern preparation package by exploring interns’ self-reported knowledge, confidence, and ability in safely performing key procedural skills during the early months (4–5 months) of their subsequent internship.

Context

The participants in this study were new interns who had completed a 4-year graduate entry medical degree at Sydney Medical School. All had been attached to their “parent hospital” at CCS, at Royal Prince Alfred Hospital; all had participated in procedural skills training at CCS in years 1 and 2, and again in a year 3 orientation program prior to their clinical rotations. In addition, between 2012 and 2016, all PrInt students at CCS (n=223) participated in our new educational package. That is, an intense 3-day preparation for internship, immediately following their final barrier exams. This took place in “conference week,” a dedicated teaching time, prior to their final 4–8-week clinical attachment, known as “PrInt.” The PrInt clinical attachment was 8 weeks during 2012–2013, and 4 weeks from 2014–2016. The length of PrInt was shortened in 2014 to allow time for those who were requiring remediation to complete their PrInt term later than the remainder of the cohort. While some remained as interns at their original, “parent hospital,” inevitably, most were dispersed to regional, interstate, and international hospitals.

Educational intervention

We designed a new, 3-day intensive educational package, with a major focus on procedural skills, aimed at assisting pre-intern students in their transition to becoming safe interns. Basic procedural skills knowledge was assumed, leaving the focus on the safe application and management of procedural skills in the workplace. Issues were addressed by use of both simulation and workshops.

Teaching sessions took place in small groups, allowing for individualized practice and formative feedback, which occurred in real time with experienced clinicians. Students were presented with clinical challenges in key situations that they would be likely to encounter as new interns.¹⁴ For example, “managing” procedural skills, identifying nasogastric tube placement on X-ray, identifying and managing the deteriorating patient, clinical emergencies, and advanced communication skills. Case scenarios included routine X-ray interpretation, suturing, and effective handover utilizing ISBAR (a mnemonic created to improve safety in the transfer of critical information, ie, identify yourself, situation, background, assessment, and recommendation).¹⁵ Strategies for identifying and dealing with common clinical challenges allowed students to clarify any concerns. This style of learning encouraged reflection and integration of skills that would support their transition to real life as a junior doctor.¹⁶

Case-based teaching

We used case-based scenarios, known to be associated with errors/adverse events, such as test ordering, prescribing medications, recognition of the deteriorating patient, and accurate patient identification prior to specimen collection. Specialist clinicians, including nurses, pharmacists, and emergency physicians, delivered the teaching sessions. Additional teaching occurred while students were on their subsequent 4–8 week PrInt clinical attachments, with weekly bedside tutorials conducted by hospital clinicians.

Development of study instruments

The mixed-method study instrument was developed by use of focus groups with hospital clinicians at the CCS and was specifically tailored to elicit information of new interns’ awareness, confidence, and ability to manage core procedural skills. Following analysis of the data, we generated a 10-question survey on the commonly performed procedural skills by new interns (Supplementary materials). The scoring was on a 5 point Likert scale ranging from “strongly agree” (ie, 5) to “strongly disagree” (ie, 1).

To provide a broader understanding of intern’s early experiences in the clinical setting, each question allowed for open-ended comments. The focus group pro forma was designed to complement the survey, to elicit further opinions by using the interaction of the group to explore and clarify views.

Methods

Between 2013 and 2017, an anonymous electronic mixed-method survey and invitation to attend focus groups was sent via Survey Monkey^TM to all participants (n=223) at 4–5 months into their internship. These interns were scattered and working in the Sydney metropolitan area, rural NSW, interstate, or overseas. To simplify reporting of results, we dichotomized the responses as follows: “strongly agree” and “agree” were combined as “agree,” and “strongly disagree” and “disagree” were combined as “disagree.” We used descriptive statistics to analyze the data.¹⁷ Several responses were missing from the following questions: 1 (from questions 2, 3, 6, 7, 8 and 9), 2 responses (from questions 1, 4, and 5), and 3 from question 10.

Focus groups

After obtaining signed consent, 2 focus groups of 5 interns, and 1 of 6, (total n=16) were conducted by the first author (SM). Four interns, 2 from focus groups 1 and 3, were unable to remain for the duration due to work constraints, and all 4 subsequently returned to the clinical school for individual interviews. Data were audio recorded, transcribed verbatim, and thematically analyzed following each focus group by the first author, for identification of recurrent themes and subthemes.

Ethical considerations

The University of Sydney Human Research Ethics Committee approved this research; Approval number 2013/211. The invitation to participate in the study contained a “participant information sheet,” which included the following: submission of completed questionnaire is considered consent to participate in the study, written consent is required prior to participating in focus groups, and all data will be deidentification to protect participant privacy.

Results

Quantitative results

Figure 1 clearly shows that participants found the 3-day educational package useful to early internship.

Figure 1 Responses to the statement “The learning program is beneficial to my current practice as an intern”. Notes: Ninety-three percent of respondents agreed that the 3-day intern preparation package was useful in early internship.

Figure 2 shows the interns’ self-reported response rates for risk factors that may jeopardize patient safety. The combined results “strongly agree” and “agree” were high, ranging from 97% to 100%.

Figure 2 Intern-reported awareness of risk factors. Notes: Interns’ self-reported response rate for risk factors that could cause patient harm ranged from 97% to 100%.

Figure 3 demonstrates interns’ self-reported confidence in their ability to manage common problems involving procedural skills with the combined “strongly agree” and “agree” variable, ranging from 58% to 85%.

Figure 3 Intern-reported confidence in managing common problems involving procedural skills. Notes: The self-reported confidence in new interns’ ability to manage common problems involving procedural skills varied and ranged from 58% in identifying PICC lines on chest X-ray to 84% in identifying N/G tubes on X-ray. Abbreviations: CXR, chest X-ray; IDC, indwelling catheter; N/G, nasogastric; PICC, peripherally inserted central catheter.

Qualitative results

Respondents contextualized the open-ended questions and focus groups, and the first author analyzed and identified themes. Following negotiation of meaning with the second author, a coding framework was developed and applied to both data sets.¹⁷ Representative results from open-ended questions and focus groups produced 5 main themes. These are given in bold print and numbered, with subthemes in bold italics.

Interns reported educational benefits from the educational package

In general, interns reported favorably on the benefits of the educational package.

Hospital skills

Hospital and organizational skills workshops were reported to be particularly useful for new interns in becoming acquainted with the hospital routine.

Good for learning how a hospital works eg, consultations, ordering tests.

We were given further opportunities to reacquaint with operating equipment, run through procedures and ask questions.

Refreshing procedural skills knowledge during PrInt helped to ease anxiety and improve confidence during the early days of internship.

The course improved my confidence at the beginning of internship.

It helped to ease my anxiety when starting internship.

I was required to suture during my first term in ED [emergency department], and I was glad to be confident in this skill prior to commencing.

The timing of the preparation package

With the summative exams over, the dedicated teaching time enabled PrInt students to focus on their practical skills.

Procedural skills practice was best in PrInt – ie, better than in PrInt than in previous terms. With exams out of the way and internship looming, practical skills were my focus.

Surprised by the increased responsibility and early performance of procedural skills – from day 1

Intern autonomy increases with graded responsibility during internship. Nevertheless, interns are held legally responsible for making decisions, including performing procedures that they may have had little exposure to as medical students.

Just a few extra letters to your name and you are expected to do it.

As part of the learning experience, the intern may be called upon to review X-rays prior to appraisal by a more experienced clinician.

On my first day as a JMO [junior medical officer] I was asked to confirm the correct placement of a peripherally inserted central catheter (PICC line) on X-ray.

On day 1 I was asked to review a chest X-ray then report it back to my registrar.

X-rays/PICC lines

I still would not feel confident confirming placement of a PICC line and giving it the ok to be used.

PICC line placement and review requires further coverage in the SMP [Sydney Medical Program] program.

More teaching of this would be really useful.

I am more confident now as have had exposure as an intern.

I have had some experience since finishing the SMP in this regard.

It is in the best interest of patient safety that the junior doctor is well versed in reading X-rays so that obvious abnormalities are prioritized and not overlooked or misinterpreted.¹⁸

If you are asked to look at X- rays after hours your answer dictates what happens.

After hours you look at the X-ray first …the results do not come back with the X-ray.

Awareness of risks factors which could compromise patient safety

Fluids safety

I had to rechart fluids […] I saw the patient first and the patient had fluid overload and pitting edema […] I am cautious about blindly recharting fluids.

Prescribing errors

Interns reported prescribing as one of the most useful skills taught in the preparation course.

One of the most useful ones was pharmacy.

The safety skills that were taught during the pharmacy workshop allowed for the recognition in prescribing errors.

I have noticed prescribing errors now- ‘nit’ picking prescribing errors that I remember being told. (Intern’s own words)

Checking patients ID prior to specimen collection

While quantitative results indicated that interns checked patient identity prior to performing a procedure, 1 comment indicated omission,

In honesty, I don’t often do this properly.

Emergencies

New interns are often first on the scene to attend a medical emergency. However, with little experience as medical students, their skills to deal with an emergency may be deficient, risking patient safety and subsequent intern stress.

I can handle chest pain and SOB [shortness of breath] pretty reasonably […] Other than that, I’m very green.

This is not something I’ve had to do, and feel like I could get things started, but my first time I’d be asking for help/clarifying my plan early.

Awareness of ones’ capabilities and limitations is essential, and fundamental to safe intern performance.

I was asked to take blood from a portacath […] I had not done this before, so I got someone with more experience to supervise me.

In order to maintain patient safety and minimize student stress, procedures are initially performed on simulated models. This is most useful for familiarity with the equipment, the step-to-step protocol, and feedback, until the student is deemed safe to progress to supervised “hands-on” performance on a patient.

I have had some experience since finishing the SMP in this regard now having done this on the ward…. Nothing really sticks until you are doing it.

Manikins vs real patients

Performing skills on manikins also has perceived limitations and there was a keenness for students to perform their skills on live patients.

More hands-on. (real patients)

It’s only so helpful doing catheters on dummies etc, so just enough. Maybe stretched into the evening a little bit!

We need so much more education on suturing. Did not get the chance to do it on a real patient in the whole four years.

I am confident, but I think that is only due to practice I have had outside of formal teaching.

Early integration into and support from the clinical team

While interns work under supervision, asking for help is dependent on a number of factors including the interns’ perceived severity of the patient’s situation, the intern and supervisor relationship, and supervisor availability and proximity.¹⁹

There is always support as long as you are able to make a phone call.

However, the clinical problem could be something that they should be able to handle at their stage of training, and therefore disturbing a supervisor could risk their professional credibility; instead, peers can be a useful immediate resource.

We worked in adjacent wards and would often rely on each other for advice.

Teamwork is essential for providing safe patient and quality care. Respondents reported increased confidence after having had the support of nursing staff.

On my first shift which was […] after hours, the very experienced nurses asked me to insert an N/G [naso-gastric] tube on a patient who had had surgery and radiotherapy. ME! They supported me, and I did it […] The ‘trouble shooting’ of N/G tubes session raised my awareness.

I wasn’t confident having only performed catheterization on mannequins but have been well supported by nursing staff on the wards to complete this task and my confidence has improved.

While nurses catheterize female patients, some hospital protocols require the intern to perform catheterization on male patients.

I have only had experience in catheterizing men, as the nurses catheterize women.

I feel like a catheter queen – managing urinary retention in men.

The step-by-step process of urinary catheterization was reinforced during PrInt workshops, and this helped to facilitate students thought process.

I have had some experience since finishing the SMP in this regard and the workshops in PrInt were very helpful in thinking through the process.

Discussion

The results of this study indicate that 93% of respondents perceived the preparation for internship package to be beneficial to their practice as an intern. Interns reported enhanced knowledge of routine hospital skills, reduced anxiety when entering the ward, increased confidence in the performance of procedural skills, and a sound knowledge of their practical abilities and limitations. Overall awareness of risk factors that could compromise patient safety was universal, with 100% reporting awareness of the principles of infection control, 98% in awareness of the complications associated with cannulation, 98% in risk factors that could lead to a medication error, and 97% in cross-checking patient identity prior to taking blood (Figure 1).

Surprisingly, there was a reported omission of patient identification, a known cause of error,²⁰ defined by Vincent²¹ as, “deliberate deviations from safe operating practices, procedures, standards or rules.” Interestingly, residents prefer to greet patients by name and consider the checking of patient ID as the domain of nurses.²²

New interns are highly likely to encounter a cardiac-related emergency on the wards.²³ Despite previous intensive training in the SMP, only 59% (Figure 2) reported confidence in the initial management of a cardiac-related emergency with comments such as; “I feel like I could get things started,” which indicates that they had some confidence in the initiation of patient care. This builds on the earlier studies of Rolfe et al,²⁴ who found that interns felt that they should be able to initiate management without supervision, and Tallentire et al,²⁵ who reported that interns felt responsible solely for ensuring that the patient survived until senior help arrived.

However, only 58% felt confident in recognizing the correct placement of a PICC line on a chest X-ray (Figure 2). Open-ended comments reported lack of confidence, calling for “further coverage” and “more teaching” in the SMP. Interns were more confident in the identification of N/G tube on X-ray (Figure 2). This concern with radiology agrees with Nyhsen et al¹⁸ who reported that “radiology teaching in medical school is inadequate” and Saha et al²⁶ who found interns to be less confident with line and tube identification on X-ray.

Our results provide insight into intern adaptation to the work role during early intern training; some expressed surprise to “suddenly” be in their new role and perform procedural skills on their first day. While direct supervision allows new interns to gain skills quickly,²⁷ especially when given feedback, it is often assumed that new doctors know what they are meant to be doing. Therefore, it is debatable how much independent work interns should do,²⁸ and it has been reported that registrars consider that interns could be asked to perform “basic” procedures “that they can handle.”²⁹ This concept is also aligned to educational theory, such as Vygotsky’s concept of the zone of proximal development, whereby mentors gradually offer the newcomer lesser support.³⁰ The decision of the new intern to perform a skill is influenced by many factors including their knowledge of the procedure, the proximity and approachability of the registrar/supervisors, and whether the skill expected of them should have been mastered at their level.¹⁹

Overall, respondents reported adequate support and supervision, including members of the interprofessional team when performing procedural skills of which they were less familiar with such as indwelling urinary catheters and N/G tube insertion. Some respondents reported early signs of integration into the clinical environment, which was described by Wegner³¹ as “legitimate peripheral participation,” whereby the novice fits into the “community of practice.”

The advantages of an intern preparation course

Our data agrees with recent research indicating that preparation courses can help to even out the varied clinical experiences of students; Wayne et al³² found that those who had participated in such studies out-performed PGY-1 historical controls by P<0.01.³² In a systematic review, Blackmore et al³³ described these interventions as “an effective educational strategy to improve learners’ clinical skills, knowledge, and confidence.” The main teaching method utilized in these preparation courses is simulation, involving hands-on experience and formative feedback, allowing for what Ericsson³⁴ refers to as “deliberate practice.”

Mandatory hospital orientation

It is now mandatory that procedural skills training be provided as part of hospital intern orientation. Unfortunately, the educational content has been inconsistent,³⁵ and varies from hospital to hospital,¹⁴ leaving uncertainty surrounding the range of procedural skills taught. In consideration of patient safety, medical schools and residency programs should consider assessing competency in the performance of procedural skills to ensure new interns have mastered basic procedural skills prior to independent practice.

The importance of this study

This study highlights the benefits of preparing graduates for internship. With the reality of imminent internship, PrInt students (who have completed all their barrier assessments) are highly motivated, and more receptive to participating in learning/revising skills and applying them to the workplace. Our findings stress interns’ awareness of their procedural skills abilities and limitations, plus the importance of seeking supervision. Our study also gives insight into how often, and how soon, interns are required to perform skills and provide immediate interpretation of X-rays.

Limitations

There are several limitations to our study. While interns self-reported their confidence in their ability to perform procedural skills, self-reporting is an area of professional uncertainty.³⁶ There is strong evidence that self-assessed confidence can outweigh actual levels of performance.^37,38 Nevertheless, we were given insight into the types of skills that junior doctors were performing and how they were integrating into their clinical team.

The 41% response rate to our questionnaire was disappointing, and this was despite 3 attempts to engage the new interns. Consequently, substantial selection bias cannot be ruled out. Our response rate was slightly lower than a previous study of physicians’ response rates (54%).³⁹ The extent of further training during both the remainder of their PrInt term and subsequent intern orientation was largely unknown.

Our results add value in further appreciating the procedural skills experienced during early internship.

Recommendations

Based on the positive feedback from interns who participated in our new 3-day intern preparation package, we recommend that an intensive transition course should be implemented into the curriculum, immediately prior to commencing internship.

Conclusion

Although interns reported that their preparation course was useful to their internship, our results indicate that further training in radiology skills and managing cardiac-related emergencies on the ward is required to ensure safe patient care and to facilitate a smooth transition into postgraduate training.

Acknowledgment

We thank the new interns who have generously shared their early experiences as a junior doctor.

Author contributions

SM is a clinical lecturer at Sydney Medical School – Central, and is currently a PhD candidate. SM is responsible for the study concept, design, analysis, and interpretation of the data and drafting of the manuscript. CM is a pediatric chest physician, former associate dean and head of Central Clinical School, and is now Emeritus Professor at Sydney Medical School. CM is responsible for the analysis and interpretation of data and critical revision of the manuscript for important intellectual content. Both authors contributed toward data analysis, drafting and critically revising the paper, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Boots RJ, Egerton W, McKeering H, Winter H. They just don’t get enough! Variable intern experience in bedside procedural skills. Inter Med J. 2009;39(4):222–227.
2.		Hannon FB. A national medical education needs’ assessment of interns and the development of an intern education and training programme. Med Educ. 2000;34(4):275–284.
3.		Burford B, Whittle V, Vance GHS. The relationship between medical student learning opportunities and preparedness for practice: a questionnaire study. BMC Med Educ. 2014;14:223.
4.		Haller G, Myles PS, Taffé P, Perneger TV, Wu CL. Rate of undesirable events at beginning of academic year: retrospective cohort study. BMJ. 2009;339:b3974.
5.		Jen MH, Bottle A, Majeed A, Bell D, Aylin P. Early in-hospital mortality following trainee doctors’ first day at work. PLoS One. 2009;4(9):e7103.
6.		Soares DS, Chan L. Stress and wellbeing of junior doctors in Australia: a comparison with American doctors and population norms. BMC Med Educ. 2016;16:183.
7.		Hilmer SN, Seale JP, Le Couteur DG, Crampton R, Liddle C. Do medical courses adequately prepare interns for safe and effective prescribing in New South Wales public hospitals? Intern Med J. 2009;39(7):428–434.
8.		Kelly C, Noonan CLF, Monagle JP. Preparedness for internship: a survey of new interns in a large Victorian Health Service. Aust Health Rev. 2011;35(2):146–151.
9.		Laack TA, Newman JS, Goyal DG, Torsher LC. A 1-week simulated internship course helps prepare medical students for transition to residency. Simul Healthc. 2010;5(3):127–132.
10.		Teunissen PW, Westerman M. Opportunity or threat: the ambiguity of the consequences of transitions in medical education. Med Educ. 2011;45(1):51–59.
11.		Wenger E. Communities of Practice: Learning, Meaning, and Identity. Cambridge, UK: Cambridge University Press; 1998.
12.		Jones A, McArdle PJ, O’Neill PA. Perceptions of how well graduates are prepared for the role of pre-registration house officer: a comparison of outcomes from traditional and an integrated PBL curriculum. Med Educ. 2002;36(1):16–25.
13.		Berridge EJ, Freeth D, Sharpe J, Roberts M. Bridging the gap: supporting the transition from medical student to practising doctor – a two-week preparation programme after graduation. Med Teach. 2007;29(2–3):119–127.
14.		Graham IS, Gleason AJ, Keogh GW, et al. Australian Curriculum Framework for Junior Doctors. Med J Aust. 2007;186(Suppl 7):2.
15.		Moore M, Roberts C, Newbury J, Crossley J. Am I getting an accurate picture: a tool to assess clinical handover in remote settings? BMC Med Educ. 2017;17(1):213.
16.		Brennan N, Corrigan O, Allard J, et al. The transition from medical student to junior doctor: today’s experiences of Tomorrow’s Doctors. Med Educ. 2010;44(5):449–458.
17.		Creswell JW. Educational Research: Planning, Conducting, and Evaluating Quantitative. Upper Saddle River, NJ: Pearson Education, Inc. 2002.
18.		Nyhsen CM, Lawson C, Higginson J. Radiology teaching for junior doctors: their expectations, preferences and suggestions for improvement. Insights Imaging. 2011;2(3):261–266.
19.		Kennedy TJ, Regehr G, Baker GR, Lingard L. Preserving professional credibility: grounded theory study of medical trainees’ requests for clinical support. BMJ. 2009;338:b128.
20.		Chassin MR, Becher EC. The wrong patient. Ann Intern Med. 2002;136(11):826–833.
21.		Vincent C. Patient Safety. Edinburgh: Churchill Livingstone; 2006.
22.		Phipps E, Turkel M, Mackenzie ER, Urrea C. He thought the “lady in the door” was the “lady in the window”: a qualitative study of patient identification practices. Jt Comm J Qual Patient Saf. 2012;38(3):127–134.
23.		Callaghan A, Kinsman L, Cooper S, Radomski N. The factors that influence junior doctors’ capacity to recognise, respond and manage patient deterioration in an acute ward setting: an integrative review. Aust Crit Care. 2017; 30(4):197–209.
24.		Rolfe IE, Pearson SA, Sanson-Fisher RW, et al. Which common clinical conditions should medical students be able to manage by graduation? A perspective from Australian interns. Med Teach. 2002;24(1):16–22.
25.		Tallentire VR, Smith SE, Skinner J, Cameron HS. Understanding the behaviour of newly qualified doctors in acute care contexts. Med Educ. 2011;45(10):995–1005.
26.		Saha A, Roland RA, Hartman MS, Daffner RH. Radiology medical student education: an outcome-based survey of PGY-1 residents. Acad Radiol. 2013;20(3):284–289.
27.		Kilminster S, Cottrell D, Grant J, Jolly B. AMEE Guide No. 27: effective educational and clinical supervision. Med Teach. 2007;29(1):2–19.
28.		Rolfe I, Pearson S, Sanson-Fisher R, et al. Which common clinical conditions should medical students be able to manage by graduation? A perspective from Australian interns. Med Teach. 2002;24(1):16–22.
29.		Dally P, Ewan C, Pitney WR. Assessment of an Australian medical internship. Med Educ. 1984;18(3):181–186.
30.		Chaiklin S. Vygotsky’s Educational Theory in Cultural Context. In: Kozulin A, Gindis B, Ageyev VS, Miller SM, editors. Bath, UK: Cambridge University Press; 2003.
31.		Wenger E. Communities of practice and social learning systems: the career of a concept. In: Blackmore C, editor. Social Learning Systems and Communities of Practice. London: Springer; 2010:179–198.
32.		Wayne DB, Cohen ER, Singer BD, et al. Progress toward improving medical school graduates’ skills via a “boot camp” curriculum. Simul Healthc. 2014;9(1):33–39.
33.		Blackmore C, Austin J, Lopushinsky SR, Donnon T. Effects of postgraduate medical education “Boot Camps” on clinical skills, knowledge, and confidence: a meta-analysis. J Grad Med Educ. 2014;6(4):643–652.
34.		Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(Suppl 10):S70–S81.
35.		McKenzie S, Burgess A, Mellis C. Interns reflect: the effect of formative assessment with feedback during pre-internship. Adv Med Educ Pract. 2017;8:51–56.
36.		Eva KW, Regehr G. Knowing when to look it up: a new conception of self-assessment ability. Acad Med. 2007;82(Suppl 10):S81–S84.
37.		Barnsley L, Lyon P, Ralston S, et al. Clinical skills of junior doctors: a comparison of self reported confidence and observed competence. Med Educ. 2004;38(4):358–367.
38.		Wall D, Bolshaw A, Carolan J. From undergraduate medical education to pre-registration house officer year: how prepared are students? Med Teach. 2006;28(5):435–439.
39.		Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clinical Epidemiol. 1997;50(10):1129–1136.

Supplementary material

Figure S1 Questionnaire sent to interns via anonymous survey. Note: Interns were asked to select from a dropdown menu and invited to comment. Abbreviations: CXR, chest X-ray; PrInt, pre-internship.