Internal medicine residents’ point-of-care ultrasound skills and need assessment and the role of medical school training

Introduction

Point-of-care-ultrasound (POCUS) is the use of ultrasound at the bedside by the treating clinician to look for specific findings that help to answer specific clinical questions.^1–4 Many see it as an extension of physical examination.^5–9 The use of POCUS is expanding beyond the emergency room, obstetrical examination, and intensive care units. It is emerging as a useful tool for specialties like anesthesiology, physical medicine and rehabilitation, rheumatology, nephrology, and emergency medical services.^10–16 The utility of ultrasound to enhance success rate and safety of medical procedures is well known.^17–19 Nowadays, there is also a mounting body of evidence to support its effectiveness in general internal medicine and hospital medicine practices.^7,20–27

Given its growing applications, POCUS has become more prevalent than ever in the context of internal medicine residents beginning their training in recent times. How residents perceive the rise of POCUS as a new skill to learn during residency is not well known. Our internal medicine residency program acquired 4 handheld ultrasound devices (GE Vscan^® with dual probe) in 2016 for inpatient use. Faculty hospitalists who use POCUS in their practice noticed that residents were not using the available devices as often as faculty thought they would. Therefore, we perceived that our residents need a longitudinal curriculum in POCUS to address their concerns and to assess their skills. No formal training or longitudinal curriculum in POCUS was introduced to our residents apart from the availability of ultrasound devices in inpatient wards and intensive care unit, the intermittent interaction with faculty who are interested in POCUS during these two rotations, and the free access to a website with POCUS video modules (EMsono^®).

Few studies have examined residents’ skills and attitude towards POCUS training,^28–38 but to the best of our knowledge, no prior study has compared POCUS image interpretation skills and self-reported confidence between residents with and without POCUS training in medical school. The goal of this study was to assess our residents’ attitude, basic needs, and prior knowledge about POCUS and whether POCUS training and exposure during medical school influences their POCUS image interpretation skills and confidence level. We also wanted to compare the results of their self-reported confidence level to their actual score in a POCUS image interpretation test. This study will be part of a formal future longitudinal curriculum to train our residents on cardiovascular and lung POCUS.

Material and methods

We distributed a 24-question, anonymous, and voluntary electronic survey (using Qualtrics, LLC). A secured email was sent to residents as a group with voluntary participation encouraged. Completion of the survey was deemed to be informed consent. Respondents self-reported their attitude towards POCUS and their confidence in recognizing specific lung and cardiovascular POCUS images (using a Likert scale where 5=extremely confident, 4=mostly confident, 3=not sure, 2=mostly not confident, and 1=not confident at all). In addition to this survey, we also conducted an anonymous, voluntary, and timed image interpretation test to examine our residents’ skills in interpreting normal and abnormal common cardiovascular and lung POCUS images (total of 15 videos: 6 lung exams, 6 cardiac exams, 1 inferior vena cava [IVC] exam, 1 right internal jugular [RIJ] vein exam for jugular venous distension (JVD), and 1 lower extremity deep venous thrombosis [DVT] exam). The image interpretation test was a mixture of fill-in-the-blank and multiple-choice questions where the residents were shown 30 seconds video clips of ultrasound findings each with a question that included brief clinical scenario and the type of the exam. A typical example of the format of questions is shown in Media (Video S1). The test was reviewed by 3 faculty hospitalists who use POCUS in their practice to make sure that images are optimum in quality and interpretation. Videos were mostly recorded by the senior author, but some were also obtained from the book “Point-of-Care Ultrasound,” first edition.³⁹ Using descriptive statistics, we compared the results of the test to the residents’ self-reported confidence in their ability to perform and recognize these specific images. Using t-tests, we compared mean confidence levels and test scores between residents with and without POCUS training in medical school. Analysis was done using IBM^® SPSS Statistics software, version 25. The study was approved by the Community Regional Medical Center institutional review board.

Results

Fifty-two residents (total 74) took the survey (one resident did not fully complete the survey). The overall response rate was 68% (67% [22/33] PGY-1, 77% [17/22] PGY-2, and 58% [11/19] PGY-3). Thirteen residents (25% of respondents) indicated having some form of POCUS training in medical school (mostly lectures and hands-on experience, but also two-week elective, simulation, and exams). Residents with prior POCUS training (n=13) were either PGY-1 (9, but one did not complete the survey) or PGY-2 (4). No PGY-3 resident had prior training (Figure 1). When asked about their interest in learning POCUS during residency, 46 residents (90% overall; 86% [20/23] PGY-1, 88% [15/17] PGY-2, 100% [11/11] PGY-3) reported being extremely or mostly interested. Most residents (83%) thought that POCUS can be extremely useful in the inpatient setting versus 29% for the outpatient setting (Figure 2). About 94% of the residents felt that dedicated workshops to train them on POCUS would be extremely or most likely to be helpful in learning POCUS skills. Other opportunities residents felt can be helpful as well include one-to-one training during the Procedure Service rotation, online videos, and bedside use of ultrasound by attending physicians (ultrasound rounds). The POCUS exam skills that residents thought would be most useful to learn during residency were the cardiac exam (15.5% of respondents), IVC exam (14.8%), lung exam (13.8%), and RIJ exam for JVD (12.7%) (Figure 3). Only 1 resident mentioned that he/she knows all the keys or functions of the Vscan^® devices and 10 said they know most of them. When it comes to the knowledge about which ultrasound probe to use for which exam, most residents (63%) felt extremely or somewhat confident about choosing the right probe for the right exam.

Figure 1 Residents with and without prior POCUS training in medical school.Abbreviations: POCUS, point-of-care ultrasound; PGY, postgraduate year.

Figure 2 Residents’ perception about the usefulness of POCUS in inpatient vs outpatient setting.Abbreviation: POCUS, point-of-care ultrasound.

Figure 3 Residents’ response when asked about what POCUS exams they think are most useful to learn during residency. Abbreviations: POCUS, Point-of-care ultrasound; RIJ, Right internal jugular; JVD, Jugular venous distension; DVT, Deep venous thrombosis; MSK, Musculoskeletal.

Residents were also assessed for how confident they feel about recognizing some common pulmonary, cardiac, and vascular POCUS exams in clinical practice (Table 1). Residents rated their confidence level on scale of 1 to 5, with 1=not confident at all, 2=mostly not confident, 3=not sure, 4=mostly confident, and 5=extremely confident. PGY-1 residents with prior training had higher mean confidence level than PGY-1 without prior training, but the difference was not statistically significant (3.26 vs 2.64; p=0.08). PGY-1 residents with prior training had a mean confidence level that was close to that of PGY-3 residents, none of whom had prior training (Figure 4).

Table 1 Mean confidence levels reported by all respondents (n=52) in detecting and recognizing common POCUS applications; highest to lowest (5= extremely confident; 1= not confident at all)

Figure 4 Residents’ overall mean confidence levels in detecting and recognizing common POCUS applications (5=extremely confident; 1=not confident at all) by PGY level and prior POCUS training in medical school. *n=8 due to 1 survey not fully completed by a PGY-1.Abbreviations: POCUS, point-of-care ultrasound; PGY, postgraduate year.

Fifty-nine residents (response rate 77%) took the POCUS image interpretation test. The most correctly answered image interpretation questions are shown in Table 2. PGY-1 residents with prior training scored significantly higher than PGY-1 residents without prior training (10.25 vs 7; p=0.01) (Figure 5). Residents felt most confident in interpreting IVC images (mean 3.7), and the same exam had the highest score in the image interpretation test with a correct response of 88% (in addition to identifying small-to-moderate pericardial effusion).

Table 2 Number of residents who correctly answered each POCUS exam video question (total n=59)

Figure 5 The mean score for the image interpretation test by PGY level and prior POCUS training in medical school (maximum possible score=17).Abbreviations: POCUS, point-of-care ultrasound; PGY, postgraduate year.

Discussion

The results of this study were very helpful in planning the POCUS curriculum for our residents. It created a good platform for faculty to know which POCUS exams residents are more interested to learn and what are their baseline skills and perceptions regarding cardiovascular and lung POCUS training. Similar to other studies,^28,36 we also learned that the vast majority of our residents are either extremely or mostly interested to learn POCUS during residency. Most of them also were extremely or mostly supportive of workshops as one method to learn POCUS skills, and so arrangement will be made to have workshops as part of the curriculum.

Although our residents currently have no or minimal exposure to ultrasound training in primary care clinics, most of them still think that POCUS can be either extremely or mostly useful in outpatient practice. Not surprising, though, more residents agreed that it is more useful in the inpatient setting. This can be explained by the fact that our residents use ultrasound more during inpatient than the outpatient service, probably because of the availability of the 4 hand-held devices in inpatient rotation and most faculty who use POCUS are in the inpatient practice. In line of what our faculty hospitalists who use POCUS believe, our residents chose cardiac, inferior vena cava, and lung exams as the top three exams they think are most useful to learn during residency. Knowing this assures more participation of residents in our workshops and curriculum which will start with these three systems.

Although all residents have access to the hand-held ultrasound devices during medical wards, most of them reported that they either know some or very few of the keys and functions of the device. Non-familiarity with the available ultrasound devices can obviously be a barrier to learn POCUS. Therefore, this is an area that needs to be particularly addressed during the workshops. When it comes to the use of the probes, most residents feel confident that they know which probe to use for which exam.

The comparison between the residents’ self-reported level of confidence in image interpretation and their actual performance in the image interpretation test was instructive. It gave an idea about areas in cardiovascular and lung POCUS exams that might need more attention during the course of the POCUS curriculum. For example, the highest level of self-reported confidence in image recognition and interpretation was reported for examination of IVC diameter and collapsibility, which is one of the common applications of POCUS. In the image interpretation test, 88% of residents were able to recognize a small and collapsible IVC. This suggests that IVC exam is probably the most common exam performed by our residents and most of them might not need much help in that area, although it will still be included in the workshop to help them refine their image acquisition skills. The next highest self-reported level of confidence was reported for recognizing pleural effusion in lung exam. When shown an image of small to moderate effusion at the lung base, 67% were able to recognize the presence of the effusion. But when shown an image of normal lung base with no pleural effusion, only about one third of residents were able to recognize the absence of effusion in that image. This suggest that our residents might be more familiar with appreciating pleural effusion than recognizing the normal pattern that they might see in patients without effusion, an area that can be practiced more during the workshops with healthy live models. Also, recognizing DVT was amongst the areas that only few residents (19%) reported to be extremely or mostly confident to recognize, but when shown an image of a positive exam by compressibility criteria, 78% were able to recognize the positive exam and report the presence of DVT. This might suggest that, when it comes to lower extremity exam for DVT, our residents probably need more training on image acquisition skills more than image interpretation using the compressibility criteria. Interestingly, residents’ low confidence levels for DVT exam was also reported in other study in Canada.²⁹ Image acquisition generally seems to be more challenging than image interpretation.⁷

One interesting finding was the fact that PGY-1 residents were the most among our residents who had some form of formal or informal ultrasound training in medical school, followed by PGY-2, with no PGY-3 resident reported having any prior training. This supports the notion that more and more medical schools now are introducing at least basic ultrasound training opportunities for their students, an indicator for the growth of POCUS training. Therefore, faculty are likely to encounter more and more medical students getting into internal medicine residency with at least basic knowledge of bedside ultrasound skills. These future residents will expect their faculty to help them acquire more advanced POCUS skills during their training, and hence, faculty should be encouraged to seek development of their own POCUS skills. The subset of PGY-1 residents who had POCUS training during medical school had higher self-reported confidence in lung and cardiovascular POCUS skills and they outperformed the PGY-1 residents without prior training in the image interpretation test as well. PGY-1 residents’ confidence level and performance were comparable to that of PGY-3 residents. This is encouraging for medical schools who provide POCUS training for their students. Ultrasound skills are largely operator-dependent so these residents are expected to develop their POCUS skills even more during the course of the curriculum, an assumption that can be examined by repeating the survey and test at the end of the curriculum. Internal medicine residency programs who are interested to build POCUS curriculum can consider involving residents with prior POCUS training in medical school to assist with curriculum development since they seem to have more confidence in their skills and likely to perform better in image interpretation than those without prior training.

This study has several limitations. Although our program is considered large by number of residents, this study still reflects the attitudes and baseline ultrasound skills of residents in only one internal medicine training program.

Moreover, the survey was anonymous and so when we repeat the survey and exam at the end of the curriculum, we will not be able to compare the results of resident’s self-reported confidence level and image interpretation skills individually, but we should be able to do so for the whole group. The survey and test only examined residents’ POCUS skills in cardiovascular and lung systems. Nevertheless, these are the exams that seem to attract the most interest of the residents. Our goal is to introduce more systems as residents’ skills advance.

Conclusion

Our internal medicine residents seem quite interested in acquiring and developing POCUS skills during residency, especially cardiovascular and lung examinations. PGY-1 residents with prior POCUS training in medical school seem to have higher confidence in their POCUS skills than PGY-1 residents without prior training and outperformed them in image interpretation test. The study was very instructive in building our future POCUS curriculum for our residents. Standardization of POCUS training for internal medicine residency programs is still evolving. Other programs including Obstetrics and Gynecology, Diagnostic Radiology, and Emergency Medicine have ACGME (Accreditation on Council for Graduate Medical Education) program requirements for graduate medical education on utilizing ultrasonography. Given the many applications of ultrasonography in Internal medicine, programs interested in developing POCUS curriculum can start with similar data gathering to better understand their residents’ baseline knowledge and attitude towards POCUS skills and training.