Evaluation of Interdisciplinary Emergency Ultrasound Workshop for Primary Care Physicians in Nepal

Introduction

Point-of-care ultrasound (POCUS) is a useful bedside imaging technique to evaluate patients in the emergency department (ED)¹ and primary care setting. POCUS is quick and focused tool performed for a specific condition that adds significant value to clinical decision-making and positively influences patient outcomes.^2,3 The benefit of POCUS as the first imaging technique is acknowledged worldwide and has also been established in various studies.^1,4 According to the World Health Organization (WHO) estimates, 60% of the world’s population does not have access to means of tissue imaging like basic x-ray, computed tomography scanners in their local health centers.^1,5 Therefore, ultrasonography (USG) has immense utility and potential for developing emergency health services in low- and middle-income countries (LMICs) due to its relatively low cost and accessibility Several educational initiatives in clinical ultrasound have been reported in these parts of the world.^6–10

Standard USG or echocardiography requires the treating physician in the ED to rely on the radiologist or cardiologist to perform it in a timely and clinically relevant manner.¹¹ Provided the physicians in the ED have skill at image acquisition, image interpretation, and the cognitive elements required for using POCUS, they are able to rapidly incorporate the results of the POCUS examination into the management plan without delay. In addition to this, the radiology or cardiology consultant may not be fully aware of the clinical facts of the case. The other benefits of POCUS include low cost, availability, portability and safety. However, the proficiency of the physicians performing the scan can be highly variable according to their knowledge and experience.¹²

Training programs have been developed and endorsed by various international committees like European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB), International Federation for Emergency Medicine (IFEM), American College of Emergency Physicians (ACEP), World Interactive Network Focused On Critical Ultrasound (WINFOCUS) and other international Guidelines).^13–16 Comprehensive training in USG is currently a mandatory part of emergency medicine (EM) training in developed countries. On the contrary, there is limited effort to disseminate this knowledge and skills in LMICs.⁶ In Nepal, EM is in the preliminary phase and several EM training modules are currently practiced fragmented with different curriculum and duration.¹⁷ Most of the emergency departments and primary health care centers are staffed by general practitioners (GP) and medical officers. No studies were found related to educational intervention to enhance POCUS in the ED and primary care in Nepal. There are limited publications in regard to the use of POCUS in the ED in our setting.^18–20

Dhulikhel Hospital (DH) is a community based not for profit academic institution with high priority to rural health services through its 21 outreach centers. We recognized the need for training in POCUS for our primary care physicians. We conducted this workshop in collaboration with the three German-speaking interdisciplinary ultrasound societies: (DEGUM, SGUM and the Ultrasound Society of Austria). This collaboration focuses its effort in enhancing and disseminating the use of USG in clinical practice in ED and primary care setting. A two-day advanced emergency ultrasound workshop was designed and implemented focusing these frontline physicians working in Emergency Departments in Nepal. The aim of our study was to explore the effectiveness of interdisciplinary emergency ultrasound course. The objectives of our study were to (1) compare the perceived confidence level and perceived usefulness of the participants to perform POCUS before and after the training, (2) explore the views of the participants regarding the clinical utility of POCUS, as well as barriers and facilitators to its implementation, (3) compare the knowledge acquisition by comparing the written pretest and posttest of the participants, and (4) assess the effect of the training on behavior and performance of the participants after the training.

Materials and Methods

Study Design

This was a prospective quasi-experimental study with mixed methods. We conducted an on-site and four-week follow-up evaluation of healthcare providers who participated in the ultrasound training program in DH.

Study Site

The workshop was conducted in the ultrasound training center, Dhulikhel Hospital-Kathmandu University Hospital. Department of General Practice and Emergency Medicine led the workshop. The department has approximately 20,000 visits annually with high acuity level. Trauma patients comprises of approximately 40% of total patients.

Study Population

The workshop was designed to address the needs of primary care physicians working in the emergency departments of urban and rural health care centers of Nepal. We invited multidisciplinary physicians working in the ED and primary care facilities with priority given to those working in various rural health care facilities all over Nepal (Figure 1). The participation was voluntary with online registration. The pre and the pretest were the prerequisite for the workshop. The feedback collected onsite and at a four-week follow up were voluntary.

Figure 1 Location map of participating health care facilities.

Course Content and Design

We conducted a two-day workshop that comprised of didactic and small group hands-on sessions on November 19–20, 2019. The first pilot workshop was conducted on December 5–6, 2018. The didactic sessions elaborated on the fundamental knowledge on the topics followed by relevant hands-on sessions using the predesigned structured checklist (Supplement 1). Each hands-on practical station was practiced in a volunteer and locally made gelatin-based models (Figure 2–5).

Figure 2 Instructor demonstrating foreign body removal in a model.

Figure 3 Participant orientation about the USG probe use before the hands-on sessions.

Figure 4 Participants practicing RUSH protocol in a volunteer participant.

Figure 5 Participants practicing USG guided intervention in locally made gelatin-based model.

The core topics of POCUS were included in the workshop, namely; basic physics and knobology of ultrasound, extended focused assessment with sonography for trauma (E-FAST), focused cardiac ultrasound (FOCUS), lung ultrasound (LUS), focused abdominal ultrasound, focused obstetrical and gynecological ultrasound, rapid ultrasound for shock and hypotension (RUSH) protocol, USG guided vascular and other interventions and focused musculoskeletal (MSK) ultrasound.

Study Procedure and Data Collection Tools

The participants filled the pre-designed pre-workshop knowledge test (Supplement 2) before the workshop. They also documented their specialty, work experience and access to USG machine. The instructors (n=5) designed the questionnaire for the training which was peer-validated and pilot tested among six physicians using POCUS in the ED of the DH. It consists of 25 questions based on pictures and videos. Needful amendments were made and the questionnaire was used for the workshop via Google form. During the workshop, participants were evaluated for the skills using predesigned structured checklist (Supplement 1) during the hands-on sessions. The posttest was conducted after the workshop and was mandatory.

The onsite feedback from the participants was voluntary and anonymous (Supplement 3). This was an online survey questionnaire designed by the authors and peer validated. This explored their opinion on the relevance, content and the delivery of didactic and hands-on sessions in Likert scale of 5. Retrospective pre-post perceived confidence level and usefulness of the POCUS for various domains were recorded by using Likert scale of 5. The participants also responded to the open-ended questions as a feedback to the workshop for future improvement.

The online follow-up feedback (Supplement 4) was sent to the participants 4 weeks after the workshop. During this follow-up evaluation, the participants from the pilot workshop (December 2018) were also requested to complete the follow-up survey. This questionnaire inquired the pre- and post-workshop use of USG in their clinical work. They were also requested to share the barriers to use the USG in their practice. The details of the participants were kept anonymous.

Evaluation of the Workshop

We attempted to explore the effectiveness of this training based on validated Kirkpatrick’s 4 steps of evaluating training outcomes. Firstly, we assessed participants’ “reaction” by evaluating the feedback from the participants after the training which reflected their perceived change in the level of confidence and usefulness of the course. Open-ended questions regarding the clinical utility of POCUS, as well as barriers and facilitators of its implementation were asked to the participants. As a second step, we assessed the participants’ “learning” by comparing the written pre- and post-knowledge scores. Thirdly, we assessed the effect on the participants’ “behavior” by a follow-up survey at 4 weeks after the training by comparing the number of scans performed before and after the course by the participants. The participants from DH ED were longitudinally supervised by the GP faculties of the DH. Finally, the fourth stage, the “results,” will be assessed in the second phase by conducting future research in terms of quality improvement and improved patient’s diagnosis and reduced cost in the ED and rural centers.

Data Analysis

The data from the Google forms were downloaded in the excel spreadsheet and data analysis was done using SPSS version 21.

The categorical values were expressed as numbers with proportions and the continuous numeric value with mean with standard deviation (SD) or median with interquartile range (IQR). The pre-and the post-test scores were expressed as medians with IQR and compared with Wilcoxon signed rank test. The items rated using Likert scale were analyzed as ordinal data and expressed as median and compared using Wilcoxon signed rank test. The Kruskal–Wallis test was used to compare the profession and the work experience with the test scores.

Results

Participants

A total of 50 physicians participated in the workshops (piloted in December 2018 and continued as the first study workshop in November 2019). During the November workshop, 25 physicians participated (11 (44%) medical officers, 10 (40%) GP consultants and 4(16%) residents). Ten (40%) participants had up to 2 years of clinical experience, 7 (28%) and 8 (32%) of them had clinical experience with a range of 2–4 and more than 4 years, respectively.

The Pre- and Post-Workshop Knowledge Score

The median (IQR) pre- and posttest knowledge scores were 14 (12.75–17.75) and 24.5 (22.25–25.5) respectively, and the difference was statistically significant using Wilcoxon signed rank test (p<0.001) (Figure 6). The academic level of the participants, duration of their clinical experience and the previous use of USG were compared with the pretest and posttest knowledge score (Table 1).

Table 1 The Pre- and Posttest Knowledge Score in Relation to the Academic Level, Clinical Experience and Previous Use of USG (n=25)

Figure 6 Pre- and post-workshop knowledge scores (n=25, p<0.001, Wilcoxon signed rank test p<0.001).

On-Site Feedback

Twenty-two (88%) participants responded to the on-site feedback questionnaire. Twenty-one (96%) participants were satisfied with the workshop and thought that the workshop was relevant to practice. Twenty (92%) participants thought that the selection of topics was appropriate. The perceived confidence level and perceived usefulness of the participants to perform POCUS before and after the training are illustrated in Tables 2 and 3. Compared with the pre-workshop evaluation, there was a significant improvement in perceived confidence following the workshop (p <0.001). Participants were least confident in performing FOCUS, RUSH, scrotum, vascular and MSK before the workshop. The difference in the perceived usefulness of USG was significant for each domain using the Wilcoxon signed rank test (p<0.001).

Table 2 The Pre- and Post-Workshop Confidence Level of the Participants (n=22)

Table 3 The Pre- and Post-Workshop Perceived Usefulness of the USG (n=22)

All of the participants agreed that they would like to attend such workshops in future. When the participants’ choices for future courses were analyzed, the topics enlisted were advanced echocardiogram, ocular ultrasound, USG in pregnancy, USG guided nerve blocks and central line insertion.

Follow-Up Feedback

The follow-up feedback was collected from both the pilot and study workshop (n=50). Forty (80%) participants responded to the follow-up questionnaire. The change in frequency of the pre- and post-workshop use of USG in their clinical work and different domains is depicted in Figures 7 and 8. The participants reported that the barriers to use of USG in current clinical practice were unavailability of USG, old USG machine, lack of knowledge and/or skills and time constraints.

Figure 7 Pre- and post-workshop frequency of USG use in routine clinical practice of participants (n=40, Wilcoxon signed rank test p<0.001).

Figure 8 Pre- and post-workshop self-reported USG use in various domains of clinical practice in Likert scale of 5 (1 being never and 5 being frequently) n=40 (Wilcoxon signed rank test p<0.001). Abbreviations: EFAST, extended focused assessment with sonography for trauma; FOCUS, focused cardiac ultrasonography; LUS, lung ultrasonography; RUSH, rapid ultrasound for shock and hypotension; US, ultrasonography; Vasc, vascular; MSK, musculoskeletal.

Discussion

This study demonstrated improved knowledge among the participants before and after the workshop. The change in pre- and post-workshop knowledge scores was statistically significant irrespective of the academic level of the participants, duration of their clinical experience and the previous use of USG. A study that implemented MSK POCUS curriculum showed that the written test scores improved significantly from 50.7 ± 17.0% before to 84.0 ± 10.7% immediately after the course (p < 0.001) and suffered no significant drop at 6 months (score 75.0 ± 8.7%; p = 0.303).²¹

Our study showed a significant increase in the self-reported frequency of use of USG in routine clinical practice. This increment was significant in all domains introduced in the workshop. Another study also demonstrated that the proportion of participants who self-reported using POCUS increased significantly between pre-survey and post-post survey (29.7% to 63.2%, P=0.0161).²² In contrast, the study done by Larrivee et al showed that residents significantly improved their subjective comfort level with all aspects of USG use at 6 months (p = 0.007–0.018) but did not significantly increase clinical usage frequency.²¹

The self-reported confidence level increased significantly in all areas of POCUS after the workshop. Similar results were found in health care workers in Cusco, Peru, where they reported increased confidence in their FAST scan ability and in their comfort in using this exam for clinical decision-making after a training session focusing on the use of USG in trauma.²³

Lack of access to machines and old machine, limited time and inadequate knowledge and skills were the substantial barriers to increasing POCUS use among the responders in our study. A survey done by Shah et al also found a lack of training and the equipment cost to be the primary barrier to the use of USG.²⁴ Another study²⁵ showed that the most significant barriers to POCUS documentation were time constraints, the existence of a separate computer program for POCUS documentation, and problems with the USG machines. One significant barrier according to a pilot study²⁶ relates to the need for dedicated time to learn and practice POCUS to achieve sufficient levels of proficiency for use in practice. In our context, with the help of the collaborators, addressing the barrier, DH is able to equip all the outreach centers with basic USG facility. Likewise, basic and other advanced USG training workshops have been conducted on a regular basis in order to close the gap in knowledge and skills dissemination. Though time constraints have been reported as one of the perceived barriers, prior studies have shown that use of POCUS at ED not only improves diagnostic accuracy but also decreases time to consultation and length of stay in the emergency department.^27,28

A study done in the ED at DH²⁹ found that 8.9% of patients presenting to ED at DH had dyspnea as their primary reason for attending ED among which the respiratory with other mixed causes were 22.4%. The LUS and the FOCUS were one of the highlights of this workshop. Participants also reported them to be useful in clinical practice. Future studies can be done highlighting the role of POCUS in undifferentiated dyspnea in the ED. Another area of interest where USG has revolutionized the diagnosis was MSK disorders. These are just two of the numerous other potential conditions where POCUS can be a quick and cheap modality for rapid clinical decision-making.

The number of participants in our study was limited. Long term follow-up for the retention of knowledge/skills, patient outcome measures, and cost-effectiveness was not measured in this phase. Future long term follow-up is planned. However, our study may serve as a basic needs assessment to an implementation of an EM POCUS training curriculum. Further, follow-up study can be tailored to address the clinical performance of the study participants longitudinally. As the participants are from different remote health care institutions, in-person follow-up trainings may not be possible. Tele-sonography and distance learning can be a cost-effective strategy to enhance USG use in low-resource settings like ours. Previous study has shown that the remotely delivered quality assurance feedback is an effective educational tool to enhance provider skills and foster continued and sustainable use of USG in LMICs.⁸ Similar strategy to support and provide feedback to the participants would be the next project.

Conclusion

Implementation of a structured emergency ultrasound course resulted in a trend towards improved trainee confidence, perceived usefulness, and clinical use of POCUS. The overall knowledge of the participants also increased after the workshop. Future research should focus on the objective longitudinal follow-up of participants’ skills as well as incorporation of POCUS into participants’ clinical practice.

This study will prime future research in terms of use of POCUS in the ED and Primary care for quality improvement in patient care and reduced costs in the ED and rural centers. Moreover, this study may be used as a prototype in performing educational activities to improve the use of POCUS in developing countries with resource constraints.