Peri-Operative Patient Safety – An Interactive Workshop for Section 3 CPD Credits Developed in Collaboration with the CMPA

Background

The Royal College of Physicians and Surgeons of Canada has implemented a Maintenance of Certification (MOC) Program that requires physicians to participate, and earn credit in, three continuing professional development (CPD) sections. The CPD section of this study is interested in is Section 3, which requires a certain number of hours directed towards self-assessments that involve data and feedback to identify and address unperceived professional practice needs. The CPD programs in medical education have been shown to be an effective method of improving physician’s application and retention of knowledge, attitudes, skills, behaviours and clinical outcomes.^1,12 Moreover, physicians are interested in more opportunities to earn formal credit for learning, and they consider the need of medical knowledge and skills as the highest-priority.³

Every year there are adverse events that cause harm to patients who undergo surgical procedures, more than half of these occur pre-operatively and post-operatively.⁶ Non-technical skills are an important part of surgical competence and patient safety and many adverse events arise from gaps in these skills.⁷ Even experienced surgeons can have deficiencies in these skills, putting their patients at risk.⁷ It is increasingly important to develop CPD programs focusing on surgical safety and non-technical skills, regardless of the level of professional experience.^7,17 This study uses medicolegal topics as a source for obtaining section three CPD credits.

More research is needed to study the most effective ways to deliver CPD programs. There are limited studies available on CPD programs in medical conferences, especially those focusing on surgical safety and self-assessment, however there have been some positive movement towards offering CPD credits and workshops at conferences.^8,9 This study aims to examine the effectiveness and the attitudes towards an interactive workshop to be used for accreditation of Section 3 CPD credits at a medical conference, with the goal of providing a template for future workshops at national medical conferences. This workshop was in collaboration with the CMPA to provide additional training to physicians in medicolegal topics and surgical safety.

Methods

This project was a qualitative pre/post-test analysis to assess the effectiveness of an interactive case studies workshop on medicolegal liability and intraoperative surgical safety. This workshop was implemented at the Canadian Society of Otolaryngology Head and Neck Surgery (CSOHNS) annual meeting in Saskatoon, Saskatchewan in 2017. It was designed to give Section 3 CPD credits for participants and was implemented in collaboration with the CMPA. REB was obtained from the University of Manitoba Bannatyne Campus research ethics boards. Consent was obtained from all participants prior to the workshop. All 22 participants of the workshop were voluntary participants attending the CSOHNS annual meeting who were either staff, residents, or fellows in Otolaryngology and all with a variety of years’ experience. This workshop was 2 hrs long and covered three topics with multiple opportunities for self-assessment, discussion and feedback which are the essential components to obtaining Section 3 CPD credits. The first topic was wait times which included duty of care, liability and regulatory authorities. The second case described procedural error with regards to NOTSS which stands for non-technical skills for surgeons including situational awareness, decision-making, communication/teamwork, and leadership. Lastly, follow-up management was discussed involving barriers to follow-up and resolutions. The inclusion of these topics is unrelated to acquiring Section 3 CPD credits and was an educational lecture given at the annual meeting. Participants were aligned in self-chosen groups of 3–6 around circular tables within an auditorium. The tables were not analyzed in comparison to each other, they were used only to help facilitate efficient discussions around topics. True and False questionnaires were distributed to each participant prior to each of these three surgical medicolegal cases to gauge knowledge on the medicolegal issues. Forty minutes were spent on each topic including writing the self-assessment questionnaire, case presentation by an expert in the field, small group discussion amongst each table separately and then large group discussions amongst all participants. The participants then received answer keys and further discussion was aimed around teaching points and reflection. The participants completed an evaluation at the end of the workshop to study the perceptions on learning and the effectiveness of the workshop to obtain Section 3 CPD credits. In the workshop evaluation, we graded the responses of “agree” and “strongly agree” as a positive response whereas “disagree” and “strongly disagree” were graded as negative responses. The comments section was qualitative and was left open-ended for feedback from the participants.

The self-assessment questionnaire regarding learning was formatted as 3-point likert scale and the workshop evaluation questions were formatted as 5-point Likert and open answer questions. All data were collected after the session, de-identified prior to analysis and were analyzed using ANOVA then follow-up t-tests. Likert data were analysed with Student’s t-tests.

Results

The results are broken into the perceptions of knowledge acquisition during the workshop as well as the attitudes of the workshop as a whole. Of the 22 participants of an interactive workshop to obtain Section 3 CPD credits at the national CSOHNS annual meeting. All the participants were either otolaryngology staff (71%), residents (22%) or fellows (7%) with a variety of years’ experience. Over 50% of participants held an academic position. Most of the participants knew the requirements for Section 3 CPD credits (58%) but only 36% knew how to obtain them. Out of the participants, 79% reported interest in obtaining more Section 3 CPD credits. In further report, 89% recorded previously witnessing at-risk surgical behaviour, and 58% admitted to previously having performed at-risk surgical behaviours.

In the end of the workshop evaluation, we had participants rate their knowledge in the three cases from before and after the discussion as “minimal”, “good”, or “excellent”. We then used those who reported “excellent” as having a strong understanding of the same topics. As shown in Figure 1, from the beginning to the end of the workshop, there was an increase in participants who felt they could analyze the impact of at-risk behaviours on patient care. They reported an improved understanding in identifying and developing strategies to address at-risk behaviours in surgical practise, with an increased ability to improve patient care and decrease exposure to medico-legal liability.

Figure 1 Self-assessment of abilities before and after course. The orange bar indicates before the course, the blue bar indicates after the course. 1-Minimal, 2-Good, 3-Excellent. All differences were statistically significant (p<0.001).

The post-workshop questionnaire also inquired about the attitudes towards the workshop. These questions demonstrated that more than 78% of participants responded positively to the interactivity, workshop content and teaching. The majority of participants (71% and above) agreed that the course was relevant and useful to their work and that they would like more similar workshops to be included in future CSOHNS annual meetings. 71% of participants felt conferences should include more opportunities to obtain Section 3 CPD credits. These results are demonstrated in Table 1.

Table 1 Pre-Workshop Experience

In addition, most participants commented on how to change their practise to apply principles they learnt at the workshop. These included templates for referrals, more verbal communication with teams, slowing down at critical times, as well as NOTSS.

Comments from participants on the strengths of the presentation included excellent discussion, strong communication and questions, as well as clear explanations and organization. Seven participants left comments regarding the weaknesses of the workshop and five of these comments described wanting a more positive “feel” to the topic of physician medicolegal liability and to increase the amount of workshop time dedicated to teaching the topic of otolaryngology medicolegal liability.

Discussion

In this study, we collaborated with the CMPA to assess how participants of the CSOHNS annual meeting perceived an interactive case studies workshop on medicolegal issues as a program to obtain Section 3 CPD credits. The Royal College of Physicians and Surgeons of Canada Maintenance of Certification Program requires physicians to participate in CPD credits in order to improve physician knowledge and skills.⁴ A list of current ways people can acquire Section 3 CPD credits is in Table 2. Section 3 CPD credits focus on knowledge and performance assessment with feedback.⁵ There have been previous concerns about the physician’s ability to obtain self-assessment credits.¹⁰ There is minimal literature available on obtaining Section 3 credits which help to explain why out of 22 participants at the CSOHNS workshop, 58% knew about Section 3 CPD credits but only 36% knew how to obtain such credits. With the need for more awareness and access to section 3 CPD credits, some programs have offered CPD credits and self-assessment programs in conferences.¹¹

Table 2 How to Acquire Section 3 CPD Credits Derived from Entcanada.org and Royalcollege.ca

The interactive CSOHNS workshop included three surgical safety cases as a pilot workshop to obtain Section 3 CPD credits at a national medical conference setting. Over 77% of workshop participants viewed the workshop content positively including the interactivity, questionnaires, teaching, small and large group discussions, and the feedback. These results are consistent with previous findings. Previous studies have shown the use of multiple-choice questions with feedback to assess and compare the knowledge amongst conference participants is an effective method to increase participation and receive positive feedback from participants.⁸ In the literature, self-assessment was demonstrated to be more effective in improving physicians' practice behaviours over traditional didactic lectures which supports the necessity to meet the section 3 CPD requirements.¹² Providing performance data and formal feedback has been suggested to be a more reliable evaluation than unguided self-assessment.¹³ Furthermore, when performance measurement is provided with expert feedback and standard measures, then self-assessment can be essential for physicians to close performance gaps and meet Continuing Medical Education (CME) requirements.¹⁴ In this study, the workshop included pre- and post-case questionnaires, discussion and expert feedback to gauge learning and to aid in self-assessment which have not only shown to be beneficial through literature but also are specific requirements outlined by the Royal College of Physicians and Surgeons of Canada.

Some literature suggests concern over the number of conferences and online courses offered to physicians to update their knowledge, as physicians might not be choosing conferences that provide the most effective and up to date information.¹⁵ Other articles have combatted this viewpoint by explaining CME conferences can be improved by limiting the total number of conferences that provide CME as well as organizing the CME workshops into areas that enable skill development.⁹

In addition, it is increasingly important to develop CPD programs focusing on surgical safety and non-technical skills, regardless of the level of professional experience.⁷ Our CSOHNS workshop discussed several medical legal issues including wait times, follow-up management, and NOTSS as a basis for providing Section 3 CPD credits. These topics received positive feedback from workshop participants. The quantity of participants who felt they knew about surgical safety from the beginning to the end of the workshop increased at least 50% for every learning objective including identifying at-risk behaviours in surgical practise that expose surgeons to medico-legal liability, analyzing the impact of at-risk behaviours on patient care and developing strategies to address at – risk behaviours in surgical practise and improve patient care. These findings are important because studies have shown that higher levels of non-technical skills can lead to quicker crisis management in operating rooms.¹⁶

By the end of the workshop, most participants commented on ways in which they can change their practise to apply principles they learnt during the workshop. These included templates for referrals, more verbal communication with teams, and slowing down at critical times. Studies have shown that adequate feedback and setting appropriate goals for improvement is a strategy to increase the quality of self-assessment.¹⁸ Overall, over 70% of participants felt the workshop was relevant and useful to their work and expressed interest to have more similar workshops at future CSOHS conferences to obtain Section 3 CPD credits. Previous studies suggest multiple exposures to continuting medical education can help optimize educational results, suggesting further workshops may be beneficial.²

A few concerns brought forth by participants about the workshop included comments on putting a more positive spin on medicolegal liability as well as the need for more time to discuss the topics of medicolegal liability. Some literature has mentioned the use of online programs to obtain Section 3 CPD credits, including questionnaires to assess knowledge, supporting references and critiques as well as a clinical simulation focusing on patient management.¹⁹ This may be beneficial as participants can complete the Section 3 CPD credits at their own pace and time. A detriment to this method is the lack of peer discussion which was one of the most positively reviewed aspects of the interactive workshop.

A limitation of our study involves the use of survey questions documenting how much participants felt they knew about medicolegal topics before and after the workshop, which may not represent participant’s actual knowledge of the topics. Although, there have been studies showing self-assessments to improve knowledge and technical skills in surgery are reliable.²⁰ To further assess participants’ knowledge, in future studies we would like to administer test questions on medicolegal topics before and after the workshop and use the test scores to document improvement in understanding of the topics.

In future studies we hope to follow up with the participating physicians to inquire about the changes they have made to facilitate more surgical safety in their practise and if they are still utilizing these changes. Furthermore, future conferences should allocate more time to talk about medicolegal cases to ensure participants have enough time to adequately learn and discuss the learning objectives and topics of the workshops.

Conclusion

To conclude, this method of obtaining section three CPD credits in a conference setting received positive feedback from the workshop participants in numerous categories including workshop content and teaching. Participants also reported self-improvement in knowledge of areas highlighted in this workshop including wait times, procedural error including non-technical skills for surgeons, and follow-up care. With the use of self – assessment, discussion and feedback, participants were able to obtain Section 3 CPD credits at the annual CSOHNS meeting and the majority of participants requested to have more opportunities for similar workshops in future conferences.