Student Perceptions of a New Course Using Argumentation in Medical Education

Introduction

Following the Flexner Report of 1910, pre-clinical medical education in America mainly followed a transmission model of learning that leaned heavily on reductionism.¹ However, in the 1970’s several medical schools began pioneering a greater role for discovery and inquiry in the educational process, mainly Problem Based Learning (PBL) and later Team-Based Learning (TBL), and these learning strategies have achieved considerable traction nationally and internationally.² Many medical colleges combined PBL and TBL, but neither approach explicitly offered or encouraged students the opportunity to engage in argumentation. The absence of argumentation (i.e., a productive exchange of opposing views aimed at improved understanding of a given issue) in medical education may leave physicians susceptible to medical marketing, and incapable of both resolving industry claims and adapting to changing paradigms.^3–7 Additionally, engaging in civil disagreements with colleagues is fundamental to team-based practice. Without explicit instruction in techniques to discuss alternate viewpoints and divergent approaches, physicians may find themselves without skills to negotiate routine disagreements associated with contemporary health care contexts.⁸

Educational researchers over the last 20 years have reported that argumentation helps the learner understand the epistemic implications of taking one position over another.^9,10 It fosters complex critical thinking and aims to amplify students’ reasoning skills when faced with complicated questions, issues, and/or problems.¹¹ Argumentation can thus enhance medical education, especially insofar as physicians need a comprehensive and skill-based understanding of how to translate research findings to clinical practice and public health policy. Though argumentation has been described and studied outside medical education, we found no evidence in the medical education literature for such a program.

Argue-to-Learn (A2L) is a new education series in the Penn State College of Medicine pre-clerkship curriculum intended to engage medical students in argumentation involving real-world medical and public health issues. The 90-minute sessions begin with a short debate by two faculty members that is followed by a small group breakout session, and closes with a moderated large group discussion. The learning objectives for each session include concepts discussed within the organ system course and require students to engage actively with issues on a deeper level than occurs with standard courses. The sessions augment existing curricula, and are intended to create a curricular thread throughout the pre-clerkship phase by addressing controversies within each course. The objective of this study was to assess student perceptions of the first two A2L sessions that were integrated into the standard first year medical education curriculum using quantitative and qualitative data obtained from standardized post-course surveys.

Methods

Background

The idea of A2L first developed during faculty brainstorming sessions on how to get medical students to think scientifically and pursue careers in research. In the process of pilot testing several sessions that involved faculty debaters and medical student volunteers it was felt that the different components of the sessions, including constructive controversy (debate) and collaborative discourse are important to all physicians in training and not just those interested in pursuing careers in research. Ultimately, these voluntary sessions and the planning that went into them resulted in the format below.

Session Description

We piloted two A2L sessions in organ systems courses offered in 2021 to first year MD students at the Penn State College of Medicine. The general format for the sessions is outlined in Table 1. Reading material to prime students on the content of the sessions was distributed within one week prior to each session.

Table 1 General Format for A2L Sessions

One session was in the “Form and Function” course that focused on the advantages and disadvantages of two treatments for bone loss, encouraging students to consider whether denosumab or bisphosphonates should be the first-line agent of choice for post-menopausal osteoporosis. Denosumab is a relatively new monoclonal antibody that costs significantly more than bisphosphonates, the traditional standard of care. Faculty “debaters” presented the case for each. For example, one presented data showing that denosumab increased bone mineral density (BMD) better than bisphosphonates.¹² The other argued that there is little evidence that it actually reduces fractures.¹³ The claims and counter-claims of the debaters, along with the learning objectives of the session, are presented in Table 2. The overall purpose of the session was to demonstrate productive argument and encourage critical thinking in a real-life scenario when considering which drug should be the treatment of choice and how physicians might form a basis for their decisions.

Table 2 Outline of Session 1

The other session in the MS-I “Host Defense/Host Response” course asked students to consider whether the COVID-19 vaccine should be mandated for all US residents. In this session, one debater presented the scientific rationale for making COVID-19 vaccination mandatory; the other debater presented the longstanding historical resistance to vaccine mandates and ethical concerns about compulsion. The claims and counter claims of the debaters along with the learning objectives of the session are presented in Table 3.

Table 3 Outline of Session 2

In both sessions, learners were given the opportunity to express their opinion on the topic of debate through an online poll that was administered at the start of the session, prior to the debaters’ presentations. Seeking these “pre-debate opinions” was designed to arouse curiosity, activate pre-existing knowledge, and increase engagement in the session. After listening to the debaters present their claims and counter claims, learners gathered in small groups in breakout rooms with a faculty facilitator. The purpose of this peer-to-peer dialogue was to support learners’ analysis of the evidence presented in order to identify strengths and weaknesses of each position and to explain and elaborate upon their reasoning. The facilitators (all of whom were coached not to provide their own opinion) promoted constructive dialogue among the learners and encouraged critical thinking and thoughtful appraisal of the various claims. Moreover, the small groups consisted of on-going PBL teams, thus helping to offer a safe environment in which the learners were accustomed to working together as a learning team.

A facilitator’s guide was developed during the planning period to help facilitators with content- and process-specific background to promote discussion through the use of probing questions, such as:

• What points did each of the speakers make that were particularly convincing? What made it convincing? Try to describe the highlights of each “side” of the debate.
• If your patient comes to you and has heard about this new drug (denosumab) and would like to try it, how would you explain to them the relative merits of denosumab vs bisphosphonate?
• Which do you think is the better approach: denosumab or bisphosphonates, and why?

After the period of small group discussion, all participants reconvened in a large group session. This session began with another online poll that presented a second opportunity for learners to vote on the debaters’ positions. Poll results showed that many learners changed their opinion. The purpose of the large group session was for learners to engage in discussion about the reasons why they had changed their opinion, or why it had remained unchanged, based on the debaters’ presentations and small group discussion. The key to success in this portion of the session was to elicit rich explanations of the learners’ reasoning process and to have them describe the features of the information presented that they found most convincing. Since the topics were selected to represent questions of true scientific or ethical uncertainty, elaboration upon the learners’ reasoning was the goal rather than consensus around a single correct answer.

Ethics

This study used data collected from deidentified post-course surveys. The same survey (described below) is used for every course in the medical education curriculum and course instructors and directors are provided with the results following the completion of the learning block. In addition, each course instructor and director is provided with cumulative metrics on their course and all other courses combined, in the learning block. No additional manipulation of data, with exception of statistical tests based on the cumulative data, was performed: thus, IRB approval was not sought.

Sampling methods The selection of students who are assigned to complete post-course surveys is done at the level of administration. Every week 50 students are selected randomly from the student body of 140 to complete an evaluation survey of every assigned course for the week and receive links by email to the online surveys. Students are mandated to complete the surveys as part of professional behaviors expected in the course. Institutional protocols were used to administer and collect surveys for the Argue to Learn sessions in the same manner as all other courses.

Evaluation Questions

After every medical school course, a randomly assigned subset of students completed standardized surveys. Surveys include four quality-related questions about the presentation scored on a 5 point Likert scale (1 = poor, 2 = fair, 3 = average, 4 = good, 5 = excellent) followed by an open-ended response that offered them the opportunity to provide “constructive comments about the strengths and areas to improve the learning session(s)”. The four quantitative responses were: 1) Rate the extent to which this faculty’s presentation(s) supported your mastery of the learning objectives. 2) Rate the quality of the learning resources (ppt slides, readings, handouts). 3) The presenter exhibited professional behavior in interactions with the class and treated students with respect; and 4) Rate your perceptions of the faculty’s presentation(s) OVERALL.

Statistics

Mixed methods were used to analyze survey data. Student’s t-test (α=0.05) was used to compare responses on a question-by-question basis for the A2L sessions versus the average score for all other sessions contained within the block. Thematic analysis was performed by one author (A.J.F.) for open-ended responses.

Results

Forty surveys (80% response rate) were completed for the A2L session on Denosumab versus bisphosphonates for postmenopausal osteoporosis, and 1387 surveys were completed for all other sessions within the Form and Function course. Eighty-four surveys were completed for the Argue-to-Learn (84% response rate) session on the COVID-19 vaccine mandate and 2206 surveys were completed for all other sessions within the Host Defense/Host Response course. The discrepancy between survey totals for the two Argue-to-Learn sessions is due to the number of lecturers who were specifically evaluated (eg, 1 vs 2). For all questions, the Argue-to-Learn session scores were the same or numerically higher than the cumulative scores of all other lectures in the learning block with no statistically significant differences between groups. (Figure 1) For “supporting the learning objectives of the sessions”, scores were 3.8 (± 0.8) vs 3.8 (± 0.8) (p=0.49) and 4.5 (± 0.6) vs 4.3 (± 0.8) (p=0.09), respectively. For “quality of lecture resources”, scores were 3.9 vs 3.8 (p=0.44) and 4.5 (± 0.6) vs 4.3 (± 0.6) (p=0.09), respectively. For “professionalism and respect”, scores were 4.2 (± 0.7) vs 4.2 (± 0.8) (p=1.00) and 4.7 (± 0.5) vs 4.6 (± 0.6) (p=0.45), respectively. Finally, for “overall perception of the presentation”, scores were 3.9 (± 0.7) vs 3.8 (± 0.8) (p=0.44) and 4.5 (± 0.7) vs 4.4 (± 0.7) (p=0.52), respectively.

Figure 1 Quantitative Results.

Forty six survey respondents provided open ended comments for thematic analysis (37% response rate). Two important themes were identified in the qualitative analysis. No student explicitly or implicitly expressed the idea that this format detracted from their ability to learn information needed for test-taking or was an ineffective use of time. The first theme was “enjoyment and engagement”. Students repeatedly expressed that they liked the A2L format, that they found it interesting and engaging, and that they wanted more classes like this in the curriculum. One student wrote,

Argue to learn sessions were more engaging than the normal lectures for me.

Another wrote,

The argue to learn idea is one my favorite lecture strategies that we have had. I would love to have more lectures like this!

A collection of statements representing this theme are presented in Table 4.

Table 4 Representative Quotes for Qualitative Theme 1

The second main theme involved students’ appreciation for seeing multiple viewpoints debated in a nonthreatening fashion and having their own opportunity to engage in debate with peers in a supportive environment. One student wrote,

It is fantastic to see two different perspectives in healthcare… getting the opportunity to discuss it afterwards with a facilitator and my classmates enhanced my critical thinking skills and helped me to see the array of opinions of the class in a safe space.

Another student wrote,

It is important to see different opinions shared with respect. I really liked getting the opportunity to have a faculty member facilitate small group discussions.

A collection of statements representing this theme are presented in Table 5.

Table 5 Quotes for Qualitative Theme 2

Discussion

This manuscript presents data showing that medical education classes centered on argumentation are perceived to be at least as effective as other more traditional forms of medical education by first year medical students. Furthermore, evidence from our qualitative analysis suggests this format may be more enjoyable and engaging to students than more traditional forms of learning. Perhaps most important, we found no evidence of significant resistance to the format.

The use of argumentation as a learning tool has been studied outside the medical education setting for several decades with positive results. Five learning mechanisms have been identified that are associated with using argumentation as an education tool: change in view, making knowledge explicit, conceptual change, co-elaboration of new knowledge, and increasing articulation.^9,14,15 A strong theoretical case can be made that physicians must be able to argue effectively and learn from it. However, students coming out of college may not have engaged in much debate when they were there, and they may need more practice in order to be effective in these kinds of deliberations. In law school students learn to argue because that is seen as central to the law. Conversely, in medical education, science learning tends to be passive and somewhat rote, and there is little attention to engaged disagreement. Many have offered that recent generations of students often have few Socratic experiences in college and have not learned to disagree civilly.⁷ This flies in the face of the fact that team-based medical practice demands that providers communicate well about reasons for and against particular practices.

The format we have developed for Argue-to-Learn sessions in medical education combines two approaches for using argumentation in education that have been studied and elaborated over the last several decades. One aspect involves constructive controversy and corresponds to the first part of each session that is dedicated to staged debates between experts (Table 1).¹⁶ The function of the expert’s debate is to model exemplary arguments, demonstrate the norms of social interaction, and help students understand that the function of their discussion is to persuade others of the validity of their arguments rather than simply express their opinions. According to prominent scholars in the field,

Conflict is to student learning what internal combustion engine is to the automobile… [it] provides the motivation that energizes students to seek out new information and study harder and longer.¹⁶

In a meta-analysis of different classroom learning strategies, Johnson et al¹⁷ found that controversy produced higher achievement than did all other forms of classroom learning. They found that students involved in controversy were more immersed in self-learning activities and more motivated to arrive at the best reasoned judgment than were students in other environments.¹⁷ Thus, a critical aspect of each A2L session is to choose areas of controversy that are consistent with the overall goals of the learning courses. “Organizing conflicts” has been recognized as a central challenge for creating curriculum centered on controversy by several educational scholars and should be an area of focus in medical education as well.¹⁸ We have chosen conflicts that are timely and relevant to the field of study where decision-making involves a recognition of trade-offs in various domains (eg, pathophysiologic basis for effect; efficacy in hard versus soft endpoints; cost considerations, both personal and societal; patient preferences; ethical considerations; etc.).

The second aspect of the sessions involves collaborative discourse using dialectical arguments and is achieved in small group breakouts sessions following the debate presentations (Table 2).¹⁹ Dialectical arguments represent a dialogue between proponents of alternative claims whose purpose is to resolve differences of opinions.¹⁹ Breakout groups are facilitated by senior educators and students are encouraged to present arguments and counterarguments along with their supporting rationale. The facilitator’s role is to keep discussion flowing by providing specific questions to students such as,

What might somebody else, who does not agree with you, think the benefit is of using X drug (mandate) over Y drug (non-mandate)?²⁰

Facilitators report that participation in the small groups is high and students are engaged in critical thinking. This is consistent with student reports of high engagement in our qualitative analysis. The goal of small group breakout sessions is for each student to reach a deeper understanding and best-reasoned judgment of the issue that is reflected by an ability to articulate a multitude of claims and counterclaims.

Our analysis has several limitations. It is based on a small sample size that included only two sessions with two different groups of students. Results are based on student perceptions and not objective evidence of achievement. Finally, the qualitative methodology may be subject to sampling bias in that learners who responded to the open-ended survey item were most likely those who had a strongly positive reaction. Despite these limitations we find these results encouraging and consistent with educational scholarship outside of medicine showing that argumentation in the classroom leads to deep, civil engagement and improved critical thinking. Whether argumentation ultimately improves learning in medicine and patient care are questions that should be explored. Several frameworks have been established for assessing argument quality, and we believe that next steps should include incorporating student arguments as part of medical course tests and comparing learning gains between courses with and without argumentation in their curriculum.^21–24 In conclusion, medical education classes centered on argumentation are perceived to be at least as effective as other more traditional forms of medical education by first year medical students. Furthermore, evidence from our qualitative analysis suggests this format may be more enjoyable and engaging to students than more traditional forms of learning. Furthermore, we found no evidence of significant resistance to the format. Instructors should consider using argumentation in medical education courses and future work should be done to assess whether such formats are associated with medical student improvements in critical thinking.