What Aspects of Supervised Patient Encounters Affect Students’ Perception of Having an Excellent Learning Outcome? A Survey Among European Medical Students

Background

To work as doctors, medical students must gain the “know how” of doctoring which can be acquired through supervised clinical experiences. Supervision can take place before, during or after patient encounters to help students improve and ensure high-quality patient care. According to Stalmeijer et al,^1–3 modelling, coaching, articulation and exploration are methods doctors use when supervising medical students. In the European Union (EU) and associated countries, supervised clinical experience during undergraduate medical training is required to become qualified as a medical doctor.⁴ Thus, it is important to know which supervised clinical experiences are perceived as useful for learning by students.

Qualitative studies have identified a range of factors students say helps them learn. Central are supervisors’ behavior,^5,6 and students’ approach^7–9 and participation.^6,10,11 Supervisors’ behavior includes feedback,^12,13 clarifying what students are supposed to learn,^14,15 asking questions^16–18 and involving students in patient interactions.¹⁹ Students’ approach includes preparation,^7,8,17 proactivity²⁰ and display of confidence.^7,9 In addition, whether the atmosphere is experienced to be psychologically safe, has been found to be important during medical students’ clinical learning.^21,22

Few quantitative studies have looked at the relationship between any of these factors and students’ perception of learning benefits. The three studies identified analyzed post-rotation questionnaires filled out by medical students at Maastricht University in the Netherlands.^23–25 They found that supervision was associated with students’ perception of instructional quality or rotation effectiveness. However, they did not examine which aspects of supervision are perceived as beneficial to learning. As students learn something from nearly any activity,²⁶ focusing on excellent learning outcome can identify the factors with the most impact.

We have not identified studies looking at students’ clinical learning experiences across countries: neither among the articles on clinical medical education that were collected systematically until 2018 to develop Experience-based Learning theory^6,21 (reference list obtained from the first author) nor through our own searches for more recent literature. With increasing globalization and students and doctors moving between countries, knowing what works is a global concern.²⁷ Thus, research across cultures and contexts is needed.

Our aim was to investigate whether supervisor behavior, students’ participation and approach, and psychological safety were associated with self-reported excellent learning outcome from supervised encounters with patients among European medical students.

Methods

Design and Ethics

This was a cross-sectional survey among medical students in Europe using a self-administered online questionnaire. The data collection took place from the 17th of March 2020 until 11th of October 2020 due to covid-19, as there were students who did not have clinical teaching in the spring.

As the research was conducted in Norway and recruiting was conducted online through an international organization, ethical approval was only sought in Norway. The study was discussed with the Norwegian Centre for Research Data and considered to be exempt from ethical review (reference number 235211) due to the anonymity of participants. An information letter based on the template for Norwegian ethical requirements and European data protection requirements was displayed on the first page of the survey. On the final page, respondents were informed that by clicking “send” they agreed to their responses being used in research.

Participants and Recruitment

Inclusion criteria were being a medical student at a medical school in Europe, having met patients through the medical school curriculum, and having received clinical supervision in at least one such patient encounter.

To recruit participants, the survey was distributed through the International Federation of Medical Students Associations (IFMSA, who have medical student associations as members) and its national member associations. The study was enrolled as an IFMSA project and was endorsed by the organization. Author VA, a medical student, e-mailed the representative for each European IFMSA member country to ask for their help in distributing the survey. Representatives were offered the opportunity to ask questions by e-mail or through scheduled e-meetings. They were promised results for their country if there were at least 50 respondents from their country. Afterwards, links to the survey, and suggestions for texts and an illustration to be used in promoting the survey were sent to all the representatives. Representatives were requested to promote the study through e-mails to members of their respective national association and on social media. The same procedure was used for reminders: Once after 2–3 weeks and then once in the fall a few weeks after the beginning of term.

Data Collection

A secure-server survey-tool (www.nettskjema.no), which did not store identifying information about respondents, was used.

The questionnaire was in English to ensure uniformity and as it was expected that all European medical students can comprehend English. It contained 62 items and took about 10 minutes to complete. When asked questions concerning their experience with clinical supervision, students were instructed to think about “the last time you were in a situation with a patient and where you received supervision. By supervision, we refer to encounters between you and another health care professional that aim to improve your performance and ensure patient safety and high-quality care”. The definition of supervision was based on Launer²⁸ and Spence et al.²⁹

Variables

Dependent Variable – Excellent Learning Outcome

The dependent variable was measured with the single self-made question “How was the learning outcome compared to other supervision sessions?”. The response categories were “poor”, “fair”, “good” “very good” and “excellent”. We dichotomized students’ answers into “excellent” and those answering any other category, “not excellent”.

Supervisor Behavior

Supervisor’s behavior was measured using four of the five sub-scales of the Maastricht Clinical Teaching Questionnaire (MCTQ): “modelling”, “coaching”, “articulation” and “exploration”.² It has proven reliable and valid within a population of 4th and 5th year medical students in a 6-year program in the Netherlands.² The sub-scale “learning environment” was not included as it overlaps with psychological safety. Questions are rated on a 5-point balanced Likert scale.

Whether the supervisor was present was evaluated by the question “for how much of the time you were with the patient was the supervisor present” divided into “50% or more” and “less than 50% of the time”.

Students’ Approach to Clinical Learning

To measure students’ approach to clinical learning, the “motivation to learn” sub-scale of the Clinical Learning Evaluation Questionnaire (CLEQ) was used.³⁰ This includes questions on enthusiasm, independence, preparation, and confidence with regards to clinical learning. Validity and reliability were reported in one study from Saudi Arabia.³⁰ The CLEQ questions are rated on a 5-point balanced Likert scale.

Student Participation

To measure student participation during the supervised clinical experience, understood as taking on parts of the role of a doctor, we asked: “did you take the history?” and “did you do a clinical examination?”. An affirmative response to either question or both, was categorized as “participation”.

Psychological Safety

Psychological safety, defined as “the degree to which people view the environment as conducive to interpersonally risky behaviors like speaking up or asking for help”,³¹ was measured with the widely used psychological safety scale,³² which has consistently demonstrated acceptable reliability.³³ The wording in the questionnaire was adjusted to the context of clinical supervision, eg, “People on this team sometimes …” was changed to “Those present sometimes …”. Each statement is rated on a 7-point balanced Likert scale.

Proportion of Medical School Completed

Due to differences in duration of the medical programs, the proportion of medical school completed was categorized into less than one-third, between one-third and two-thirds, and above two-thirds. This meant, for example, that students in year 5 or 6 of a six-year program were placed in the above two-thirds category.

Gender

Gender was based on the question “which gender do you identify with”, with response options “female”, “male”, “non-binary” and “do not wish to specify”. This was dichotomized into “female” and “not female”.

Place of Study

Place of study was determined based on the question “which country do you study in?” with fixed answering options listing European countries listed in the World Directory of Medical Schools,³⁴ and “other European country”. As about half of the respondents either studied in Norway or was a Norwegian studying abroad, place of study was categorized as “non-Norwegian studying outside Norway”, “studies in Norway”, and “Norwegian student abroad”.

Age

Respondents were asked how old they were in years between 18 and 30, or over 30. The categorization of over 30 was to ensure anonymity.

Patient Contact and Supervision Setting

Respondents were asked to report on when they started seeing patients, how many students were present during the supervision encounter, on how many previous occasions the supervisor had supervised them, the supervisor’s profession and in what subject/specialty supervision took place.

Analysis

Data were analyzed using Stata version 15 (StataCorp, College Station, Texas, USA). Descriptive statistics, Pearson chi squared, and two-sample t-test were used to present the variables and their relationship with perceived excellent learning outcome. The variables in the bivariable analysis had p-values <0.2 were included in a multivariable logistic regression model together with proportion of medical school completed and gender, to look at the association between these variables and the learning outcome being rated as excellent (dependent variable). Odds ratios (OR) with 95% confidence intervals (95% CI) were used to describe the results of the multivariable analysis. Variance inflation factors (VIFs) were estimated to evaluate multicollinearity, and VIFs were from 1.03 to 3.61 indicating no multicollinearity.

Results

Characteristics of the Sample

We received 908 questionnaires fitting inclusion criteria (Figure 1).

Figure 1 Flow chart of inclusion process.

Overall, 72% of the respondents identified as female, with over half the respondents being between 23 and 26 years old (Table 1).

Table 1 Characteristics of Sample and Those Who Scored the Learning Outcome as Excellent

Respondents were from more than 25 different European countries, with number of respondents for each country ranging from 1 to 229 (median 17, Table 1). The countries with the most respondents were Norway (n=229), France (n=141) and Poland (n=103). Almost half of the respondents either studied in Norway or were Norwegians studying abroad (n=455).

Respondents studied in programs of 2–7 years duration, with 89% of the respondents being enrolled in 6-year programs (Additional file 1). Over half were in the final two-thirds of their program (Table 1). About a third (32%) started seeing patients in their first year of medical school (regardless of the length of their program), and nearly all (93%) did so by their third year (Table 1).

Supervision Encounters

Students reported on supervised clinical experiences from a range of specialties (Table 1). In 32% of these encounters, there was only one student together with the supervisor, while 30% were in a group of five or more. Nearly all (94%) were supervised by a doctor, and most students had little previous contact with the supervisor, with 36% never having met the supervisor before. In most cases (93%), they had participated by taking a history and/or performed a clinical examination. The supervisor was present ≥50% of the time in 58% of the encounters reported (Table 1).

Excellent Learning Outcome

Bivariable Analysis

The learning outcome of the supervised clinical experience was rated as excellent by 154 respondents (17%, Table 1), very good by 233 respondents (26%), good by 306 respondents (34%), fair by 138 respondents (15%) and poor by 70 respondents (8%). The largest percentage point difference in proportion reporting excellent learning outcome was for whether the supervisor was present most of the time (26% excellent) or not (12% excellent).

The score on the sub-scales of MCTQ, CLEQ motivation to learn, and psychological safety were higher among respondents who rated the learning outcome as excellent (Table 2). Among the MCTQ subscales, both modelling and exploration had nearly 1 point difference in mean score on the 5-point scale between those who rated the learning outcome as excellent and those who did not. For psychological safety, there was a 0.77 difference on a 7-point scale and for CLEQ motivation a 0.31 difference on a 5-point scale.

Table 2 Supervisor Behavior (MCTQ), Students’ Approach to Clinical Learning (CLEQ) and Psychological Safety Scores (N=908).

Logistic Regression Analyses

All independent variables had p-values under 0.2 for bivariable logistic regressions and were therefore included in the multivariable logistic regression. In the multivariable logistic regression MCTQ modelling, MCTQ exploration, CLEQ motivation subscale, psychological safety, and studying in Norway had 95% CIs that did not include 1 (Table 3), indicating that they were independently associated with the learning outcome being rated as excellent. The adjusted ORs for one unit increase in MCTQ modelling was 2.1 (95% CI 1.5–3.0), for MCTQ exploration 1.4 (95% CI 1.1–1.7), for CLEQ motivation 1.7 (95% CI 1.0–3.0), and for psychological safety 1.5 (95% CI 1.1–2.0). Respondents studying in Norway had an adjusted OR compared to non-Norwegians studying in another country of 1.7 (95% CI 1.0–2.9).

Table 3 Bivariable and Multivariable Logistic Regression of Variables Associated with the Dependent Variable Excellent Learning Outcome

Discussion

In summary, what was associated with European medical students experiencing their latest clinical supervision encounter as having an excellent learning outcome was supervisors being rated higher on positive role modelling behavior and addressing learning goals (MCTQ modelling and exploration), students approaching clinical learning with more proactivity and enjoyment (CLEQ motivation), and psychological safety being perceived as higher.

To the best of our knowledge, this is the first study of its type. Three other publications with similar aims^23–25 found that supervision was the most important factor affecting medical students’ perception of the effectiveness or instructional quality of their clinical learning experiences. Our study adds to this knowledge by identifying what supervisor behaviors are perceived as useful by students as discussed below. Our study also explored factors that have been found in qualitative studies, but that, as far as we have been able to identify, have not been tested quantitatively. Especially, students’ approach to clinical learning,^7–9 operationalized in our study with CLEQ motivation to learn subscale, was consistent with the qualitative studies. Other findings from qualitative studies were not found to be associated, like participation,^6,10,11 operationalized with students’ history taking and examination, and MCTQ coaching, and supervisor providing feedback^12,13 and asking questions,^16–18 which we operationalized with MCTQ coaching and articulation.

In light of Experience-based Learning theory (ExBL), our findings give some food for thought. ExBL places observing (role models) as the first rung of a ladder to independent practice, the second being students performing tasks without contributing to patient care, and the third and final is students contributing to patient care.²¹ In this study, MCTQ modelling – which equates to the first rung – was the variable with the largest odds for reported excellent learning outcome. MCTQ coaching (practice with feedback) and articulation (supervisors asking questions to increase student understanding), and students taking a history or doing an examination (practicing or contributing to patient care), which are higher up the ExBL ladder, were not associated. According to ExBL, students should observe what the supervisor does if they are not prepared to participate more fully²¹ and previous qualitative research suggest that students prefer being active participants^11,16 and feel held back when only allowed to observe.³⁵ However, the findings in this study are consistent with van der Zwet et al, who did not find an association between independent practice and students' perception of instructional quality.²⁵ This suggests that opportunities for independent practice alone do not contribute to students’ perception of learning from clinical experiences.

One possibility for makings sense of our findings, is that the students in our study felt that they were in an unfamiliar situation during their last supervised encounter with a patient. When being in an unfamiliar situation, having the possibility to observe is likely experienced as the best option. The unfamiliarity of the situation cannot be explained by exposure to patients in clinical setting being new to most of the respondents, as 86% started seeing patients at least a year before. Sixty-four percent had met the supervisor on at least one previous occasion. We have no data on whether other factors like the ward, clinic, department, or health care institution was new to them, but if they had been supervised by the supervisor before, they were probably not completely new to the clinic/department. One possible explanation, then, is that the complexity of clinical situations with new configurations of staff, patients, and symptoms makes the students experience most supervised clinical experiences as novel.

If correct, this would mean that most clinical supervised learning situations place students in a beginner state. Accordingly, in line with ExBL theory, they then need the basic steps of identifying what is to be learned (MCTQ exploration) and how it is supposed to be performed (MCTQ Modelling) before they can move on to more advanced participation. This is also the case when they are in the later years of medical school, as was controlled for in this analysis. Supporting this interpretation are two studies looking at medical students’ roles in block and longitudinal integrated clerkships (where students spent a year with the same supervisor or clinic) which suggest that students in longitudinal integrated clerkships are more likely to progress into roles where they participate in patient care,^35,36 suggesting it takes many months in the same place before students participate fully.

However, the importance of modelling and exploration is not the whole story. CLEQ motivation and psychological safety were also independently associated with perceived excellent learning outcome. CLEQ motivation measures whether students came well prepared, were proactive, and enjoyed learning. Thus, there is an independent contribution of the student’s own approach. Psychological safety has been described as working like the breaks on a car: taking them off is necessary for driving, but it does not drive the car forwards.³³ The analogy to our study would be that the students need to feel psychologically safe (take the breaks off) as an early step in the learning process.

Altogether our interpretation of these findings, which builds on an understanding that students in medical school are frequently in a beginner state, is that the first step to participation in practice – being engaged, feeling safe, and being introduced to what to do (through role modelling and addressing learning goals) – are most important for students’ perception of having an excellent learning outcome.

We think these findings convey an important message to clinical supervisors; they should recognize that students are beginners when supervising them in clinical settings and focus on addressing learning goals and allow for active observation in a safe environment before advancing to increased participation.

Strengths and Weaknesses

Our study’s strength lies in its inclusion of over 900 respondents from over 25 European countries and focuses on concrete experiences using validated measures. While there were many respondents, the response rate is unknown, making selection bias a threat. There is also a risk of recall bias, as the survey was sent out shortly after the COVID-19 pandemic hit and our collaborators informed us that many students were excluded from the clinical environment during spring 2020. Many of the variables in our analysis had a much larger effect in bivariable than multivariable analysis. Testing did not indicate multicollinearity. The data are cross-sectional, so no causal inferences can be drawn, and our findings must be seen as hypothesis of relationships to be further explored.

Conclusions

To the best of our knowledge, this is the first study of its type. We found that supervisor role modelling especially, but also supervisors addressing learning goals, students’ approach and psychological safety were associated with students’ perception of having an excellent learning outcome from a supervised patient encounter. This indicates that supervisors should recognize that students are beginners in most supervised clinical settings and often appreciate that learning goals are addressed, behavior and thinking role modelled and psychological safety established before they participate more fully.