Exploring Fairness in Scholarly Development: Are We Creating Knowledge Storing Zombies or Curious, Creative and Critical Healthcare Professionals?

“Education is not the learning of facts, but the training of the mind to think.” – Albert Einstein (1879-1955).

Developing scholars is an essential part of medical doctors’ training. The scholar is, rightfully so, one of the roles outlined in the CanMEDS competency framework.¹ Scholarly doctors require the retention of research knowledge and skills, as well as an academic mindset, which includes curiosity, creativity, and critical thinking. Research projects are widely used to develop future doctors into scholars. However, in contrast to knowledge and skills, assessment of development of an academic mindset in these projects is not so easy in a fair way. We explore and illustrate the issues that arise when focusing on objective, i.e. equal and unbiased, summative assessment of knowledge and skills in scholarly projects in order to safeguard fairness.

In the Netherlands, all medical schools provide a mandatory individual research project to develop scholarly abilities of future doctors.^2–4 During this project, students work individually full time for 16 up to 26 weeks on their own research. They go through all phases of the empirical cycle in an authentic setting and develop practical research skills such as searching and critically appraising literature, designing research, analyzing and interpreting data, and academic writing. Moreover, to shape an academic mindset, they receive training-on-the-job in research integrity, ethics, and philosophy and get challenged on their curiosity, creativity, and critical thinking. A staff member, PhD candidate or clinician-scientist supervises the students individually. To complete their project, students write a report formatted as a scientific paper and present the results at the department where they conducted their research.

We facilitated a few roundtable sessions for medical educators from all eight Dutch medical schools. The participants were course coordinators, supervisors and independent assessors in the mandatory research projects. During these sessions, participants shared their opinions, experiences, and challenges regarding supervision and assessment of the research projects. Participants deemed, or even convicted, that training scholarly skills, knowledge, and attitude were the main aims of the course. Regarding assessment of the learning goals, the vast majority considered fairness a fundamental quality of assessment and hence a main guiding principle, also well known as a general principle in (medical) education.^5–8 It is difficult to provide a precise definition of fairness as there is no all-encompassing consensus definition described in the literature. The Cambridge Academic Content Dictionary describes fairness as ‘the quality of treating people equally or in a way that is right or reasonable’. Participants perceived objective approaches as imperative for fairness in assessment, perhaps even the only way to achieve fairness, which is in line with common beliefs in higher education.⁵

To pursue fairness in assessment through objectivity, participants deemed standardized, equal, and unbiased educational procedures pivotal: all students should be treated similarly regarding, among others, supervision and assessment. This was perceived particularly important since all students carry out their research project individually at different departments. To guide supervisors and teachers in supervising and assessing students equally, all institutes use standardized procedures, quality measures, protocols, and rubrics. While we fully acknowledge the value of fairness in assessment, some dilemmas in shaping scholars arise when objectivity is assumed to be a prerequisite for fairness. We illustrate this with three practical examples that were discussed during the roundtable sessions.

The first example of objective assessment regards the standardized amount of provided study time. Some faculties set deadlines for submission of research reports to ensure that all students receive the same amount of study time. Exceeding the deadline influences their grades adversely. The second example concerns supervision: to further safeguard objectivity in assessment, some schools standardize and regulate the frequency and amount of feedback the supervisor provides on drafts of the written report. The more feedback is given, the more the research report may well be effectively (co-)authored by the supervisor, which likely improves the quality of the research report and could result in higher grades. Most roundtable participants considered this unfair and argued that frequency and amount of feedback should be standardized for every student, and even restricted to e.g. two times max. 1 hour of feedback. Therefore, medical schools provide guidelines, for instance for deadlines and feedback. Finally, to objectively assess scholarly learning outcomes, schools commonly use summative testing with rubrics. As summative assessment of the research report can be subjected to the favor of the supervisor, a four-eyes principle is used to strengthen objective assessment. Therefore, a second, independent assessor appraises the research paper as well. In this way, the chance of subjective assessment by supervisors favoring their own students is believed to decrease, as objective assessment of the research product is paramount. This is reinforced by visitation committees, who often evaluate students’ written reports as part of national quality assurance. These committees commonly note that grades given by supervisors are higher than committee’s own grades, and this way of assessing emphasizes the importance of objective assessment in medical education.

These examples illustrate that all participants considered standardized educational procedures essential for objective and, thus, fair assessment of scholarly abilities. To us, at first, this seemed reassuring – who could oppose an objective assessment of scholarly learning outcomes? On second thought, however, some dilemmas emerge. For instance, if the research report is assessed by an independent reviewer after a structured process with strict timelines and a standardized (often limited) amount of feedback, does this educational format properly reflect the learning objectives of a scholarly project? Even so, what are the consequences for growth and development of future scholars when objectivity is the main guiding principle to safeguard fairness?

First, we will elaborate on the consequences of the examples of objective assessment. We, as project supervisors ourselves, experience that some students are willing to maximize their learning potential; the discovery of the fun nature of conducting research during the project is intrinsically motivating and makes them eager to invest even more time and effort in their project.⁹ Providing extra time in such cases may boost their learning curve and could take them to a next level.⁴ This additional time even enables a significant subset of students to reach such academic levels that they publish their research in a peer-reviewed journal.^2,10,11 Moreover, these experiences increase the likelihood of postgraduate research activities.¹² Regarding frequency and amount of feedback, some students continuously improve upon every feedback session, i.e. their academic growth will benefit from more feedback.¹³ From this perspective, the process of students’ academic growth and degree of feedback provided during their research project is beneficial for shaping their academic mindset, as they are willing to make the most of this academic opportunity. Although this dedication and eagerness could be considered as the highest achievable scholarly development, when safeguarding fair assessment objectively, this seems to be undesired learning behaviour. Some faculties might even consider to give a lower grade in such situations, which, by emphasizing the objective measurable research product, complicates the learning process.

Within the context of individual research projects, standardized procedures to achieve objective assessment irrevocably imply fixed and regulated study time, standardized quantity of feedback, same curricular timing, and, at the end, assessment of learning only by an independent assessor. One could even argue that, in order to provide students with maximum standardized opportunities, they should all work alone on an identical research topic. Consequently, a research project that aims to shape curious, creative, and critical doctors with research skills and knowledge devaluates into an almost fully standardized writing assignment. In this way, in fact, standardized procedures for objective as mean for fair assessment are placed above the scholarly learning objectives. While standardization and procedural approaches concerning feedback and limited study time seem appropriate for the development of research knowledge and skills, there is friction when applying these in the context of shaping true scholars with academic mindsets.

Two complementary concepts, Ausbildung and Bildung, illustrate the friction of scholarly learning objectives eminently. Ausbildung, achieved through vocational training, provides students mainly with theoretical knowledge and practical skills. Within this context, modules are measurable units of learning and assessment of this learning leads to the awarding of credit. In addition, students learn along a fixed, regulated timely pathway defined by national standards. Bildung, on the other hand, involves the process of continuous individual development, shaping, and growth of an academic mindset. This cannot be easily measured, let alone objectively assessed, and its goal is different from provision of knowledge and skills. Bildung focusses on the journey (i.e. learning process) rather than the destination (i.e. research product). It is a lifelong process without regulated and fixed learning pathways or measurable and known attainment levels. Pace, tendency, and final attainment levels depend solely on the individual, and flexible learning pathways can support students in their journey. Treating students fairly based on their needs facilitates, supports, and stimulates this growth.

Wilhelm von Humboldt, founder of the modern university, linked the concept of Bildung to academic education in the early 1800s. He envisaged university education as a student-centred activity of research. Up until today, medicine is considered to be an academic discipline, taught at university. The word ‘university’' is derived from the Latin ‘universitas magistrorum et scholarium’', which roughly means ‘community of teachers and scholars’'. Von Humboldt believed that universities should enable students to become individuals with an academic mindset by developing their own reasoning powers and choosing their own way in an environment of academic freedom, as he captures in his ‘Theorie der Bildung’'¹⁴

Applying Ausbildung and Bildung to our national academic research project, Ausbildung reflects vocational research knowledge and skills, with standardized educational learning pathways, using objective assessment instruments. Equally important, however, is our aim to shape academic mindsets, in line with Von Humboldt’s Bildung. As we aim to fairly assess the scholarly development of medical students, learning objectives that can be measured (e.g. research papers), as well as standardized learning pathways, are useful. From an Ausbildung perspective, objective assessment of knowledge and skills is aligned. Bildung, however, hardly seems compatible with standardized educational procedures and objective assessments. Bildung implies academic freedom and is supposed to shape students with academic mindsets. It requires a merely formative approach based on academic freedom, flexible learning pathways without fixed study time or limited amount of feedback, and thus, ultimately, assessment for learning.

As our examples illustrate, the combined Bildung and Ausbildung approach in academic courses is reduced to Ausbildung only, especially when academic education is strictly regulated and objectively assessed to safeguard so-called fairness. Even more, only educating and assessing Ausbildung inevitably spoils and harms Bildung. When doing so, we risk to equip future doctors with theoretical knowledge and practical skills only, rather than training doctors with truly academic mindsets. Consequently, harsh tongues talk about marketized regimes of massification, evaluation, accreditation, and quality assurance that limits academic freedom and growth in medical education. In addition, medical students are referred to as malleable and manageable zombies, trained to store knowledge rather than shaping a curious, creative, and critical mindset.^15–17 In line with this, the competency of scholar is being referred to as ‘the neglected competency’' and medical schools are considered to be ‘degree mills’' and ‘uniformity factories without leaving any room for creative, independent, critical, and confident individuals’'.^15–17

In short, over the past few decades a contradiction has emerged within the field of scholarly competencies, as attempts to standardize the unstandardizable in the name of so-called fair assessment have inadvertently created a disconnect between education and practice. While objectivity is frequently viewed as a prerequisite for fairness and commonly used when designing educational guidelines, protocols and rubrics, our examples illustrate that excessive reliance on objectivity can actually undermine fairness as it only measures what can be measured quantitatively.^5,17 We demonstrated that the development of a true academic mindset requires individually tailored discussion and feedback, which cannot be adequately achieved through standardized approaches. Returning to the principle of fairness, objective assessment with standardized learning pathways is considered fair, but in practice learning activities are directed to passing writing assessments rather than training real scholars. Consequently, an objective approach may even result in hampered development of a genuine academic mindset. In this way, objectivity can be considered unfair rather than fair.

To rebalance Ausbildung and Bildung in academic education we should go beyond the assumption that objectivity is a prerequisite for achieving fairness in assessment. We mainly focused on the contradiction and consequences of striving for fairness and objectivity within assessment in the light of developing scholarly competencies, but it is likely to be applicable to the development of other competencies within medical education – or maybe even all educational domains – as well. There has already been an increasing push in the literature to re-set the traditional objective approach and to be more open to an equal role of subjectivity in assessment.^5,17,18 We would like to go one step further and besides changing the focus on fairness in assessment, apply this focus to learning pathways as well.

Scholarly doctors with an academic mindset in the realm of patient care are crucial to ensure, critically appraise, and advance the quality of patient care. To develop genuine academic mindsets within academia, all students deserve to have their unique abilities recognized and be intellectually stimulated at their own level. Therefore, we encourage medical teachers, curriculum coordinators, and faculty members to support diverse, flexible, and individual learning paths, including procedural variation with a more subjective, programmatic approach with feedback and feedforward conversations between a scholarly supervisor and his/her pupil. In practice, this suggests a minimum rather than a maximum of, among others, study time and feedback and less standardized educational procedures in scholarly courses to allow academic freedom to flourish. Shifting the focus from pure objectivity to both objectivity and subjectivity in assessment as well as learning pathways can assist in protecting fairness and, as a result, bring back Bildung to medical education to ensure future doctors to be truescholars.