A Paradigm Shift in Assessment of Scientific Skills in Undergraduate Medical Education

Introduction

The General Medical Council’s (GMC) publication ‘Outcomes for Graduates’ outlines the knowledge and skills required of newly qualified doctors in the United Kingdom (UK).¹ These guidelines focus on three main domains spanning the breadth of good clinical practice: professional values and behaviors, professional skills and professional knowledge.¹ Across these domains there is emphasis on doctors having the ability to integrate biomedical science, research and scholarship with clinical practice. The concept of evidence-based medicine underpins modern clinical practice. It is grounded on the principle of implementing current best evidence to provide the highest standards of care and achieve the best outcomes for patients.^2,3 Constant advances in medical knowledge demand that all clinicians are equipped with a skillset that enables them to access, interpret and evaluate new evidence as it arises, and appropriately integrate this with their practice. These skills are of paramount importance to any clinician who is directly involved in conducting academic research, as a robust scientific approach assures the quality of the evidence on which we base our practice.⁴ Medical schools must, therefore, have a curriculum and assessment strategy that is directed towards students developing the knowledge and skills to practice evidence-based medicine and the potential to conduct high caliber scientific research.³

Aligned with the GMC’s outcomes relating to the integration of biomedical science with clinical practice, a new paradigm of assessment was introduced for fourth year undergraduate medical students on the intercalated Bachelor of Science (iBSc) program at Imperial College School of Medicine (ICSM) in 2019. Assessment is recognized to have significant influence over students’ learning behaviors, and those which are authentic and have practical application encourage a deeper approach to learning.^5–7 Our novel approach therefore moves away from conventional written examinations to an evidence-based system of applied, authentic “active learning” assessments.

In contrast to the traditional learning format where students form a “passive” audience who listen to information delivered by a lecturer, active learning is achieved through completion of meaningful learning exercises (individually or in groups) which require students to think about, and directly apply, the knowledge and concepts that are intended to be taught.^8,9 Active learning can therefore be used to simultaneously teach applied skills and cognitive processes such as critical thinking and complex problem-solving, alongside subject-specific knowledge.^8–10 These are important skills to foster in future doctors as they are integral to accurate clinical decision making, potentially reducing diagnostic errors and improving patient safety.^11,12 Taking an active approach to learning has shown to improve academic outcomes for students across scientific disciplines, and there is a large pedagogical literature base (including a meta-analysis of 225 studies) supporting improved assessment scores, reduced failure rates and reductions in attainment gaps.^8,10,13,14

Cognitive psychology research has demonstrated that active learning tasks should be designed to specifically practise the skills that students need to learn, as this leads to development of expertise in those areas.^9,15 If we wish to equip our students with the ability to search the literature, critically appraise research and apply scientific reasoning, we must provide learning opportunities that specifically support the development of these skills. We therefore designed this new system of assessments with assignments that are analogous to tasks encountered in a research environment (Figure 1).⁸ Through authentic replication of academic activities such as scientific writing and presentation, literature searches and data analysis, this novel approach intends to teach and test a wider range of applied scientific skills than is possible with traditional essay-based examinations.

Figure 1 Overview of assessments of scientific skills in the iBSc program at Imperial College School of Medicine.

Our Innovative Approach

The iBSc program constitutes the fourth year of the six-year undergraduate medical degree at ICSM. The program includes 15 pathways that are delivered via three modules. “Module 1” develops core research skills alongside specialism-specific knowledge. Assessments include a written task, oral presentation and a data management assignment. The written assignment involves students writing a “Letter to the Editor” of a journal or a commentary addressing a topic within their chosen field. This letter is based on a topical or contentious subject, research article or a report of a relevant major clinical study. The presentation assessment focuses either on comparing aspects of their specialism or a controversial topic in the field, and these tasks can include discussion of an area of research, therapies or diagnostics, a pitch for a research proposal, or review of a paper. This work is presented via oral or poster presentation, or using digital story telling. Finally, students perform analysis on a data set they are given or have gathered through their own laboratory work. They generate a hypothesis, perform appropriate analysis, and draw conclusions. This is assessed through three written tasks: a results compendium (a succinct but comprehensive account of their data management and analysis), a scientific abstract and production of a lay summary. Alongside subject-specific knowledge, these assessments are designed to target the development of fundamental scientific skills including scientific writing, consulting literature, critical appraisal, understanding of research methodologies and data analysis and interpretation. Furthermore, the focus on written and verbal communication of scientific information to different audiences is directly transferrable to clinical practice: doctors must be able to communicate clinical and scientific information safely and effectively to a wide range of colleagues and patients.

“Module 2” consists of two written tasks: a group literature review and a patient case report titled “Science in Context”. The group literature review assessment is designed to emulate the academic publishing process and is based on evidence that completing active learning tasks in small groups significantly increases learning.⁹ Groups perform an in-depth, self-directed literature search on an assigned topic within their specialism and submit a written literature review, plus an accompanying cover letter to “the Editor”. Each initial submission is marked by a faculty member (“the Editor”) and undergoes reciprocal peer review by another student group. In response to peer and “Editor” feedback, groups revise their literature review and re-submit it along with a rebuttal letter detailing how reviewers’ comments were addressed, thus simulating the stages of publication from manuscript preparation through to acceptance.

Marks for teamwork in this assessment are generated using a modified comprehensive assessment of team member effectiveness (CATME) score, which evaluates performance within a team by rating each individual across five behavioral domains that have been shown to correlate with effective teamwork (Figure 2).^16,17 Students are required to submit peer scores and feedback comments for each member of their group across all five domains, as well as a self-evaluation of their own performance. An overall group mark is awarded for the written review by the faculty. This mark is subsequently adjusted up or down for each student by their individual modified CATME peer score (awarded by their team) to generate a final individual mark (Figure 2). A high modified CATME peer score reflects good teamwork and would result in the group mark being adjusted upwards, resulting in a higher final individual assessment mark for that student. A student who received a lower modified CATME peer score would have the group mark adjusted down, receiving a lower final mark for this assessment. The CATME system has shown to be a reliable method of assessment with good interrater agreement and correlate with final course results.^16–18 In addition to building skills in literature searching, scientific writing and critical appraisal, this task also fosters professional skills in providing feedback, teamworking, and accountability, all of which are essential to clinical practice.

Figure 2 Example of generating individual assessment marks from group work using the modified CATME Score17.

The second assessment of “Module 2”, “Science in Context”, is a written clinical case study based on a real or virtual patient. Students write a report which includes a clinical case summary of the patient’s presentation and a discussion of the current and emerging evidence for either the underlying pathophysiology or the clinical management of the condition, commenting on the level of available evidence. This task serves to highlight the inextricable link between scientific research and evidence-based clinical practice while developing students’ ability to systematically present clinical information.

The aim of Module 3 is to consolidate learning across the iBSc program with students conducting an original research project either in a clinical or laboratory-based setting. This is assessed via a written report in the style of a research paper and an oral presentation. In the oral presentation, the students should present a critical analysis of their project including limitations and future research directions, alongside a reflective account of their research experience and skills gained throughout the iBSc program.

Conclusion

At Imperial College School of Medicine, we aspire to train future clinicians who apply scientific principles to critique the evidence base of their medical practice, work as reflective team-players, and possess the skillset to pursue a clinical academic career. Using this innovative approach to assessment of the fourth year undergraduate iBSc degree program, we can assess specialism-specific knowledge within an applied scientific context, while also fostering essential professional skills for practicing safe, evidence-based medicine. We have therefore moved away from traditional assessment of learning to a program using authentic active learning assessments for learning. Supported by extensive pedagogical research we believe this is an effective approach to achieving the GMC outcomes and training our students to graduate as scholars as well as clinical practitioners and professionals.