Involving medical students in service improvement: evaluation of a student-led, extracurricular, multidisciplinary quality improvement initiative

Introduction

The NHS faces an aging population with complex, long-term health conditions, in conjunction with insufficient capacity, stretched resources and financial constraint.¹ Despite these challenges, it is essential that quality of services are maintained and improved.^2,3 As such, service improvement is considered an essential duty of every doctor.⁴

Alongside doctors, other groups involved in the delivery of healthcare can influence service improvement. In particular, medical students possess attributes and are exposed to opportunities which place them in a strong position to improve the quality of care. For instance, they: regularly rotate between different institutions and specialties; have frequent contact with patients, families and front-line staff; are not yet burdened with clinical commitments; and are unbiased and fresh observers of health-care teams. As a result, they are more likely to identify outdated practice and produce innovative solutions to quality issues.^5,6 Indeed, evidence suggests they are capable of completing successful quality improvement projects (QIPs),² in addition to advancing the quality of care during clinical placements.^6,7 It is essential, therefore, to nurture the quality improvement (QI) skills of medical students.

In recognition of this, a number of medical schools have introduced QI training programs into their curricula. However, this still constitutes a minority, offering limited guidance to other institutions.^8,9 In contrast, at University College London Medical School (UCLMS), students have engaged in QI through an extracurricular student-led initiative; providing a unique environment to explore how they can be involved in service improvement. This paper describes and qualitatively evaluates the initiative, including its organization and outputs. In doing so, it aims to add to the experiences gained by other groups, and offer guidance for establishing medical student QI programs.

Methods

The initiative’s design

Leadership

The initiative comprized of a core leadership team of four medical students, named the initiative’s leaders, ranging from their second to their fifth year of study. One out of the four students acted as the president, and retained ultimate control over the initiative.

Student recruitment

Involvement in the initiative was advertised to students via social media, a website, campus posters, email bulletins and the medical school student magazine. The applicants were required to provide a curriculum vitae (CV), and successful candidates selected to participate by the initiative’s leaders. Students from Medicine, Pharmacy, Medicine with Data Science, Computer Science and Engineering, and Data Science were invited to apply. Furthermore, students from all year groups were invited to apply.

Student education

Prior to project commencement, students were educated on QI methodology through a half day seminar or lecture organized by the initiative’s leaders, depending on when they joined the initiative.

Project conceptualization

The initiative’s leadership sourced QIPs through attending the University College London Hospital (UCLH) board meetings, as well as keeping regular contact with the medical school faculty, local research hospital leaders, Health Foundation fellows and third parties. QIPs also originated from students’ own observations from clinical placements, as well as those gained during participation in the Trust’s Improving Care Walk Rounds (Figure 1). These Walk Rounds are led by the Trust’s Quality and Safety Committee and Department of Informatics, and involve staff auditing the standard of care on a regular basis.

Figure 1 Illustrative example of the initiative’s structure and operation. Firstly, the initiative’s leaders seek to conceptualize projects. This is done in two ways: route (1) the initiative’s leaders regularly attend UCLH (University College London Hospital) board and research meetings in order to find suitable service improvement projects; or route (2) the initiative’s leaders recruit a small number of students who conceptualize projects from their observations of the clinical environment, which are gained by attending the UCLH Improving Care Walk Rounds or from their medical school clinical placements. Secondly, the supervision of projects is decided. Those projects conceptualized in route (1) can be supervised by senior clinical staff/researchers, or the initiative’s leaders. Alternatively, projects conceptualized in route (2) remain supervised by the initiative’s leaders. After then, students from a range of university disciplines are recruited to join the projects and form MDTs (multidisciplinary teams). Depending on the presence of a senior academic supervisor, teams take the form of either Project Structure A or Project Structure B. In both project structures, a recruited student is selected to act as the leader for that project.

Project supervision

Throughout project work, students were mainly supervised by the initiative’s leaders and student project leaders, or more senior academic supervisors if present. Progress was monitored through regular meetings, and communication was maintained via Slack (a messaging platform with file sharing capabilities and team communication tools) (Figure 1).

Timeframe for project completion

Involvement in the initiative was extracurricular, and not constrained by the academic calendar.

Participant incentives

Following project completion, students were encouraged by the initiative’s leaders to write up their project and consequently present and publish it.

Study design and data collection

In order to measure the effectiveness of the initiative, retrospective data was collected on the proportion of the initiative’s projects which: 1) reached completion (for example, projects which resulted in the implementation of a change and where post change data was collected and analyzed); 2) resulted in a significant improvement in their primary outcome; 3) had sustained results at follow-up (conducted 3 years post project completion for the earliest projects, and only on projects which resulted in the implementation of a change (regardless of whether they achieved full completion)); 4) achieved publication; and 5) contributed towards a prize or conference presentation. The data collection period was from the initiative’s launch in 2015, until 2019.

In addition to the aforementioned points, retrospective data was collected on how external parties evaluated the effectiveness of the initiative, as evidenced through prizes awarded to the initiative’s concept and output as a whole. Furthermore, how the initiative has influenced practice locally, such as the medical school and local hospital, has been described.

Ethical approval

Where required, each individual project produced by the initiative sought ethical approval. Considering this study evaluates these individual projects in the aggregate, we feel that ethical approval is not required.

Results

The initiative’s design

The initiative’s leadership, student education, project timeframe and participant incentives occurred as described in the methods section.

Student recruitment

There were 109 students involved in the initiative from its launch in 2015 until 2019. Students came from a range of university courses including 19 Pharmacy (17.5%), 75 Medicine (68.8%), 6 Data Science (5.5%), and 10 Computer Science and Engineering (9.2%) students (Table 1).

Table 1 Detailed team information of the student teams working on initiative’s projects and the prizes, posters and publications of each project

The recruitment process occurred as described in the methods section, however, it attracted a greater number of students than places available. Subsequently, applicants were screened and selected based on their CVs, and the process became competitive.

Project conceptualization

Seven out of 10 projects had their initial concepts generated by students.

Project supervision

Nine out of 10 projects were academically supervised by students, with the presence of more senior medical and faculty staff only acting as clinical facilitators.

Effectiveness of the initiative

During the time period, the initiative generated 10 projects, two of which are still currently ongoing. A brief description of these projects can be found in Table 2. Two types of methodology were used, seven based on “Plan-Do-Study-Act” (PDSA) cycles, and two using a conventional research format. Four projects reached completion, half of which achieved a significant improvement in their primary outcome (two out of four). At follow-up, the changes implemented were still in place in all applicable projects (five out of five applicable projects). Four projects did not reach completion, however, two of these projects (half of uncompleted projects) still resulted in the implementation of a change; only failing to generate post-change data. Two out of 10 projects (20%) have been published in journals, and six have been presented at at least one conference in addition to directly receiving, or contributing, towards a prize (60%) (Table 1).

Table 2 Brief descriptions of individual projects organized by BTC UCL with achieved outcomes

The initiative has in itself resulted in five awards, as well as three presentations, and catalyzed change at UCLMS (Table 3). After the initiative was successful at the NHS Quality Improvement Championships in 2016, the students were invited to hold the first ever student-led Grand Round to talk about their success. Key faculty representatives were present, which sparked discussion surrounding involvement of QI education into the curriculum. The conclusions of these discussions are currently still in development. It also led to the Change-Makers project which set about surveying the baseline QI knowledge of UCLMS students, and educating them on the key principles. Most importantly, it led to the establishment of an official collaboration with the Chief Executive’s Office of the Local Hospital and the Deans of Medical and Pharmacy schools. Students from the initiative are now involved in the Research Hospital initiative at UCLH and play a fundamental role in clinical audit and patient safety. The student representative now sits on the Quality and Safety Committee and in the Research and Innovation Committee of the Trust.

Table 3 Selected initiative’s achievements 2016–2019

Discussion

The initiative’s design

Student recruitment

The majority of student recruitment was done through social media, with an active website, Facebook page, and Twitter page providing multiple media through which the initiative gained exposure. Additionally, interest was generated via an active blog, with content redistributed through social media pages which showcased the achievements of the initiative. Since recruitment was carried out by a student team rather than senior supervisors, there was greater ease of access to the student network. One of the main advantages of this type of digital communication is that it provides a cheap and efficient vehicle through which many university students can be contacted.

It was also acknowledged that, although social media is ubiquitous, it is important to employ multiple avenues of advertising as a sole reliance on social media has the potential to exclude students and key non-student stakeholders not part of this social media network. The initiative was, thereby, also advertised in the medical school’s weekly newsletter, the medical school student magazine, and public spaces.

As demonstrated by the oversubscription of places on all projects, this strategy overcame issues experienced by other extracurricular research initiatives, which describe difficulty surrounding student awareness of available opportunities, and an inability to attract enough interest from students.¹⁰

As previously established, our initiative utilized and observed the unique attributes of medical students as agents of change. In particular, this includes greater flexibility, availability and ease of adoption of new technologies and methods of inter-team communication. We suspect that the extracurricular and competitive recruitment nature of the initiative meant that students were more likely to be self-motivated, ambitious and possess prior leadership, research or QI experience;¹⁰ and we believe this may have benefited the QIP outcomes. As such, we feel it is a factor worth formally testing in future research.

The multidisciplinary approach including students from a variety of university disciplines is an entirely novel concept.² Students from different courses possess different skills, capabilities and knowledge and, subsequently, we speculate this will have widened the potential focus, solutions and impact of the projects. Furthermore, working in a team composed of people from different academic and professional backgrounds more closely resembles real working environments, which may have helped to equip the initiative’s participants with valuable professional skills.

Student education

The initiative educated medical students on the basics of QI methodology using either a half day seminar or lecture, organized by the initiative’s leaders. This provided considerably less QI education compared to curriculum based programs, which involved more detailed and regular QI training delivered by teachers with extensive QI experience.¹¹ Such extensive QI training will have, undoubtedly, enhanced students’ ability to carry out effective QIPs, and motivated them to participate above and beyond what was provided by the initiative described here.¹²

Despite this, we assume that students will still have gained a range of skills and knowledge through involvement in the initiative, even if projects were not completed. We believe this, not only because of the success of the initiative, but due to the fact students were at the forefront of the majority of the various projects’ conception and organization; thus, allowing them to develop and apply knowledge on the methodology and importance of QI. However, the gain of such knowledge was not formally tested in this study. In the same regard, considering many of the projects required front-line interaction, we assume that students’ will have gained a better understanding of the clinical environment and healthcare system, as well as the chance to relate to patients on a more patient-centered level. This is opposed to the purely medical perspective that students are exposed to during medical school, and such benefits are also reflected in previous studies.^7,13,14 Collectively, this is likely to improve the students’ future medical practice and generate doctors who practice evidence-based medicine.¹⁰

Project concepts can be generated either by students or by senior clinical supervisors; while the study was not designed to directly compare the two methods, based on previous literature, we speculate that both have the potential to influence QIP outcomes. For instance, we suspect that student-generated projects were advantageous in that the involved students likely engaged more, experienced greater ownership, and gained a deeper understanding of the clinical problem. This is in accordance with Morrison & Sullivan (1997),¹³ who found that students learn less when joining preconstructed projects. Additionally, the conceptualization of projects may have been aided by students’ unbiased and unprejudiced view of healthcare environments. This is supported by Ibrahim et al (2013),⁵ who showed that medical students were more likely than experienced clinicians to notice potential areas of improvement. On the other hand, students’ limited clinical experience may have risked student QIPs possessing lower clinical significance. Similar difficulties have been described by Jackson et al (2018),¹² and Morrison & Sullivan (1997).¹³

Students were also able to conceptualize projects through their participation in key hospital quality improvement initiatives. For example, students were included in UCLH’s Improving Care Walk Rounds, led by the Quality and Safety Committee and Department of Informatics. These initiatives, despite being lightly resourced, generate essential reviews on a regular basis which ensure the quality of care. Consequently, this provides these initiatives with additional resources in the form of individuals who are interested and impartial.

Project supervision

The majority of the initiative’s projects were supervised by the students themselves, with senior clinicians only acting as clinical facilitators. Projects which received supervision or facilitation by senior clinicians may have benefited from their experience, connections within the clinical environment, access to resources and continued presence beyond project completion which could have enhanced sustainability. However, we suspect a number of ways through which senior supervision could have hindered the progress of projects; first of which is their lack of availability. For instance, when projects were directed by senior clinicians, students lacked the autonomy to proceed with project development in their absence. This had the tendency to delay the progression or commencement of projects depending on the period of their unavailability. Secondly, is senior clinicians’ occasional negative perception surrounding medical student involvement, as they play a pivotal role in encouraging the adoption of changes within a healthcare environment. If projects are to be supervised by senior clinicians, Jackson et al (2018),¹² Nikkar-Esfahani et al (2012)¹⁰ and Morrison & Sullivan (1997)¹³ suggest these clinicians should be both adequately trained in QIP leadership, and appreciative that medical student projects can improve care.

Timeframe for project completion

Time restriction, as mentioned by previous curriculum-based program, is one of the most influential factors affecting completion and potential scope of medical student QIPs. Furthermore, timeframe restriction has been reported to discourage students pursuing QIPs altogether.^10,12,13 Due to the extracurricular nature of the initiative described here, students were not constrained within a curriculum-set timeframe and had the flexibility to choose how much time to dedicate. As supported by previous literature, we believe this may have allowed a higher rate of project completion, sustainability and primary outcome success.^10,12,13

Equally, due to the initiative’s strong connections with executives at UCLH, students were able to be transiently involved with larger scale, long-term QIPs. This allowed students unable to commit to the entirety of an individual QIP the opportunity to gain QI skills and experience, as well as providing valuable input towards project progress.

Participant incentives

At the initiative’s conception, we hypothesized that student engagement would decline throughout the year without adequate incentives. This was based on the leaders’ previous experience that students’ interest in extracurricular activities tends to wane throughout the year due to growing coursework and clinical responsibilities. Therefore, the initiative incentivized students with the opportunity to present their work at a conference or aim for publication, and to plan this from the outset. Whilst this study did not formally assess the effectiveness of this incentivization, considering the high output of projects (10) within the study period (2015–2019), it is unlikely that student engagement was difficult to maintain.

Additionally, we believe incentivizing students to publish and present their work will aid the initiative in the future, by disseminating its success and thus attracting future participants, stakeholders and supervisors.

Effectiveness of the initiative

We believe the initiative was effective at involving medical students in QI because of the high number of projects generated by the initiative (10 projects) during the timeframe (2015 to 2019), as well as the proportion of completed projects which achieved a significant improvement in their primary outcome (50%). This is supported by the proportion of projects which resulted in the implementation of a change which was sustained at follow-up (100%); a measure that has been difficult to obtain when measuring the effectiveness of previous medical student QIPs.² Furthermore, 20% of projects were published in journals, and 60% were presented at atleast one conference, in addition to directly receiving, or contributing, towards a prize. This further validates the quality of the QIPs, as judged by the respective journals and judgement panels. Collectively, this suggests that the quality of projects were such that they correctly identified practice that could be improved, and implemented strategies which resulted in change.

The initiative’s quality is further affirmed by the number of awards achieved, especially considering the leadership and QI expertise possessed by the judging panels. Furthermore, the local impact of the initiative is important because it gained the attention of key local stakeholders, such as the medical school faculty, executive hospital board and QI team; which will provide ongoing support to the initiative, and facilitate its sustainability. Moreover, the fact that it was able to gain the support of such stakeholders, reflects the strength of the initiative as judged by them.

Study strengths and limitations

The strength of this study comes from the initiative’s novel design. In particular, this includes the use of students from a range of university subjects and its extracurricular nature, as well as considering elements such as participant incentives, and innovative ways of both recruiting students and utilizing technology. It also builds upon knowledge surrounding the effectiveness of other QI student program characteristics, such as project conceptualization and student supervision. Unfortunately, this study does not test each of these characteristics in turn, and therefore the results cannot be directly attributed to any single characteristic. As such, we suggest that further research should look at testing these factors in turn.

Another weakness of this study is due to the unvalidated method used to evaluate the quality of QIPs. Due to the retrospective nature of the study, crucial data required for such methods, for some of the projects, were not available. Future studies should incorporate validated methods, such as SQUIRE,¹⁵ to assess the quality of QIPs when testing the characteristics of their programs.

Moreover, the study did not formally test the students’ gain of skills and knowledge, and the initiative’s design incorporated far less QI training and support as offered by other programs. Therefore, we plan to adjust the initiative’s education process in the future and collect data to support how students gained skills through participation.

The final limitation is that our work, although conducted over several years and in various medical fields, comes from a single university, and different local regulations may be in place at other institutions. Therefore, the model described in the paper may need to be adjusted based on the local needs.

Conclusion

The results of this study suggest that the unique and innovative design of this quality improvement initiative has the ability to generate quality QIPs, and successfully involve medical students in QI in the context of a London Medical School and its local teaching hospital. Such features of our initiative’s design include its: student leadership; multidisciplinary approach; extracurricular nature; utilization of technology and social media to recruit students and aid team communication; use of student incentives such as publication and presentation; and formation of connections with key stakeholders such as the medical school faculty and local hospital executives and QI team. Furthermore, it builds upon existing knowledge provided by the analysis of other medical student QI programs, indicating that both student- and supervisor-generated or supervised projects can be equally successful. Moreover, due to the involvement of students in all stages of the initiative, it is highly likely that the initiative provided students with QI skills, knowledge and experience.

As such, other institutions building their own QI programs or involving medical students in QI, may wish to use some, or all, of the aspects of our program to enhance their own; depending on the local requirements and restrictions.

Future research should be aimed at further elucidating which individual factors of our program has the greatest effect on QIP quality, as well as quantifying and validating the skills, knowledge and experiences gained by students, and the effect this has on their future careers.