Virtual Interview, Real Anxiety: Prospective Evaluation of a Focused Teaching Programme on Confidence Levels Among Medical Students Applying for Academic Clinical Posts

Introduction

The General Medical Council outlines the ability to rigorously evaluate scientific evidence and contribute to research among key expectations of any clinician practising medicine in the United Kingdom (UK).¹ However, only a small fraction of doctors consistently dedicate a substantial proportion of their time to research activities and posts with an integrated academic pathway have been historically very competitive.² In particular, only 7.9% of Foundation Programme (FP) jobs have an integrated academic component and fewer than 250 Academic Clinical Fellowships at the post-FP level are advertised each year across the UK.^3,4

While securing a standard FP job at a competitive deanery solely depends on educational achievements and a standardized written assessment judging one’s interpersonal skills, Academic Foundation Programme (AFP) posts are largely allocated based on candidate’s performance during a short structured interview.^2,5 In fact, for some of the UK’s most competitive deaneries, including London, the interview score contributes over 60% to candidate’s overall rank.³ As it is well established that anxiety is more prevalent among medical students than the general population,⁶ it is conceivable that high-stake interviews with the potential of setting a favorable career trajectory may lead to considerable anxiety among this cohort.

We have previously demonstrated that a structured course on academic medicine can increase confidence, knowledge and preparedness among motivated candidates wishing to apply for an academic training post.^7,8 However, the 2020/2021 cohort is faced with an unprecedented challenge due to the Coronavirus Disease 2019 (COVID-19) pandemic and the move to online interviews. Online interviewing among medical trainees has been previously trialled in speciality interviews in the United States and has been well received by both faculty members and prospective applicants, but the impact on the junior and largely unexperienced cohort of final year medical students is currently unclear.⁹ Additionally, whilst many resources on improving application scores and maximizing performance in medical interviews have been devised to date, few guides have been published for online interviews and, to the best of our knowledge, no such materials specifically tailored at applicants for junior clinical academic posts in the UK exist.^10–14

In this study, we built on our experience of delivering teaching using an online platform^15,16 and designed a focused full-day course aimed specifically at improving candidate’s performance in academic clinical interviews conducted online in these unprecedented times. We then assessed the impact of this on candidates’ anxiety and confidence regarding the new format.

Materials and Methods

Study Design

This study utilized a cross-sectional design in which the impact of a day-long national teaching programme for prospective AFP applicants on participant’s anxiety related to the change in interview format was assessed. This teaching course has been developed over the previous five years and newly tailored to what will likely be expected of the 2020/2021 cohort, namely participating in an online interviewing process.^7,8 The new interventions included a full course delivered online using the Zoom platform (Zoom Video Communications, Inc.; Eric Yuan) and an online mock interview.

Study participants were asked to fill in three questionnaires: pre-course questionnaire collecting information on demographic data and general perception of the interviewing process; pre-interview questionnaire in the MASI format (please see below) and post-course questionnaire that combined the MASI tool and additional questions relating to candidate’s confidence levels.

The Intervention

The course began with focused lectures on clinical interviews and critical appraisal, followed by a practical critical appraisal workshop. Subsequently, participants were provided with an abstract of a research paper and given a reading time of 15 minutes. The candidates were then asked to attend a 15-minute interview on Zoom with one of the faculty. Prior to the interview day, the candidates had notified the course organizers which deaneries they wished to apply to, so that interviews were bespoke. Each interview was composed of two 7-minute-long sections considering emergency clinical scenarios and the critical appraisal of a research abstract. The examination format was viva-style. The interviews were followed by 5 minutes of personalized feedback from the interviewer focusing on the aspects of performance unique to the online format. The feedback was delivered using the validated Promoting Excellence and Reflective Learning in Simulation (PEARLS) debriefing script.¹⁷ All teachers and interviewers on the course were either current or former AFP trainees.

Anxiety and Confidence Measures

Participant’s anxiety was assessed using the Measure of Anxiety in Selection Interviews (MASI) score and subjective confidence questionnaires (Appendices A–C).¹⁸ The MASI consists of a 30-point questionnaire completed before and after the mock interview (Appendix B). It evaluates 5 domains of anxiety: Communication (D1); Appearance (D2); Social (D3); Performance (D4); and Behavioral (D5). The MASI questionnaire was originally published by John Wiley & Sons Incl. press and the right to reproduce it for the purpose of this study was kindly granted under the license agreement 5047721451965 (RightsLink). Components were scored on the scale of 1 to 5, as in the original study. The MASI questionnaire was modified in order to relate specifically to the online delivery of the academic interviews. As well as confidence, the pre-course questionnaires also assessed participant’s baseline experience in online and in-person interviews and included a free-text section.

Data was also collected on age, sex, year group, medical school, course type, and additional qualifications. The small size of our cohort allowed us to match answers from pre-course and post-course questionnaires in 68% of cases based on demographic identifiers for the subjective confidence rating and in 76.5% for the MASI questionnaires. In order to maintain participants’ anonymity, participants were assigned prior to data analysis a randomly generated candidate identifier. In this way, the individual conducting data analysis was blinded on the demographic characteristics and candidate’s anonymity was maintained.

Statistical Analysis

All statistical parameters (averages, means, medians, interquartile ranges [IQR] and standard deviations) were calculated in Origin Pro (OriginLab). When two parameters for the same individual were compared, Paired Sample Wilcoxon Signed Rank test was used. All quantitative questions in the three questionnaires were graded on a scale 1–5, apart from question 6 on the pre-interview MASI questionnaire. In this case, a 6-point scale was standardized accordingly before the answers were compared. Fisher’s exact test was used to compare qualitative answers among the two study cohorts. Statistical significance was calculated in Origin Pro and the exact p values are cited in the main text. p < 0.05 was considered statistically significant. All box plots show median (line), mean (small square), 5th, 95th (whiskers) and 25th and 75th percentiles. Participation in the questionnaires was voluntary and was not a requirement to complete the course.

Results

Study Cohort Characteristics and Baseline Interview Experience

Thirty-one individuals attended the course (Table 1), with 27 participants (87.1%) completing the pre-course questionnaire. Individuals came from 17 different universities across the UK, with most individuals attending Imperial College London (5), University of Birmingham (4) and University of Cambridge (3). The median age was 23 (IQR = 23–24). Males and females were almost equally represented in this cohort (female = 44%) and 93% of individuals were in their final or penultimate year of medical education. All participants were enrolled at an undergraduate-entry medical degree, with an additional 81.4% holding an intercalated degree.

Table 1 Baseline Characteristics of the Study Participants, Including Demographics and Educational Background

All respondents had experience at in-person interviews, with the median number of medical school interviews attended being 3 (Figure 1A; IQR = 2–4). However, 93% had not had an interview within the last year and 59% had not had an academic interview since application to medical school 5–6 years previously (Figure 1B). Conversely to the high experience in in-person interviews, only 45% had any experience in online interviews, with only 7.4% having experienced more than two interviews in this format (Figure 1C). Furthermore, most candidates stated that they would have strongly preferred to have an interview in-person rather than online (Figure 1D; median = 4, IQR = 3–5, where 1 corresponds to “definitely prefer online” and 5 corresponds to “definitely prefer in-person”).

Figure 1 Baseline interview experience of the study cohort. (A) Number of in-person medical school interviews attended by study participants. (B) Evaluation of when the last in-person interview experience was. (C) Number of online medical school interviews attended by study participants. (D) Preference of online versus in-person interviews by the study cohort expressed on a 5 point scale, where 1 = much prefer online and 5 = much prefer in person. n = 27.

Effects of the Intervention: Confidence Levels and MASI Anxiety Score

To evaluate the effects of the intervention, we compared questionnaire results before and after the online mock interview. Among course participants, 28 individuals (90.3%) accepted the invitation to attend a mock interview conducted online. Of these, 23 (82.1%) completed the pre-interview MASI questionnaire. Additionally, 22 (71.0%) filled in the post-course questionnaire.

Subjective confidence varied greatly across participants, although confidence was significantly higher for interviews in-person compared with online (Figure 2A; p = 0.005, Paired Sample Wilcoxon Signed Rank test). Furthermore, there was a significantly lower uptake of self-directed learning in the form of books or online resources among study subjects preparing them for interviews online than in-person (Figures 2B; 11.3% vs 59.3%; Fisher’s exact test, p < 0.001), and none of the participants had previously attended a structured course focusing on online interviews. Moreover, only 3.5% of study participants (1/27) had received guidance from their university on how to succeed at online interviews (Figure 2C). Study participants were also unsatisfied with the information regarding the interview provided by the body in charge of the official AFP interviews (Figure 2D; median = 2, IQR = 2–3; where 1 = very insufficient and 5 = extremely comprehensive), with only 48.1% reported having received information from the AFP deaneries on which internet platform the interviews will be taking place on (data not shown).

Figure 2 Pre-course awareness and confidence specific to the different interview formats. (A) Confidence levels pertaining to interview in-person and online compared at the level of individual course participants (paired data points; Paired Sample Wilcoxon Signed Rank test; p = 0.005) (B) Degree of self-directed preparation done prior to the course, relative to the interview format (Fisher’s exact test; p <0.001) (C) Fraction of course participants who received guidance from their own medical school on how to succeed at online interviews. (D) Satisfaction regarding the information obtained from the body organising the official Academic Foundation Programme (AFP) interviews, where 1 = very insufficient and 5 = extremely comprehensive. n =27.

We then evaluated whether having experienced an online mock interview can increase subjective confidence levels among the study cohort. The comparison of pre- and post-course questionnaires revealed that a single online mock interview did not affect the strong preference for having an interview in-person over an interview online (Figure 3A; p >0.05, Paired Sample Wilcoxon Signed Rank test). In fact, despite the experience candidates still felt that the online format would likely negatively affect their performance and the resultant interview score (Figure 3B; p >0.05, Paired Sample Wilcoxon Signed Rank test). However, when compared with pre-interview confidence, there was a significant increase specific to confidence at interviews online after attendance at the course (Figure 3C and D; Paired Sample Wilcoxon Signed Rank test: p >0.05 for interviews in-person; p = 0.009 for interviews online).

Figure 3 Impact of the intervention on subjective confidence levels pertaining to interviews on-line/in person. (A) Candidate’s self-reported preference for the Academic Foundation Programme (AFP) interview being conducted in-person rather than online (p = 0.718, not significant [n.s.]) (B) Candidate’s perception that the online interview format is likely going to affect the overall interview score (p = 0.276, n.s.) (C) Candidates confidence for interviews conducted in person (p = 0.058, n.s.) (D) Candidates confidence for interviews conducted online (p = 0.009). n = 15. All statistical analysis was done on paired data points (Paired Sample Wilcoxon Signed Rank test).

Following this, we compared the effect of the intervention on anxiety levels, as assessed by the MASI tool. Pre-interview, 35% of the answers to the MASI questions (n=390) scored above 3 on the 1–5 scale (where 1 = not anxious; 5 = very anxious), indicating that candidates displayed substantial anxiety in at least some aspects of the online interview process. Similarly to subjective confidence levels, overall baseline anxiety levels pre-interview differed greatly between individuals, with some scoring a median of 1 (IQR = 1–1), while others scoring a median of 5 (IQR = 3–5) across the 30 MASI questions.

Comparison with the post-interview MASI score (Figure 4) in those who could be reliably linked identified a reduction in the MASI score across all 5 domains and meeting statistical significance in 3 of those (D3 = Social Anxiety, D4 = Performance Anxiety and D5 = Behavioral Anxiety; D3: p = 0.004, D4: p <0.001, D5: p = 0.003).

Figure 4 Effects of the intervention on the Measure of Anxiety in Selection Interviews (MASI) score. (A) Example of how the MASI score changes following the intervention at the level of an individual candidate (Q21: “I worry that my online Academic Foundation Programme (AFP) interview performance will be lower than that of other applicants, who may be more experienced in the online format”; D4 – Performance Anxiety) (B) Average MASI score across the 5 domains, where 1 indicates low and 5 indicates high anxiety levels. D1 Communication Anxiety = 0.350 = not significant [n.s.]; D2 = Appearance Anxiety = 0.730 = n.s.; D3 = Social Anxiety = 0.004; D4 = Performance Anxiety <0.001; D5 = Behavioral Anxiety = 0.003). n =13. All statistical analyses were done on paired data points (Paired Sample Wilcoxon Signed Rank test).

Discussion

The selection process for the academic training pathway aims to identify individuals who are capable of combining clinical excellence with academic diligence.¹⁹ However, many of the parameters used in the scoring criteria are confounded by factors such as medical school attended, in that studying at a top medical school is likely to on average lower your decile ranking, whilst attending a less academically focused university may reduce the chance of additional presentations or publications. In contrast to this, interview-based selection may provide a standardized way of comparing the performance of all candidates using the same strictly defined metrics, thus alleviating some of the bias in the selection process.

The results of our study suggest that the online format of the AFP interviews imposed for the first time on the 2020/2021 cohort may, at least partially, defeat this goal. We identify that although experience in in-person interviews was ubiquitous, less than half of individuals had ever had an online interview. Individuals concordantly reported significantly lower confidence in online interviews, suggesting that the lower confidence in interviews may be due to unfamiliarity with the format. The baseline scores varied greatly between different individuals, which is expected given that both confidence and self-esteem levels are multi-factorial.²⁰ Crucially, however, we found that attendance at a one-day structured online course improved confidence in online interview performance and led to a reduction in three out of five key anxiety domains assessed by the MASI tool. As a content-rich course delivered online is an intervention that could be relatively easily executed UK-wide, our study suggests that effective tools to address this problem already exist.

Our study might be limited by the fact that our cohort may not be representative of the medical student population in the UK as a whole. 81.4% of course participants have intercalated an additional degree during their medical studies, which indicates that our course pre-selected for highly motivated individuals with a strong academic potential.²¹ This, together with the fact that our study cohort was relatively small, could potentially indicate that the intervention may not be equally effective at a population level. Thus, a larger study that included more participants of more diverse academic backgrounds might enhance the generalizability of our results. However, the AFP programme itself selects for highly motivated individuals and our results are thus likely to be applicable to the cohort of UK students interested in the academic pathway.²² Additionally, our course participants came from 17 different medical schools indicating comprehensive coverage across UK universities. Another limitation could be that in a few cases, we could not link the answers to pre-course and post-course questionnaires due to insufficient demographic information. The methodology could be further improved by allocating a unique identifier to each study participant before the commencement of the course, so that answers to all three questionnaires can be reliably linked.

While our study focused on prospective candidates for the AFP, it is likely that similar trends may be observed for higher-level academic posts as this year Academic Clinical Fellowships and Clinical Lecturer interviews will also be conducted largely online. Future studies evaluating the impact of a focused teaching programme on doctors applying for these posts might provide additional insight into the effectiveness of our intervention. Furthermore, at present, it is unclear when the in-person interview format for the UK’s academic pathways will be restored. As a simple and effective tool to improve confidence and reduce some of the anxiety associated with changes in academic pathway selection that were imposed by the COVID-19 pandemic exist, we believe that implementation of a similar tool in the curriculum of medical schools across the UK could go a long way in reducing the bias and increasing the likelihood of selecting the candidates that are most suitable for the academic training posts.

Conclusions

The results of this study demonstrate that there is a significant experience gap in online interviews amongst prospective Academic Foundation Programme trainees, but that this can be mitigated by a one-day structured online course with a personalized mock interview. It remains to be seen whether this effect can be replicated in different academic cohorts.