The medical specialty of faculty interviewers does not influence scores in the internal medicine residency interview

Introduction

The faculty interview plays a critical role in the internal medicine (IM) residency selection process.¹ With the increasing number of applicants and applicant preference for 1:1 interviews, there is a growing pressure on residency programs to expand their interviewer pool to accommodate this increased number of interviews.^2,3 As such, many IM residency programs are inviting medicine specialists (MS) in addition to general internal medicine (GIM) physicians (ie, primary care physicians and hospitalists) to conduct these interviews.

Although this intervention increases the pool of interviewers, it may also introduce inconsistencies in the evaluation process, as MS may rate applicants differently than GIM interviewers. As MS focus their clinical practice on a subset of IM, they may value applicant characteristics that are more favorable to their own specialty rather than IM as a whole. In addition, MS likely spend more time supervising subspecialty fellows rather than IM residents, which may affect their expectations for applicants. Having spent additional years of training, MS are also more distanced from their IM residency training and thus may have a different understanding of the needs of the IM program compared with their GIM colleagues. Given the importance of reliability and consistency across interviews to assure fairness, we sought to explore this potential bias within our own institution. To our knowledge, this is the first published study to investigate this question.

Methods

We performed a retrospective review of the Brigham and Women’s Hospital IM residency interview evaluation forms for the 2017–2018 application season. The faculty interviewers (FI) received a prepopulated form containing the applicant’s medical school, undergraduate degree, clerkship grades and USMLE test results. Prior to the interview, the FI were expected to review the full application and assign a “pre-interview” score based on all written material. After the 25 min interview with the applicant, the FI selected a final interview score based on both the paper record and the interview. Assigned scores ranged from 1 to 5 in 0.5-point increments, with 1 defined as “absolutely top candidate” and 5 as “not suitable.” All applicants underwent at least two interviews. We collected information on the GIM vs MS status, gender, medical school, and IM residency site of each FI through our faculty profile database.

We used means and standard deviations (SD) to describe continuous variables and counts and percentages for categorical variables. We separated interviewers into four categories based on mean final interview score given (ie, 1.0–1.5, 1.5–2.0, 2.0–2.5, and 2.5–3.0). We then compared baseline characteristics between the four subgroups of interviewers using trend tests based on linear regression, Cuzick’s non-parametric trend test, and the Chi-squared test for trend. To control for the observation that conducting more interviews was associated with assigning worse interview scores, we used a mixed-effects linear regression model with both applicant and interviewer ID modeled as random effects and interviewer specialty and total number of interviews as fixed effects. All analyses were conducted using STATA version 14 (College Station, TX). P-values less than 0.05 were considered significant. The study was reviewed and approved by the institutional review board of Partner’s Healthcare, who determined that FI consent was not necessary as faculty demographic information was de-identified.

Results

Over the 2017–18 recruitment period, there were a total of 634 interviews of 274 applicants by 72 FI, among whom 43 (59.7%) practiced in GIM and 29 (40.3%) practiced as MS (Table 1). Most FI were female (38, 52.8%) and prior graduates of the institution’s IM residency program (54, 75%).

Table 1 Baseline characteristics of faculty interviewers

The mean interview score (Figure 1) given by an FI was 2.0 (SD 0.4). Trend test analyses showed no association between an FI’s medicine specialty status (p=0.09) and the mean interview score given. Further, no significant trends were seen based on an FI’s gender (p=0.88), medical school (p=0.54), residency (p=0.49), or year of residency graduation (p=0.91). On linear regression, there was no significant difference in interview scores given by FI who practiced GIM vs those who practiced as MS (−0.13 change, p=0.168).

Figure 1 Distribution of mean interview scores by faculty interviewers.

Discussion

As IM residency programs seek to provide 1:1 interviews to the ever-increasing number of applicants, there is a growing practice of including MS in the FI pool. To our knowledge, this single center study is the first to investigate the potential impact of a faculty member’s medicine specialty on the IM applicant interview process. In this retrospective study, we found there was no significant difference in the interview scores assigned by MS compared with GIM interviewers.

In this institution, MS and GIM interviewers rate applicants similarly for a variety of potential reasons. For one, the IM residency program provides frame-of-reference training to all FI on the interview scoring system, which increases the likelihood that MS and GIM interviewers are using the scale in the same manner. Second, the evaluation form has anchors for each score on the interview rating scale (eg, “absolutely top candidate” for a score of 1), which helps improve the response process and accuracy of the interview scores given by both groups.⁴ Third, our institution has an integrated teaching unit model in which MS and GIM co-attend on the IM service.⁵ As many of our FI (both MS and GIM) attend on this service, we suspect that such collaboration and direct involvement in the IM residency helps create a shared perspective on the program that leads to similar evaluations of residency applicants.

The results of this study thus support the practice of including both MS and GIM in the IM residency selection process. By expanding the interviewer pool with MS, programs can interview more applicants while preserving the desired 1:1 interview. Including MS in the interview process allows programs to introduce applicants to a more diverse faculty as well as to match candidates with faculty in their expressed area of interest. Finally, pairing applicants with faculty interviewers who share interests may allow for early identification of mentoring relationships that can develop regardless of where an applicant eventually matches.

Although not the primary aim of the study, it is important to note that we also found no association between an FI’s gender and the mean interview score given. This is consistent with a prior single-center study that similarly found no such relationship at their institution.⁶ These results should help allay concerns that faculty gender biases interview scores in the IM residency selection process.

There are important limitations of this study that warrant discussion. The main limitation is that this was a single-center study conducted over the course of one applicant cycle, which raises questions about the generalizability of the results. In addition, a large proportion of FI were graduates of this institution’s IM residency program, which may not be the case at other sites. Third, variable faculty schedules make consistent distribution of FI across the applicant pool difficult. It was therefore not possible for each applicant in this study to have been interviewed by exactly one MS and GIM interviewer. However, given the large number of interviews analyzed in the study, we believe that we would have been able to detect a difference if one existed. Given these limitations, we intend to continue investigating this question over consecutive years and across multiple programs in subsequent studies.

Conclusion

In this single-center retrospective study, we did not find any significant difference in the interview scores given to internal medicine residency applicants by general internal medicine physicians vs medicine specialists. This lack of identified bias supports the inclusion of medicine specialists in the internal medicine residency selection process.