Exploring the Validity of an Instrument That Measures Final Year Medical Students’ Career Choice Preferences

Introduction

Medical professionals are key members of the health workforce, but there is often a mismatch between which speciality they elect to work in and the healthcare needs of the population. While many medical students have an idea of future speciality preference, more than half of these career preferences change over the course of their medical degree, and many remain undecided at graduation.^1,2 Even those students who start with a strong preference towards a speciality can change their preference as there are many factors that influence career choice.³ Often, speciality training choices are difficult to reverse once made, as changes are time-consuming and may be associated with emotional and financial stress.¹

Speciality career choice is also important for the individual as it affects career satisfaction and personal well-being later in life.¹ At a system level, career choices also shape the future medical workforce as they determine the availability and distribution of clinicians to serve the healthcare needs of the population.^4,5 Shortages and maldistribution of doctors across various specialties, particularly general practice and in rural areas, are global concerns.^5–7 High levels of interest for certain specialties, along with increased competition, also appear to skew the distribution of medical graduates across some specialities, while other specialties, such as psychiatry and general practice, are unable to attract enough graduates to meet the needs of the population.^3,8 In addition, there is now an increase in chronic diseases, and more healthcare is delivered in community-based care.⁹ It is, therefore, essential that populations are served with an adequate mix of medical professionals and specialties to address the demands of the population.^{4–6,10–12}

Bland et al proposed a model based on the factors influencing medical students’ career choice literature in the 1990s to address shortages in general practice in the United States.¹³ The model demonstrates that career choice is shaped by the medical students’ preferences and values are affected by the students’ characteristics such as age, gender, educational background, characteristics of their medical program, exposure to various specialties, mentors, social influences and future career needs.¹³ Furthermore, a strong match between personal perceived future career needs and the characteristics of a specialty can determine students’ future career choices. Although the applicability of this model across various specialties is unclear, it is plausible that the dynamics of factors contributing to career choice decisions vary among individuals, communities and populations. Consequently, medical educators have continued to explore the factors that influence medical graduates’ career choices in an attempt to address these needs.¹⁴

A range of methods can be used to evaluate medical students’ career choices including document analysis, self-report questionnaires, and retrospective interviews conducted later in the medical career. In collaboration with Australia, the NZ Medical Schools Outcomes Database (MSOD) was established in 2005 as a prospective longitudinal multiple cohort study of medical graduates to collect national-level data using surveys to evaluate demographic backgrounds, influential career choice factors and future career intentions.^3,15 The information gained from the longitudinal research directly informs curriculum development and future medical workforce planning, particularly with respect to careers where there are shortages.¹⁵ Within the MSOD questionnaire, there are 24 items that focus on Factors Influencing Career Choice (FICC). Development of these items was based on a Delphi method drawing on expert advice from key stakeholders and existing literature from the Australasian medical education field in the mid 2000’s.¹⁵ However, a comprehensive evaluation of the psychometric properties of these items has not been undertaken. Specifically, there has not been an evaluation of the effectiveness of the 24 items within the FICC domain in capturing the range of factors, nor if there are any correlations and latent constructs among the items to better understand factors influencing NZ medical students’ career choice preferences. Therefore, the aim of this study was to (a) explore factors underlying the 24 items and (b) to psychometrically explore the validity of the questionnaire to assess the model fit of the factors identified.

Methods

Participants

All medical students and graduates from each of NZ’s two medical programmes, offered at The University of Auckland | Waipapa Taumata Rau and The University of Otago | Ōtākou Whakaihu Waka were invited to participate in the MSOD survey on entry into medical school, on exiting, and further at one, three, five, and eight-years post-graduation.¹⁵ In this study, we focused on the exit data points. Note, regardless of entry pathway (undergraduate or graduate entry), the total duration of the NZ medical programme is six years. This study focused on two cohorts of final-year medical students: those graduating in 2017 (n=314) and 2018 (n=404) were available in their entirety at the time of analysis.

Data Source

The items contained in the FICC section of the MSOD questionnaire are displayed in (Table 1). Respondents are asked to rate the importance of each of the 24 question items on a 5-point Likert scale (1 = not at all, 3 = in the middle, 5 = a great deal). For this study, students’ age, gender, and responses to the FICC question item stems were extracted. Cases with missing values were removed to ensure consistency in the results; this decision is in accordance with the recommendation that deleting missing values is often a prudent course of action.¹⁶

Table 1 Factors Influencing Career Choice (FICC) Domain of the MSOD Questionnaire at Exit from Medical School

Research and Ethics Approval

This study was conducted with the ethics approval from the Human Participants Ethics Committees from the University of Auckland (Ref: 022338) and University of Otago (Ref: 07/155), and all participants gave written informed consent. The anonymity of respondents was assured.

Analysis

Exploratory Factor Analysis Conducted on the 2017 Cohort

The Statistical Package for the Social Sciences (SPSS version 26, IMB Corporation, New York, USA) was used to perform Exploratory Factor Analysis (EFA), descriptive frequency data generation, and reliability analyses. Descriptive statistics were obtained for each of the 24 items. EFA was undertaken on the 2017 student cohort to identify factors or underlying constructs that best represented the 24 items addressed in the FICC questionnaire.¹⁶ Principal component analysis was performed after confirming initial checks for sample adequacy using Kaiser–Meyer–Olkin (KMO), Bartlett’s test, and the R-matrix to ensure that the dataset was appropriate for the EFA.¹⁶ The Scree plot was visually inspected to identify the number of factors likely to best represent the 24 items, and factors with eigenvalues >1 were highlighted. The varimax (orthogonal) option was employed based on the assumption that the factors were independent of each other.¹⁶ Once the factors were established, Cronbach’s Alpha (α) reliability coefficients for each domain were calculated. Factor labels were established by examining common linguistic themes underlying item stems within the factors identified.

Confirmatory Factor Analysis Conducted on the 2018 Cohort

Analysis of Moment Structures (AMOS version 25, IBM corporation, New York) was used to undertake the Confirmatory Factor Analysis (CFA), which was implemented to assess the factor structure developed in the EFA. Model fit parameters, including Chi-Square, Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), and Standardised Root Mean Square Residual (SRMR), were used to evaluate the goodness of fit of the CFA structure based on those recommended by Kline.¹⁷ Modification fit indices were considered to optimise the model fit of the 2018 dataset. In order to achieve a parsimonious model, an iterative process was employed by auditing the model fit indices against Kline¹⁷ best fitted criteria by adding or removing paths between latent variables (factors) and observed variables (items). Factor loadings below 0.4 were removed as they indicated a weak prediction of the latent variable.¹⁷

Results

A total of 643 participant records were utilised in this study after excluding 75 records due to missing data from both cohorts. The EFA was conducted with the 314 final-year medical students who completed the 2017 MSOD-Exit Questionnaire (62% response rate); however, only 300 entries were included (14 records with missing values were excluded). There were 170 female and 130 male respondents with a mean age of 26 years (Table 2). For the CFA, the 2018 cohort contained 404 entries (76% response rate); however, after excluding cases with missing values (61 entries), a total of 340 records were analysed. The 2018 cohort had 190 female and 150 male respondents, with a mean age of 25 years.

Table 2 Demographic Information for 2017 and 2018 Graduating Medical Students

Exploratory Factor Analysis

The coefficient generated from the KMO test analysis was 0.80, and Bartlett’s significance value (p<0.001). Both these values indicated that the EFA was appropriate for analyzing the 2017 dataset. The R-matrix values illustrated adequate correlations (around 0.3) between the variables. The resultant Scree plots indicated the potential for four well-defined factors with eigenvalues >1. Details of the factor loadings are presented in Table 3. Variance explained (VE) statistics indicated that this four-factor structure had a total variance of 50.2%. The internal consistency (Cronbach’s alpha) measures were promising. The main underlying factors identified in this study were as follows: Finances and Rewards (Factor 1, VE = 24.02%, mean = 2.02, SD = 0.51 α = 0.81), Work Flexibility (Factor 2, VE = 11.3%, mean = 2.90, SD = 0.12, α= 0.73), Vocational Characteristics (Factor 3, VE = 9.26%, mean = 3.10, SD = 0.16, α= 0.73), Placement Experiences (Factor 4, VE = 5.63%, mean = 3.9, SD = 0.04, α= 0.67).

Table 3 EFA: Principal Component Analysis

Confirmation of the Best-Fitted Model

The standardised estimates provided by AMOS were used to describe the CFA model for the 2018 cohort (Figure 1). The initial CFA results indicated a poor model fit of CFI = 0.710, RMSEA = 0.09, SRMR = 0.108 and χ² (340) = 945.334 (p>0.05). A closer investigation of the CFA results also indicated potential issues in relation to wording flaws in the FICC question items. Following the process of model specifications, only four error variances (e1 to e4) were required to be correlated. Nine items from the original 24-item set were removed, given that their factor loadings did not improve or were below the 0.4 threshold. The nine deleted items were as follows: “Self-appraisal”, “Parents and relatives”, “Domestic circumstances”, “Intellectual content”, “Location”, “Typical patients”, “Work culture”, and “Helping people”. The final CFA model (Figure 1) confirmed that the four-factor 15-item model formed a reasonable fit on two of the four indices employed, ie, CFI = 0.90 and RMSEA = 0.08. The two other indices were marginally outside the required specifications, ie, with SRMR = 0.09 and χ² (340) = 273.910 (p>0.05). The highest factor loading is observed between Factor 4 Placement Experiences and Item 5 “medical school experiences” (0.88).

Figure 1 Best fitted CFA model for the 2018 cohort. Latent variables (factors) are indicated by ovals, observed variables (items) are shown within the boxes, and the unique error covariances are illustrated with circles labelled e1-e21. The direction of the straight arrows indicates the relationship between the observed and latent variables. Factor loadings for predicting the indicators from the latent variables are displayed by the straight arrows.

Discussion

This study aimed to explore the validity of an instrument measuring factors that influence NZ medical students’ career choices. The EFA was conducted to identify any meaningful constructs, ie factors within the 24 items. Our study identified four main factors: Placement Experiences, Work Flexibility, Vocational Characteristics, and Finances and Rewards. The CFA was then employed further to validate the identified factor structures from the EFA. The best-fit model (Figure 1) indicated a modest but meaningful statistical verification of the four-factor EFA structure. The results helped to identify possible semantic issues that could contribute to the marginal fit of SRMR in the overall CFA model. The overall utility of the four-factor model enables dimensional reduction of the item variables for analyses, particularly for longitudinal datasets.¹⁶ In other words, making it easier to model career choices and understand the key driving factor(s) that matter most across various groups and to track these changes over time.

Placement Experiences were one of the main factors in our study that illustrated high values for both the mean rating scores and factor loadings towards the latent construct of “clinical exposure”. A systematic review by Querido et al reviewed and evaluated 57 independent articles focusing on factors associated with undergraduate medical students’ career choice literature found that 30 out of 57 studies reported that experience in courses and rotations informed the perception of specialty characteristics.¹⁸ Our study’s EFA and CFA findings further indicate that exposure to the clinical setting of any given specialty is one of the most influential factors that predict future career choice. These findings are consistent with other studies that have reported this factor’s importance, particularly in the early phases of medical graduates’ career path.^19,20

Items within the factor of Work Flexibility encompassed some of the central aspects of day-to-day working conditions, including external influences such as “Location” and “Domestic Circumstances”. Variants of the item “working conditions” are often ranked among the top 5 factors influencing medical students’ career choice decisions; such conditions include the number of working hours, short working days, a stable or predictable lifestyle, enough time for leisure activities, and compatibility with family life and responsibilities.^1,12,21 High CFA factor loadings in our study for the items “working hours” and “flexible hours” (see Figure 1) indicate that having manageable working conditions that allow time for leisure and family life outside working hours is significantly valued.

Items within the factor Vocational Characteristics illustrated similar patterns as reported in other studies that relate to future employment and career prospects. Takeda et al classified similar items among the top two constructs in their study that employed EFA to identify factors influencing specialty preference.²² These elements were associated with their construct of “Bioscientific orientation” (in relation to career advancement opportunities) and “Fulfilling life with job security”.

The CFA results in our study further confirmed that the factor Finances and Rewards had the highest correlation (see Figure 1) with the factor Vocational Characteristics in comparison to the correlations among the four-factor structure. Our findings are also consistent with Bland’s career choice model whereby some future personal needs seem to be taken in consideration along with specialty characteristics. Items within the factor Finances and Rewards demonstrated low ranking scores towards the least influential end of the Likert Scale. These results are consistent with a similar NZ-based study by Webster et al which concluded that the number of sources of financial support matter more than the level of debt or expenses when choosing a future medical specialty.²³ Furthermore, even though financial aspects are important, our results suggest they may not be critical when deciding on a particular future career choice.

Nine out of 24 items were removed from the CFA model based on low regression weights (<0.4), and this amendment was used to explore a more acceptable model fit. However, some deleted items such as “Work culture”, “Intellectual content”, “Helping people”, “Self-appraisal”, “Location” and “Domestic circumstances” are often linked to important influencers in the annual MSOD reports as well as in the wider medical graduates’ career choice literature.^9,10,24,25 The initial CFA model, without any adjustments, indicated insufficient statistical support of the original EFA four-factor model, suggesting modifications to enable a more parsimonious model fit. For example, the item “Interest in helping people” was highly ranked as nearly all respondents significantly value this item resulting in a ceiling effect providing little statistical discriminatory effect.²⁶ Thus, while this factor is clearly important, it is not discriminating.

Nonetheless, it should be noted that while a few key items were removed to improve the final CFA model fit indices, these adjustments do not diminish the value of the identified factors. Rather these findings indicate that the current items may need to be reworded to capture a clearer understanding while analysing large sets of longitudinal data.²⁶ Particularly for key items such as “Atmosphere/work culture” and “General medical school experiences (eg, mentors, discipline placements)”. Medical students often assess their abilities and interest in relation to the clinical environment (positive and encouraging versus hostile and demanding in nature) and how they fit into it.^1,4 Future research could explore the significance of these influencers and how they contribute to various facets of the latent factors identified in this study. Addressing these semantic issues in relation to the wording of items, including triple-barrelled items, might be important along with conducting criterion-related validity checks against other validated instruments used in this area.²⁷ This is especially important when delving into the underlying complexities of the factors influencing career choice involving specialties in need of more doctors.^16,17,28 In addition, correlations of the items and some of the relationships underlying the factors are of benefit for medical schools and vocational training colleges as they could address these needs. For example, incorporating flexible training options.

Overall, the four identified factors show alignment with Bland’s proposed model as well as the range of factors commonly reported in the current literature^1,6,7,13,15 and hence were used to frame and explain our understanding of the items within the constructs. Placement Experiences align with characteristics of the medical programme and exposure to various specialties. Work Flexibility may align with societal and personal needs to satisfy, while the factor Vocational Characteristics reflect unique aspects pertained within a specialty such as type of patients and level of contact, employment and financial aspects. The factor Finances and Reward is, however, less well represented in Bland’s model. These results can therefore add to our understanding of factors influencing medical students’ career choice.

Strengths and Limitations

This novel study is the first to examine the MSOD questionnaire addressing the influences underlying medical students’ career choices in NZ. One of the main limitations of the questionnaire is the lack of conceptual foundation such that the FICC section was originally developed based on a Delphi method in the mid-2000s and no empirical findings related to the development process were published at this time. Nonetheless, our EFA results illustrated that the questionnaire performs well in capturing a variety of facets over the years.^3,11 Moreover, very limited studies have investigated the utility of such questionnaires using CFA within the medical graduates’ career choice literature. An orthogonal approach was used as we rationalized that the factors were likely independent. However, the factors Work Flexibility and Vocational Characteristics could be viewed as interrelated, while other factors are clearly distinct. Future research using an oblique method could be trialled to assess for critical differences. Additionally, we acknowledge that the removal of 75 entries due to missing data could also be further explored for any discernible patterns. Lastly, the MSOD questionnaire is a self-reported instrument and as such response and social desirability bias could possibly affect the observed responses.²⁶

Conclusion

The current FICC section of the MSOD and its items fit within established theoretical frameworks pertaining to medical students’ career choices. Our findings provide empirical evidence for their utility, with potential revisions being suggested. While some of the highest-ranked items were not present in the final CFA model, these items indicate the need for future research addressing the linguistic properties of the item stem. Nonetheless, the significance of the constructs identified remains reflective of the underlying influences considered by the NZ medical students. Enhancing the utility of this questionnaire could improve the scope of the factors identified and generate information that accurately describes the career-defining decisions made by medical students. This information additionally informs decisions made regarding workforce planning and the development of medical education curricula.