Sustained Impact of an Intramural Faculty Leadership Training Program at an Academic Medical Center

Introduction

As academic medical centers (AMCs) continue to grow in size and complexity, there has been an expanding call to develop diverse, multi-disciplinary leadership teams.^1,2 The critical role of physician leaders in ensuring team and organizational effectiveness has specifically been emphasized.¹ Yet, formal leadership training has typically not been integrated into the core curriculum for medical trainees and faculty.³ Experts have cited several reasons why leadership training for physicians is critical – more effective management of physician practices have been associated with better clinical outcomes, higher patient satisfaction,³ and better financial performance. Furthermore, stronger leadership is associated with better physician well-being.³ Indeed, young physicians have reported a desire for more leadership training.⁴

Much of this training has been delivered through intensive executive degree programs,⁵ limiting the reach and potential impact. Over the past two decades, an increasing number of leadership programs have been created within institutions and/or professional societies for physician faculty, trainees, and medical students.^6–11 Several competencies have been addressed in these training programs, with wide variability in content across programs. Many programs include education on operational/financial considerations and strategic planning, in addition to training on emotional intelligence, communication, and problem-solving.^2,12 The format has also been widely variable, with some programs using a predominantly didactic approach in large group settings, and others employing hybrid (independent + group learning)¹³ and/or newer approaches, such as flipped classroom models (ie, where students are asked to review curricula in advance to facilitate engagement and discussion during class time).⁹

While research has demonstrated promising outcomes after these leadership programs, many gaps remain. First, many studies have compared self-reported competencies at baseline with those immediately post-program completion,⁶ with few studies examining outcomes greater than a few years. Second, most studies have included small sample sizes and have lacked a control group. Given the time, effort, and financial investment involved in developing and sustaining academic leadership programs for physicians, more rigorous outcomes data are critical.

The Faculty Leadership Institute (FLI) was developed to provide leadership training to interested faculty members (both physician and non-physician faculty), as part of the Ohio State University (OSU) College of Medicine’s Center for Faculty Advancement, Mentoring, and Engagement (FAME).^14,15 Applications were invited from all full-time College of Medicine faculty members, with 30 participants selected each year. FLI included a rigorous educational component prior to and in between monthly group sessions; sessions were facilitated by local experts, along with an expectation to complete homework, self-study, and a group Capstone Project. The three-hour classes focused on topics in five core areas, including character, interpersonal skills, business acumen, healthcare leadership, and change management. While the FLI curriculum covered many of the core topics included in other national leadership programs, education on healthcare law and human resources was also provided. Additional resources included access to Harvard Business Review, leadership books, and 360-degree evaluations with coaching during the program. Participants also received bi-monthly articles on leadership topics during and after program completion. From 2011 to 2018, more than 150 faculty (primarily M.D.s along with some Ph.D.s) from diverse areas within academic medicine completed the leadership program. Evaluations after completion of the program showed a significant increase in mean knowledge of core areas compared to pre-session scores (p<0.001), with no difference based on gender or faculty rank.¹⁶

Given the lack of longitudinal evaluations of leadership training, the primary aim of this study was to examine the long-term impact of the FLI program on participants in comparison to faculty members who did not complete the program. We hypothesized that FLI participants would be more likely to report more effective leadership skills and attain leadership positions in a variety of areas, relative to non-participants. We also explored group differences in attainment of different leadership positions and in perceived effectiveness of leadership skills based on gender and race (ie, group × gender and group × race interactions). To address noted gaps in the literature and test these hypotheses, a cross-sectional analysis of long-term follow-up data from FLI graduates and faculty controls was conducted.

Materials and Methods

Population and Sampling Procedures

Our target population for this study included individuals who held a faculty appointment from the Ohio State University College of Medicine and were on the College faculty roster as of the final day (December 31) of 2013 or 2018. The total number of faculty in the database (with redundancies removed) was N=2477. We further screened the faculty database to only include individuals who were alive and still working (whether at OSU or not) and individuals for whom we had valid contact information. Since we were only interested in evaluating the FLI program as it existed between 2011 and 2018, we removed faculty from the database who participated in FLI after 2018 prior to drawing a control group sample. The final database contained 1563 faculty members comprised of 148 FLI graduates and 1415 potential controls. Table 1 shows the demographic characteristics of the population prior to study implementation.

Table 1 Comparisons of Background Characteristics Between FLI Cohort and Control Group

With the remaining n=1415 faculty candidates who had never participated in the FLI program, we sought to create a sizeable control group of at least n=450 to compare to our faculty who had participated in the FLI program from 2011 through 2018 (n=148). To obtain a control group of this size, we chose to include the entire database of n=1415 candidates for this study instead of drawing a random sample. To engage faculty who had not participated in our intramural FLI program, we offered an incentive, offering to donate $10 to one of three charities for subject participation. This study was determined to be a category 1 exemption from review by The Ohio State University Wexner Medical Center Institutional Review Board, meaning that the research was considered a study of effectiveness of a program conducted in an established education setting.

Measures

We developed a leadership training evaluation instrument for use in this study. The core of the instrument was derived from the FLI curriculum and adapted for use with non-FLI faculty. Items for this portion of the instrument included FLI learning objectives or topics along with a self-rating scale. Other portions of the instrument included a section designed to assess the subject’s trajectory as a leader in academic medicine or higher education, and a section designed to obtain basic demographic information.

Self-Perceived Leadership Effectiveness

The survey was designed to glean the effectiveness of the core leadership skills taught in the FLI program and whether individuals who did not participate in FLI would consider themselves as effective in those skills. Participants rated their perceived effectiveness as a leader on 26 core leadership skills, using the following scale: 1=Not effective at all, 2=Slightly effective, 3=Moderately effective 4=Very Effective, or NA=Not sure, do not remember, cannot assess. Items represented skills in areas such as strategic planning (eg, “Creating organizational vision, mission, objectives, and priorities”), business acumen (eg, “Understanding basic principles of accounting”), change management (eg, Identifying and mobilizing stakeholders”), negotiation and conflict resolution “Applying mediation and negotiation technique”), communication (eg, Demonstrating sensitivity in communication with individuals of varying cultures and backgrounds”), and healthcare law (eg, Knowledge of False Claims Act, Anti-kickback and Stark laws”). To aid interpretation, average scores were calculated across all items for an overall perceived leadership effectiveness score (ie, range 1–4) if 80% of items were completed.

Attainment of Leadership Positions

We used the formal leadership positions and titles held by participants during their career thus far as an indicator of leadership attainment or trajectory. We asked participants to indicate whether they had held leadership positions across eight different levels within the university or academic health center setting, including University/Higher Education (eg, President, Vice President, Chancellor, Provost, Faculty Senator), Academic Health Center (eg, CEO, Chief of Staff, Committee Chair, etc.), College or School (eg, Dean, Vice Dean, Associate Dean, Unit or Block Director), Graduate Medical Education (eg, Residency Program Director, Fellowship Program Director, Academic Program Director), Department (eg, Chair, Vice Chair, Associate Division Chief), Clinical Operations (eg, Medical Director, Associate Medical Director, Clinical Program Director), Research Program (eg, Research Director, Institute Director, Center Director), or Professional Organization (eg, President, Board of Directors, Committee Chair). Primary leadership items were quantified as “yes” = 1, and “no” = 0. Then, an overall leadership summary score was calculated by counting the number of leadership positions endorsed across all eight categories (ie, range 0–8).

Demographic Background

Demographic characteristics regarding gender, race, ethnicity, and age, as well as faculty rank, track, and years at the institution were obtained from the college faculty database. In addition to asking participants to report on their perceived leadership skills, we also asked them to report on other formal leadership training, outside of the FLI program, that they may have received.

Survey Implementation

We implemented our survey instrument using the REDCap electronic data capture software hosted at The Ohio State University.^17,18 We surveyed all n=1563 faculty members, including the 148 who participated in FLI. Data collection took place from April-August 2023, with reminders distributed to non-respondents every three weeks for up to four months. Consent for the use of participants’ answers to the leadership questionnaire was sought through REDCap in advance of their completion of the questionnaire.

Analysis Plan

Descriptive statistics were calculated using means (standard deviation [SD]) or frequencies (relative frequency [%]) to summarize continuous and categorical demographic characteristics, leadership positions attained, and leadership skills, respectively. Chi-square analyses, Fisher exact test (where expected counts < 5), and independent sample t-tests (α =0.05, two-way) were used as appropriate to evaluate group differences in primary outcomes. Cohen’s d was also provided for primary outcomes to describe the mean standardized difference of perceived leadership effectiveness between groups. Linear and logistic regressions were used to explore interactions between group and demographic characteristics (ie, gender, race) on both leadership positions attained and perceived leadership effectiveness. To account for the increased likelihood of type I errors when performing multiple statistical tests, the false discovery rate (FDR) adjustment was applied using the Benjamini–Hochberg procedure for all statistical tests. The FDR adjustment is a method for controlling the expected proportion of incorrectly rejected null hypotheses (ie, false positives) among all rejected hypotheses. Specifically, p-values from all statistical tests were ranked in ascending order, and each was compared to a threshold calculated as (i/m)⋅α, where i is the rank of the p-value, m is the total number of tests, and α is the desired FDR threshold (eg, 0.05). Adjusted p-values were computed to reflect the corrected significance levels, ensuring that findings are robust to multiple testing. All analyses were conducted using SAS v9.4 (SAS Institute Inc., Cary, NC, USA).

Results

Participants

The final eligible sample consisted of 1563 faculty members, 148 of whom were FLI graduates and 1415 controls. We received surveys from 106 FLI graduates for a 72% rate of return and 212 controls for a 15% rate of return. We dropped 11 FLI and 56 controls due to incomplete data on primary variables of interest. This resulted in a final sample of 251 participants (251 of 1563 = 16%) for analyses, comprised of 95 FLI graduates (95 of 148 = 64%) and 156 controls (156 of 1415 = 11%).

The overall sample was 47% female (n = 118), 76% White (n = 190), and 87% non-Hispanic (n = 219), with an average age of 52.39 years (SD = 9.32) (Table 1). Faculty appointments included 13% Assistant Professor (n = 33), 34% Associate Professor (n = 86), and 51% Professor (n = 127), where 68% (n = 170) reported having a medical degree (eg, MD, DO, MBBS) and 26% (n = 66) reported having a PhD only. The remaining 6% (n = 15) checked Master’s degree (n = 4) or did not indicate their degree (n = 11). On average, FLI participants were followed for 7.4 years (SD = 1.7) after enrollment in the leadership program. Overall, the two groups of faculty members were similar on background factors, except FLI participants were less likely to be Assistant Professors and more likely to be Full Professors relative to controls (Table 1). Specifically, 2% of FLI graduates and 20% of controls were Assistant Professors, while 62% of FLI graduates and 44% of controls were Full Professors (p < 0.001).

Self-Perceived Leadership Effectiveness

Group comparisons of perceived effectiveness in leadership skills are in Table 2. FLI participants reported moderate to high effectiveness in overall leadership skills (M = 3.17; SD = 0.41), which was significantly greater than controls (M = 2.85; SD = 0.56) based on average scores across all items (p < 0.001; d = 0.62). Exploratory comparisons on individual items indicated FLI participants had higher self-reports of effectiveness on 25 of 26 (96%) leadership skills relative to controls, and these were statistically significant in 16 of the 26 (62%) leadership skills relative to controls (p < 0.034 to 0.001, see Figure 1), with small to moderate effect sizes (ds = 0.34 to 0.57). FLI scores were highest for recognizing different leadership styles (M = 3.57, SD = 0.52) and creating, participating in, and leading teams (M = 3.61, SD = 0.59). The largest group differences were seen in understanding effective strategies for improving patient satisfaction (p = 0.001), applying tools and techniques to effectively communicate in the workplace (p = 0.001), familiarity with national quality initiatives (p = 0.003), and applying conflict resolution (p = 0.005), tools for quality improvement (p = 0.005), knowledge of false claims act, anti-kickback and stark laws (p = 0.007), and negotiation (p = 0.008) techniques.

Table 2 Comparison of Self-Perceived Leadership Effectiveness Between FLI Cohort and Control Group

Figure 1 Significant Variables of Self-Perceived Effectiveness between FLI Cohort and Control Group. Notes: The bars in red represent faculty members who have completed the Faculty Leadership Institute program and the bars in gray represent faculty controls. A=Impact and consequences of decision-making (internal and external); B=Identifying and mobilizing stakeholders; C=Securing commitment and achieving results through skillful communication; D=Managing performance through coaching and feedback, developing talent; E=Knowing factors that influence team dynamics; F=Creating, participating in, and leading teams; G=Identifying types of negotiations; H=Applying mediation and negotiation techniques; I=Applying conflict resolution techniques; J=Stating and applying principles of communications; K=Demonstrating sensitivity in communication with individuals of varying cultures and backgrounds; L=Applying appropriate communication tools and techniques to effectively communicate in the workplace; M=Effectively applying tools for quality improvement; N=Familiarity with national quality initiatives such as national quality forum; O=Understanding effective strategies for improving patient satisfaction; P=Knowledge of False Claims Act, Anti-kickback and Stark laws; Q=Overall Average Leadership Effectiveness.

Leadership Positions Attained

Participants in both groups were more likely to hold leadership positions in Professional Organizations (62%; n = 157), followed by Departmental Leadership (55%; n = 138) and Clinical Operations (40%; n = 101) (Table 3). The overall number of leadership positions held by FLI participants (M = 3.14; SD = 1.49) was greater than controls (M = 2.65; SD = 1.88) but after controlling for multiple comparisons, the difference was not significantly different relative to controls (t (248) = 2.13, p = 0.1). The FLI group was more likely to hold leadership positions in Clinical Operations compared to controls (53% vs 33%, p = 0.018), but there were no group differences on other individual leadership outcomes, including University/Higher Education, Academic Health Center, College or School, Graduate Medical Education, Department, Research Program, or Professional Organization (Table 3).

Table 3 Comparisons of Leadership Positions Attained Between FLI Cohort and Control Group

Group Differences Based on Gender and Race

Regressions were conducted to explore interactions between group (FLI vs control) and sex (female vs male; see Table 4). No interaction effects were found for race on leadership positions (ps = ns). In addition, group differences in perceived leadership effectiveness did not differ based on gender or race (ps = ns).

Table 4 Interaction Effects Between Group and Sex or Race on Overall Average Leadership Effectiveness and Total Leadership/Achievements

Discussion

Our findings demonstrated sustained benefit of a faculty leadership program, in that faculty participants self-reported moderate-to-high effectiveness of leadership skills at an average of 7 years after program completion, which was significantly higher than the control group. FLI participants were more likely to attain leadership positions in clinical operations, but overall number of positions did not differ significantly between groups. Notably, group differences in leadership positions and perceived effectiveness in leadership skills generally did not vary based on gender or race.

In terms of participant characteristics, there were important group differences in academic rank, in that the FLI participants were more likely to hold the rank of Full Professor, while the faculty members in the control group were more likely to be Assistant Professors. This greater academic advancement was seen despite the lack of significant age differences between the FLI cohort and the control group and is similar to other studies examining effects of a faculty leadership program on promotion.¹⁹ While we cannot assess whether this advancement was a direct result of FLI program participation (ie, program participants may have been at higher rank than their peers at the time of program completion and/or higher-rank individuals may already have stronger networks or leadership access), it is an important difference given that promotion has been linked to retention, career satisfaction, offers financial reward, and also more leadership opportunities.²⁰

Regarding self-perceived effectiveness in leadership, FLI graduates reported greater competency than the control participants in 96% of the leadership skills that were assessed, and these were significant in nearly two-thirds of these skills. They also reported a higher total score compared to the control group. Some of the biggest differences were seen in applying negotiation and conflict resolution techniques, decision-making, mobilizing stakeholders, skillful communication, talent development, team dynamics, and creating, participating, and leading teams – skills that are critical for physicians to acquire to maximize their own well-being and increase effectiveness of AHCs.^1,3 These results suggest that the FLI program is meeting its goal (ie, to provide evidence-based leadership education designed to cultivate leaders capable of addressing the evolving challenges within AHCs) and is aligned with a priority that has been underscored by the National Academy of Medicine.²¹ It is also important to note, however, that there were no group differences in domains pertaining to accounting, financial statements and strategic plans, indicating these to be potential areas to strengthen in the FLI curriculum.

Findings regarding program effectiveness are similar to previous studies that have shown self-reported benefits of participating in a faculty leadership programs.⁷ These results also extend the literature by assessing these skills many years after program completion and comparing outcomes to those in a control group.^22,23 Recent systematic reviews have shown that most evaluations of faculty leadership programs rely solely on immediate post-program self-evaluations and that a control group is rarely included.^22,24 These reviews have highlighted the importance of using more rigorous approaches to assess whether these leadership development programs yield true value in healthcare and justify the financial and time investments of those who create the programs and those who participate.^22,24

Contrary to our second hypothesis, total number of leadership positions, although higher for FLI graduates, did not significantly differ between groups. Further, we did not see outcome differences by race or gender. Prior research has shown that development of a physician leadership program creates a culture where physician leadership is valued, prepares physicians for their leadership roles and strengthens the institutional leadership pipeline.²⁵ Recent qualitative studies have highlighted the importance of institutional leadership training opportunities for women and other communities underrepresented in leadership positions, in order to achieve more equitable access to these positions.^26–29 Although not the focus of this study, in a previous study we have shown that FLI-trained faculty had a lower attrition rate (3.6%) than our medical center as a whole (5.3%) and also lower than the national physician attrition rate of 7.8%.¹⁵ Further research is needed to determine how to better support these communities beyond leadership training at our institution and at other AHCs, such as developing an infrastructure for ongoing mentorship and sponsorship for years after program completion,^26,30 competency-based offering of positions with holistic evaluation, and broader calls for positions to capture more underrepresented groups. Additionally, since leadership roles were self-reported in this study, future research should integrate other sources of data (eg, records from Human Resources) to assess leadership attainment more objectively and comprehensively.

There are some study limitations to consider. First, this was a single-site study and while many of the position types and leadership principles may be applicable to other AHCs, they may not translate to all types of medical centers. The self-evaluation survey methodology is subject to response rate limitations and recall bias, which may be affected by social desirability, as it relies on a subjective perspective of leadership effectiveness. The low percentage of responses from control faculty members (a product of sampling all non-FLI faculty) might have also introduced selection/nonresponse bias in that those who participated may not be representative of the entire faculty group. Furthermore, the leadership domains assessed were aligned with the FLI’s learning objectives rather than a validated measure, which may limit generalizability. The Cohen’s d effect sizes for some of the leadership effectiveness differences were small, suggesting that meaningful differences between groups in those domains may be modest. Additionally, since we did not have baseline data on leadership roles or competencies, we cannot determine whether FLI program participation was the sole factor in explaining group differences. Perhaps, program participants were more knowledgeable of these domains than their peers to start and/or perhaps they pursued additional leadership training after completing the FLI program. Finally, attainment of leadership positions was difficult to assess and compare given the heterogeneity of position types and inability to assess timeline of attainment (ie, before vs after FLI program completion). Further investigation using objective outcome measures is warranted, to assess how perceived effectiveness in leadership skills translate into tangible organizational outcomes. Our small sample sizes for gender and race differences may have contributed to our inability to find differences in these groups.

Despite these limitations, the long-term evaluation and use of a control group provide a more rigorous approach to mitigate issues like inflated efficacy scores that may be seen immediately after the completion of a leadership program.²⁴ In addition to subjective effectiveness metrics, there was also a focus on objective leadership position outcomes, giving multiple perspectives regarding program evaluation. This study contributes to the growing body of knowledge on physician leadership education, highlighting how programs like FLI can create a culture where physician leadership is valued and encouraged, in addition to empowering physicians to pursue higher-level leadership roles within academic institutions, achieve higher academic ranks, and become more effective leaders.^25,31,32 Future research will be aimed at prospectively following FLI program participants’ career trajectories to assess how their self-rated skills translate to attainment of leadership roles and system-level outcomes (eg, mentoring, research funding, patient care). Studies should also be conducted to determine how our findings may or may not translate to other types of institutions/regions. Ultimately, continued efforts to develop talented faculty members, with a particular focus on underrepresented groups, will ensure a pipeline of leaders well prepared to meet the demands of academic medicine and ensure its bright future.

Conclusion

These findings verify the sustained benefits of our FLI program. These benefits were found in the effectiveness of overall leadership skills, with substantially higher self-reports of effectiveness compared to controls, demonstrating the real-world benefit of FLI program participation. Although not statistically significant, FLI participants attained a greater number of leadership positions. From an equity standpoint, these differences did not vary by gender and race. Although causal inferences may be impacted by selection/nonresponse bias and lack of baseline comparison data. Additional research may help demonstrate how positive effects translate to system-level outcomes (eg, research funding, patient care) across various types of institutions. Thus, continued efforts to develop talented faculty members is important to ensure a succession of well-prepared leaders at AHCs.