“Service” versus “Education” in Anesthesiology: Domain Classification of Activities Based on Perceptions from Learners and Teachers

Introduction

The Accreditation Council for Graduate Medical Education (ACGME) emphasizes the necessity of maintaining a balance between education and service in graduate medical education (GME). However, the definitions of “education” and “service” remain ambiguous. This lack of clarity can lead to challenges in training and expectations across various residency programs globally. The major objective of the Next Accreditation System (NAS) implemented by the ACGME in 2013 is to maintain a balance between education versus service in graduate medical education. ACGME common program requirements for Anesthesiology (IV.C.18) states “The program director must ensure that service commitments do not compromise the achievement of educational goals and objectives”, with further specifications that the program director must “adjust schedules as necessary to mitigate excessive service demands and/or fatigue”. This requirement is exemplified in the annual resident and fellow content survey questions with wording such as “education compromised by mandatory training-related obligations” and “appropriate balance between education and patient care”.¹

While there is extensive overlap between the two concepts, the vague delineation of “service” versus “education” creates some controversy. This can contribute to a false dichotomy that holds education and service as mutually exclusive. This ambiguity has been widely documented in the literature of graduate training across medical specialties for the past 10 years since NAS implementation, including Internal Medicine, Emergency Medicine, General Surgery, Pediatrics, and Radiology. This has implications for trainee education globally.² However, in-depth study of these domains in Anesthesiology remains sparse.^3–7

To address this question, we designed and implemented a survey among teachers and learners over a range of typical clinical and non-clinical anesthesia activities at a major academic department of Anesthesiology in a tertiary care hospital.

The primary hypothesis is that the perceived educational value of routine anesthesia training activities by learners and teachers vary with the type of activity, learner levels of training and teacher-years of clinical practice. The primary objective was to identify the anesthesia training activities associated with the perceived lowest level of educational value universally. The secondary objectives were to explore the perceptional differences among anesthesia learners and teachers, and the impacts of learning climate on the perception of educational value of various routine training activities.

Materials and Methods

Survey Instrument Design, Validation, and Distribution

The main objectives were to 1) Identify common activities that teachers or learners believe have lower educational value and 2) Assess perception on what constitutes education versus service among teachers (attending anesthesiologists in the Department of Anesthesiology) and learners (residents and fellows).

The study was granted exempt status by the Yale University Institutional Review Board (IRB) under 45CFR46.104. The survey questions were designed by the study team based on routine training activities and were grossly divided into 3 sections, perioperative responsibilities, outside of operating room teaching activities, and administrative obligations. Additional survey questions included commonly encountered clinical scenarios frequently associated with dilemmas and external factors in the learning environment that may impact perception of educational values in line with similar survey instruments in other medical specialties.^6,8–10 Positive and negative phrases were utilized to minimize distraction and reduce ambiguity. Qualtrics software was used to facilitate survey completion on computer, tablet, and smart phones. The survey (Appendix 1) was first reviewed and edited by two expert medical educators within the Department of Anesthesiology at Yale University (RHL, RG), followed by two external medical educational experts from the Teaching and Learning Center at Yale University (JH, JE). The survey was modified based on feedback, including removal of non-essential questions to minimize survey length and increase compliance, clarifications on the wording and format of four questions and two answers to ensure readability and minimize confusions. The questions were reordered to improve the flow of the survey instrument. The survey was next tested internally, and externally validated by a pilot survey to two teachers, one resident and one fellow, all reported the survey was easy to understand, convenient to perform taking less than 10 minutes to complete.

Our target population and sampling frame included anesthesia teachers (N = 160) and learners (N = 80). The learners included individuals in postgraduate year (PGY) 1–5 (ie, Clinical Anesthesia (CA) 0–3 and fellows). A few days before the official deployment of the anonymous survey, an Email letter was sent to introduce the study to potential participants. The final survey was distributed to members of the Anesthesiology Department right before departmental grand rounds. It was announced at the beginning and at the end of the session to maximize the response rate. To facilitate participation, a second Email letter was sent out to the participants 10 days later, and reminders for the survey announced at departmental grand rounds and other departmental level activities such as faculty meetings. To encourage participation, the investigators offered an anonymous raffle. There were five random winners, each received a $50 electronic gift card. The survey was closed after four weeks.

Statistical Analysis

Quantitative data from Qualtrics surveys were summarized descriptively as number of observations (%) for categorical characteristics, mean values and standard deviation (SD) for continuous characteristics, for both learners and teachers. Bivariate associations among categorical characteristics were compared using Chi-Square test statistics. Group-level comparisons of continuous characteristics were made using the two-sample Welch t-test.

To understand the similarities among and clustering features of the survey items, latent variable exploratory factor analysis (EFA) was performed. Factor analysis assumes that the relationship (correlation) between variables is due to a set of latent variables. Because the survey items used in EFA all had polytomous responses, the polychoric correlation was estimated to determine the standardized factor loadings using the minimal residual method. Factor loading is basically the correlation coefficient between the individual survey item and the “latent” factor. An absolute value of 0.7 or higher factor loading represents that the factor extracts sufficient variance from that survey item. Then, the regression method was used to calculate the factor scores, in which a higher factor score represents greater educational value. In addition, the promax rotation method was chosen to better discriminate the factors, resulting in oblique factors that were correlated. Number of factors were determined by the Scree plot.

After factors were identified, names were created to represent different domains in anesthesia activity based on the meanings of those high factor loading items in the respective domains. Then, the mixed effects models were used to jointly investigate the scores from the domains and their relationships to experience level classified in three ways: learner vs teachers, years of independent clinical practice for teachers, or seniority (CA0-CA4) for learners. The model included the domain indicator, experience level, and domain by experience level interaction, gender and age as fixed effects and survey respondents as random effects. All analyses were performed in RStudio (2023.03.01), using the fac function in “psych” package (v2.3.6) for factor analysis and the lmer function in “lme4” package for mixed effects modeling. All tests were 2-sided using p < 0.05 for statistical significance.

Results

Demographics and Characteristics of Survey Participants

In total, 126 responses to the survey were received with a completion rate of 52.5%, including 76 of 160 teachers (47.5%) and 50 of 80 learners (62.5%). Among the survey participants, 44% were female. The distribution by racial domains was Caucasian (66.4%), African American (4.9%), Asian (24.3%) and Mixed/Other (4.7%), respectively. Non-Hispanic and Hispanic represent 89.7% and 10.3% of the total survey participants. There were no significant differences between teachers and learners in terms of gender, race, and ethnicity distributions (Table 1).

Table 1 Demographics and Characteristics of Survey Participants

Domain Classification of Anesthesia Activities

Survey response patterns by learners and teachers were first summarized in univariate comparisons by individual survey items. Based on the magnitude of the factor loadings for the survey items (Table 2) from the latent variable factor analysis, three domains of activities were identified: “procedural responsibilities” of which an individual’s factor score was mostly influenced by greater loadings of their responses to the peri-operative hands on experience survey questions; “classroom activities”, by the outside of operating room teaching related responses, and “mandatory training-related obligations”, by the administrative duty and non-regular hour shifts. Some activities fell between domains, for example “discussion of anesthesia plan with patient before surgery” received similar factor loading in the “procedural responsibilities” and “mandatory training-related obligations” domains. The domain classification also does not always correlate with the three sections in survey design. One example being both preoperative patient transport and intraoperative record documentation were placed in the perioperative section in the survey design, and case log and duty hours were placed in the teaching activities section, they ended up having greater factor loadings towards mandatory training-related obligations domain. The amount of variability explained by the three domains were 21.2%, 22.3%, and 19.3%, respectively.

Table 2 Factor Analysis on the Survey Items

Perceived Educational Value Among Learners and Teachers Varied with the Domains of Anesthesia Activities

Both teachers and learners scored similarly in procedural responsibilities and classroom activities. The largest discrepancy was seen in the mandatory training-related obligations, where learners tended to perceive administrative duty and non-regular hours as service rather than being educational. In the analysis of joint modeling of the three domain scores, senior learners had similar perception of the educational values as teachers. PGY-1 learners tended to score lowest across all three domains. The educational values of mandatory training-related obligations increased as training levels increased in learners. Gender and race/ethnicity were not associated with any of the three domain areas (Figure 1). Fellows and mid-year teachers deemed procedural responsibilities most educational.

Figure 1 Summary plot of learner and teacher responses in the three major domains.

Time Allocation on Service and Education During Medical Training

Scales were assigned to reflect the respondents’ perception on the ideal proportion of time that should be spent on service versus education during the graduate anesthesia training, with 100% Service = 1, 75–25% Service-Education = 2, 50–50% Service-Education = 3, 25–75% Service-Education = 4, and 100% Education = 5, respectively. There was no difference in the perceived ideal proportion of time (learners: mean (SD) 3.5 (0.7) vs teachers: 3.3 (0.8); P = 0.094). However, the perception of the actual time spent on education was significantly shorter in learners than in teachers (mean 2.4 (0.8) vs 3.2 (0.8), P < 0.001, respectively). This was confirmed by learners’ dissatisfaction of insufficient educational time (learners: 2.6 (1.0) vs teachers: 3.4 (1.0); P < 0.001, based on a scale of 1 to 5 ranging from very dissatisfied as 1 to very satisfied as 5).

Factors in Learning Environment Affected the Perception of Educational Value Differently in Learners and Teachers

To investigate the impact of external circumstances that might alter the perception of an activity as toward more service oriented or more educational, learners and teachers were queried on potential changes in their perception with versus without several external factors, including lack of enthusiasm for teaching in attending anesthesiologists, lack of supervision from teachers, lack of respect from attending anesthesiologists and surgeons, poor learning environment, excessive work load, and prolonged work-hours. In general, teachers were more likely to treat external adverse learning climate factors as neutral while the perceptions of learners were more likely to be affected negatively (Figure 2).

Figure 2 Distributions of potential factors that affect the perceptions of service versus education by teachers and learners. Abbreviations: Poor-Learn-Env, poor learning environment; Little-Respect_SurgTeam, little respect from surgical team; Little-Respect, little respect from attending anesthesiologists; Little-Interest, attending anesthesiologists had little interest in teaching.

Perceptions of Educational Value in Common Clinical Scenarios

Several classic scenarios that are frequently associated dilemma and challenges in decision-making in Anesthesiology graduate training are presented in this study, with the value 1 being assigned to purely service, and the value 10 being assigned to purely educational. The clinical scenarios further attested to earlier findings in this study that significant differences on the perception of educational values persisted between teachers and learners in real time clinical scenarios, with all anesthesia activities universally viewed as having lower educational values by learners than teachers. In addition, the educational value rating for certain activities had nearly no overlap between teachers and learners. A few intriguing examples are the learning value in “easy” cases and pure observation, exemplified in scenario D question 1 when a resident on pediatric rotation was asked to staff operating rooms for straightforward cases such as laparoscopic appendectomy while taking in house call, and in scenario F question 1, when an attending anesthesiologist took over the procedural opportunity of arterial line and central line placement quickly, allowing the learners only a few attempts due to production pressure from the surgical team. These samples further illustrated the complexity in defining education versus service in clinical setting in graduate anesthesia training (Table 3 and Figure 3).

Table 3 Summary of Learner and Teacher Responses in Real Time Clinical Scenarios

Figure 3 Perceptions of Educational Value in Clinical Scenarios (see Table 3). Abbreviations: Scen, scenario; Q, question.

Discussion

This study is the first of its kind in Anesthesiology assessing the perceived educational value of anesthesia activities across the spectrum of training in graduate medical education. It is also, the first attempt to classify routine anesthesia activities into three relatively distinct domains, procedural responsibilities, classroom activities and mandatory training related obligations. This study confirmed previous findings reported in other medical specialties, that teachers perceive higher educational values in more activities than learners also holds true for the specialty of Anesthesiology.^5,9,11,12 In addition, it also revealed the domain of mandatory training related obligations such as night and weekend call and maintaining case logs were not only viewed as having low educational values, but also statistically lower by learners than by teachers. In contrast, the domains of procedural responsibilities and classroom activities were perceived to have high educational value by both learners as well as teachers. These findings are consistent with the shift reported in a letter to the Editor by Lam ST that the focus of residency training is shifting towards didactic education and away from clinical duties and irregular life schedule.¹³

A large and important part of graduate medical education occurs through active learning in the provision of patient care through observation, debriefing, reflection, feedback, and repetition. It should be noted patient care-related activities span all three major domains in our analysis. In other words, the insight that we got from this study is that learners do not view all patient care related activities as service, which is somehow a misunderstanding or at least unclear in current literature. However, they appreciate the educational value of certain patient care activities much less than teachers, as an example, educators appreciate much more than learners that service can be educational in the absence of formal teaching, such as clinical observation of master techniques illustrated in scenario F. This finding is in line with results from a similar study in pediatric residents and faculty.⁶ Educators probably tend to believe repetition in multiple and varying contexts is critical to acquisition of certain skills and ultimately to demonstration of competence under various circumstances,⁴ as demonstrated in the scenario D where repetition was considered education (probably as deliberate practice) by teachers, but service (probably as accomplishing a task) by learners. This is also in line with insights across countries that both educational and service activities affect trainee satisfaction.²

Clinical care activities can range in the spectrum of actual and perceived educational value. The importance of learning climate has prompted accreditation bodies such as the Liaison Committee for Medical Education (LCME) to require schools conduct a “periodic evaluation of the learning environment”. Student perception of the academic learning environment has also been linked to academic performance on standardized tests.¹⁴ This study highlighted that the learning climate affects the perception of educational value of a specific activity more in learners than what the teachers believe. Therefore, the spectra of what constitutes service and education is not rigid, and certainly should not be perceived as mutually exclusive, rather they should be regarded as being mutually dependent and critical to both physician education and practice.¹³

Our study also indicates that it is important to develop a balanced curriculum that emphasizes high-value educational activities while at the same time finding ways to enhance the educational appeal of mandatory training related activities. It is important to explicitly discuss expectations and importance of activities such as call responsibilities with learners, particularly at the start of the residency when they experience the least appreciation of the educational values of these responsibilities, as these differences can contribute to potential concerns for program accreditation maintenance and these concerns may be reflected in answers to NAS surveys from ACGME. It also emphasizes the need to focus on improving the learning environment to decrease fatigue and excessive caseloads to positively impact the perception of educational value of training activities. These are important concepts that need clarity at the policy level as core competencies require that patient care is not an optional service, rather a professional obligation and privilege for a physician. In addition, we posit that some of the identified activities could be modified to improve quality of training and learner satisfaction. For example, the nearly universal adoption of electronic medical record (EMR) across North America may allow secure electronic communication and automatic data transfer between the EMR and the ACGME database to minimize the workload of manual case log entry.

The primary limitations of this study were that it involved a single center and therefore a small sample size. However, our study employs a detailed and in-depth analysis such as latent variable exploratory factor analysis, which can be effectively conducted even with smaller sample sizes, focusing on the quality and specificity of the data rather than quantity. Additionally, this is the first study in Anesthesiology graduate training that demonstrates the importance and challenges in maintaining the interdependent yet balanced relationship between education and service. It is also the first study to analyze routine anesthesia training activities based on three major domains. While a deliberate balance needs to be carefully maintained between education and service in an Anesthesiology training program to graduate the most competent anesthesiologists, the perception of what constitutes education versus service might have a subjective component, illustrated in the clinical scenarios presented in our study. The challenge of defining “education” and “service” components in resident training has implications for residency programs globally. Though our results can be generalized to medium to large size academic anesthesia programs in the USA, it may not hold true for smaller programs. In addition, due to varying training requirements in different countries, programs outside of USA may have unique social and cultural factors that have not been assessed in this study. The next step would be to implement a larger scale survey study using selected representative questions from each of the 3 major domains without the need to include all common anesthesia activities as in this original study, with a rigorous sampling design at national and/or international level, to better understand the complicated and dynamic balance between education and service by creating a framework to define these terms and communicating to learners, teachers and the program leadership to ensure the quality of graduate education in Anesthesiology.^4,9

Practice Points

The key points from our study are,

1. There is disparity in the perception of educational value for various training activities between learners and teachers. Learners perceive lower educational value in anesthesia training activities compared to teachers, especially in mandatory tasks.
2. Training activities can be categorized into three domains such as mandatory obligations, procedural responsibilities, and classroom activities, with classroom activities rated highest in educational value.
3. Poor learning environment factors, such as fatigue and high caseloads, significantly diminish perceived educational value, particularly for learners.
4. Procedural responsibilities and classroom-based activities were seen as having higher educational value than routine, obligatory tasks.
5. Modifiable elements of the learning climate should be addressed to enhance the educational experience for learners.