Internal Medicine Residents’ Perceptions of the Learning Environment of a Residency Training Program in Ethiopia: a Mixed Methods Study

Introduction

The learning environment (LE) is an important determinant of the quality of medical education in a post-graduate residency program.¹ It has been defined as a set of factors that describe a learner’s experiences.² It has three components; the physical environment, safety, sheltering and other facilities; emotional environment, support and feedback; and intellectual component, evidence-based practice, planned education and patient learning.²

Having a good learning climate leads to active participation by the trainees.³ It affects the standard of the learning process,^4,5 trainees’ behavior,⁵ satisfaction with education,^5,6 burnout level^7,8 and determines the final outcome of the curriculum.^5,6 It also influences academic performance.^7,8 Career choice and satisfaction with it seems to be influenced by experience during education.⁷ It not only affects trainees but also has a direct impact on the quality of healthcare delivered.^4,9

Assessment of the quality of learning environment helps with the identification of areas that need improvement.⁶ It subsequently needs continuous monitoring with the aim of improving the performance of the institution and refining the learning experience.⁴ It is also the basis for curriculum and policy development and revision.¹⁰

Several tools have been developed to assess LE.⁹ Out of these tools, the most widely used and recognized one is Postgraduate Hospital Educational Environment Measure (PHEEM).^8,9 PHEEM was developed specifically for postgraduate trainees in 2005 in the United Kingdom.¹¹ The tool has been tested in multiple studies worldwide and has shown good reliability, content validity and reproducibility.^2,4,8,10 It has also been used in a wide range of specialties.^6,8,10,12,13 To our knowledge, the psychometric properties of PHEEM have not been assessed in an Ethiopian setting. Nevertheless, it has been used to assess residency programs previously.^14,15 The mentioned attributes, along with the previous experience in the Ethiopian setting made PHEEM the most appropriate tool for assessment of LE in this study.

An explanatory mixed methods study design was conducted to elaborate on the significant challenges identified. Findings from quantitative data were supplemented with a focus group discussion to help comprehend participants’ points of view and to further explore new concepts that affect learning environment. This study design has been reported to provide opportunity for more exploration of concepts and provide stronger conclusions.⁷

The residents’ perception of LE and factors associated with it has not been studied in detail in an internal medicine residency program in Ethiopia. Knowledge of perception of LE can help identify areas that need improvement and can help recommend ways to strengthen the LE for future residents. This can help better achieve the targets of the curriculum.

The objective of this study was to determine the residents’ perception of the LE of internal medicine department.

Methods

An explanatory mixed methods study design utilizing both quantitative and qualitative measurements was used to assess the learning environment of internal medicine training program of St. Paul’s Hospital Millennium Medical College (SPHMMC) located in Addis Ababa, Ethiopia. SPHMMC is a tertiary and teaching hospital established in 2007. It started as a medical school for undergraduate studies but has expanded to postgraduate and fellowship programs. It has postgraduate programs in 20 specialties.¹⁶ Internal medicine department has been training postgraduate students and is currently instructing its eighth batch. There are a total of 127 residents in the department, with 27 in third year, 33 in second year and 67 in first year, from two overlapping consecutive years. It has different units including cardiology, neurology, gastroenterology, nephrology, endocrinology, pulmonology and critical care and hematology units. In addition to working in the units, the residents also have rotations in emergency and critical care medicine, radiology, dermatology, anesthesiology and psychiatry departments with rotations in two affiliate hospitals in the city.

A cross-sectional survey that included PHEEM questionnaire along with sociodemographic data including age, gender, marital status, year of residency and prior work-experience was administered using an electronic online questionnaire from December 2020 to May 2021 on all willing residents from the department. The PHEEM consists of 40 items with 3 subscales each assessing perception of role autonomy, perceptions of teaching and perceptions of social support, respectively. It is scored on a 5-point Likert scale ranging from “strongly disagree” (0) to “strongly agree” (4), with a maximum score of 160. Interpretation of the findings was made according to the original study that suggested a scoring of excellent score (>120), more positive than negative but room for improvement (81–120), plenty of problems (41–80) and very poor (0–40).¹¹ There were four negative questions (questions 7, 8, 11, 13) that were scored in reverse. Slight modifications of the questionnaire were made, including changing the word “bleeped” on question 11 to “called” and “New deal” on question 17 to “national limits of working hour of physicians”, to suit the local context. PHEEM has been validated to assess the post-graduate clinical learning environment after the original study, including after translation to different languages.^2,4,11,17,18 Multiple subsequent studies have shown a good internal reliability.^3,5,8,10

The sample size for the study was computed from the following formula: n = P × (1 − P) × Zα/2²/d². Z α/2 is the critical value of standard normal deviation at 95% confidence interval (1.96), and d is precision (the marginal error) of 10%. Level of perception of the learning environment was taken as 50%, since it has not been studied in detail in the past in Ethiopian internal medicine residency programs. This led to a sample size of 96.

After the questionnaire, a focused group discussion was conducted among selected group of residents, facilitated by trained data collectors. Focus group discussion was conducted as it provides in-depth discussion and understanding of perceptions, attitudes and experiences of all participants at the same time. Residents who have worked in administrative works, including serving as chief residents, in the department were selected, and the rest were selected by random sampling. The residents stated their gender, age and year of residency, and signed an informed consent. The findings of the PHEEM questionnaire were presented, and they were asked to deliberate on additional points not included in the questionnaire. They were also asked to discuss on the most significant challenges that they have faced and the area of the difficulties. The participants were asked to provide consent to have audio recording of the discussion. Focus groups were conducted until new insights were no longer attained reaching saturation. Two focused group discussions were conducted with sessions lasting from 60 to 90 minutes. Transcriptions of the interview from the audio recording were made by an independent data collector, and its accuracy was confirmed by the focused group discussion facilitators. The transcripts were coded into common themes independently by two investigators and discussed using thematic analysis.

Data were analyzed using Statistical Package for Social Sciences version 25. Descriptive statistical tests were used to describe the findings of the measurement tool using percentages, means, standard deviation (SD) and interquartile range (IQR). Internal reliability was assessed using cronbach’s alpha coefficient. Comparison of quantitative data was done using Mann–Whitney U-Test and Kruskal–Wallis H-test, followed by post hoc test. A p-value <0.05 was considered statistically significant.

Before conducting the research, permission was obtained from SPHMMC Institutional Review Board. Participants were sent an invitation that includes information about the objectives and significance of the study. If they were willing to participate in the study, they were invited to sign an electronic informed consent to open the link to the questionnaire. Every participant was given chance to deliberate on their rights to participate, refuse, or withdraw at any time they wish to do so.

Results

A total of 100 responses were returned resulting in a response rate of 75.6%. Out of the participants, 80% were male. The mean (SD) age of the participants was 28.8 (2) years and ranged from 25 to 36 years. Around 13% were above 30 years of age. First-year residents had the highest response from the total responses (44%), followed by second-year (32%) and third-year residents (24%). Participants had worked for a median duration of 2 years (IQR 1.5 to 3 years) before starting residency.

A high level of internal reliability of the PHEEM questionnaire was seen with cronbach’s alpha coefficient of 0.915, and no significant improvement was seen with removal of any of the 40 questions.

The overall total mean score (SD) of the responses of the participants was 70.87 (19.8), which lies in the category of 41 to 80. This is interpreted as having plenty of problems according to the original PHEEM study.¹¹ The mean (SD) perceptions of role autonomy was 25.9 (7.1), while perceptions of teaching and perceptions of social support were 27.1 (10.2) and 17.9 (5.1), respectively. The mean of the subscales is in congruence to the overall score and lies in the category of having negative views of one’s autonomous roles, teaching perception being in need of retraining and social support not being in a pleasant place. These findings illustrate the need for significant improvement of the learning environment.

There were seven items on role of autonomy that scored a mean value below 2 which indicates an area of significant challenges. Similarly, 10 items among perceptions of teaching had mean value of less than 2. Finally, 6 highly problematic areas were identified for perception of social support. On the contrary, 3 items from social support had mean scores above 3 suggesting positive attributes (Tables 1–3).

Table 1 Summary of Responses on Perceptions of Role Autonomy Subscale

Table 2 Summary of Perceptions of Teaching Subscale

Table 3 Summary of Perceptions of Social Support Subscale

The overall mean result of learning environment was significantly higher in first-year residents compared to second- or third-year batch (p = 0.008). Perceptions of role autonomy was also significantly associated with year of residency, with first-year residents reporting higher means compared to other years of residency (P = 0.003), and higher mean value was reported in males compared to females (P = 0.025). The mean score for perception of teaching was significantly lower for third-year residents compared to both first-year and second-year ones (P = 0.008) (Table 4).

Table 4 Factors Associated with Perception

Two focus group discussions were conducted involving 5 residents in each group. Two of the participants were chief residents from second and third year and the rest were randomly selected from each year of residency. Two females and eight males participated. Four major themes were identified as barriers to having a good LE and are presented with selected supporting quotations.

The first major theme identified was excessive workload, which was beyond residents’ capacity to handle, and lack of clear job description in the hospital. This was mentioned by every participant with different views on how excessive workload was affecting their education, including on how it encroached on their protected education time. One resident described his frustration by saying he does not feel like a resident while he was in one of the affiliated hospitals. “The work load and patient number that I take care of is so high that I’m always exhausted; so much that I feel like an intern. It’s hard to say it’s even a teaching environment” (Resident 3). Another participant asked “sometimes I get confused, am I hear to do work or is my primary objective learning?” He continued and said, “we need job descriptions and it should be clear” (Resident 8). His idea was supported by resident 5 who said “we don’t order our nurses; they order us, which clearly shows the lack of job description.”

The second theme identified was the inadequate teaching activity. The major aim of a residency program is to provide with skills and knowledge in the study area. There needs to be a balance between work and education that can help attain the education goals. The participants saw their environment as a workplace with low opportunity to learn. “There is a huge gap with teaching, particularly for first-year residents. Based on my estimate, I say its 80% work and 20% learning” (Resident 1). Another resident described it as “we are not given assignment to read and the rounds are short and patient focused. I am not motivated to read further on the patient” (Resident 4). “My opinion is that the majority of the staff are not motivated to teach, and it’s discouraging us. I’m always tired when I get home; and if I know no one will ask me anything in the morning I will sleep” (Resident 6).

Another participant (Resident 2) had a different perspective and said, “the increased patient load causes the seniors to rush during round. The round will be focused on addressing patient’s issues rather than leaving time for academic activities, like detailed discussion on the case”.

The hospital environment is not physically organized to ease the burden of residents and provide a comfortable environment. This non-conducive physical environment can bring avoidable hassle and fatigue, while taking up valuable time. This was the third theme identified and can potentially contribute to burnout and fatigue. In addition, resting and reading areas are not well arranged. “We don’t have study and resting area. The duty rooms are crowded and we always have to move around to find a place to study” said resident 9.

The work environment was also not conducive to work. “There’s a lot of moving around, going up and down stairs to go to laboratory or radiology. The clinic is also far from the hospital. This causes physical and mental fatigue” (Resident 7). Another participant added “I believe most hospitals should have a centralized system to retrieve imaging, laboratory and other results minimizing delayed results. Whatever has an impact on patient care has an impact on the learning environment; our education is patient care!” (Resident 2).

The last theme identified was the lack of diagnostic and therapeutic options for patient care. In light of the resource limitation the country has, the hospital is not able to provide the best possible care for patients. This puts a toll on the learning opportunities available for residents. Resident 6 expressed it as “most of our patients cannot pay for diagnostic investigations and treatments outside the hospital. This forces us to give empirical treatment in the majority of cases and affects our clinical practice”.

Resident 1 gave a supporting statement for this by saying “advanced diagnostic investigations are needed so we can see what we read.”

“All of our guidelines advocate for evidence based patient care. Unfortunately, we don’t have these tests or treatments” (Resident 10).

Discussion

This study revealed that the LE was afflicted with numerous challenges, with reports of negative attributes in all three components of the PHEEM subscales. First-year residents and males reported better perception of the LE.

The overall PHEEM score of 70.87 in this study showed that more negative attributes were expressed by residents compared to positive ones. A study on internal medicine residents from Sudan has reported a similar score of 72.4.⁹ Another study from Ethiopia that included 57 internal medicine residents done in a different medical school reported one of the lowest scores of 69.74.¹⁴ On the contrary, two reports from Pakistan and Singapore reported more positive results of 92 and 112.2, respectively.^1,7 A study done in the study hospital over 6 years ago reported findings from fewer number of internal medicine residents of a mean score of 87, showing the worsening of the perception about the LE.¹⁵ Comparing our findings to studies from other departments, including Pediatrics, Surgery, Obstetrics and Gynecology and Intensive Care training, more positive overall scores of 85 to 103 have been reported.^1,3,5 Local studies have also suggested internal medicine to have lower scores compared to other departments.^14,15 The differences could have resulted from variation in number and experience of academic staff, the different workload residents are expected to handle, and differences in teaching methods among the departments.

Perception of the LE was better among male participants in our study. This finding is similar to reports from a study in Saudi Arabia and the two local studies.^14,15,19 Differences in the responses from the specific questions among genders have been identified, which were seen without difference in the overall subscales and overall scores. Other publications reported no gender differences at all.^1,4,7,9 It has been proposed that cultural differences can influence perception between genders.¹⁹ Most studies did not find year of residency to affect perception.^1,9,15 A report from a family medicine program showed that fourth-year residents reported higher scores.¹⁰ This is in contrast to another study and our finding that suggest first-year residents to have higher perception.¹⁴ Increasing years in the training program can give better opportunities to identify gaps and can give time to adopt to the environment. This along with the ability of final-year residents to give comment on the entire training program might explain the differences of scores among different years.

Subscales of the PHEEM including perception of autonomy, perception of teaching and perception of social support were reported to be in the categories of negative view of one’s role, in need of retraining and not a socially pleasant place, respectively. Similar results were also reported from Sudan and a medical school in Ethiopia.^9,14 On the other hand, results in the higher category of all three subscales were published from studies in Singapore and Brazil.^7,13 The previous study from the current study setting reported higher category of score on all three subscales.¹⁵ This shows that the training program is in urgent need of improvements and has failed to meet the needs, desires and expectation of residents. Worsening of the findings compared to the previous study could suggest that appropriate follow-up of the program has not been done. However, the larger number of participants could have also resulted in different findings. Although studies that exclusively evaluated LE of internal medicine are scarce, all of these reports from internal medicine residents including our report suggest that LE and residents’ views are given less time and concern.

Autonomy in medical education can lead to improved decision-making skills, increased preparation for work independence, increased self-confidence, enhanced responsibility, satisfaction with work and reduced burnout.²² A good teaching climate influences the standard of the learning process,^4,5 satisfaction with education, and determines the final outcome of the curriculum.^5,6 Similarly, the role of social support in a residency program is essential as it is needed to mitigate the effects of decreased sleep, excess stress, depression and burnout.¹³

A review of post-graduate medical education in sub-Saharan Africa published in 2019 identified challenges such as inadequate opportunity for attaining procedural skills, excessive workload, inadequate supervision, lack of quality catering services, poor accommodation and inadequate education materials as some major obstacles.²⁰ These hurdles have also been identified in this study. Suboptimal conditions regarding the hospital’s diagnostic and treatment modalities have been reported, which is one of the themes identified in this study.²⁰ The participants also reported not having enough clinical learning opportunities on the questionnaire, which contrasts to previous reports, but is in harmony with one theme identified.^1,4,5 The low perception of learning environment from Sudan and Ethiopia might suggest the consequence resource limitation brings in these settings. Various solutions such as private sector engagement, public subsidy, utilization of international partners, donations, health insurance and public investment in healthcare have been used in the past.^20,21 Using these potential solutions can help patients get the care they deserve, while benefiting residents in the process.

Among the questions that had the lowest mean values were lack of catering facilities when on call and lack of an informative handbook and clear clinical protocols. Similar studies have identified these factors as major challenges.^4,6,7,14 Residents prefer to have guiding documents for patient management. These should be made easily available and accessible by the faculty and administrators. Being on call can lead physicians to have increased stress, disturb sleep patterns and affect general mental wellbeing.²³ This on top of inadequate catering services can further exacerbate the problem and should be solved with hospital site management. Our focus group participants have further emphasized the importance of the physical environment. In addition to just the catering and accommodation aspects that are included in PHEEM, the hospital physical environment has also been reported to uniquely add to challenges faced. A poor LE has been linked to burnout, one component of which is physical and emotional exhaustion.²⁴ A hospital environment that has easy access to all available services can help reduce the emotional and physical burden on physicians and can help save valuable time.

Residents have reported their workload to be inappropriate.^1,10,14,15 The excess workload can interfere with the protected education time demonstrated in this and previous studies.⁷ Clearly set workload and regulations must be put in place, including setting limits to working hours. This has been practiced in some European countries.¹⁰ There should be clearly set job descriptions, accountability, proper shift management with timely and adequate compensation for extra work.

Supervision and subsequent constructive feedback is an essential part of learning process, and helps create a good relationship with seniors, particularly, for residents who fail to satisfactorily complete their trainings. Feedback is important for academic and professional growth. Inadequate feedback and decreased contact with supervisors have been reported.^5,7,10,14 Residents also reported inadequate bedside teaching. Rushing of seniors to finish rounds was reported to be a negative attribute in this study and a previous study assessing satisfaction of internal medicine residents.²⁵ Our participants described an imbalance between teaching activities and patient service. This imbalance has been investigated in the past and has even been described as exploitative work, as it led to less time available for studies.²⁶ Our participants did not feel ready to be consultants, which is the essential target of their training. Most previous studies reported mean scores of above 2 on this item reflecting the need for LE that is tailored to meet their needs.^4,7,9 Efforts should be made to establish an environment that gives emphasis to resident training without compromising patient care. Changes that ascertain routine work activities are integrated with resident education should be enhanced.

Hospitals can have difficulty with monitoring the quality of clinical teaching.²⁰ These can be mitigated with increasing education and overall contact time, establishing mentoring programs, enhancing pedagogic skills, establishing faculty advisory program, and creating an assessment and feedback method.^7,10 This will also help engage academic staff in teaching activities and improve their teaching skills.

This study included a larger number of internal medicine residents compared to previous local studies. The mixed quantitative and qualitative nature of the study helped identify the effect that resource limitation and associated lack of adequate diagnostic and therapeutic interventions has on residency. It also helped further underscore some of the components of PHEEM and showed it to be a reliable instrument. There are however several limitations to the study. This was a cross-sectional study, and causative inferences cannot be made. There was a relatively low response rate. In addition, it was a single-center study, and further studies including other available teaching institutions in different cities should be done.

Conclusions

In conclusion, the internal medicine residency LE is plagued with many challenges and needs significant improvements in all components. There are several areas that need prompt corrective measures to achieve the targets of the curriculum and the needs of residents. There needs to be a collaborative effort among residents, academic staff, college administrators and all other concerned bodies. Further follow-up and monitoring of improvement efforts should be made.