Asking what do residents value most: a recent overview of internal medicine residents’ learning preferences

Introduction

Residency is a critical, formative, multidimensional educational experience that bridges the gap from medical student to independent practitioner. A resident is the consumer of numerous learning sessions designed to provide the knowledge, communication skills, and hands-on “know-how” to succeed as a high-caliber, informed, and caring physician. The Accreditation Council for Graduate Medical Education (ACGME) mandates that internal medicine residency programs provide a myriad of educational experiences to support resident learning.^1,2 However, little is known about which educational experiences internal medicine residents value most.

The prior literature has evaluated and described medical student and resident learning style preferences;^3–10 however, there are few papers analyzing residents’ perceptions of the most valuable learning experiences in their daily work.¹¹ Several published studies do examine individual aspects of the resident educational experience, including learning preferences on rounds; reading habits; preferences in noon-conference presentations; educational experiences as they relate to ACGME competencies; and the relationship between resident reading, conference attendance, and in-training exam scores.^12–16 These studies offer important contributions to our understanding of resident learning. However, understanding which educational methods and strategies today’s residents value most remains a significant challenge facing the graduate medical education community. There are two studies in the surgical literature which take on this question, asking residents to reflect on their most significant learning experiences.^17,18 However, to our knowledge, there are no prior studies examining the learning preferences of internal medicine residents. The purpose of this study is to provide a current overview of what educational experiences internal medicine residents value most and why.

Methods

Survey distribution

This study was approved by the Partners Human Research Committee, in July 2016. We conducted an anonymous survey study of the Brigham and Women’s Hospital internal medicine residency during the 2016–2017 academic year. The survey instrument was created by a third-year resident (JC) in the Brigham and Women’s Hospital internal medicine residency and was reviewed by a senior faculty member (HS), our statistician (SP), and the residency program director (Dr. Joel Katz). The survey instrument was created using Qualtrics, a computer software that makes respondents’ email addresses anonymous, available at Harvard University and Harvard Medical School. Completion of the survey was deemed to be informed consent. Because the email addresses were made completely anonymous by the Qualtrics software, it was not possible to determine on how many days each resident completed the survey.

All members of the internal medicine residency (N=182) received a daily email with a link to an online survey on nine separate weekdays. The survey was distributed Monday through Thursday during week 1 and Monday through Friday during week 2. The survey administration period included a rotation switch day so that most residents participated in more than one rotation during the study. Residents were informed that their responses would be confidential. The total number of possible participants on each day of the survey administration varied from 116 to 120 residents. This reflected that only residents who were participating in internal medicine inpatient or ambulatory rotations at Brigham and Women’s Hospital, or an elective rotation in research at Brigham and Women’s Hospital, at the time of survey administration, were eligible for the study. This means that residents who were on vacation, sick call, a non-internal medicine rotation, or on a rotation at an unaffiliated site were excluded from this study. All residents received a link to the online survey, but if a resident noted that he or she was in one of these ineligible groups, then the survey was terminated automatically.

Residents were asked, “Please choose your most valuable learning experience in the past 24 hours and give a one sentence description of why it was valuable to you”. They were then asked to rate their chosen learning experience on a 5-point Likert-type scale and were asked two follow-up questions about the experience: “Where did this experience occur?”, and asked to choose only one location, and “Who else was involved?”, and invited to mark all that applied. They were also asked to provide their postgraduate year and current rotation with each survey (Table S1).

Data analysis

After data collection, qualitative analysis of resident responses to the question “Please choose your most valuable learning experience in the past 24 hours and give a one sentence description of why it was valuable to you” was performed to look for repeated concepts and ideas. These initial concepts were organized into a series of categories, which were then grouped together into themes.^19,20 Open coding was initially performed by one of the study investigators (JC), a third-year internal medicine resident, to identify why respondents found their chosen learning experience to be valuable. Based on the identification of keywords, phrases, and concepts in the text, 22 codes were generated (Table 1). Each response could be coded with one, several, or zero codes depending on the content of the response. A senior faculty member from the department of medicine (HS) then reviewed each response and assigned codes with the first coder (JC). Each response was discussed until consensus was reached between JC and HS. Axial coding was then performed, wherein the responses were all reexamined in the context of their assigned codes and based on commonalities the specific codes were divided into five thematic groups (Table 1). Next, the responses were reviewed by a second coder (SS), a second-year internal medicine resident, who independently assigned codes to each response from the list of codes generated by JC and HS. The two sets of coded data (produced by JC and SS, respectively) were then assessed for agreement. A third coder (ED), also a second-year internal medicine resident, then reviewed and independently coded all discordant responses. The second and third coders (SS and ED) also reviewed each of their assigned codes with the same senior faculty member from the department of medicine (HS) to ensure that each code was thoughtfully assigned. For a response to be included in analysis, there needed to be agreement between two out of the three coders. We considered agreement to mean either assignment of the same code or assignment of two codes from the same thematic group. Any responses that were discarded because of discordance among coders were reviewed to ensure that they were not systematically different from those that were included.

Table 1 List of codes and thematic groupings

In summary, our coding was done methodically by having each coder read the item out loud, process it, discuss it, and then jointly assign it a code. We did not use a systematized rubric or other validated tool that could easily be translated into numeric values. Thus, our data do not lend themselves to calculation of Cohen’s kappa or Fleiss score when analyzing interrater reliability.

Results

Resident respondents

In total, the 182 internal medicine residents completed 303 surveys during the nine-day study period. The overall response rate varied by day and ranged from 20% to 49% during the study period (Table S2). Respondents were from all years of internal medicine training and participating in a wide variety of rotations (Table 2). Because the survey software made the respondent’s email addresses completely anonymous, it was not possible to know how many residents were repeat responders and exactly how many times each resident responded.

Table 2 Participating residents’ year of training and current rotation Abbreviation: VA, Veterans Health Administration Medical Center.

Characteristics of residents’ most valuable learning experiences

The 182 residents completed a total of 303 surveys. Of the 303 surveys, 92% (N=277) of the responses noted their chosen learning experience was useful, with 63% (N=191) reporting the experience to be “very useful” and 29% (N=86) reporting it to be “somewhat useful” (Figure 1). The residents’ most valuable learning experiences included a wide variety of other individuals. An attending was cited most frequently, and involved in 50% of total experiences (N=152). The patient was involved in 8% (N=25) of all experiences. Nurses (N=14, 5%) and other allied health care professionals (N=12, 4%) were the least frequently cited individuals (Table 3). The experiences occurred in a variety of settings. The most commonly cited setting was the team workroom (N=94, 31%; Table 3).

Figure 1 Internal medicine residents were surveyed during the 2016–2017 academic year about their most valuable learning experiences and were asked to rate these experiences on a 5-point Likert-type scale. Note: This figure depicts the frequency of each response on that scale (N=303).

Table 3 Participants’ most valuable learning experience and experience setting Notes: aMultiple answers were allowed. bOnly one answer was allowed.

Coding

Of the 303 responses, 266 surveys (88%) included a response to the free-text question, “Please choose your most valuable learning experience in the past 24 hours and give a one sentence description of why it was valuable to you”. Of these 266 responses, 152 respondents (54%) provided a description of why the experience was valuable. The remaining 114 responses named a valuable experience without providing an insight into why it was valuable. Of these 152 complete responses, 138 were assigned open codes with agreement between at least two out of the three coders (91%), leaving only 14 responses uncoded. These 14 responses were reviewed to see if they were somehow different, more interesting, or important compared to the other responses. We found that the discarded responses tended to be shorter and more vague than the included responses. In 13 of the 14 discarded responses, at least one of the coders did not feel that a code could be assigned based on the information provided, suggesting that these responses were more vague than the responses included in the dataset. Because each response could be assigned more than one code, there were a total of 162 codes with agreement between at least two out of the three coders. As described above, the open codes were divided into five thematic groups representing the key feature of the resident’s most valuable learning experience: Repetition in Learning, Bedside Learning, Clinical Problem Solving as an Individual or Collaboratively, Effective Pedagogy, and Opportunity for Active Engagement (Table 1). Figure S1 depicts the distribution of each thematic group within the sample. Of interest, the five thematic groups represented different aspects of resident learning and pedagogy. The most commonly identified themes were Repetition in Learning (N=48, 35%) and Effective Pedagogy (N=47, 34%), followed by Clinical Problem Solving as an Individual or Collaboratively (N=34, 25%), Opportunity for Active Engagement (N=22, 16%), and Bedside Learning (N=16, 12%). Of note, each survey response could be coded into zero, one, or multiple thematic groups.

Thematic groups

Verbatim examples of resident responses categorized into each of the five thematic groupings are presented in Table 4.

Table 4 Internal medicine residents were surveyed during the 2016–2017 academic year about their most valuable learning experiences Notes: Internal medicine residents were surveyed during the 2016–2017 academic year about their most valuable learning experiences. Responses were coded and categorized into five thematic groups. This Table lists representative resident responses for each thematic grouping. *Quotations could be coded into more than one thematic group. This is noted throughout the table. Abbreviations: AM, morning; CRT-D, cardiac resynchronization therapy defibrillator; ED, emergency department; RVR, rapid ventricular response.

Repetition in Learning

This theme captures the idea that residents found value in repeated exposure to or practice with core concepts and skills. Residents valued learning through a large volume of clinical experience with exposure to various clinical topics repeatedly and through a variety of modalities: at the bedside, at the microscope, in clinical team discussion, through self-guided learning, or formal didactics.

Effective Pedagogy

Several residents described what they felt was effective learning facilitated by either great teachers or successful teaching methods. The exemplary teachers described were most often attendings with specific expertise, but also included coresidents who used engaging teaching strategies.

Clinical Problem Solving as an Individual or Collaboratively

Many responses focused on learning how to tackle a specific clinical problem. Residents valued group brainstorming, opportunities for individual creative thinking, and the acquisition of new skills or tools to use in clinical decision making.

Opportunity for Active Engagement

Some residents felt that they were learning best because they had the opportunity to participate actively in the learning process. They described interactive lectures, small group didactics, or team rounds where they were given space to participate, discuss, and ask questions.

Bedside Learning

Several responses touched on the salience of learning that occurs while taking care of a patient in real time. Residents found that active engagement in individual patient care helped make learning memorable.

Discussion

To our knowledge, this is the first study to provide an overview of internal medicine residents’ perceptions of what learning experiences are most valuable and what makes them valuable. The strengths of our study are the breadth of the experiences we captured, novel delineation of the setting and cast of characters in resident learning, as opposed to just the content, and exploration of why residents find more value in certain learning experiences. Our findings depict a rich tapestry of learning opportunities within residency education. The image of learning in residency that emerges from this single-academic-center study highlights the importance of diversity of experiences and educational modalities.

Specifically, and surprisingly, we noted a conspicuous absence of the patient as a central figure in the residents’ reported most valued learning experiences. Residents cited the patient as one of the people involved in their chosen learning experiences in only 8% (N=25) of responses (Table 3). Forty percent (N=121) of the most valuable learning experiences occurred in either the team workroom (31%, N=94) or on the medical floor (9%, N=27), locations where the residents engage in work and learning related to patient care; yet, the patient is not given credit as a major educator in these experiences. Examining the thematic groupings used to code our resident responses, “Bedside Learning” was the least well-represented category, accounting for only 12% of codes (Figure S1). While it seems that many of residents’ high-yield learning experiences relate specifically to care of individual patients, residents are only associating the patient as a key figure in their learning experiences a minority of the time.

Bedside learning, defined as learning in the presence of the patient, is recognized as an integral and essential component of resident learning^21–25 and has been the focus of several recent, published interventions.^26–28 Furthermore, spending time at the bedside engaged in patient care has been shown to combat physician burnout.²¹ The divide between enthusiasm about learning from clinical care and identifying the patient as a central part of that learning is demonstrated in our study. We believe this is a relevant and unexpected finding that should encourage residency programs to actively incorporate the patient as a teaching partner.²⁸

In addition, it is notable that the residents surveyed mention nurses and other allied health care professionals infrequently as participants in their useful learning experiences. Nurses are noted as a participant in the most valuable learning experience in 5% of responses (N=14), and other allied health care professionals are noted in 4% of experiences (N=12; Table 3). Interprofessional learning is emerging as a key educational goal for today’s residents and has been shown to improve patient care outcomes.^29–31 Thus, our finding of only few resident surveys noting interprofessional learning experiences is of significant importance and provides a rationale for the development of increased and innovative opportunities for interprofessional education.

It is important to note that on a survey administered by a residency program, there is a possible source of implicit bias on the part of respondents toward recollecting educational experiences that involve their formal educators. This potential bias may partially explain the prevalence of formal educators as participants in residents’ most valued experiences, and may have overshadowed the patient and other allied health care professionals as additional possible valuable contributors. However, our survey asked that residents select as many individuals as were involved in their chosen learning experience. Thus, the rare mention of patients, nurses, and allied health care professionals is an important finding.

Our data describe the educational methods and strategies that today’s residents value most, and in doing so provide an opportunity to develop new strategies that residency programs can use to increase the visibility of the patient and allied health care professionals in valuable resident learning experiences. Based on our data, we challenge educators to create space for all team members, including the patient, to engage in clinical interactions and discussions and ask questions both during rounds and throughout the day. The next steps in our research will include evaluation of whether tailoring residents’ educational experiences to those they find most valuable will have a positive outcome on resident learning, job satisfaction, and quality of resident life.

Our study has several limitations. First, the daily response rate averaged 37% over the nine days of survey administration. However, this response rate is within the range of what is reported in the literature for trainee and physician response rates to surveys.^32,33 This may be attributed to survey fatigue, especially noting that the response rate trended down over successive days in the study period.³⁴ Perhaps, this would be mitigated in future studies if the same number of surveys were spaced out over a longer period. Second, we surveyed residents from a single internal medicine residency program at a single academic hospital. Third, our study was a snapshot capturing nine days. It would be more informative to collect data over a longer period of time. Fourth, because our survey instrument anonymized the email addresses of respondents, we do not know how many times each resident responded to our survey over the nine-day study period. This is a limitation that would be correctable when future computerized survey instruments are programmed for anonymous distribution. Fifth, our study examined resident impressions of learning, but did not measure resident learning itself as an outcome. Future studies are needed to assess the relationship between resident learning preferences and specific identified learning outcomes.

Conclusion

Our data and detailed analyses provide a new and broader recognition of the learning preferences of today’s internal medicine residents. Our results indicate that internal medicine residents value exposure to a wide variety of patient care-centered and core didactic activities, and value learning that is relevant to patient care, actively engages them in the learning process, and is delivered by skilled and expert physician educators. We clearly illustrate that the residents surveyed value learning experiences centered on patient care, but they do not recognize the patient as a central educational figure. In addition, residents are rarely engaging in, recognizing, or finding value in interprofessional educational experiences. In the future, educators will need to partner with the patient and other members of the health care team to deliberately design learning experiences that are focused, high yield, and inclusive. We look forward to bridging this divide by creating, designing, and implementing educational experiences that inspire today’s residents and faculty, based on these results.

Acknowledgments

The authors would like to thank Dr. Marshall Wolf and Dr. Joel Katz for their guidance and support, as well as the data coders, Dr. Ersilia Defilippis and Dr. Sanjay Salgado. Funding was provided by the Myra and Robert Kraft Fellowship in Medical Education. This work was presented at Harvard Medical School Medical Education Day in October 2017.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Accreditation Council for Graduate Medical Education. ACGME Common Program Requirements. Chicago, IL: Accreditation Council for Graduate Medical Education; 2017. Available from: http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf. Accessed October 31, 2017.
2.		Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Internal Medicine. Chicago, IL: Accreditation Council for Graduate Medical Education; 2017. Available from: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/140_internal_medicine_2017-07-01.pdf. Accessed June 28, 2017.
3.		Turner DA, Narayan AP, Whicker SA, Bookman J, McGann KA. Do pediatric residents prefer interactive learning? Educational challenges in the duty hours era. Med Teach. 2011;33(6):494–496.
4.		Shrewsbury DD, Wiskin DC. Medical student preference in teaching methods and educational support. Health Soc Care Educ. 2013;2(2):11–15.
5.		Liew SC, Sidhu J, Barua A. The relationship between learning preferences (styles and approaches) and learning outcomes among pre-clinical undergraduate medical students. BMC Med Educ. 2015;15:44.
6.		Modi N, Williams O, Swampillai AJ, et al. Learning styles and the prospective ophthalmologist. Med Teach. 2015;37(4):344–347.
7.		Kim RH, Gilbert T. Learning style preferences of surgical residency applicants. J Surg Res. 2015;198(1):61–65.
8.		Neuhauser C. Learning style and effectiveness of online and face-to-face instruction. Am J Distance Educ. 2002;16(2):99–113.
9.		Pashler H, McDaniel M, Rohrer D, Bjork R. Learning styles: concepts and evidence. Psychol Sci Public Interest. 2008;9(3):105–119.
10.		Rogowsky BA, Calhoun BM, Tallal P. Matching learning style to instructional method: effects on comprehension. J Educ Psychol. 2015;107(1):64–78.
11.		Martin SK, Farnan JM, Arora VM. Future: new strategies for hospitalists to overcome challenges in teaching on today’s wards. J Hosp Med. 2013;8(7):409–413.
12.		Edson RS, Beckman TJ, West CP, et al. A multi-institutional survey of internal medicine residents’ learning habits. Med Teach. 2010;32(9):773–775.
13.		Fraser T, Sargsyan Z, Baggett TP, Baggett M. Quantitative study of the characteristics of effective internal medicine noon conference presentations. J Grad Med Educ. 2016;8(2):185–190.
14.		McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22(7):962–968.
15.		Roy B, Huff N, Estrada C, Castiglioni A, Willett L, Centor R. Contextual influences of trainee characteristics and daily workload on trainee learning preferences. J Hosp Med. 2017;12(7):558–561.
16.		Guerrero LR, Baillie S, Wimmers P, Parker N. Educational experiences residents perceive as most helpful for the acquisition of the ACGME competencies. J Grad Med Educ. 2012;4(2):176–183.
17.		Monaghan SF, Blakely AM, Richardson PJ, Miner TJ, Cioffi WG, Harrington DT. The reflective statement: a new tool to assess resident learning. J Surg Res. 2012;178(2):618–622.
18.		Michel E, Meyerson SL. Resident identification of significant learning experiences: a qualitative analysis. Ann Thorac Surg. 2016;102(5):1731–1735.
19.		Weisberg H, Krosnick JA, Bowen BD. An Introduction to Survey Research, Polling, and Data Analysis. 3rd ed. Thousand Oaks, CA: SAGE; 1996.
20.		Vogt WP, Vogt ER, Gardner DC, Haeffele LM. Selecting the Right Analyses for Your Data: Quantitative, Qualitative, and Mixed Methods. New York, NY: Guilford Press; 2014.
21.		Crumlish CM, Yialamas MA, McMahon GT. Quantification of bedside teaching by an academic hospitalist group. J Hosp Med. 2009;4(5):304–307.
22.		Williams KN, Ramani S, Fraser B, Orlander JD. Improving bedside teaching: findings from a focus group study of learners. Acad Med. 2008;83(3):257–264.
23.		LaCombe MA. On bedside teaching. Ann Intern Med. 1997;126(3):217–220.
24.		Ramani S, Orlander JD, Strunin L, Barber TW. Whither bedside teaching? A focus-group study of clinical teachers. Acad Med. 2003;78(4):384–390.
25.		Gonzalo JD, Masters PA, Simons RJ, Chuang CH. Attending rounds and bedside case presentations: medical student and medicine resident experiences and attitudes. Teach Learn Med. 2009;21(2):105–110.
26.		Ali NB, Pelletier SR, Shields HM. Innovative curriculum for second-year Harvard-MIT medical students: practicing communication skills with volunteer patients giving immediate feedback. Adv Med Educ Pract. 2017;8:337–345.
27.		Gonzalo JD, Chuang CH, Huang G, Smith C. The return of bedside rounds: an educational intervention. J Gen Intern Med. 2010;25(8):792–798.
28.		Perez AA, Pelletier SR, Klein IA, Shields HM. Patient as co-teacher on rounds: a pilot study. J Gen Intern Med. Epub 2018 Apr 9.
29.		Reeves S, Perrier L, Goldman J, Freeth D, Zwarenstein M. Interprofessional education: effects on professional practice and healthcare outcomes (update). Cochrane Database Syst Rev. 2013;(3):CD002213.
30.		West C, Landry K, Graham A, et al. Conceptualizing interprofessional teams as multi-team systems-implications for assessment and training. Teach Learn Med. 2015;27(4):366–369.
31.		Zabar S, Adams J, Kurland S, et al. Charting a key competency domain: understanding resident physician interprofessional collaboration (IPC) skills. J Gen Intern Med. 2016;31(8):846–853.
32.		Kellerman SE, Herold J. Physician response to surveys. A review of the literature. Am J Prev Med. 2001;20(1):61–67.
33.		Phillips AW, Friedman BT, Utrankar A, Ta AQ, Reddy ST, Durning SJ. Surveys of health professions trainees: prevalence, response rates, and predictive factors to guide researchers. Acad Med. 2017;92(2):222–228.
34.		Porter S, Whitcomb M, Weitzer W. Multiple surveys of students and survey fatigue. New Directions for Institutional Research. 2004;121:63–73.

Supplementary materials

Table S1 Survey instrument Abbreviation: VA, Veterans Health Administration Medical Center.

Table S2 Daily survey response rate

Figure S1 Distribution of thematic groupings.