Does how much a resident teaches impact performance? A comparison of preclinical teaching hours to pathology residents’ in-service examination scores

Introduction

Educating others is itself a form of active learning, a key component in training adults.¹ The very act of preparing educational material to present to others, analyzing the value of and distilling down practical, experiential, and didactic knowledge into a usable form, are functions that represent higher levels of learning in Bloom’s taxonomy.²

Graduate medical education has long recognized the inherent value of residents as educators. Data suggest that as much as 20%–25% of a resident’s time may involve teaching,^3,4 and medical students may receive a significant portion of their clinical instruction from resident educators.⁵

While many authors have examined the effects of resident teaching on medical student performance or methods to improve residents’ teaching skills, few have specifically examined the potential academic benefits to the residents themselves.^2,4,6 However, these and other such studies concentrate on education in the clinical years and often compare academic or procedural “performance” to student evaluations. To our knowledge, no study has examined residents’ involvement in preclinical didactic teaching, whether the amount of teaching is correlated with academic performance, or if potential benefit of teaching changes with year in training. Our study compared the quantity of pathology residents’ preclinical didactic teaching over 10 years with their performance on a nationally administered standardized in-service examinations. We hypothesized that the amount of teaching in which a resident participated was positively associated with in-service examination scores and that the effect differed as residents progressed in training.

Methods

Pathology residents at the University of Nebraska Medical Center (UNMC) teach pathology to medical and allied health students on a voluntary basis. As a metric for departmental compensation, all teaching activity is monitored by the Educational Support Office of the College of Medicine. This office coordinates all of the college’s educational efforts across the entire UNMC campus. The office quantifies the number of preclinical contact hours for each educator (faculty or resident) and compiles these data into a report for the dean and departmental chairs on an annual basis. The amount of pathology residents’ teaching from 2003 through 2012 was garnered from the spreadsheets used to generate these reports and included the number of hours of lecture, problem-based learning facilitation, and small group leadership provided by the trainees.

The academic records of those enrolled in the anatomic and clinical residency program at UNMC over the 10-year period were reviewed. The residents’ highest scores on step I of the United States Medical Licensing Examination (USMLE^®) were obtained. In addition, each trainee’s overall percentile (as compared to the residents at the same level of training) on the annual Resident In-Service Examination (RISE) administered by the American Society for Clinical Pathology for each year in their training was retrieved.

Correlation between the number of teaching hours and RISE percentile was examined in two ways. First, the average RISE percentile for each resident over their 4 years of training was compared to their individual average number of recorded annual didactic teaching hours over their residency tenure to assess for global trends. Overall differences in means were compared using Student’s t-test as applicable.

Second, the change in RISE percentile for each hour of teaching was compared for each year in training. Statistical analysis was performed using a compound symmetry correlation structure for each of the paired observations (Statistical Analysis Software). Briefly, four independent, generalized, linear models were used to estimate the association of the number of a resident’s teaching hours in a given year of training (first through fourth) to their RISE percentile for that year, not including USMLE data. Afterward, four similar models were used that included step I results. For these eight models, each resident’s RISE percentile and teaching hours for each year were treated as an independent data point.

Results

The records of all 35 residents in the 4-year, combined anatomic and clinical pathology program during the above-mentioned period were included. Each trainee on the program was represented at least once in the data set. Instances in which either RISE score, step I score, or teaching hour data were not available were excluded. In total, 73 individual instances were identified in which all three items were available.

The average number of teaching hours in which the 35 residents participated over their tenure was 9.7 per year (median 9.25, standard deviation 4.35), ranging from 2.5–22.1. The mean number of teaching hours by year in training was as follows: first, 6.4; second, 8.2; third, 10.3; fourth, 10.0. The mean step I score for trainees during this interval was 222 (range 188–254). Residents’ mean career RISE percentile and annual teaching hours averaged over a resident’s 4 years of training showed a weak positive correlation (Figure 1). A similar weak association was noted when comparing residents’ average annual number of hours taught with step 1 scores (Figure 2).

Figure 1 Scatterplot depicting residents’ average annual teaching hours versus mean overall RISE percentile. Abbreviation: RISE, Resident In-Service Examination.

Figure 2 Residents’ average overall RISE percentile compared to admission USMLE step I scores. Abbreviations: RISE, Resident In-Service Examination; USMLE, United States Medical Licensing Examination.

The association between RISE score and teaching hours was examined in two ways: overall career means of teaching hours and RISE score and incremental effect of the former on the latter (Figure 3). To address the first, a comparison was made of each resident’s average annual teaching hours to their mean percentile from their in-service examinations, as shown in Figure 1. Two clusters of data points are evident: those with an average overall RISE score less than the 50th percentile and residents scoring greater than the 50th percentile. The mean number of annual teaching hours for the latter group was 10.4 (median 10.3, standard deviation 4.6) and the former 7.8 (median 7.3, standard deviation 3.0, P=0.01). The mean USMLE scores for both groups were 207 and 230, respectively (P<0.01).

Figure 3 Average annual teaching hours as a function of admission USMLE step I score. Abbreviation: USMLE, United States Medical Licensing Examination.

Second, using the statistical models described above, the incremental difference in RISE percentile per hour taught was examined by year in training (Figure 4). Not taking USMLE performance into account, each hour of teaching performed by interns was associated with a decrease in RISE percentile of 0.6%. In contrast, an hour of preclinical teaching correlated with an almost 3% increase in overall RISE percentile for senior residents (P<0.01), and there is a similar (although lesser) positive association controlling for USMLE performance in this cohort (Table 1).

Figure 4 Linear regressions of scatterplots comparing overall RISE percentile to annual teaching hours by year of training (individual data points not shown). Abbreviations: PGY, postgraduate year; RISE, Resident In-Service Examination.

Table 1 Change in overall RISE percentile per hour of preclinical didactic teaching by year in program with and without controlling for USMLE Step I performance Note: *Expressed as absolute change in percentile score per hour taught. Abbreviations: PGY, postgraduate year; RISE, Resident In-Service Examination; USMLE, United States Medical Licensing Examination.

Discussion

A considerable proportion of most residents’ time is spent teaching medical students and peers. In many instances, this is out of necessity due to competing time demands on faculty and the more consistent presence of residents on the “front lines” of patient care. Some literature has been devoted to examining the effect of residents on student education. This body of work has primarily reviewed topics such as the potential value of giving trainees instruction in teaching methods² or how students’ perceived quality of resident teaching impacts student examination scores.^7–9

It is a truism that the act of instructing others benefits the educator as well as the student. However, only a few authors have explored the effect that teaching has on the academic or clinical performance of residents. Seely et al found that there was a correlation between students’ evaluation of surgery residents’ teaching and in-service examination results, although the trend was not statistically significant.⁴ Similarly, another study showed somewhat indirect evidence that those residents who were “superior teachers” exhibited better didactic knowledge and surgical skills.⁶ In contrast, Morrison et al found that students’ evaluation of a resident’s teaching skills was not associated with scores on a standardized examination.¹⁰

Our study is unique in three ways. First, while much of the literature regarding residents-as-teachers concentrates on medical student instruction in the clinical years, we focused on preclinical didactic education. Second, we attempted to ascertain whether the amount of teaching impacted academic performance, as we hypothesized that each quantum of teaching experience had value to the resident educator. Third, we aimed to examine the degree to which the academic skills of the resident (measured by USMLE step I score) impacted both the amount of teaching in which a resident participated and their in-service examination score. Step I was chosen to evaluate this, as the material in this exercise more closely simulates the content of the pathology in-service examination than the content of the other USMLE tests.

Several factors are important to note when interpreting the presented data. At our institution, pathology residents’ participation in preclinical education is entirely voluntary, hopefully meaning that those who took part in teaching activities were more invested in the process, and residents do not receive formal instruction in educational methods. Further, the medical school and residency’s curricula were virtually unchanged during the examined time frame. In addition, the average RISE score for the program was between the 50th and 75th percentile for each year that data was available (the decrease seen in 2008 and 2009 was primarily the result of two outliers), and average admission USMLE for residents admitted to the program remained between 200–230 between 2003 and 2012. As such, any differences observed were less likely to be a result of a sudden change in the academic “prowess” of the resident pool.

When comparing an individual resident’s average annual teaching hours and mean RISE percentile over their 4 years of training, as shown in Figure 1, two groups can be identified in the scatterplot. Of those whose mean RISE performance was less than the 50th percentile and those above, there was a statistically significant difference between the average number of teaching hours per year and step I score (7.8 hours versus 10.4 hours and 207 versus 230, respectively).

The correlation between the mean number of hours a resident was involved in teaching and their in-service examination scores appeared to increase over the course of their residency (Figure 4). During the intern year there was no significant correlation between the two. In the final year of residency, each hour of student contact was significantly associated with a higher overall RISE percentile with. This association was retained, although of lesser statistical significance, even when taking step I performance into account (Table 1).

An observation that is apparent from the data is that third and fourth year residents took part in more didactic teaching than those earlier in their residency. The reasons for this are uncertain but may represent differences in resident schedules (ie, more elective time than their junior peers) or reflect more free time due to graduated responsibility.

Step I performance is an important confounding variable, as differences in RISE performance may be a reflection of test-taking ability or underlying academic skill. Indeed, we noted a weak positive correlation between step I and average career RISE percentile (Figure 2). Further, even though those residents with an average career RISE score >50th percentile had a greater mean number of teaching hours over their 4 years, this group also had significantly higher average USMLE scores. In addition, the incremental change in RISE percentile per hour taught, as noted in Figure 4, slightly lessened when USMLE data were taken into account. There are several additional limitations to our small, retrospective study. To begin, this study represents the experience of a small cohort of residents from a single institution. The education and background of the residents was fairly homogeneous: more than 75% were graduates of United States medical schools, and at least 50% attended UNMC for undergraduate medical education. Our study did not include clinical/“scope-side” teaching, which likely represents the majority of education in which a pathology resident participates, as this was more difficult to definitively and reproducibly quantify. Indeed, the time that a pathology resident spends on didactic teaching represents a small fraction (<1%) of their residency. It was chosen for evaluation as the data were easiest to obtain, were not affected by considerations such as the number of senior medical students taking part in elective rotations, and represented a voluntary experience. Another question that remains unanswered is if there are other reasons that those with higher USMLE scores participate in more teaching.

It is critical to note that amount of teaching does not necessarily equate to quality of instruction. This was not examined here, as students do not provide written feedback for almost all of the activities in which pathology residents participate, nor were performance metrics available on medical student examination items linked to resident-taught experiences. Data presented by others suggest that more skilled teachers (as evidenced by student evaluations and other metrics) may demonstrate superior performance.^4,6 One study suggested that student ratings of clinical teachers were lower for senior residents than for residents 1 year their junior,⁵ and it would be interesting to compare the students’ perception of the instructor based on year in training.

Most importantly, however, USMLE and RISE scores serve as indirect assessment of a resident’s performance at best. As the majority of residents’ faculty evaluations were at best semiquantitative and/or narrative for a portion of the examined period, correlating these measures with teaching experience is difficult. American Council on Graduate Medical Education Milestones evaluation data may prove a more robust means of comparing resident performance for future studies. Finally, a causal relationship cannot be established; it may be that “stronger” residents partake in more teaching opportunities than those that struggle rather than the teaching experience making an individual resident perform better.

In conclusion, in our data set, there was an association between a resident’s hours of preclinical educational experience and their mean performance on an annual nationally administered standardized examination. A resident that had a mean in-service examination score less than the 50th percentile over their 4 years undertook significantly fewer teaching hours each year on average. The strongest positive correlation between RISE score and teaching hours was seen with fourth year residents, also significant when controlling for USMLE performance. Based on the information from this limited study, there is insufficient information to draw firm conclusions about the strength or reproducibility of the trends noted. However, these data do suggest that there may be an association between the volume of teaching that a resident undertakes and their academic performance, particularly later in their training, and provide interesting potential directions for future investigation.

Disclosure

No external funding was used in the preparation of this manuscript. The authors report no conflicts of interest in this work.

---

References

1.	Weiss V, Needlman R. To teach is to learn twice. Resident teachers learn more. Arch Pediatr Adolesc Med. 1998;152(2):190–192.
2.	Bloom BS. Taxonomy of Educational Objectives. Book 1: The Cognitive Domain. New York; David McKay Co, Inc.; 1956.
3.	Wamsley MA, Julian KA, Wipf JE. A literature review of “resident-as-teacher” curricula: do teaching courses make a difference? J Gen Intern Med. 2004;19(5 Pt 2):574–581.
4.	Seely AJ, Pelletier MP, Snell LS, Trudel JL. Do surgical residents rated as better teachers perform better on in-training examinations? Am J Surg. 1999;177(1):33–37.
5.	Wilson FC. Teaching by residents. Clin Orthop Relat Res. 2007;454:247–250.
6.	Falcone JL, Ferson PF, Hamad GG. S/he who can, does and teaches. S/he who cannot, doesn’t. J Surg Educ. 2014;71(1):96–101.
7.	Langenfeld SJ, Helmer SD, Cusick TE, Smith RS. Do strong resident teachers help medical students on objective examinations of knowledge? J Surg Educ. 2011;68(5):350–354.
8.	Roop SA, Pangaro L. Effect of clinical teaching on student performance during a medicine clerkship. Am J Med. 2001;110(3):205–209.
9.	Stern DT, Williams BC, Gill A, Gruppen LD, Woolliscroft JO, Grum CM. Is there a relationship between attending physicians’ and residents’ teaching skills and students’ examination scores? Acad Med. 2000;75(11):1144–1146.
10.	Morrison EH, Rucker L, Prislin MD, Castro CS. Lack of correlation of residents’ academic performance and teaching skills. Am J Med. 2000;109(3):238–240.