Hospitalist workload influences faculty evaluations by internal medicine clerkship students

Background

The last decade has brought significant changes to internal medicine clerkships through resident work-hour restrictions and the widespread adoption of hospitalists as medical educators.¹ The 2010 annual Clerkship Directors in Internal Medicine survey indicated that 91% of internal medicine clerkships use hospitalists as teaching physicians.¹ Observational studies^2–4 and a systematic review⁵ have shown hospitalists to be effective medical-student educators, and nearly two-thirds of academic hospitalists are classified as clinician-educators (CEs).⁶

The role of the CE seems ideally suited to academic hospitalists, who have intense day-to-day clinical contact and myriad opportunities for clinical bedside teaching.

The majority of US medical schools have CE faculty tracks that emphasize teaching and clinical service over research.⁷ Surveys of department and promotion committee chairs indicate that teaching skills have the highest priority in promotion decisions regarding CEs,^8,9 with the majority of promotion committee chairs (72%) using learner evaluations as a means of assessing teaching skills.⁸ Department and promotion committee chairs identify learner evaluations as very important, but of low quality, in the promotion decision-making process.⁹

The quality and utility of learner evaluations of faculty has been an area of extensive and often contradictory research.^10,11 A review of larger studies of learner evaluations from a diverse range of educational settings indicates that up to 25% of the variability in learner ratings is due to factors such as class size, workload, difficulty of the subject material, and faculty grading leniency.¹² Investigations into medical-student learner evaluations have shown positive influences on scores due to factors such as clinical rotations occurring earlier in the academic year¹³ and teacher attributes of academic rank,¹⁴ recent board certification, attendance at a teacher training program, and prior receipt of a teaching award.¹⁵ A greater time commitment to teaching has a positive influence on learner evaluations, but this directly conflicts with the clinical service demands faced by many hospitalist CEs.^14–19

Objective

The study reported here was conducted to explore the potential relationship between clinical service intensity and teaching evaluations of hospitalist CEs by internal medicine clerkship students.

Design

The study was a retrospective review of clinical productivity and teaching evaluations of faculty by clerkship students (third-year medical students) prospectively collected during the 2009 to 2013 academic years (July 1 to June 30 of each year) for the hospitalists in the Department of Internal Medicine of the Southern Illinois University School of Medicine (SIU-SOM). After linkage of productivity data to teaching data, the dataset was de-identified prior to analysis.

To protect the anonymity of individual faculty evaluations, no demographic data for the hospitalists were included in the dataset for analysis.

The study was approved as an exempt study by the Springfield Committee for Research Involving Human Subjects, the local institutional review board.

Study participants

SIU-SOM has a total enrollment of 298 medical students who have inpatient rotations at two university-affiliated hospitals. Supervision of the students during internal medicine inpatient rotations is provided by the hospitalists of the internal medicine department at SIU-SOM. All hospitalists are board certified or board eligible in internal medicine, and virtually all inpatients are seen with medical students. A nonacademic hospitalist service does not exist at SIU-SOM for comparison. Non-hospitalist faculty provides occasional weekend coverage, but are not evaluated by medical students for inpatient teaching.

Measures

Clinical service intensity data in terms of work relative value units (wRVUs), the number of patient encounters, and number of days of inpatient duty were collected for all members of the hospitalist service from administrative databases. wRVU values for medical services were determined by the current Center for Medicare and Medicaid Services physician fee schedule. Average wRVUs per day (RVUs/day) and encounters per day (encs/day) were calculated for each hospitalist.

Teaching evaluations of the hospitalist faculty by students were compiled by the clerkship office in an anonymous fashion. Each faculty member was rated on a Likert scale of 1–5 by the students in the areas of patient rapport, enthusiasm about the profession, clinical skills, sharing knowledge and skills, encouraging the students, probing student knowledge, stimulating independent learning, providing timely feedback, providing constructive criticism, and observing patient encounters with students. The average score for the faculty member for each area was used for analysis.

Annual datasets (productivity data matched with teaching scores for a specific hospitalist) from all hospitalists were analyzed with SPSS software (v 20; IBM Corporation, Armonk, NY, USA) for the 2009 to 2013 academic years. A total of 32 datasets were available for analysis. Spearman correlations were used to assess the strength and direction of the association of clinical service intensity variables with teaching scores.

Results

A total of 32 sets of annual learner evaluations and clinical service intensity data were reviewed, representing data for 18 individual hospitalist CEs. The 18 hospitalists included six women (33%), three hospitalists (17%) with 10 or more years of practice after completion of residency, and four hospitalists (22%) with a large percentage of their time spent on administrative duties. Two hospitalists held the rank of associate professor (11%), and the remainder held the rank of assistant professor.

Hospitalists in this study had a median annual productivity of 3,139 wRVUs, had 1,454 patient encounters, and worked in the hospital for 166 days (Table 1). On a daily basis, hospitalists generated 20 wRVUs and had nine patient encounters. The distributions of all evaluated measures of clinical service intensity were skewed toward higher intensity. The greatest degrees of skewing can be seen in duty days and RVUs/day.

Table 1 Clinical service intensity measures Abbreviations: Encs/day, patient encounters per duty day; RVUs, relative value units.

Learner evaluation scores in each area ranged from 3 to 5 out of a maximum possible score of 5, with skewing of the distributions toward higher scores (Table 2). There was greater variability in individual scores, particularly in the areas of clinical skills and sharing knowledge.

Table 2 Annual teaching scores

Correlation analysis (Table 3, Figure 1) showed significant negative correlations between higher wRVUs, total patient encounters, duty days, and learner evaluation scores for enthusiasm about the profession, clinical skills, probing the student for knowledge and judgment, and observing a patient encounter with the student. Higher duty days had a significant negative correlation with sharing knowledge/skills and encouraging student initiative. Higher wRVUs and encounters were negatively correlated with timely feedback and constructive criticism.

Table 3 Spearman correlations between clinical service intensity and teaching scores Note: ‡P<0.050. Abbreviations: Encs/day, patient encounters per duty day; wRVUs, work relative value units; RVUs/day, work relative value units per duty day.

Figure 1 Scatterplot analysis of clinical service intensity vs faculty ratings. Abbreviation: RVUs, relative value units.

In addition to these indicators of clinical service intensity on an annual basis, higher average encs/day was correlated with lower scores on timely feedback. Ratings related to simulating independent learning were insensitive to any of the measures of clinical service intensity.

Discussion

This study showed significant correlations between long-term measures of service intensity such as annual wRVUs, total patient encounters, and duty days, with lower teaching evaluation scores in most areas evaluated. These results differ from data reported for emergency medicine physicians^19,20 and anesthesiologists.²¹ Higher daily wRVUs and daily patient encounters were only correlated with lower teaching scores related to timely feedback.

Potential explanations of the correlation of lower teaching scores with higher clinical service intensity are diverse, including faculty burnout and a local trend for more experienced faculty to spend less time in direct patient care. Emergency medicine and anesthesiology differ significantly from hospital-based medicine because they are geographically organized, procedural, and involve only short periods of patient contact. These factors may attenuate the risk for physician burnout and lower-quality teaching for these specialties.

The SIU-SOM hospitalists’ median wRVUs (3,150) is lower than the national median (4,159) for all hospitalists as reported in the 2012 State of Hospital Medicine Report.²² This is likely due to the academic nature of this service, which is focused on meeting the needs of the learners in the system and has the ability to redirect patients to other independent hospitalist practices. This may allow the hospitalists in this system to spend more time on teaching, which is a known influencer of learner evaluations.^14–16

A wide range (median: 166 days, range: 123–175 days) of annual inpatient duty days was also seen for this group of hospitalists, reflecting the diversity of service assignments for members of the hospitalist practice. Four out of 18 hospitalists (22%) in this study had significant administrative responsibilities, which reduced the time spent on inpatient duty and inpatient teaching.

These factors, which may not be present in other academic hospitalist groups, limit the generalizability of the results of this investigation. Furthermore, this study was a retrospective analysis that looked for correlations between multiple variables, thus it is not likely to account for all potential cofounding and difficult to quantify variables such as teaching experience or skill.

Further investigation of academic hospitalists with wRVU productivity closer to the national median and a larger sample size to investigate the influences of other factors such as academic rank, years of teaching experience, and formal teaching training is essential to determine if medical schools should alter clinical service intensity expectations for teaching faculty to maximize the student-rated quality of the inpatient component of the internal medicine clerkship.

Conclusion

The results of this study suggest that internal medicine clerkship student evaluations of hospitalist faculty may be negatively influenced by high clinical service intensity measured in terms of annual wRVUs, patient encounters, and duty days. Further investigation is needed to determine what level of clinical service intensity begins to affect the learner-rated quality of the internal medicine clerkship inpatient experience.

Disclosure

The author declares no conflicts of interest in this work.

---

References

1.	Liston BW, O’Dorisio N, Walker C, Torre D, Papp KK. Hospital medicine in the internal medicine clerkship: results from a national survey. J Hosp Med. 2012;7(7):557–561.
2.	Kripalani S, Pope AC, Rask K, et al. Hospitalists as teachers. J Gen Intern Med. 2004;19(1):8–15.
3.	Hauer KE, Wachter RM, McCulloch CE, Woo GA, Auerbach AD. Effects of hospitalist attending physicians on trainee satisfaction with teaching and with internal medicine rotations. Arch Intern Med. 2004;164(17):1866–1871.
4.	Hunter AJ, Desai SS, Harrison RA, Chan BK. Medical student evaluation of the quality of hospitalist and nonhospitalist teaching faculty on inpatient medicine rotations. Acad Med. 2004;79(1):78–82.
5.	Natarajan P, Ranji SR, Auerbach AD, Hauer KE. Effect of hospitalist attending physicians on trainee educational experiences: a systematic review. J Hosp Med. 2009;4(8):490–498.
6.	Glasheen JJ, Misky GJ, Reid MB, Harrison RA, Sharpe B, Auerbach A. Career satisfaction and burnout in academic hospital medicine. Arch Intern Med. 2011;171(8):782–785.
7.	Fleming VM, Schindler N, Martin GJ, DaRosa DA. Separate and equitable promotion tracks for clinician-educators. JAMA. 2005;294(9):1101–1104.
8.	Beasley BW, Wright SM, Cofrancesco J Jr, Babbott SF, Thomas PA, Bass EB. Promotion criteria for clinician-educators in the United States and Canada. A survey of promotion committee chairpersons. JAMA. 1997;278(9):723–728.
9.	Atasoylu AA, Wright SM, Beasley BW, et al. Promotion criteria for clinician-educators. J Gen Intern Med. 2003;18(9):711–716.
10.	Greenwald AG. Validity concerns and usefulness of student ratings of instruction. Am Psychol. 1997;52(11):1182–1186.
11.	Marsh HW, Roche LA. Making students’ evaluations of teaching effectiveness effective – The critical issues of validity, bias, and utility. Am Psychol. 1997;52(11):1187–1197.
12.	Marsh HW. Students’ evaluations of university teaching: research findings, methodological issues, and directions for future research. Int J Educ Res. 1987;11(3):253–388.
13.	Shea JA, Bellini LM. Evaluations of clinical faculty: the impact of level of learner and time of year. Teach Learn Med. 2002;14(2):87–91.
14.	Ramsey PG, Gillmore GM, Irby DM. Evaluating clinical teaching in the medicine clerkship: relationship of instructor experience and training setting to ratings of teaching effectiveness. J Gen Intern Med. 1988;3(4):351–355.
15.	Arah OA, Heineman MJ, Lombarts KM. Factors influencing residents’ evaluations of clinical faculty member teaching qualities and role model status. Med Educ. 2012;46(4):381–389.
16.	Ramsbottom-Lucier MT, Gillmore GM, Irby DM, Ramsey PG. Evaluation of clinical teaching by general internal medicine faculty in outpatient and inpatient settings. Acad Med. 1994;69(2):152–154.
17.	Irby DM, Gillmore GM, Ramsey PG. Factors affecting ratings of clinical teachers by medical students and residents. J Med Educ. 1987;62(1):1–7.
18.	Crumlish CM, Yialamas MA, McMahon GT. Quantification of bedside teaching by an academic hospitalist group. J Hosp Med. 2009;4(5):304–307.
19.	Hemphill RR, Heavrin BS, Lesnick J, Santen SA. Those who can, do and they teach too: faculty clinical productivity and teaching. West J Emerg Med. 2011;12(2):254–257.
20.	Begaz T, Decker MC, Treat R, Tews M. No relationship between measures of clinical efficiency and teaching effectiveness for emergency medicine faculty. Emerg Med J. 2011;28(1):37–39.
21.	Backeris ME, Patel RM, Metro DG, Sakai T. Impact of a productivity-based compensation system on faculty clinical teaching scores, as evaluated by anesthesiology residents. J Clin Anesth. 2013;25(3):209–213.
22.	Society for Hospital Medicine, Medical Group Management Association. 2012 State of Hospital Medicine Report. Englewood, CO: Medical Group Management Association; 2012.