Lecturing skills as predictors of tutoring skills in a problem-based medical curriculum

Introduction

The concept of learning rather than teaching was a major breakthrough that challenged the role of traditional teachers in the academic front.¹ An effective teacher is the one who stimulates thinking and facilitates student learning.² Therefore, the role of the teacher goes well beyond providing information, with the teacher having a range of key roles to play in the education process.³ Twelve roles for any medical teacher have been identified and grouped into six domains: information provider, role model, facilitator, student assessor and curriculum evaluator, curriculum and course planner, and resource material creator.³

Problem-based learning (PBL) is one of the learning strategies that enables the students to develop critical thinking skills through posing challenges based on clinical case scenarios.⁴ The small-group tutorial is one of the corner-stones in PBL⁵ during which students are put in an active learning situation by providing them with clinical problems.⁶ The problem comes first without advance readings, lectures, or preparation, serving as a stimulus for the need to know.⁷ The students then go through active discussion and analysis of problems, develop hypotheses, explain mechanisms, and generate learning issues.⁸ These student-generated learning needs guide self-directed learning between the tutorial sessions, and then in subsequent sessions, students reapply, synthesize, and appraise their learning.⁹ Teaching in a PBL curriculum is a much different experience than almost any other instructional method. In the PBL tutorial group, the teacher’s main role is a facilitator for the students learning. On the other hand, she/he may deliver lectures (Resource Sessions) to a large student group with a primary focus on the content expertise as lectures in PBL can also be an important learning resource.¹⁰

Therefore, the role of the PBL tutor is of pivotal importance, as student learning would depend on the tutor’s understanding and appreciation of her/his responsibilities in the small group.⁵ The tutor keeps the group focused on their tasks and guides them to achieve their goals.¹¹ The tutor’s skills used during the PBL tutorial session are: asking open-ended questions, listening to students and intervening only when needed, helping students to reflect on their experience, monitoring progress of the group, promoting the group dynamics, stimulating critical thinking, and encouraging a warm tutorial atmosphere.¹² The tutor must not dominate a session with content-specific questions and answers that convert it into a tutor-led seminar.⁹

Most of the faculty members in PBL medical programs are graduated from discipline-based programs with good familiarity in lecturing and little experience in small-group facilitation. Although faculty development programs are routinely held to train PBL tutors to better practice their role in the small group tutorials, the performance of faculty members as tutors in PBL is unpredictable. Furthermore, students’ rating of the effectiveness of faculty on PBL tutoring is influenced by their teaching styles and other dimensions such as content expertise and interpersonal attributes.¹³ On the other hand, lecturing styles can vary from being didactic, in line with discipline-based programs, to being interactive, which is more in line with the notions of knowledge and learning in PBL.¹⁴ However, there is no current body of evidence on whether aspects of lecturing skills can predict the tutoring skills of faculty members in PBL programs. This study was, therefore, designed to answer the following research question: What aspects of the lecturing skills of a faculty member can predict his/her tutoring skills in a PBL medical school?

Methods

Study context and subjects

This study is a cross-sectional survey conducted at the College of Medicine and Medical Sciences (CMMS), Arabian Gulf University, Bahrain. The study included faculty members (n=69) who participated in both tutoring and teaching a particular PBL unit (block) in the pre-clerkship phase (years 2, 3, and 4) during the academic years 2010 to 2012. At the end of each PBL unit, lecturing skills of faculty members were evaluated by students (n=45±8 per faculty), while tutoring skills of the same faculty members were evaluated by students in their tutorial groups (n=8±2 per faculty). For each faculty member, evaluations of tutoring skills and lecturing skills by students were averaged for the purpose of statistical analysis. All faculty members involved in PBL tutoring have been involved in a comprehensive faculty development program before being inducted and certified as tutors. Lecturers are the subject matter experts of disciplines related to the problems of each PBL unit. The research protocol was approved by the Curriculum Committee and Research and Ethics Committee at CMMS.

Study instruments

Two types of instruments were used in this study for evaluation of the faculty members by students. To ensure content validity of both instruments, they were developed based on a comprehensive literature review and refined by input from a group of medical education experts. The first instrument evaluates the PBL tutoring skills and includes ten items addressing the tutor’s role in the PBL tutorials. These items are: enthusiastic about educational role, stimulates group interactions, provides effective feedback, facilitates discussion of learning needs, helps students to achieve unit objectives, encourages students’ self-directed learning, communicates clearly with students, facilitates integration of knowledge, and behaves as a role model. In the last item, students were asked to provide an overall rating of the effectiveness of the tutor. The second instrument evaluated the lecturing skills of faculty members in Resource/Review Sessions. This instrument is modified from a previously validated form.¹⁵ The evaluation form included six areas: lecture organization and content, interactivity and clarity of presentation, subject matter mastery, class atmosphere, effective use of audiovisual aids, and time management in the lecture. In both study instruments, each parameter is evaluated based on a five-point Likert scale (1= poor, 2= fair, 3= good, 4= very good, 5= excellent).

Statistical analysis

The data were entered and analyzed using the Statistical Package for Social Sciences (SPSS) version 22 (IBM Corporation, Armonk, NY, USA). Data were presented as mean ± standard deviation (SD) for each parameter. Internal consistency reliability for the study instruments was measured using Cronbach alpha statistics. In order to determine the effectiveness of different lecturing skills (independent variables) in predicting PBL tutoring skills (dependent variables), stepwise multiple regression was used to estimate how much variance in tutoring skills was accounted for by items of lecturing skills. A P-value <0.05 was considered statistically significant.

Results

Descriptive statistics

The response rate from the students in PBL tutorials was 78%, while the response rate for students who evaluated the lecturing skills was 38%. However, the absolute number of responses for evaluating PBL tutoring skills of faculty was 552 and for lecturing skills of faculty was 3, 105 students. Internal consistency reliability coefficient for the tutoring skills evaluation form was 0.94 and for lecturing skills was 0.93. Table 1 shows the students’ ratings for 69 faculty members regarding their tutoring and lecturing skills. The mean scores of both tutoring and lecturing skills are considered to be relatively high. Among the tutoring skills, the ability to communicate well was rated with the highest mean scores (4.55±0.51), while providing effective feedback was the lowest (4.29±0.59). Among the lecturing skills, subject matter mastery was rated with the highest mean score (4.01±0.50), while clarity and interactivity was the lowest (3.89±0.53).

Table 1 Student-rated scores of the faculty members regarding their tutoring and lecturing skills Note: Data scores are expressed as mean ± SD. Abbreviation: SD, standard deviation.

Do lecturing skills predict tutoring skills in a PBL program?

The six items of lecturing skills (lecture content and organization, interactivity and clarity, subject matter mastery, relaxed class atmosphere, effective use of audiovisual aids, and time management) were used in a stepwise multiple regression analysis to predict each of the items in tutoring skills. The prediction models indicated that three lecturing skills, namely, subject matter mastery, relaxed class atmosphere, and effective use of media of audiovisual aids were significant predictors of PBL tutoring skills. Table 2 shows the raw and standardized regression coefficients of the predictors, their squared correlations (ΔR²) to show how much of the variance in each outcome variable can be accounted by each predictor, and the analysis of variance (ANOVA) results (F and P-values) to indicate the degree of significance of the model in predicting the outcome variables. Among the lecturing skills, subject matter mastery was the most important predictor of overall effectiveness in tutoring skills and accounted for 25% of the variance in overall effectiveness of PBL tutors (F=22.39, ΔR²=0.25, β=0.50, P=0.000). In addition, subject matter mastery positively predicted the following five PBL tutoring skills: 1) enthusiastic about his/her role; 2) helps students achieve unit objectives; 3) stimulates students’ self-directed learning; 4) facilitates integration of knowledge; and 5) behaves as a role model.

Table 2 Stepwise linear regression analysis of the relationship between students’ rated scores of lecturing skills of faculty as predictors for their PBL tutoring skills Notes: β: A measure of how strongly each predictor variable influences the criterion variable. ΔR2: Indicates the proportion of the variance in the criterion variable which is accounted for by our model. Abbreviations: ANOVA, analysis of variance; PBL, problem-based learning.

The second significant predictor of tutoring skills was relaxed class atmosphere during the lecture, which positively predicted providing effective feedback and stimulating students’ interaction in PBL tutorials. Finally, effective use of audiovisual aids in a lecture positively predicted communicating well with students and facilitating group discussion of learning needs in PBL tutorials.

Discussion

This study examined the faculty lecturing skills as predictors of their tutoring skills in a PBL medical program. The results of the regression analysis indicate that subject matter mastery in a lecture is the most significant predictor of the overall effectiveness of a PBL tutor. Therefore, according to students’ evaluations, the teacher who is able to explain difficult concepts for students and respond satisfactorily to their queries in a lecture is most likely to be a good PBL tutor. These findings add another piece to the controversy regarding the use of content versus non-content experts as PBL tutors. Previous studies indicated that students appreciated the facilitative role of faculty in PBL tutorials.^16,17,18,19 Furthermore, tutors stressing on the learning process in the tutorial group were perceived as more effective than content expert tutors.¹⁷ In contrast, other studies indicated that students rated highly tutors with content knowledge related to the problem.^13,20,21 The contradictive findings in these studies could be due to several variables including the definition of content expertise, cultural differences, level of faculty training, curriculum structure, and academic level of students.

This study demonstrates that subject matter mastery in a lecture predicted the enthusiasm of PBL tutors about their role, promoting knowledge integration, helping students to achieve unit objectives, stimulating student’s self-directed learning, and the tutor as a role model. These findings are supported by previous studies which indicated that students with expert tutors generated significantly more learning issues, which were more congruent with the faculty objectives,²² and spent longer studying time on self-directed learning.^22,23 Furthermore, a recent study in Brazil indicated that medical students perceived content expert facilitators to be more effective than their nonexpert counterparts at building knowledge, guiding the learning process, achieving cognitive learning, generating learning goals, and motivating self-study.²⁴ In contrast, PBL tutors with subject-matter expertise were reported to play a directive role in the tutoring process, supply more direct answers to questions posed by students, and suggest more points for discussion with less emphasis on group interactions.^15,25,26 However, a key issue is how the faculty uses her/his content expertise in promoting students learning in both lectures and in PBL tutorials. Therefore, it is conceivable that faculty in the current study use their content expertise in promoting students’ learning in PBL tutorials through identifying gaps of knowledge to generate learning needs and demonstrating the relevance of knowledge to promote integration, which motivate the students’ self-directed learning.

The finding that subject matter mastery explained only 25% of the variance in PBL tutor effectiveness indicates that there are other important factors which could explain tutor effectiveness. A previous study indicated that students identified three main roles for effective PBL tutors: facilitative expertise, knowledge expertise, and clinical reasoning expertise.²⁷ In addition, Maudsley⁹ claimed that effective PBL tutors promote student learning by creating a supportive environment that encourages active participation by all members of the group, by monitoring the quality of learning through questions and feedback and by encouraging the development of students’ metacognitive skills. Furthermore, one of the attributes of an effective PBL tutor was establishing rapport with students, through establishing good relationships with students, respecting students’ opinions, understanding students’ feelings, appreciating students’ performance, and being friendly and kind with students.¹³ These studies are corroborated by our finding that providing a relaxed class atmosphere significantly predicted stimulating students’ interactions and providing effective feedback in PBL tutorials. A positive learning environment is essential for feedback to be maximally effective.²⁸ In addition, feedback was termed effective when it is conducted in a private setting using a considerate tone and requires good interpersonal skills on the part of the teacher.²⁹ Similarly, climate setting was identified as one of the key elements on how the prospective PBL tutor might prepare for her/his role as a facilitator.¹² The teacher can establish a relaxed class atmosphere by demonstrating respect for the learners and their needs, praise, encouragement of participation, and lack of threat to personal integrity.³⁰ An interesting finding in the present study is that effective use of audiovisual aids by the lecturer predicted both her/his communication clearly with students and helping them to discuss learning needs in PBL tutorials. This finding indicates that faculty who encourage the use of audiovisual aids in lectures are likely to encourage the students to use them in PBL tutorial groups. A recent study indicated that students perceived that the use of diagrams as an audiovisual aid in PBL tutorials helped them to structure knowledge, to develop an overview of topics, and stimulated them to find relationships between topics.³¹ Furthermore, PBL tutors emphasized that diagramming increased interaction and enhanced the focus and detail of the discussion.³¹ Overall, this study emphasized that effective PBL tutoring requires multiple skills from faculty members that are positively related to their role in large group lectures.

There are some limitations of this study which need to be explored. The study is conducted in one PBL medical school and evaluation of lecturing and tutoring skills of faculty is done only through students’ ratings. The way PBL is implemented in that school and how students see a good PBL tutor could, therefore, affect the relationships between lecturing and tutoring skills. A larger scale study conducted on different PBL medical schools and triangulating sources of faculty evaluation will be required to ensure generalizations of the study findings. Although only two measures were taken which supported the content-validity evidence and internal consistency reliability for both instruments, a comprehensive study to evaluate the other types of validity evidence for the scores emanating from these instruments will be required.

Conclusion

Subject matter mastery, providing a relaxed class atmosphere, and effective use of audiovisual aids by faculty in lectures significantly predicted their PBL tutoring skills. However, subject matter mastery in a lecture appears to be the most important predictor of the overall effectiveness of PBL tutors. These findings could have implications for recruiting PBL tutors in medical programs and for tutor training workshops.

Acknowledgment

The authors would like to thank Professor Abdel Halim Salem, Vice Dean for Academic Affairs, College of Medicine and Medical Sciences, Arabian Gulf University, Bahrain, for his help in accessing the data related to students’ evaluations of faculty members.

Author contributions

Salah Eldin Kassab initiated the study idea, designed the research methods, and conducted the statistical analysis. Nahla Hassan wrote the initial draft of the manuscript. Marwan F Abu-Hijleh participated in study design and data analysis. Reginald Sequeira participated in study design and data analysis. All authors contributed to the interpretation of data, drafting, and critical revision of this paper and approved the final version of the manuscript.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Srivastava TK. Medical teacher as a facilitator. Natl J Physiol, Pharm Pharmacol. 2014;4(3):179–181.
2.	Jaques D. Teaching in small groups. In: Cantillon P, Wood D, editors. ABC of learning and teaching in medicine. 2nd ed. London: Blackwell Publishing Ltd; 2010:23–28.
3.	Harden RM, Crosby J. AMEE Guide No 20: The good teacher is more than a lecturer: the twelve roles of the teacher. Med Teach. 2000; 22(4):334–347.
4.	Jafri W, Mumtaz K, Burdick WP, Morahan PS, Freeman R, Zehra T. Improving the teaching skills of residents as tutors/facilitators and addressing the shortage of faculty facilitators for PBL modules. BMC Medical Education 2007;7:34.
5.	Mclean M. What can we learn from facilitator and student perceptions of facilitation skills and roles in the first year of a problem-based learning curriculum? BMC Med Educ. 2003;3:9.
6.	Das M, Mpofu DJ, Hasan MY, Stewart TS. Student perceptions of tutor skills in problem-based learning tutorials. Med Educ. 2002;36:272–278.
7.	Wood D. Problem-based learning: clinical review. ABC of learning and teaching in medicine. BMJ. 2003;326:328–330.
8.	Mennin SP, Friedman M, Skipper B, Kalishman S, Snyder J. Performances on the NBME I, II, and III by medical students in the problem-based learning and conventional tracks at the University of New Mexico. Acad Med. 1993;68(8):616–624.
9.	Maudsley G. Roles and responsibilities of the problem-based learning tutor in the undergraduate medical curriculum. BMJ. 1999;318:657–661.
10.	Albanese MA. Problem-Based Learning. An Introduction to Medical Teaching. Jeffries WB, Huggett KN, editors. Dodrecht: Springer Science Business Media B.V; 2010:41–53.
11.	Azer S. Challenges facing PBL tutors: 12 tips for successful group facilitation. Med Teach. 2005;27(8):676–681.
12.	Neville AJ. The problem-based learning tutor: teacher? facilitator? evaluator? Med Teach. 1999;21:393–401.
13.	Kassab S, Al-Shboul Q, Abu-Hijleh M, Hamdy H. Teaching styles of tutors in a problem-based curriculum: students’ and tutors’ perception. Med Teach. 2006;28(5):460–464.
14.	Malik AS, Malik RH. Twelve tips for effective lecturing in a PBL curriculum. Med Teach. 2012;34:198–204.
15.	Shevlin M, Banyard Y, Davies M, Griffiths M. The validity of student evaluation of teaching in higher education: love me, love my lectures? Assess Eval High Educ. 2000;25:4, 397–405.
16.	Kaufman DM, Holmes DB. The relationship of tutors’ content expertise to interventions and perceptions in a PBL medical curriculum. Med Educ. 1998;32(3):255–261.
17.	De Grave WS, Dolmans DH, van der Vleuten CP. Profiles of effective tutors in problem-based learning: scaffolding student learning. Med Educ. 1999;33(12):901–916.
18.	Dolmans DH, Wolfhagen IH, Scherpbier AJ, van der Vleuten CP. Relationship of tutors’ group dynamics skills to their performance ratings in problem-based learning. Acad Med. 2001;76(5):473–476.
19.	Steinert Y. Student perceptions of effective small group teaching. Med Educ. 2004;38(3):286–293.
20.	Davis WK, Nairn R, Paine ME, Anderson RM, Oh MS. Effects of expert and non-expert facilitators on the small-group process and on student performance. Acad Med. 1992;67:470–474.
21.	Schmidt HG, Moust JHC. What makes a tutor effective? A structural equations modeling approach to learning in problem based curricula. Acad Med 1995;70:708–714.
22.	Eagle CJ, Harasym PH, Mandin H. Effects of tutors with case expertise on problem-based learning issues. Acad Med. 1992;67:465–469.
23.	Schmidt HG, Van Der Arend A, Moust JHC, Koxx I, Boon L. Influence of tutors’ subject matter expertise on student effort and achievement in problem-based learning. Acad Med. 1993;68(10):784–789.
24.	Couto LB, Bestetti RB, Restini CBA, Faria-Jr M, Roma GS. Brazilian medical students’ perceptions of expert versus non-expert facilitators in a (non) problem-based learning environment. Med Educ Online. 2015;20:26893. Available from http://dx.doi.org/10.3402/meo.v20.26893. Accessed August 15, 2015.
25.	Silver M., Wilkerson L. Effects of tutors with subject expertise on the problem-based tutorial process. Acad Med. 1991;66:298–300.
26.	Gilkison A. Techniques used by “expert” and “non-expert” tutors to facilitate problem-based learning tutorials in an undergraduate medical curriculum. Med Educ. 2003:37(1):6–14.
27.	Caplow JA, Donaldson JF, Kardash C, Hosokawa M. Learning in a problem-based medical curriculum: students’ conceptions. Med Educ. 1997;31(6):440–447.
28.	Hewson MG, Little ML. Giving feedback in medical education: verification of recommended techniques. J Gen Intern Med. 1998;113:111–118.
29.	Bing-You RG, Paterson J, Levine MA. Feedback falling on deaf ears: residents’ receptivity to feedback tempered by sender credibility. Med Teach. 1997;19:40–44.
30.	Hutchinson L. ABC of learning and teaching. Educational environment. BMJ. 2003;326:810–812.
31.	De Leng B, Gijlers H. Collaborative diagramming during problem based learning in medical education: do computerized diagrams support basic science knowledge construction? Med Teach. 2015;37(5):450–456.