Preliminary investigation into application of problem-based learning in the practical teaching of diagnostics

Introduction

Problem-based learning (PBL) is a pedagogical approach based on problems. PBL essentially involves small groups of students discussing some form of trigger material (the problem) and determining what they need to study, then meeting again to share the results of their learning. PBL was initially proposed in 1969 by Barrows, a professor of neurology, and was first adopted by McMaster University in Canada. Subsequently, PBL was tried and popularized in the USA and Europe throughout the 1970s. PBL advocates active learning, emphasizes problem-solving, combines various learning approaches, encourages mutual exploration and academic debate, and emphasizes the role of communication and cooperation. PBL has received high-level attention in the education sector and is advocated across the world.¹ To date, PBL has been implemented either partially or fully in more than 200 medical colleges.² In fact, Harvard Medical School has fully adopted PBL to replace traditional teaching methods.³

The key feature of PBL is the use of problems as a starting point to acquire knowledge and develop students’ problem-solving skills. Diagnostics is an introductory discipline of clinical medicine, and key aspects of teaching diagnostics involve a combination of basic theoretical knowledge with clinical practice and the development of students’ clinical thinking skills and ability to solve clinical problems, which represent the advantages of the PBL method. However, national and international studies on the application of PBL in teaching diagnostics are rare. In addition, the results of systematic evaluation have shown that although a large number of studies focus on PBL, the quality of the majority of the studies is rather low.^4–7 Previous studies indicate that PBL is more effective in improving students’ information management skills and decision-making abilities.^2,7,8 However, the results of a meta-analysis showed that PBL could improve basic medical theory examination scores in medical undergraduates, but did not have a long-term effect.⁴ Other research on clinical skills teaching also showed that PBL is helpful for improving professional and clinical diagnostic skills, and other aspects of comprehensive ability of medical students, but more subjective evaluation methods used in this research lead to the limitation of poor strength of argument.⁵ However, the results of PBL have been criticized and questioned.⁹ Although studies have shown that the academic achievements obtained using PBL are not lower than those obtained using traditional teaching methods, PBL puts greater demands on teachers, teaching time, and learning space, and increases learning burdens. Therefore, PBL may seem less effective than traditional learning methods.^10,11 Against this background, we conducted a randomized controlled trial to investigate the differences in educational performance between PBL and traditional teaching methods, with the aim of providing supportive evidence for the implementation of PBL across a wide range of disciplines.

Materials and methods

Study design

A prospective randomized controlled trial was conducted using students as study subjects. A total of 80 college juniors in clinical medicine were selected from our university using the cluster sampling method. The students were grouped randomly using the computer random digital method. Grouping of the students was conducted based on student identification numbers and on the premise that the teachers at the Office of Teaching Aids were not notified. Out of the 80 students, 40 were assigned to a PBL group, which adopted PBL pedagogy, and 40 were assigned to a control group, which employed traditional teaching approaches.

Textbooks and curriculum setup

The two study groups used the same standard textbooks as other national medical colleges (Diagnostics, seventh edition), as well as the same syllabus and practice guidance. The lecturer, teaching schedule, and examination formats were identical. Students in the PBL group learned the chapter on symptomatology of fever, abdominal pain, and palpitation via the PBL method as follows. One week before the class, the teacher revealed the curriculum setup, assigned self-study materials, and selected clinical cases suitable for teaching purposes a few hours before the class. At the first class, the students were divided into groups of four. These groups conducted bedside diagnostic examinations and obtained medical histories from patients (1.5 class hours). A group discussion was then held in which the students shared ideas on diagnostic analysis, identified problems to be addressed, determined the approaches to be used, and allocated tasks within the group (1.5 class hours). The students searched and sorted through the information after class and completed the assigned tasks. At the second class (1 week later), the students engaged in group discussions, shared the learning outcomes, and summarized the situation of problem-solving and goal achievement (2.5 class hours). During the group discussion, the students were allowed to expand on or leave the questions. The teachers then made appropriate comments (0.5 class hours). Students in the control group learned via traditional methods, ie, tutors gave a lecture on key knowledge points (0.5 class hours). The students were then divided into groups of the same size as the PBL groups and collected the patient medical histories at the bedside (1.5 class hours). The teachers commented on the performance of the students and summarized the application of key knowledge points in specific clinical cases as well as diagnostic ideas (one class hour). Group discussion or self-study was arranged by the students after the class. That is, the experimental class consisted of 6 hours (not including homework) spread across two sessions 1 week apart in one topic, while the control class was only 3 hours and without homework.

Ethics statement

Ethical approval to conduct this study was obtained from the ethics committee at our institution. All students were told the purpose of this research and the methods that would be used before entering the study, and were also informed that the teaching content involved in the research was limited, and would arrange teachers’ answering questions and supplementary explanation time before the end of the course of this study, all of this let students who participated in the research totally understand the details, and voluntarily sign the informed consent. All research involving human subjects was conducted with prior institutional ethical approval under the requirements of the Chinese Prevention of Cruelty to Human Subjects and the Code of Practice for the Care and Use of Human Subjects for Scientific Purposes. All human subjects used in this study were inspected by members of the Human Subject Ethics Committee of West China Medical Centre (HSPC20130102-13477) in compliance with the Declaration of Helsinki.

Evaluation methods

At the end of the study period, a teaching feedback survey questionnaire was conducted among the students in the PBL group. In addition, medical record writing, clinical case analysis, clinical skills assessment, written examination scores in every stage and behavioral observation were compared between the two groups of students and statistically analyzed. The questionnaire included seven items graded on a 5-point Likert scale (5, strongly agree; 4, agree; 3, neither agree nor disagree; 2, disagree; 1, strongly disagree).

Medical record writing was evaluated and scored based on the integrity of content, organizational rationality, and coherence of the text (a total of 10 points). Clinical case analysis was graded based on summary of the characteristics of medical history and diagnosis, differential diagnosis (including the main differential diagnosis point), and general idea and its logicality of the idea (a total of 10 points). Medical history writing and case analysis were evaluated by all teachers according to the unified standard during pipeline operation; Although teachers know the group allocation, but it is difficult for them to check the student information and make discriminatory operation due to the large number of students and medical records. Thus, it could largely avoid the bias from the grouping information.

The clinical skills assessment was divided into two parts. First, the diagnostic section was evaluated in terms of content and skills. The scores for diagnostic content were calculated as n/(45–50). The students were assigned different standardized patient (SP) examinations by drawing lots, and clinical cases varied randomly according to the assigned SP, although the difficulties were rather similar. The scores for diagnostic skills were calculated as n/90, with each skill worth 10 points. Second, the physical examination assessment was performed in relation to content and examination skills. The scores for examination content were calculated as n/(186–192). The students were assigned different SP examinations by drawing lots, and the items and total scores varied depending on the sex of the SP. The scores for examination skills were calculated as n/45, with each skill worth 10 points. The clinical skills examination was conducted using standard patients.

The stage examination (performed 6 months after the end of the teaching period) included two parts. The first part was a written examination containing 20 objective questions (worth 20 points). All of the questions were single-answer in nature according to A2-type medical history, and the questions were used to examine the student’s ability to analyze and comprehensive application. The stage examination was done using a machine readable card. The second part of the stage examination involved behavioral observation of students during the training process. The tutors in each group subjectively evaluated the student’s overall performance during the training process (a total of 10 points), including their autonomous learning skills (2 points), ability to logically analyze a discussion (4 points), clinical problem-solving skills, and ability to cooperate with partners (4 points) according to tutors who could be in an uninformed state.

Statistical analysis

The statistical analysis was performed using Statistical Package for the Social Sciences version 17.0 statistical software (Chicago, IL, USA). The data are presented as the mean ± standard deviation and analyzed using two-group independent sample t-tests . Count data were analyzed using the χ² test. The statistical significance for all measurements was set at P<0.05.

Results

Baseline student characteristics

There were no significant differences in sex composition or performance in basic course work between the two groups (P>0.05). Therefore, the two groups were comparable (Table 1).

Table 1 Comparison of the baseline characteristics of the two groups of students Note: The scales used in the Characteristics column are as follows: humanistic quality (100 points), human morphology (100 points), physiology (100 points, pathophysiology (100 points), pathology (100 points). Abbreviation: PBL, problem-based learning.

Comparison of academic achievement (scores) in diagnostics between the two groups of students. Statistically significant differences were not identified between the two groups of students with regard to written or clinical skills examination, which included medical record writing, diagnostic content, physical examination content, and skills (P>0.05). The difference in the written examination (favoring the PBL group) could be noted as a trend (P=0.06). In contrast, scores for clinical case analysis, diagnostic skills, and behavioral observation were significantly higher in the PBL group when compared with the control group (P<0.05). The results are summarized in Figure 1 and Table 2.

Figure 1 Comparison of academic achievement (scores) in diagnostics between the two groups of students. Note: *P<0.05. Abbreviation: PBL, problem-based learning.

Table 2 Comparison of academic achievement (scores) in diagnostics between the two groups of students Note: *P<0.05 versus the control group. Abbreviation: PBL, problem-based learning.

PBL questionnaire survey

Forty completed questionnaires were returned, giving a survey response rate of 100%. In the items regarding PBL as a teaching method, including “PBL enhanced learning interest and sense of involvement”, “PBL enhanced analytical and problem-solving skills”, “PBL promoted self-directed learning”, “PBL improved communication skills”, “PBL improved team cooperative skills”, and “Compared with the traditional teaching method, I prefer PBL teaching method”, the percentages of students selecting “agree or strongly agree” were significantly higher than those selecting “disagree”. In addition, most students agreed that “Learning knowledge through PBL was fragmentary and lacking in systematicness” and “PBL increased academic burden”. The results of this survey are shown in Table 3.

Table 3 Results of the PBL evaluation questionnaire survey Abbreviations: PBL, problem-based learning; SD, standard deviation.

Time investment

A sample group of teachers and students was interviewed to assess the time investment in teaching and learning. In the PBL group, students took an average of 6 hours in class and 11 hours after school to complete a case, and teachers spent approximately 10 hours preparing for one lecture. In the control group, students spent an average of 3 hours in class and 3 hours after school on the corresponding contents, and teachers spent approximately 3 hours preparing one lecture.

Discussion

The results of this study indicate the superiority of PBL over traditional method methods, as reported by previous studies.^2,7,8,10 PBL emphasizes the core contents required by the global minimum essential requirement,¹² which include the students’ inherent development advantages essential for their overall professional attainment and also this blind zone was the focus in the traditional test-based evaluation.^13,14 Therefore, in this study, subjective evaluation by clinical tutors was added as an evaluating indicator. Student behavior was followed, observed, and comprehensively evaluated throughout the semester to compensate for the coverage limitations of the other evaluating indicators. The results showed that students in the PBL group generally received a more favorable subjective evaluation from the tutors, indicating that PBL training indeed improved the overall professional attainment of medical students. However, we also realize that due to the difficulties of implementing a blinded study, observation bias was nearly impossible to prevent. Therefore, the results of our study require further verification using more rigorously designed research.

Studies have shown that students who complete PBL training display higher professional competence and adapt better to their professional roles after graduation.^15–17 Long-term effects are normally assessed using methods that display the capacity for comprehensive evaluation, such as the Objective Structured Clinical Examination. However, because the long-term effects are often not seen for several years and are influenced by teaching and other hidden factors unrelated to teaching, it is difficult to separate out and pinpoint the effects of PBL from multiple confounding factors. Therefore, in this study, the immediate medical record writing and clinical case analysis capabilities after the teaching intervention and the clinical skills examination were utilized as evaluation indicators for short-term teaching effects. Meanwhile, the stage written examination held 6 months later and behavioral observation and assessment were used as evaluation indicators for long-term effects. The results indicated no significant differences in the grades of the clinical skills examination (which included medical record writing, diagnostic content, and physical examination content and skills) and the stage written examinations between the two groups of students. This result may be related to the fact that the examinations employed largely traditional educational assessment methods to examine the mastery of knowledge. This result also indicated that PBL pedagogy yielded the same outcomes for knowledge mastery as the traditional teaching methods.¹³ Although there were no differences in scores for most items in the clinical skills examination between the two groups of students, the scores for diagnostic skills were significantly higher in the PBL group than in the control group. In particular, the PBL group displayed better flexibility and adaptability when communicating with SPs, indicating that PBL improved the ability to communicate. We also found that the PBL group performed better in the clinical case analysis and differential diagnosis, which is consistent with published reports.^18,19 Our results also indicate that PBL enhanced grades for short-term comprehensive examination. In our study, the stage written examination, which mainly tested the students’ skills in clinical case analysis and comprehensive application, served as an evaluation indicator for long-term effects. No significant differences were found in the stage written examination scores between the two groups, which is also consistent with previous findings.⁴ This result indicated that although PBL enhanced the grades for short-term comprehensive examination, no significant impact on long-term learning effects was observed. Moreover, the effects of PBL were weakened gradually with time.

Students in the PBL group generally received a higher subjective grading from tutors in the longer-term evaluation 6 months later. Although PBL seemed to have no long-term effects on examination scores, it had a delayed effect on improvement of an individual’s comprehensive behavioral performance. This phenomenon may be related to the fact that PBL improves communication and teamwork skills, self-directed learning skills, and problem-solving skills, which are the qualities overlooked by traditional examination methods. Moreover, the relatively hysteretic evaluation system failed to reflect the advantages of PBL.¹⁴ This overall positive long-term effect indicates exciting incentives for the implementation of PBL.

The results of the present study show that the majority of students were satisfied with the PBL method. In contrast with traditional teaching methods, whereby students assume long-term passive and non-relevant roles, the PBL approach enabled students to have a sense of participation, which may be a reasonable explanation for the results. Under the PBL approach, the students were able to access real clinical cases. In addition, the students identified problems through team work and formulated ways to solve these problems. Using library and Internet resources, each group member actively and conscientiously accomplished his/her own tasks. The students then returned to their groups and shared their learning experiences; in particular, they were able to present their opinions freely in a relaxed and safe environment. Moreover, they were able to acquire knowledge and skills and offer mutual assistance through reciprocal questioning and recognition. The students not only achieved their learning goals but also proactively improved their identification, communication, and collaboration skills. In fact, the learning process was no longer tedious and instead became interesting. Compared with traditional teaching methods, the PBL approach trained the students’ clinical thinking in a better way to develop their skills in formulation of questions, comprehensive analysis, document retrieval, language performance, and exploration and innovation through the processes described above.^18,20

This study also showed that the PBL approach required time input from the students and teachers that was approximately three times more than that associated with traditional teaching methods. On the one hand, PBL requires teachers to design a curriculum and select problem-based clinical cases relevant to specific educational objectives. The teachers also need to take into account the integration of knowledge points. Therefore, the teaching burden is significantly increased because tutors may have to learn and replenish a large amount of knowledge relevant to the subject being taught.^11,13,21 In addition, the teaching process for PBL requires that the teachers possess not only professional knowledge and skills and extensive interdisciplinary knowledge, but also strong leadership and communication skills. Specifically, teachers must provide attention to each member of the group, control the overall situation, and maintain appropriate efficiency and flexibility. It is also essential that the timing and extent of the intervention and guidance are appropriate during the teaching process. These requirements produce significant challenges for PBL teachers and are unavoidable obstacles to the implementation of PBL in medical colleges. On the other hand, PBL directly avoids the traditional methods of imparting knowledge and strongly promotes self-directed learning. For Chinese students, who have received many years of indoctrination in traditional education, elimination of psychological dependence and alteration learning habits and mindset cannot be achieved instantly. Indeed, the self-exploration process requires large amounts of time and energy, which increases the already heavy academic burden on medical students. In particular, previous studies have shown that PBL may require 4–5 times more class hours than traditional teaching methods to convey the same information.¹⁴

PBL has historically been criticized for lack of systematicness in the imparting of knowledge.^22,23 Using PBL, key knowledge may be omitted,⁶ and the PBL approach is not as efficient as lecture-based teaching for the learning of basic theory. In addition, PBL demands higher requirements regarding the student’s critical thinking and inquiry abilities, and the current study confirmed this point of view.

This study was conducted as an RCT, in which an allocation concealment procedure was performed to minimize selection bias. Therefore, this study has a certain advantage in terms of its design. However, it also has some limitations, including a small sample size and failure to achieve a blinded evaluation of the study results. Moreover, the PBL teaching method aims to develop a mature, applicable scientific evaluation system, which requires a combination of self-assessment, mutual assessment, and the teacher’s evaluation of various nondominant qualities through knowledge point examination. These characteristics represent the key to solving the bottleneck of research problems on teaching methods, although all of the above points need to be explored in depth through more rigorously designed studies with a larger sample size.

Application of PBL in the practical teaching of diagnostics significantly improves medical students’ skills in comprehensive analysis, communication, and patient handling in the short-term and is likely to exert a long-term, positive effect on the overall professional performance of medical students. Thus, PBL represents a highly effective scientific teaching method and should be expanded in the future.

Conclusion

PBL is a student-centered, problem-oriented teaching method that is conducted through group discussions. The core idea uses problems as the starting point for acquisition and integration of new knowledge. Our questionnaire survey revealed that PBL could improve the interest in learning, cultivate the ability to study independently, and foster strong communication and analytical skills as well as good team cooperation spirit. However, there were some shortcomings in systematization of imparting knowledge. PBL has an obvious advantage in diagnostic practice teaching.

Acknowledgment

This research was supported by the China Medical Board (CMB-00-721).

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Newman MJ. Problem-based learning: an introduction and overview of the key features of the approach. J Vet Med Educ. 2005;32(1):12–20.
2.	Taylor D, Miflin B. Problem-based learning: where are we now? Med Teach. 2008;30(8):742–763.
3.	Peters AS, Greenberger-Rosovsky R, Crowder C, Block SD, Moore GT. Long-term outcomes of the new pathway program at Harvard Medical School: a randomized controlled trial. Acad Med. 2000;75(3):470–479.
4.	Neill PA. The role of basic sciences in problem-based learning clinical curriculum. Med Educ. 2000;34(8):608–613.
5.	Colliver JA. Effectiveness of problem-based learning curriculum: research and theory. Acad Med. 2000;75(3):259–266.
6.	Herzig S, Linke RM, Marxen B, Börner U, Antepohl W. Long-term follow up of factual knowledge after a single, randomized problem-based learning course. BMC Med Educ. 2003;3(3):1–4.
7.	Albanese MA, Mitchell S. Problem-based learning: a review of literature on its outcomes and implementation issues. Acad Med. 1993;68(1):52–81.
8.	Matthes J, Marxen B, Linke RM, et al. The influence of tutor qualification on the process and outcome of learning in a problem-based course of basic medical pharmacology. Naunyn Schmiedebergs Arch Pharmacol. 2002;366(1):58–63.
9.	Ozuah PO, Curtis J, Stein RE. Impact of problem-based learning on residents’ self-directed learning. Arch Pediatr Adolesc Med. 2001;155(6):669–672.
10.	Vernon DT, Blake RL. Does problem-based learning work? A meta-analysis of evaluative research. Acad Med. 1993;68(7):550–563.
11.	Beachey WD. A comparison of problem-based learning and traditional curricula in baccalaureate respiratory therapy education. Respir Care. 2007;52(11):1497–1506.
12.	Rodriguez R, Campos-Sepulveda E, Vidrio H, Contreras E, Valenzuela F. Evaluating knowledge retention of third-year medical students taught with an innovative pharmacology program. Acad Med. 2002;77(6):574–577.
13.	Silvia M, Schmidt HG, Norman GR. Innovations in problem-based learning: what can we learn from recent studies? Adv Health Sci Educ Theory Pract. 2006;11(4):403–422.
14.	Major CH, Palmer B. Assessing the effectiveness of problem-based learning in higher education: lessons from the literature. Academic Exchange Quarterly. 2001;5:4–9.
15.	Norman GR, Wenghofer E, Klass D. Predicting doctor performance outcomes of curriculum interventions: problem-based learning and continuing competence. Med Educ. 2008;42(8):794–799.
16.	Neville AJ. Problem-based learning and medical education forty years on. A review of its effects on knowledge and clinical performance. Med Princ Pract. 2009;18(1):1–9.
17.	Norman GR, Schmidt HG. Effectiveness of problem-based learning curricula: theory, practice and paper darts. Med Educ. 2000;34(8):721–728.
18.	Wolfram A, Stefan H. Problem-based learning versus lecture-based learning in a course of basic pharmacology: a controlled, randomized study. Med Educ. 1999;33(2):106–113.
19.	Shin JH, Haynes B, Johnson ME. Effect of problem-based, self-directed undergraduate education on life-long learning. CMAJ. 1993; 148(6):969–976.
20.	Houlden RL, Collier CP, Frid PJ, John SL, Pross H. Problems identified by tutors in a hybrid problem-based learning curriculum. Acad Med. 2001;76(1):81.
21.	Thomas RE. Problem-based learning: measurable outcomes Med Educ. 1997;31(4):320–329.
22.	Maudsley G. Making sense of trying not to teach: an interview study of tutors’ ideas of problem-based learning. Acad Med. 2002;77: 162–172.
23.	Kilroy DA. Problem-based learning. Emerg Med J. 2004;21(4):411–413.