The Effect of Portfolio-Based Education and Evaluation on Clinical Competence of Nursing Students: A Pretest–Posttest Quasiexperimental Crossover Study
Authors Assadi Hoveyzian S, Shariati A, Haghighi S, Latifi SM, Ayoubi M
Received 19 September 2019
Accepted for publication 18 January 2021
Published 19 February 2021 Volume 2021:12 Pages 175—182
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Md Anwarul Azim Majumder
Shahla Assadi Hoveyzian,1 Abdolali Shariati,2 Shayeste Haghighi,2 Sayed Mahmud Latifi,2 Mohammad Ayoubi3
1Abadan Faculty of Medical Sciences, Academic Board, Nursing and Midwifery School, Abadan, Iran; 2Ahvaz Jundishapur of Medical Sciences, Ahvaz, Iran; 3Cellular and Molecular Biology Branch, Science School, Tehran University, Tehran, Iran
Correspondence: Shahla Assadi Hoveyzian
Academic Board, Nursing and Midwifery School, Abadan Faculty of Medical Sciences, Abadan, Iran
Tel + 98 916-304-6425
Purpose: Currently, revisions in education and evaluation in the nursing domain are necessary to increase the clinical competence of graduates. The aim of this study was to compare nursing students’ clinical competence using the portfolio method with conventional education and evaluation methods in clinical education.
Methods: In this pretest–posttest quasiexperimental crossover study, all six-semester undergraduate nursing students of Ahvaz Jundishapur University of Medical Sciences were classified into four groups. They were selected based on the use of portfolio and conventional methods during their apprenticeship. Then, they were educated using conventional and portfolio methods. Their clinical competence was measured by a self-appraisal questionnaire. Data were statistically analyzed with independent and paired t-tests.
Results: There were significant differences between average scores of clinical competence in all three domains before and after study in portfolio-based education method (P=0.0001). Furthermore, the average difference in scores before and after internship between the two educational groups in all three domains — cognitive (P=0.002), affective (P=0.0001), and psychomotor — (P=0001) was significant. Similarly, there was a significant difference in total average scores of clinical competence between the two methods (P=0.0001).
Conclusion: The results showed the effect of educational method on increasing the clinical competence of nursing students in a clinical unit. This new method can be used as a combination of education, evaluation and student-centered methods.
Keywords: clinical competence, education, nursing student, portfolio
Nowadays, due to the practical orientation of the nursing profession and the increased information and techniques of nurses, educational considerations in this domain have high importance.1–3 The goal of nursing education as a practical discipline is to increase students’ awareness at the highest level of clinical competence in the domain of nursing care.4–7
Competence in nursing has many definitions. For example, according to the World Health Organization, competence in nursing is defined as a level of performance that represents the effective use of knowledge, skills, and judgment and as the ultimate efficiency of any educational system encompassing a variety of learning domains.8 The UK Nursing and Midwifery Council uses the term “competence” simply to describe the skills and ability to practice safely and effectively without the need for direct supervision.9 Nurse competence is determined through an assessment of the nurse’s personal characteritics, clinical observations, and practical skills in contact with patients and colleagues and written examinations.10
Studies in different countries have indicated that impediments toclinical education can be lack of access to adequate clinical trainers and lack of proper coordination between clinical education in schools. Facilities in hospitals are mong the clinical education problems of students in achieving professional competence.2,11 According to the findings of some studies, nursing students’ clinical competence is moderate and poor.12
Based on studies in Iran, most nursing students believed that they could not learn the skills needed for employment in nursing during the courses.13 Furthermore, Imanipiur and Jalili showed that 79.6% of students believed that current evaluations indicated the personal opinion of teachers versus the clinical competence of students.2
Nowadays, expectations of care service-recipients of service-provider systems with experienced employees and high clinical competence have increased. This issue is important in the domain of health care, particularly in nursing education and management.14,15
In recent years, extensive studies have been performed to modify or replace clinical education methods. These methods have advantages and disadvantages. They require planning and preparation to be executable while study using the different systems affords different educational opportunities and reduces weaknesses related to the separate use of each educational method.16
One of the new combined methods of education and evaluation is the portfolio method, which has been presented in the domain of medical education during the last few years.11 Various studies and investigations on the impacts of the portfolio method on different aspects of nursing students’ education and evaluation have been carried out. For example, McCready and Pennbrant and Nunstedt indicated that clear guidelines for portfolio construction and qualitative assessment are important to competence in practice. The portfolio method can help nursing students to develop understanding of their future profession.17,18 Byrne et al indicated that the portfolio process is a validated method encompassing personal and professional aspects in interactive and multidimensional manner in nursing domains, and may be used for other organizations based on their own portfolio models.19 Buckly et al indicated that portfolios can improve theoretical and practical skills of students in different emotional situations and also their relationships with trainers.20 Halfling et al indicated that portfolios can improve self-awareness of feelings, attitudes, and concerns. Students’ reflections on their consultation skills and the importance of active listening in the patient–doctor relationship were presented.21
In Iran, Latify et al foun thatd critical thinking scores increased in both portfolio and conventional groups after intervention, but progress in critical thinking was lower in the portfolio group.1 Also in Iran, Habibzadeh et al found that students’ understanding using the portfolio method about clinical education was significantly higher than the conventional education method. In other study in Iran,22 Studies by Assadi-Hoveizian et al and Azadi et al in Iran indicated that satisfaction of students using the portfolio method was higher than in the conventional method and recommended it for use as a learning and assessment tool of nursing students in apprenticeship courses.23,24
The portfolio method has some important advantages. It creates a link between theory and the clinical circumstance through reflection on application of theory to clinical circumstance.s Students’ awareness of their strengths, weaknesses, and limitations is increased. It encourages learners to strengthen their modes of learning trend. A sense of responsibility and self-confidence is increased in the learners. With active participation of the educational trainer in the process of evaluation, feedback is presented.25–28
Despite the aforementioned advantages, the use of this method in nursing education is ambiguous, and studies have been incomplete and limited to few aspects of education or evaluation. For example, studies have shown that instructions for constructing the content of the portfolio are not clear and their structure and process still in development at universities and centers of clinical internship.29
The objective of the present study was to evaluate the effectiveness of education with the portfolio method on the clinical competence of nursing students.
The study was conducted on sixth-semester undergraduate nursing students during the internship course in the Nursing and Midwifery School, Ahvaz Jundishapur University of Medical Sciences. Internship is a course/program for bachelor of science nursing students during the 4 years of their academic career in Iran. During the first 3 years of their education, the students take an internship course in relation to other defined educational courses. For the last year (fourth year) of the program, the students spend the whole year in hospitals and medical centers for practical education and to gain different skills in the nursing domains.30
Sample size was calculated based on mean scores for clinical competence and portfolio (n=29). To prevent the reduction of deleted items, the sample was increased to 37 students. To justify the sample size, we used a crossover design, which is conventional in studies with small samples. Power analysis between the means of the two educational methods after intervention was calculated to be 0.87, which is higher than the suggested suitable range (0.80) for power in statistical analysis.
This was a quasiexperimental study to evaluate the association between an intervention (not randomly assigned) and an outcome. The pretest–posttest quasiexperimental crossover design used in this research compared groups and measured changes between pretest and posttest data among all groups.31
After the methods had been explained to them, 34 of the 37 students filled and signed the written consent. Then, students were divided into four groups of eight, nine, seven, and eight without any researcher interference. Two students who had not provided their first portfolio report were excluded from the study and passed their internship through the conventional method. For elimination, external factors, and the previous educational method, the crossover method was used to examine both educational methods in each group (see Table 1 for classification of studied groups).
Table 1 Classification of Groups Studied Using the Crossover Method
The length of the study was two periods of internship for 20 days (each 10 days). At the beginning and end of internship programs, two groups of four students were educated in the conventional trend by the university’s coaches for 10 days and the other two educated by the researchers with the portfolio method for 10 days. Then, in the next 10 days, interns that had been educated with the portfolio method were taught by the college instructor with the conventional method. The groups that had been educated with conventional methodology were educated by the researchers with the portfolio method. Students in the portfolio groups were supervised by the same clinical trainer as the conventional group.
In the conventional group, clinical education was provided by the trainer on the first day of the internship in an introduction. Educational goals and course assignments were explained, and then lecture topics associated with that internship course were split between students to present conferences in classrooms in hospital departments. One hour per day was assigned for conference presentation, and then group discussions were organized to get feedback from the trainer. Furthermore, with the help of a trainer, students performed therapeutic procedures and nursing care for patients in the ward, and finally an evaluation was carried out in conventional form. At the end of the internship course, a clinical competence form was given to the students to assess themselves.
In the portfolio group on the first day of internship, a clinical course lecture plan, and a brochure describing the portfolio method was given to the students. This brochure contained information according to the lecture plans to determine their learning needs and educational and internship courses and effects on the practical skills. Additionally, the portfolio was reviewed every week by the researcher and returned to the students after feedback to address deficiencies.
In this type of portfolio, the learner presents his/her abilities and progress, and gives examples of the tasks performed during a certain period for evaluation by the trainer. This collection needed to include the title, contents, objectives, reasons for choosing each goal and how to achieve that goal, feedback from the trainer, self-evaluation at the end of each goal, valid scientific resources, and finally an overall evaluation of the educational course.26
Moreover, they were asked to write all activities and ways to achieve the aforementioned purposes and collect contents of selected topics related to theory according to the needs and objectives and with emphasis on the evaluation of nursing care through reviewing theoretical content presented in past semesters, as well as scientific literature and Internet resources, and present reports of the portfolio to their coaches, with finalized feedback delivered at the end of the first week of the internship.
At the end of the second week, students wrote a report about practical activities relating to their goals, with help from the trainer to improve defects based on the feedback. The nursing process was reported in the form of three cases of nursing care patient during the nursing. On the last day of the internship, all written material, diagnostic and therapeutic tests, and patient-related therapies were integrated in a booklet that included evaluation at the end of each topic, ultimately evaluating the entire course of the internship, whether the course had achieved its goal, and if not, what the reason for this was. They were asked to delivering a list of references to their coaches for evaluation and scoring. Then, a self-evaluation checklist of clinical competence was filled out by the students at the end of each programs and data analyzed. Evaluation was based on this checklist to compare clinical competence between the conventional and portfolio groups.
To improve the internal and external validity of this study, we assessed the groups at the beginning and end of the internship program to compare results and used crossover statistical methods. Clinical competence was calculated and compared before and after the study in each group. We also used an experienced trainer to deliver both conventional and portfolio education methods.
Demographic data (containing questions about age, sex, semester, the average of the previous semester grades, location of internship, interest in nursing, and possible familiarity with the portfolio method) were collected and a self-assessment questionnaire on clinical competence delivered. This questionnaire was used in Hakimzadeh et al.32 Its content validity had been confirmed by ten professors in the Faculty of Nursing, University of Tarbiat-e-Modarres. Its reliability had been determined with Cronbach’s α of 0.94, which indicates high internal consistency and reliability.
The questionnaire has 31 items in three domains — cognitive (12) affective (ten) and psychomotor (nine) —set up to follow the steps of the nursing process. For score assessment, a Likert scale with five response options was used: very low (0–20%), low (21%–41%), high (41%–60%) and very high (81%–100%). The score of one and five were assigned to very low and very high skills, respectively.32
To teaching the portfolio, the researcher used educational steps based on prior research.2 For evaluation of student portfolios, a checklist developed by Imanipour and Jalili was used.2 For evaluation of the portfolio method, a checklist was provided based on three criteria: portfolio content (eg, knowledge and clinical skills in specialized subjects, ways to improve of behavioral skills, and patient physical and emotional state, social problems, cares, and drug use), portfolio structure (eg, report format), and students’ clinical skills (eg, attendance at clinical and practical workshops and relationships with the medical team and patients).
The data were analyzed using SPSS version 18. Independent t-tests were used to compare average scores before and after the courses. Paired t-tests were used to compare average scores before and after internship. Subjects in the conventional and portfolio groups numbered 34 and 32, respectively.
Demographic variables and baseline characteristics of the students are presented in Table 2.
Calculation of the clinical competence of students and comparison of differences between before and after study using the conventional and portfolio methods indicated that there were significant differences in all three domains: cognitive (P=0.0001), affective and psychomotor (P=0.0001), and clinical competence overall (P=0.0001; Tables 3–6). Scoresbefore and after internship also showed a significant differences in all three domains — cognitive (P=0.002), affective (P=0.0001), and psychomotor (P=0.0001) — and clinical competence overall (P=0.0001; Tables 3–table 6, table 4, table 5). Therefore, implementation of the portfolioprogram and evaluation procedures affected the clinical competenceof the students.
Mean clinical competence scores before and after study using the conventional and portfolio programs showed significant differences in all three domains — cognitive (P=0.0001), affective, and psycho-motor (both P=0.0001) — and clinical competence overall (P=0.0001; Tables 3–6). Differences between the two groups before and after education showed significant differences in all three domains — cognitive (P=0.002), affective (P=0.0001), and psychomotor (P=0.0001) — and clinical competence overall (P=0.0001; Tables 3–6). Therefore, implementation of education and evaluation using the portfolio method had an impact on the clinical competence of the students.
In this study, the students experienced the portfolio and conventional education methods together, and intragroup comparison was performed. The increased clinical competenc scores after using the portfolio method indicates the effect of this type of education in increasing the clinical competence of nursing students in a clinical unit.
Demographic profiles in the two groups were similar and had no effect on our study. On the other hand, the shift between the two groups in any of the previous internship was carried out for the elimination of the effects of the previous intervention.
Implementation of portfolio education and evaluation methods was effective in terms of the cognitive aspect of students’ clinical competence, consistent with,Valizadeh et al, which concluded that portfolio method promoted cognitive learning of students significantly more than the combined approach.13
Habibzadeh et al concluded that portfolio education enhanced students’ understanding of the clinical environment more than conventional education, consistent with our study.22 Latifi et al investigated the effects of portfolio evaluation on critical thinking of nursing students. Critical thinking scores for their conventional group whigher than the portfolio group, inconsistent with our study. However, the large number of students in each group, too many duties, and their exhaustion may may explain the lack of effect found in their portfolio group.1 In our study, the portfolio method had an effect on the affective dimension of clinical competence (P=0.0001), consistent with Buckley et al, showing that this method enhances theoretical power in combination with clinical circumstances. It also improves students’ relationships with the instructor, self-evaluation, and dealing with affective situations.20
Haffling et al investigated students’ reflections on the pilot portfolio program at a Swedish University and concluded that the portfolio had an effect on affective issues, especially self-awareness of emotions, attitudes, and concerns, and increased interview skills and communication between patient and teacher.21
Implementation of the portfolio method in our study was effective in psychomotor aspects of clinical competence (P=0.0001). In a qualitative study using the content-analysis method, Hekmatpou concluded that portfolio learning enhanced the psychomotor aspect, consistent with our study.26
The impact of the portfolio method on all dimensions of clinical competence indicated that it is effective in improving the clinical competence of students during the internship program. Bahraini et al found that the clinical competence of nurses increased using theportfolio method.11 Similarly, Haffling et al successfully introduced the portfolio method in clinical education that causes to incorporate the cognitive, affective and practical domains in the clinical circumstances.21
The limitation of this study was the intervention of the college’s education in the division of students according to the planning with the relevant trainers, which limited the possibility of a researcher’s change that was used to overcome this defect in the student’s cross-choice approach. However, for justifying the sample size, we used a crossover study design, which is conventional in smaller studies, and power analysis was strong scientifically, though the small sample may have influenced the final results.
Portfolio-based education and evaluation can be used at different scientific and academic levels and lectures for nursing students to provide and improve evaluation, assessment, education, and efficiency in nursing domains. With this method, it will possible to monitor nurses for professional development and scientific management. It can also improve clinical competence for nursing students, and we recommended its design and implementation in the nursing educational system.
The results of a new educational approach, changing the model of education from a deterministic model to one constructivism and ownership of learning from teacher to student, can be used as a basis for further research in the domain of teaching methods. The results of this research can also be exploited for later studies that explore the impact of the portfolio methodology.
Self-assessment of clinical competence of nursing students and student feedback on portfolio-based education and evaluation as a combined approach for internship and externship programs indicated that students identified their strengths and weaknesses. As such, they did not contest their evaluation scores. Furthermore, they also worked harder to obtain better results. Since the portfolio method is the application of a student-centered learning model, it enhances students’ appreciation of clinical circumstances and can be used as a beneficial and effective method to optimize the quality and quantity of clinical education and motivate students for self-directed learning.
Ethics and Consent Statement
This study was approved by the ethics committee of Ahvaz Jundishapur University of Medical Sciences (ajums.REC.1392.315). The objectives of the study were explained to the students and informed consent obtained from all of them before distributing the questionnaires.
This study is part of a master’s thesis in nursing at Ahvaz Jundishapur University of Medical Sciences. Hereby, the authors express thanks to Jundishapur University of Medical Sciences for financial support, as well as the Department of Nursing, professors, and students who participated in this project and helped us. Also, our special thanks go to members of the Ahvaz Golestan Hospital Clinical Research Development Unit.
The authors report no conflicts of interest in this work.
1. Latifi M, Shaban M, Nikbakht Nasrabadi A, Mehran A, Parsa Yekta Z. Effect of clinical evaluation with portfolio on critical thinking skills of nursing students. Iran J Med Educ. 2011;11(4):368–381.
2. Imanipour M, Jalili M. Nursing students’ clinical evaluation in students and teachers views. Iran J Nurs Res. 2012;7(25):17–26.
3. Hoseini BL, Jafarnejad F, Mazloum SR, Foroughipour M, Karimi MH. Midwifery students’ satisfaction with logbook as a clinical assessment means in Mashhad university of medical sciences, 2010. Iran J Med Educ. 2012;11(8):933–941.
4. Dehghany Z, Abaszadeh A, Moattari M, Bahreini M. Effective reflection on clinical competency of nursing students in Shiraz university. Mag E-Learning Dist Academy. 2011;2(1):12–18.
5. Pazargadi M, Ashktorab T, Khosravi S. Multi-rater evaluation in the clinical evaluation of nursing students: instructors, experiences and viewpoints. J Qual Res Health Sci. 2012;1(2):102–111.
6. Hemmati Maslakpak M, khalilzadeh H. Nursing students’ perspectives on effective clinical education: A qualitative study. Iran J Med Educ. 2012;11(7):718–727.
7. Khademian Z. Nursing students’ initial clinical experiences. Iran J Nurs Res. 2012;7(26):23–33.
8. Bahreini M, Moattari M, Kaveh M, Ahmadi F. Self-assessment of the clinical competence of nurses in a major educational hospital of Shiraz University of Medical Sciences. J Jahrom Uni Med Sci. 2010;8(1):28–36.
9. Nursing and Midwifery Council (NMC). Requirements for Pre-Registration Nursing Programmes. London: Nursing and Midwifery Council; 2002.
10. Cowan DT, Norman I, Coopamah VP. Competence in nursing practice: A controversial concept – A focused review of literature. Nurs Educ Today. 2005;25(5):355–362. doi:10.1016/j.nedt.2005.03.002
11. Bahreini M, Ahmadi F, Shahamat S, Behzadi S. The Impact of professional portfolio on nurses’ clinical competence. Stride Dev Med Educ. 2012;8(2):107–114.
12. Zhang C, Fan H, Xia J, Guo H, Jiang X, Yan Y. The effects of reflective education on the disposition of critical thinking for nursing students in China: A controlled trial. Asian Nurs Res. 2017;11(3):194–200. doi:10.1016/j.anr.2017.07.002
13. Valizadeh S, Rahmani A, Davoodi A, Aminiaee N. Comparison of the effect of clinical education and assessment using portfolio and blended methods on cognitive learning of nursing students in clinical settings. Iran J Med Educ. 2011;11(3):245–253.
14. Jaffari Golestan N, Vanaki Z, Memarian R. Organizing nursing mentors committee: an effective strategy for improving novice nurses’ clinical competency. Iran J Med Educ. 2008;7(2):237–247.
15. Mardani Hamuleh M, Heidari H, Changiz T. Evaluation of clinical education status from the viewpoints of nursing students. Iran J Med Educ. 2011;10(4):500–511.
16. Arfaie K. Priorities of clinical education evaluation from nursing and midwifery students’ perspective. Iran J Nurs. 2012;25(75):71–77.
17. McCready T. Portfolios and the assessment of competence in nursing: A literature review. Int J Nurs Stud. 2007;44:143–151. doi:10.1016/j.ijnurstu.2006.01.013
18. Pennbrant S, Nunstedt H. The work-integrated learning combined with the portfolio method-A pedagogical strategy and tool in nursing education for developing professional competence. J Nurs Educ Pract. 2017;8:8. doi:10.5430/jnep.v8n2p8
19. Byrne M, Schroeter K, Carter S, Mower J. The professional portfolio: an evidence-based assessment method. J Contin Educ Nurs. 2009;40:545–552. doi:10.3928/00220124-20091119-07
20. Buckley S, Coleman J, Khan K. Best evidence on the educational effects of undergraduate portfolios. Clin Teach. 2010;7(3):187–191. doi:10.1111/j.1743-498X.2010.00364.x
21. Haffling AC, Beckman A, Pahlmblad A, Edgren G. Students’ reflections in a portfolio pilot: highlighting professional issues. Med Teach. 2010;32(12):e532–e40. doi:10.3109/0142159X.2010.509420
22. Habibzade H, Kazemi N, Mhammadpr Y, Khalkhali H. The effect of portfolio educational plan on the perception of nursing students in a clinical educational environment: based on dream model. Nurs Midwifery J. 2013;11:1.
23. Assadi-Hoveizian SH, Shariati AA, Haghighi SH, Latifi SM, Sheini-Jaberi P. Effects of clinical education and evaluation with portfolio method on nursing students’ satisfaction: a clinical trial. J Clin Nurs Midwife. 2014;3(3):70–79.
24. Azadi A, Bastami M, Bastami M. The Effect of clinical learning and assessment of community health nursing apprenticeship using Portfolio method on nursing students’ satisfaction. Iran J Nurs Res. 2017;11:1–6. doi:10.21859/ijnr-11061.
25. Tsai PR, Lee -T-T, Lin H-R, Lee-Hsieh J, Mills ME. Nurses’ perceptions of e-portfolio use for on-the-job education in Taiwan. CIN- Comput Inform Nurs. 2015;33(1):21–27. doi:10.1097/CIN.0000000000000112
26. Hekmatpou D. Effect of Portfolio based evaluation on accuracy of clinical evaluation of nursing students during internship in Arak university of medical sciences, Iran. Stride Dev Med Educ …. 2013;10(1):60–69.
27. Kariman N, Heidari T. The effect of Portfolio’s evaluation on learning and satisfaction of midwifery students. Arak MedUni J. 2010;12(4):81–88.
28. Nafisi ZMN. Improve students’ skills in reading comprehension in English as a foreign language through the portfolio evaluation outlining the conceptual diagrams. J Lang Stud Uni. 2012;2:4.
29. Timmins F, Dunne PJ. An exploration of the current use and benefit of nursing student portfolios. Nurs Educ Today. 2009;29(3):330–341. doi:10.1016/j.nedt.2008.12.010
30. Nikravan Mofrad M. The law and regulation for attendance in internship for nursing domain. Shahid Beheshti Med Sci Univ. 2015;1–103.
31. Levy Y, Ellis T. A guide for Novice researchers on experimental and quasi-experimental studies in information systems research. Interdiscip Inf Sci. 2011;6. doi: 10.28945/1373.
32. Hakimzadeh R, Karamdost N, Memarian R, Ghodrati A, Mirmosavi J. Assessing nursing students’ clinical competency: self-assessment. Q J Nurs Manag. 2012;1(1):17–25.
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]