Medical Students’ Perspectives on an Assessment of Reflective Portfolios [Letter]

Dear editor

We read with great interest the study by Kassab et al¹ which discusses the implementation of an instrument for assessing reflective portfolios in medical school curriculums. As fifth-year UK-based medical students, we have engaged in reflective writing of clinical experiences and would like to outline some of the limitations in the suggested instrument which we note have not been fully discussed.

Firstly, Kassab et al¹ attribute student reflections as the largest single factor for the variance, accounting for 46.6%, and therefore raters were able to effectively use the instrument to determine the calibre of the reflections. However, the majority of the variance comes from other factors, such as the student–rater interaction which accounts for 17.7%. We note that it is unclear whether the reflections were anonymised which, as described by Brennan, may leave room for bias in assessment and may, in part, account for this large variance.² We suggest that anonymised reflections may reduce the variance for a future study and improve the validity of the instrument. Another significant proportion of the variance was due to unexplained sources of error, which accounted for 27.7%. It appears as though the authors have not considered an important facet as noted by Shavelson and Webb, which is the variation in difficulty of the experiences that students have chosen to reflect on.³ A lack of prompt for reflections may have contributed to this.

The authors do not provide students with prompts in order to give them “liberty to reflect on their own personal experiences.”¹ Whilst this does not restrict students, some form of prompt is required in order for students to produce reflections which fulfill the assessment criteria. Therefore, the lack of prompt may have limited some scores due to limited guidance. Furthermore, studies such as that of Renner et al suggest prompts stimulate reflection by guiding the process and encouraging students to evaluate their experiences in greater depth.⁴

The study was restricted to pre-clinical students, with the aims of applying the instrument to a clinical environment. Background for the basis, or circumstances around the reflective pieces would be beneficial, as authors do not state whether reflective pieces were based on personal and familial clinical experiences or problem-based learning experiences. Research by Lutz et al describes the inherent complexity of clinical environments, therefore the described instrument created for pre-clinical students may not be replicable for clinical students.⁵ They also expressed that reflective practice on clinical encounters improves communication and professionalism therefore an instrument aimed towards clinical students may be of more benefit.

We congratulate the authors for developing a complex tool for assessing reflective portfolios, a key element in medical curriculums. However, the instrument may be limited due to lack of anonymised reflections and prompts. Additionally, the proposed instrument may not be replicable for clinical students due to increased complex environments. Nevertheless, we believe the primary focus of such tools should aim to promote in-depth reflections to develop professionalism and competence as opposed to their use purely for assessment purposes.