Suggestions for Improving the Assessment of a Learning Management System Used for Clinical Curriculum Development [Response to Letter]

Dear editor

Thank you for the opportunity to respond to the observations of Oliveira et al We fully agree with their recommendations for further study of higher educational outcomes when implementing and developing the use of a learning management system (LMS) in clinical learning environments. As the authors correctly noted, our study focused on an early phase of an educational design research study cycle,¹ and we primarily used student evaluations as outcome parameters. Importantly, our conclusion that using an LMS appeared to support student learning was not based on a single Likert item, but also on the improvement of our teaching hospital’s overall ranking. This ranking was calculated based on a 30-item evaluation that included questions on perceptions of learning goal achievements, educational support and learning climate, workplace-based assessments, global evaluation items, and narrative student feedback. It was administered through an external evaluation department at our University; therefore, the details were not included in our data.

Furthermore, we would like to emphasize that the development and implementation of an LMS prototype at our teaching hospital was not intended to replace clinical training. In contrast, it was developed to support and scaffold self-regulated learning by novice clinical students while they participated in clinical work for the first time. We agree with Oliveira et al that future studies should carefully examine which clinical learning contents can be effectively taught online or in a blended format, and which cannot. An additionally important aspect is the role of the clinical supervisor and how they facilitate the educational experience by enabling a community of inquiry.²

We commend Oliveira et al for their critical comments with regard to using Likert-item data and its implications for statistical analysis. While we agree that a numerical value of 3.9 or 4.4 does not convey much meaning in and of itself, it can be interpreted as a trend. As researchers in other fields have shown, treating Likert-item data as numerical is acceptable.³ However, we agree that the consequences of choosing one statistical test over another need to be carefully considered depending on the context and the potential implications of the research findings. Moreover, even if acquiescence bias is relevant, it should apply to both cohorts. As our study was designed as an exploratory pilot project, we considered the implications of using Likert-item data in this way as acceptable.

Finally, we would like to thank the authors for their insightful feedback and wish them all the best in their upcoming graduation and professional career, and we encourage their scholarly interest in medical education.