Very Short Answer Questions: The Answer to the Perfect Medical Assessment? [Letter]

Dear editor

We read with great interest the study by Sam et al regarding a novel method of assessment in pathology, Very Short Answer Questions (VSAQs).¹ The reliability shown by this method to discriminate between students and potentially improve learning behaviours were fascinating. We appreciate the authors’ efforts in conducting this study; however, certain aspects need addressing.

Given that the technological aspects of this assessment produced some of the main benefits (such as faster marking), we wonder how widespread it would be adopted. As current and previous medical students at Imperial College London, we are aware that the medical school establishes an ongoing relationship between students and technology, as free iPads are issued to all students.² However, such an integrated familiarity with technology does not exist at other medical schools, such as the University of Leeds. This must be considered in light of the positive feedback attained from students in the authors’ previous study where they determined the validity of VSAQs;³ thus, attempting to generalise the utility of VSAQs may be prone to bias.

Along this vein, the lack of external validation measures to evaluate exam performance calls for a need for research to examine the utility of VSAQs in a multicentre study. However, instituting such a novel assessment at other universities may experience more friction during the implementation stage. A 2011 paper from the University of Leeds, which extensively studies the effectiveness of technology-enhanced learning, discussed the financial costs associated with developing such ventures, stating: “the costs can only be recovered by providers if there is the economy of scale […] or when they are extensively subsidised through grants or advertising.”⁴ Such a change would also call for increased work at the curriculum level, for example, to write tailored VSAQs and issue practice questions for students, and at the systems level, to establish an understanding of the software.

Additionally, it is worthwhile noting that UK junior doctor job allocations are considerably dependent on a student’s academic ranking. Therefore, among medical students who wish to distinguish themselves, VSAQs might encourage recall of the academic minutiae characteristic of pathology. Studies have shown that medical school is already a stressful environment as students are expected to retain a large amount of information within a limited time frame.⁵ Thus, we fear that VSAQs may raise the risk of burnout as the volume of work in the penultimate year, during which they were proposed, is particularly large. If an institution was determined to use an assessment that tests recall, we would encourage the authors to explore employing VSAQs specifically in physiology and clinical scenarios of acutely unwell patients, critical areas to master as junior doctors.

The findings of this study are critical and the implementation of VSAQs may pay dividends in our future clinical practice. Nevertheless, we question the utility of such a novel approach regarding its impact on stress and wonder how it could accurately be assessed in a multicentre study given the barriers to uptake that may probably exist at other institutions.