Parallel to the Situational Judgement Test: is the Educational Performance Measure fair in ranking medical students on the UK Foundation Programme?

Dear editor

We readily appraised the expert opinion by Singagireson et al which explored the fairness of using the Situational Judgement Test (SJT) to rank medical students as part of the UK Foundation Programme.¹ Although we agree with many of the points raised, we feel it is equally important to discuss the role of the Educational Performance Measure (EPM), which has comparable weighting to the SJT in ranking medical students. Therefore, we aim to explore whether the EPM is a fair measure in determining the allocation of foundation-training jobs to newly qualified doctors.

Academic ranking

The UK Foundation Programme aims to reward students who have attained success in academic performance. Its largest component is a student’s intra-university ranking. Subsequently, it can be argued that this creates an environment of healthy competition where hard work is rewarded. This is in stark contrast to the SJT, as the author clearly documents, where students were largely “concerned about the score weighting, ranking format, and subjectivity of the assessment”.² Furthermore, Simon et al concluded that there was no correlation between EPM and SJT scores.³ It follows that a model for segregating students is required, of which the EPM is one such method.  

Singagireson et al correctly raise concerns about inter-university disparities; while the EPM seems to award students who rank higher at their specific university, it may not be truly representative of top-notch achievers nationwide. A study in 2008 analyzing 5,287 doctors’ performances in the Royal College of Physicians postgraduate examinations found a significant difference in results across university cohorts, in which 76% and 67% of graduates from Cambridge and Newcastle universities, respectively, passed their written examination on the first attempt, whereas the figures were 32% for Liverpool, 38% for Dundee, and 37% for Belfast.⁴ This, we fathom, highlights the possible need for the UK to adopt a similar approach to the United States in implementing standardized medical licensing examinations across medical schools.

In addition, the author raises an argument that an “off-day” during the SJT could potentially jeopardize their ranking; this is also applicable to students sitting their university examinations. Moreover, practical assessments such as the Objective Structured Clinical Examinations (OSCEs) are troublesome to rank due to their subjective nature alongside inter-examiner differences. Despite the former ultimately having a more direct impact on a student’s application for foundation jobs, the role of university examinations should not be underestimated. Interestingly, many believe global undergraduate performance should be factored into their EPM score as opposed to the method currently employed, which analyzes performance over two pre-agreed years.

Extracurricular achievement

Further educational accomplishments, such as extra degrees and publications, also contribute to one’s EPM score. Certain universities now offer students an opportunity to study an intercalated BSc (iBSc) and its benefits have been widely documented, including the development of strategic thinking and enhancement of scientific rigour.^5-7 The EPM encourages students to explore and pursue alternative courses, ultimately helping to create more well-rounded doctors in the process. Still, as pointed out by Singagereson et al, there are significant disparities in opportunities to intercalate between universities; while some medical schools offer a compulsory iBSc, others restrict this option to a limited number of students.¹ This variability potentially translates into lower total EPM scores for those who are not presented with the chance to intercalate.

Similarly, the EPM acknowledges students with publications, providing a rationale for students to delve into the arena of evidence-based medicine and academia. On the other hand, numerous students feel that research opportunities are difficult to obtain without adequate connections to senior academic personnel. Moreover, it can be argued that those with an additional degree, such as an iBSc, are provided with a platform to undertake such research projects and as a result, are dually rewarded.

Conclusion

Singagireson et al provide an interesting and thorough analysis with respect to the fairness of the SJT as part of the UK Foundation Programme.¹ However, the fact that most students attain the same score in the SJT combined with the scope to attain extra points through the EPM raises the question of whether the SJT is in fact such a large player in determining student deanery placements.⁸ Overall, we feel the EPM is as significant as the SJT in the UK Foundation Programme, but is equally controversial. As it stands, the EPM utilizes a variety of criteria and through the allocation of points for extracurricular performance, we feel that part of the aforementioned bias is minimized. Therefore, we believe the EPM is an appropriate component of the UK Foundation Programme application.

Disclosure

The authors report no conflicts of interest in this communication.

References

1.		Singagireson S, Ramjeeawon N, Ravindra S, Shah N, Singh B. Is it fair for a junior doctor’s deanery to be largely based on one test: a student’s perspective. Adv Med Educ Pract. 2015;6:499–500.
2.		Sharma N. Medical students’ perceptions of the situational judgement test: a mixed methods study. Br J Hosp Med (Lond). 2015;76(4):234-238.
3.		Simon E, Walsh K, Paterson-Brown F, Cahill D. Does a high ranking mean success in the Situational Judgement Test? Clin Teach. 2015;12(1):42-45.
4.		McManus IC, Elder AT, de Champlain A, Dacre JE, Mollon J, Chis L. Graduates of different UK medical schools show substantial differences in performance on MRCP (UK) Part 1, Part 2 and PACES examinations. BMC Med. 2008;6:5.
5.		Cleland JA, Milne A, Sinclair H, Lee AJ. An intercalated BSc degree is associated with higher marks in subsequent medical school examinations. BMC Med Educ. 2009;9:24.
6.		Mahesan N, Crichton S, Sewell H, Howell S. The effect of an intercalated BSc on subsequent academic performance. BMC Med Educ. 2011;11:76.
7.		McManus IC, Richards P, Winder BC. Intercalated degrees, learning styles, and career preferences: prospective longitudinal study of UK medical students. BMJ. 1999;319(7209):542-546.
8.		The UK Foundation Programme. Foundation Programme 2016 Recruitment Stats and Facts Interim report. UK Foundation Programme; 2016. Available from: http://www.foundationprogramme.nhs.uk/pages/home/keydocs. Accessed October 4, 2016.