Do Medical Students in Their Fifth Year of Undergraduate Training Differ in Their Suitability to Become a “Good Doctor” Depending on Their Admission Criteria? A Pilot Study
Received 21 October 2019
Accepted for publication 24 January 2020
Published 7 February 2020 Volume 2020:11 Pages 109—112
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Md Anwarul Azim Majumder
Thomas Kötter, 1, 2 Silvia Isabelle Rose, 1, 2 Annika Waldmann, 2 Jost Steinhäuser 1
1Institute of Family Medicine, University Medical Centre Schleswig-Holstein, Campus Lübeck, Lübeck, Germany; 2Institute of Social Medicine and Epidemiology, University of Lübeck, Lübeck, Germany
Correspondence: Thomas Kötter
Institute of Family Medicine, University Medical Centre Schleswig-Holstein, Campus Lübeck, Ratzeburger Allee 160, Lübeck 23562, Germany
Tel +49 451 3101 8006
Fax +49 451 3101 8004
Email [email protected]
Background: In Germany, the pre-university grade point average (pu-GPA) has to be the main criterion for medical school applicant selection. This is also mandatory in the university-specific selection procedures (Auswahlverfahren der Hochschulen [AdH]). The admission framework has now been reworked following a judgement by the German Federal Constitutional Court. From 2020, more students will be admitted based solely on the pu-GPA and at least two selection criteria independent of the pu-GPA have to be considered in the AdH. However, the question whether an AdH (the core of the AdH at Lübeck Medical School [LMS], Germany, is a 30-mins panel interview led by two faculty members and one student) leads to better doctors as compared to pu-GPA-based selection, remains unanswered.
Objective: To compare students selected based either on their pu-GPA alone (“pu-GPA-students”) or based on the result of the AdH at LMS (“AdH-students”) regarding their suitability to become a good doctor.
Design: We conducted a cross-sectional observational pilot study at LMS. Students were judged regarding their overall suitability to become a good doctor by their supervising general practitioners after a two-week internship in their last year of theoretical medical education. The scores were matched to the selection procedure and compared between the pu-GPA-students and AdH-students.
Results: In all, 79% of the AdH-students were rated as “absolutely suitable” for the medical profession, as compared to 42% of the pu-GPA-students (p = 0.01, odds ratio 5.17, 95% confidence interval = 1.41, 18.99). We did not find any association between gender or age and the suitability rating.
Conclusion: Despite the small sample size, our results indicate that it could be favourable to select medical students not only based on their pu-GPA but also using additional selection criteria.
Keywords: medical education, medical students, students selection, educational assessment
In Germany, there are five applicants per study place at medical schools.1 At the moment, the statutory framework for medical school admission in Germany is in a transitional phase. To date, the pre-university grade point average (pu-GPA) is the main selection criterion, as 20% of the study places are allocated to the applications via the pu-GPA.2 Universities also allocate 60% of their study places via university-specific selection procedures (Auswahlverfahren der Hochschulen [AdH]). However, even with the AdH, the pu-GPA has to be the decisive selection criterion.2 The remaining 20% of the study places are mainly allocated to applicants on a waiting list (current waiting time: 7 years). This framework has recently been judged as unconstitutional by the German Federal Constitutional Court.3 The restructured legal requirements for the selection process include a pu-GPA rate of 30%, no waiting time quota and the consideration of at least two selection criteria, which are independent of the pu-GPA, in the AdH.
However, selecting those applicants most likely to become good students and, more importantly, good doctors is very difficult; there neither a generally accepted definition of a “good doctor”4 nor does a set of characteristics exist that allow for the prediction of who will succeed in medical education. Several studies have shown that, for academic performance (eg, knowledge test scores, grades, and study progress), especially in the preclinical stage of medical school, the pu-GPA is the best predictor.5,6 However, the evidence base for the predictive validity of certain applicant characteristics for developing into a good doctor is small, and methodological issues make conclusive studies difficult. Considering the immense effort for the AdH [the core of the AdH at Lübeck Medical School (LMS), Germany, is a 30-mins panel interview led by two faculty members and one student] and the necessity to select those who are most likely to become good doctors, the question arises whether students selected by this kind of procedure are more likely to become good doctors, as compared to students selected solely on the basis of their pu-GPA.
There is some evidence that selection procedures not solely based on cognitive criteria are efficient in identifying applicants with desirable skills. For instance, knowledge test scores, study progress and professionalism scores during the pre-clinical education of medical students admitted to one Dutch university were found to be better in those students selected via a multifaceted selection procedure as compared to those selected via a lottery.7 In a study at another Dutch university, those students admitted via a selection procedure aiming at non-cognitive characteristics had a higher mean grade for the practical clinical course in year 3 than those admitted via a cognitive selection.8 In a third study, students selected via an elaborate, multistage selection procedure outperformed students who did not pass this selection procedure but were admitted to medical school through a lottery regarding a number of outcomes (eg, cognitive and [inter-]personal skills) during a three-year programme.9 However, the selection procedures in these studies were heterogeneous and not readily comparable to the German framework. Additionally, none of the cited studies compared students in different admission quotas after the third year of undergraduate training.
Therefore, the aim of our study was to compare students selected based either on their pu-GPA alone (“pu-GPA-students”) or based on the result of the AdH of LMS (“AdH-students”) regarding their suitability to become a good doctor (rated in the last year of theoretical studies).
Materials and Methods
We conducted a cross-sectional observational pilot study including medical students in their last year of theoretical medical education at one public German medical school (LMS). Students, who had given written consent to participate in the first session of the general practice course after receiving information on the study, were rated by the supervising general practitioner (GP) at the end of a two-week internship in a general practice regarding their overall suitability to become a good doctor; a 4-point Likert scale from “not at all suitable” to “absolutely suitable” was employed for the rating system. There were no exclusion criteria. This question is part of a questionnaire developed by Knorr et al10 containing 11 items on different aspects, such as communicational and interpersonal skills.
The ratings and selection approach were matched by a data custodian and subsequently anonymized.
Data analyses were conducted in IBM SPSS Statistics for Windows Version 22.0 (IBM Inc., Armonk, New York, United States). We used χ2-tests with an alpha of 0.05. After dichotomising the suitability rating into “absolutely suitable” and “less than absolutely suitable”, the odds ratio (OR) was calculated as a measure for the effect size.
Seventy-three ratings by GPs could be matched to the selection approach (N = 61 AdH-students quota, N = 12 pu-GPA-students). The gender distribution in the two groups did not differ significantly (AdH-students: 66% female; pu-GPA-students: 67% female; χ2 [1, N = 73] = 0.01, p = 1.0). The mean age of the AdH-students was 24.1 years (Standard deviation [SD]: 1.51) as compared to 23.6 years for the pu-GPA-students (SD: 1.24).
The majority (73%) of the students were rated as “absolutely suitable” by their supervising GP. Forty-eight (79%) of the AdH-students were rated as “absolutely suitable” for the medical profession, as compared to five (42%) of the pu-GPA-students (χ2 [1, N = 73] = 6.91, p = 0.01, OR 5.17, 95% confidence interval = 1.41, 18.99). We did not find any statistically significant association between gender or age and the suitability rating.
In our cross-sectional observational pilot study at one German medical school, medical students who had been admitted through the AdH, including panel interviews, were rated significantly better by supervising GPs regarding their suitability to become a good doctor.
Our results are in line with the results of earlier studies on the predictive validity of selection procedures employing non-academic criteria. The studies conducted at different Dutch universities cited above showed consistently, despite heterogeneous selection procedures, that the consideration of non-academic criteria leads to better “patient-relevant” outcomes, such as interpersonal skills.6–9 Knorr et al (2018) examined the suitability of medical students for the medical profession in order to validate their selection procedure based on multiple mini-interviews (MMIs).10 In their study, supervising GPs were asked to rate medical students after a one-week internship in their practices, following the first year of medical education. The MMI score showed a significant correlation with the suitability (assessed by the same question we used) rating.
Due to the small sample size, limiting the possibility of more in-depth analyses, our results have to be interpreted with caution. However, the difference in competence ratings between the students selected via the different approaches is anything but negligible. Another limitation of our study is its single-centred nature. Due to the unique nature of the AdH at LMS, the results cannot be generalised to other medical schools. The global, single-item suitability rating is prone to the subjective views of the raters. However, to our knowledge, this is the first study showing differences between students selected for medical school by different criteria in the later years of medical education, just before their final undergraduate year. We chose the fifth year of undergraduate training for the assessment, because at LMS, students attend a two-week internship in a general practice in this year. The rating by GPs, who come to know the students for these 2 weeks during their consultation hours/home visits and teach them in a 1:1 setting, should, therefore, be comparatively valid, as no other supervisor spends this amount of time with single students. Thus, our results could be a signal and motivation for other universities to employ similar studies for the evaluation of their often resource-intensive AdHs.
Our results indicate that it could be worth the effort to select medical students not only based on their pu-GPA but also using certain criteria in a well-designed selection process. The probative value of our results is clearly limited by the small sample size. Hence, we will resume collecting data in a broader context. Further research (preferably by means of prospective, longitudinal studies of adequate size) on suitable selection criteria for medical education is needed.
AdH, Auswahlverfahren der Hochschulen (university-specific selection procedures); GP, General Practitioner; LMS, Lübeck Medical School; MMI, Multiple Mini-Interview; OR, Odds Ratio; pu-GPA, Pre-university Grade Point Average; SD, Standard Deviation.
The study protocol was approved by the Ethics Committee of the University of Lübeck (file reference 16-143). All participants provided written informed consent.
Data Sharing Statement
The dataset generated during this study is not publicly available due to ethico-legal reasons (lack of consent of the study participants) but will be available from the corresponding author upon reasonable request to all interested researchers.
The authors declare no competing interests.
1. Foundation for Higher Education Admission. Daten der bundesweit zulassungsbeschränktenStudiengänge an Hochschulen [Data on nationwide admission restricted study programmes at universities]. 2018. Available from: https://hochschulstart.de/fileadmin/user_upload/bew_zv_ws19.pdf.
2. Federal Ministry of Justice and Consumer Protection. Verordnung über die zentrale Vergabe von Studienplätzen durch die Stiftung für Hochschulzulassung (VergabeVO Stiftung) [Regulation on the central allocation of study places through the Foundation for Higher Education Admission]. 2019. Available from: https://hochschulstart.de/fileadmin/downloads/gesetze/g03.pdf.
3. Federal Constitutional Court. Urteil des Ersten Senats vom 19. Dezember 2017 [Judgement of the First Senate on 19 Dec 2017]. 2018. Available from: https://www.bundesverfassungsgericht.de/e/ls20171219_1bvl000314.html.
4. Steiner-Hofbauer V, Schrank B, Holzinger A. What is a good doctor? Wien Med Wochenschr. 2018;168(15–16):398–405. doi:10.1007/s10354-017-0597-8
5. Ferguson F, James D, Madeley L. Factors associated with success in medical school: systematic review of the literature. BMJ. 2002;324(7343):952–957. doi:10.1136/bmj.324.7343.952
6. Lucieer SM, Stegers-Jager KM, Rikers RMJP, Themmen APN. Non-cognitive selected students do not outperform lottery-admitted students in the pre-clinical stage of medical school. Advances in Health Sciences Education. 2016;21(1):51–61. doi:10.1007/s10459-015-9610-4
7. de Visser M, Fluit C, Cohen-Schotanus J, Laan R. The effects of a non-cognitive versus cognitive admission procedure within cohorts in one medical school. Adv Health Sci Educ Theory Pract. 2018;23(1):187–200. doi:10.1007/s10459-017-9782-1
8. Schripsema NR, van Trigt AM, Borleffs JC, Cohen-Schotanus J. Selection and study performance: comparing three admission processes within one medical school. Med Educ. 2014;48(12):1201–1210. doi:10.1111/medu.12537
9. Schreurs S, Cleutjens KB, Muijtjens AM, Cleland J, oude Egbrink MG. Selection into medicine: the predictive validity of an outcome-based procedure. BMC Med Educ. 2018;18:214. doi:10.1186/s12909-018-1316-x
10. Knorr M, Schwibbe A, Ehrhardt M, Lackamp J, Zimmermann S, Hampe W. Validity evidence for the Hamburg multiple mini-interview. BMC Med Educ. 2018;18:106. doi:10.1186/s12909-018-1208-0
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]