What factors affect the quality of medical students’ doctoral theses? A comparative study in affiliated hospitals of a Chinese university

Background

The quality of medical doctoral thesis is highly valued among medical educational institutions around the world. It determines whether medical students are cultivated to be qualified high-level medicine talents. A great many efforts have been put into improving quality of medical theses.^1–3 Since the reform and open-up in 1978, rapid development of society in China has gained growing demands for high-level talent in medicine.⁴ Medical doctoral degrees in China have been divided into two types since 1997: Doctor of Medicine (MD) and Doctor of Philosophy (PhD). For PhD students, the fundamental training goal is to gain the ability to conduct scientific research independently. In practice, quality of thesis and impact factor (IF) of published articles on impacted scientific journals are used for evaluating the accomplishment of the training goal of PhD candidates. While for MD students, the primary training goal is to achieve excellent professional clinical skills. Further, in order to be able to promote the development of the specialty in their future career, MD candidates should also gain the basic ability to perform clinical research, pose scientific questions and utilize appropriate methods. Quality of thesis has also been widely used to evaluate the scientific training of MD candidates. Moreover, writing a doctoral thesis in good quality and passing a blind review by nationalwide experts has become one of the requisites for obtaining doctoral degree not only for PhD candidates, but also for MD candidates. Many medical schools award excellent doctoral theses to encourage the improvement of thesis quality. Questions regarding factors influencing the quality of medical students’ theses have been frequently asked by students, mentors and administrators. However, no answer based on data analysis has been provided. In this study, we aim to explore factors affecting the quality of medical doctoral thesis by a matched comparative study design.

Methods

Materials

Medical doctoral theses from 2012–2016 were available online at Peking University Health Science Library. A total of 50 doctoral theses from affiliated hospitals of Peking University were honored by the university as “excellent theses” (honored theses). Forty-one honored theses with open full texts were downloaded from the online system of the university library. Specialty- and year of submission-matched nonhonored theses with open full texts were randomly chosen in the online library system in a 3:1 nonhonored:honored ratio. For two honored theses, only one matching thesis was downloaded from online library system. Information including name, department, specialty and title of thesis were hidden from two statisticians who performed data collection and analyses. Potential factors affecting thesis quality were determined through an intensive brainstorm.

Analysis

To determine whether the honored theses group has distinct characteristics from nonhonored theses group, indicators for evaluating theses quality were put forward through brainstorming and went for first evaluation by univariate conditional logistic regression. Indicators with P<0.05 and OR >2 were then subjected to a multivariate conditional logistic regression model to select indicators at a significance level of 0.05. STATA 13.1 (STATA Corporation, College Station, TX, USA) was utilized for all statistical analysis. All tests were two-sided and had a significance level of 0.05.

Results

A total of 160 doctoral theses including 41 honored theses and 119 matched nonhonored theses were subjected as study materials. Five domains comprising 17 indicators were put forward by brainstorming.

Results of univariate conditional logistic analyses were summarized in Table 1. There was no statistical difference between the two groups in indicators including number of instructors, word count of review, percentage of last three years references, introduction of sample source, provide sample size estimation and provide ethic approval by the univariate logistic analysis (P>0.05). Among these indicators, both honored and nonhonored groups had similar word count of review (10,600 vs 8,700 words). Nearly half of the candidates put efforts in reading the latest references and providing ethic approval in their theses. Most candidates (honored vs nonhonored group: 97.56% vs 94.96%) have introduced sources of samples, making it clear whether study subjects come from hospitals or communities, what genetic backgrounds of laboratory animals are and what the origins of cell lines are. Less than a third of the candidates utilized appropriate statistical methods for their study, and fewer than 5% of candidates provided sample size estimation in their doctoral theses. The honored group demonstrated statistical significance in writing more words on abstract (P=0.001) and main text (P<0.001), and in citing more references (P<0.001) than the nonhonored group. From the univariate analysis, seven indicators with P<0.05 and OR: >2 were selected for the multivariate logistic regression analysis: degree type, research classification, first author impact factor, scholarship, standard format, provide clear diagnostic criteria and correctly report statistic results (Table 2). Three indicators showed statistical significance in the final multivariate conditional logistic regression model: degree type (OR: 107.56, 95%CI: 1.20–9632.70, P=0.041), first author IF (OR: 1.24, 95%CI: 1.01–1.53, P=0.040) and correct statistic report (OR: 43.18, 95%CI: 1.88–991.61, P=0.019).

Table 1 Comparison of thesis quality indicators between honored and nonhonored group (univariate analysis)

Table 2 Multivariate conditional logistic analysis of indicators affecting quality of medical doctoral theses

Discussion

The present descriptive cross-sectional study was conducted to determine factors influencing thesis quality of medical doctoral candidates in Peking University. Independent factors included degree type, first author IF, and correct statistic report.

Like most universities in China, an open and stepwise recommendation system is adopted by Peking University in the process of selecting excellent medical doctoral theses. Both PhD and MD candidates could be recommended by their mentors, reviewers and schools. However, there are also opinions that it is unfair to compare the two degree types. PhD candidates have spent most of their time in scientific training and have more time to polish their scientific writing, whereas MD candidates have spent most of their time in clinical training and have limited time for research and writing theses. Results of this study suggested that PhD candidates have significant advantage in winning the honor of an excellent doctoral thesis than MD candidates. A previous survey⁵ has demonstrated widespread incompetency in conducting research among Chinese doctors. This situation has aroused demands for clinical doctors to conduct scientific research. However, various factors including shortage of time, knowledge and academic resources contribute to enormous pressure for doctors to do research. Currently, such academic pressure has shifted to MD candidates, making MD candidates facing similar research obstructions with clinical doctors. Comments have been pointed out that it is necessary to avoid the confusion between PhD and MD degrees on training objective and pattern.⁶ Using same criterion to evaluate scientific trainings as well as thesis quality of MD and PhD students may not be appropriate. Theses evaluations for the two degree types should be considered differently. More investigations on training differences between the two degree types should be conducted in the future.

The median first author IF in the nonhonored group was zero which indicates that most candidates have not made their scientific work visible to academic community upon graduation. A study from Finland found that fewer than 25% of medical students published their theses in impacted scientific journals.⁷ Data from a specific field of research shows that 86% of MD theses failed to result in publication.⁸ Most opinions are that the intrinsic value of a doctoral thesis is closely related to the impact factor of published academic articles. When multiple recommended theses are presented, it could be simple and easy to select one or two as excellent theses according to the author’s accumulated impact factors by graduation. However, mentors and students have raised concerns that using first author IF upon graduation may underestimate a good quality thesis, which will result in publication after graduation. Furthermore, using single selecting indicators may cause deviation from the original intention for excellent thesis evaluation. Quality-oriented and comprehensive evaluation system should be studied and put into application.

It seems that one common deficiency in methodological training for medical students is statistics. Earlier studies on German medical doctor candidates showed that more than half of the students need more assistance with statistics.⁹ A measure of attitudes found that medical students were not very active in their study of statistics.¹⁰ The present study shows that the majority of medical doctoral candidates have difficulties in choosing suitable statistical methods and providing sample size estimation. Using correct statistical methods is crucial to give credible answers to study hypotheses. Furthermore, correctly reporting statistical results could significantly improve thesis quality. Medical students need more help from medical statisticians to improve their understanding and application of medical statistics.

The present study demonstrated important factors influencing quality of medical doctoral theses and provided evidence-based suggestions for medical education administrators to consider. There are some limitations of the present study. First, Peking University has been one of the leading universities in China and may represent the top drawer of medical education. However, since China has seen multiple medical education reform,¹¹ cultivation for medical doctoral talent can vary from place to place, which might make the result of this study not applicable to other higher medical education institutions. Second, since the current analyses were based on objective indicators, assessment of subjective attitudes and difficulties in writing and supervising doctoral theses were not reflected in the present study. Last but not least, in the absence of standard evaluation criteria, honored theses are often chosen based on expert judgment. To some extent, factors affecting thesis quality might reflect expert consensus rather than objective standards. Objective evaluation system for medical doctoral theses still needs further research and construction in the future.

Conclusions

Within the limitation of this matched comparative study, it might be concluded that:

(1) PhD candidates might have a distinct advantage in winning the honor of an excellent thesis than MD candidates.

(2) The rewarded theses have a feature of high first author impact factor.

(3) Assistance in statistics could be helpful for medical doctoral students to improve the quality of their doctoral theses.