Assessing DeepSeek-R1 for Clinical Decision Support in Multidisciplinary Laboratory Medicine

Introduction

The application of artificial intelligence (AI) in healthcare is transforming diagnostic methodologies,^1,2 enabling streamlined laboratory processes and more efficient clinical workflows.^2,3 Among AI technologies, large language models (LLMs), such as Generative Pre-trained Transformers (GPT), have attracted significant attention for their ability to perform complex natural language processing tasks with contextual understanding.⁴ These models are being rapidly adapted across multiple domains due to their versatility and linguistic fluency.

In medicine, LLMs are being increasingly explored for their potential to support clinical decision-making, assist in diagnosis, and facilitate information delivery.⁵ ChatGPT, a widely used LLM, has been applied in clinical tasks such as drug development, image recognition, data analysis, medical report enhancement, and literature review.^6–12 Several studies have evaluated its capacity to answer multiple-choice medical questions¹³ and provide explanatory reasoning with accuracy comparable to healthcare professionals.¹⁴ Furthermore, LLMs can help reduce unnecessary clinical visits by offering accessible and understandable medical information,^15–17 thus potentially improving patient engagement and outcomes.^18,19 However, their performance in specialized fields such as laboratory medicine remains underexplored, highlighting the need for focused evaluation.

Despite these advancements, clinical decision-making is often influenced by cognitive bias and complex patient conditions. LLMs can draw on a wide range of medical knowledge and use algorithms to simulate clinicians’ clinical diagnostic and therapeutic thinking, providing users with medical information and diagnostic and therapeutic recommendations.^13,20–22 However, these high-performance LLMs typically come with high computational demands and expensive subscription fees. DeepSeek-R1, a new and affordable LLM, has gained global attention for its ability to perform complex tasks, including logical reasoning, mathematical computations, and code generation while maintaining contextual coherence in multi-turn conversations.^23–25 Moreover, Deepseek’s open-source and free nature significantly reduces deployment costs, making it an attractive option for hospitals and healthcare facilities. Its flexibility enables easy integration into hospital systems, facilitating the broader adoption of advanced diagnostic and therapeutic support, especially in resource-limited settings. Despite these strengths, its capability to support healthcare decision-making, akin to ChatGPT, remains under investigation.

This study aims to evaluate DeepSeek-R1’s performance in clinical laboratory medicine, specifically assessing its ability to analyze case reports, generate diagnostic hypotheses, suggest differential diagnoses, and recommend appropriate diagnostic workups.

Methods

Study Design

We conducted a study to evaluate the diagnostic, differential diagnostic, and next-step examination analysis capabilities of DeepSeek-R1 based on medical case history. The case analysis reports were from the textbook Clinical Laboratory Medicine Case Studies, edited by esteemed experts Tiesheng Zheng and Yan Li. This textbook is primarily used by senior students in medical laboratory technology programs and also serves as a clinical reference for practicing laboratory physicians.

A total of 100 case reports were analyzed. Each case includes a concise summary of medical history, presenting symptoms, physical signs, and auxiliary examination results. Although structured for educational purposes, these cases are derived from anonymized real-world clinical cases, which were professionally reviewed and curated by experienced clinicians to reflect representative diagnostic patterns and reasoning pathways.

In this study, each case was independently submitted to DeepSeek-R1 three separate times using standardized prompts, and the model’s responses were carefully recorded. By referencing the case analysis results, we assessed the accuracy and completeness of the responses provided by DeepSeek-R1.

Study Data

Disease-related data from 100 case reports were entered into DeepSeek-R1 three times independently; three questions were asked: 1. What is the most likely diagnosis for this patient based on the history and physical examination? What is the diagnostic basis? (Diagnosis Hypothesis and Basis, Dx Hypothesis & Basis); 2. What differential diagnoses should be considered? (Differential Diagnoses, DDx); 3. What diagnostic tests should be performed to confirm the diagnosis? (Diagnostic Workup, Dx Workup). The outputs from DeepSeek-R1 were recorded for further analysis to assess the consistency and accuracy of its responses across multiple queries for the same case.

Assessment of Outputs

Three senior clinical laboratory physicians independently evaluated the accuracy and completeness of DeepSeek-R1’s responses. All evaluators are from the Department of Clinical Laboratory Medicine, Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, and each has 5–10 years of clinical diagnostic experience in laboratory medicine, with expertise in hematology, biochemical diagnostics, and molecular/genetic testing.

The Likert scale for evaluating accuracy using a six-point scale can be described as follows: (1) Completely incorrect response; (2) More errors than correct elements; (3) Balance between correct and incorrect elements; (4) More correct elements than errors; (5) Nearly completely correct response; (6) Completely correct response.

The three-point Likert scale for evaluating completeness can be expressed as: (1) Incomplete answer, addressing only some aspects of the question and missing important parts; (2) Sufficient answer covering all necessary aspects of the question; (3) Comprehensive response covering all aspects of the question with additional information or context beyond expectations.

Statistical Methods

Statistical analyses were performed using SPSS (version 25.0; IBM Corp). The accuracy and completeness scores were reported as mean ± standard error (SEM). The percentage of correct answers for each type of question was calculated based on the number of cases where the score was 5 or 6 (accuracy) and 2 or 3 (completeness). The Kruskal–Wallis test was used for omnibus group comparisons of non-normally distributed data. Wilson score confidence intervals were calculated for each group to estimate the accuracy rates with uncertainty quantification. Pearson’s chi-square test was employed to assess statistical significance between different groups. A significance level of P <0.05 was considered statistically significant.

Results

Overview of DeepSeek-R1’s Performance

The accuracy and completeness scores for the 100 cases analyzed are presented in Figure 1, with a detailed list of the scores in Supplementary Table 1. These cases were categorized into seven distinct groups: Organ-specific Pathology & Diagnostics (ORG), Hematologic Evaluation & Management (HEM), Metabolic & Endocrine Testing (MET), Infectious Diseases & Antimicrobial Testing (INF-ANT), Genetic & Obstetric Diagnostics (GEN-OB), Immune-mediated Disorders (IMM) and Oncology Laboratory Diagnostics (ONC). The distribution of cases across these categories was as follows: ORG (18%), HEM (24%), MET (20%), INF-ANT (16%), GEN-OB (8%), IMM (7%) and ONC (7%).

Figure 1 DeepSeek-R1’s performance for 100 cases. (A) Accuracy scores for 100 cases. (B) Completeness scores for 100 cases. Error bars are 1 SEM of the mean.

Performance by Question Type

DeepSeek-R1 was evaluated based on three specific queries for each case: (1) the most likely diagnosis based on the clinical history and examination (Dx Hypothesis & Basis), (2) differential diagnoses to consider (DDx), and (3) recommended diagnostic tests (Dx Workup).

The results revealed significant variation in DeepSeek-R1’s performance depending on the question type. As shown in Figure 2A, DeepSeek-R1 demonstrated significantly higher accuracy for Dx Hypothesis & Basis compared to DDx (P <0.0001) and Dx Workup (P <0.01). Additionally, Dx Workup significantly outperformed DDx in accuracy (P <0.01), highlighting a distinct hierarchy in the model’s performance across diagnostic reasoning tasks.

Figure 2 Comparison of Mean Scores in question type. (A) Comparisons of the accuracy performance of DeepSeek-R1 in question type. (B) Comparisons of the completeness performance of DeepSeek-R1 in question type. Abbreviations: aDx Hypothesis & Basis, Diagnosis Hypothesis and Basis; DDx, Differential Diagnoses; Dx Workup: Diagnostic Workup. Notes: bP-value calculated with Kruskal–Wallis test, ns = not significant (P ≥.05); **P <0.01; ***P <0.001, ****P <0.0001.

Regarding completeness, as seen in Figure 2B, a similar trend was observed: Dx Hypothesis & Basis achieved the highest completeness, significantly outperforming DDx (P <0.0001) and Dx Workup (P <0.001). However, there was no statistically significant difference in completeness between DDx and Dx Workup (P ≥.05).

Moreover, we calculated the overall accuracy and completeness of DeepSeek-R1’s outputs in all cases. We found that, on average, across all cases, DeepSeek-R1 achieved an average accuracy of 72.9% (95% CI [69.9%, 75.7%]) and an average completeness of 73.4% (95% CI [70.5%, 76.2%]). In addition, we calculated the percentage of correct answers for each of the three question types, covering both accuracy and completeness. As shown in Table 1 and Table 2, there was a statistically significant variation in DeepSeek-R1’s performance across the question categories, with P <0.001 for both accuracy and completeness. Among the question categories, Dx Hypothesis & Basis yielded the highest performance, achieving an accuracy of 85.7% and completeness of 87.7%. In contrast, DDx (Differential Diagnoses) exhibited the lowest performance, with only 55.0% accuracy and 55.3% completeness.

Table 1 Comparison of Accuracy in Question Type

Table 2 Comparison of Completeness in Question Type

Performance by Disease Category

The case analyses (n = 100) were categorized into seven distinct disease categories, allowing for an evaluation of DeepSeek-R1’s performance not only at the case level but also at the disease category level. Figures 3A and B present the trends in the accuracy and completeness of DeepSeek-R1’s responses across various disease types.

Figure 3 Comparison of Mean Scores in disease type. (A) Comparisons of the average accuracy performance of DeepSeek-R1 in disease type.(B) Comparisons of the average completeness performance of DeepSeek-R1 in disease type. Abbreviations: aHEM, Hematologic Evaluation & Management; ORG, Organ-specific Pathology & Diagnostics; MET, Metabolic & Endocrine Testing; IMM, Immune-mediated Disorders; INF-ANT, Infectious Diseases & Antimicrobial Testing; GEN-OB, Genetic & Obstetric Diagnostics; ONC, Oncology Laboratory Diagnostics. Note: b P-value calculated with Kruskal–Wallis test, *P <0.05, ***P <0.001.

Across all disease categories, as shown in Figure 3A, DeepSeek-R1 achieved the highest accuracy scores in the GEN-OB category and the lowest in HEM, while there were no statistically significant differences between the groups (P >0.05). In contrast, Figure 3B reveals that completeness scores varied significantly across disease types. GEN-OB demonstrated significantly higher completeness than several other disease categories, particularly ORG and MET (P<0.001).

Furthermore, as summarized in Tables 3 and 4, statistical analysis confirmed significant differences in the accuracy and completeness rates across the disease categories (χ²=36.524, P<0.001 for accuracy and χ²=19.693, P =0.003 for completeness). Across all categories, DeepSeek-R1 achieved correctness rates exceeding 60%, with GEN-OB attaining the highest accuracy (93.1%) and completeness (86.1%) rates, as detailed in Tables 3 and 4.

Table 3 Comparison of Accuracy in Disease Type

Table 4 Comparison of Completeness in Disease Type

Discussion

Principal Results

Similar predictive models in decision support have been explored in other clinical domains, such as advanced systems for heart disease detection²⁶ and AI-based diagnostic frameworks for diabetic retinopathy,²⁷ demonstrating the growing potential of LLMs and multimodal data integration in medical diagnostics.

This study demonstrates that DeepSeek-R1, an advanced large language model (LLM), holds considerable potential as a decision-support tool in clinical laboratory medicine. With an overall accuracy of 72.9% (95% CI [69.9%, 75.7%]) and completeness of 73.4% (95% CI [70.5%, 76.2%]), DeepSeek-R1 performs reasonably well in generating diagnostic hypotheses and recommending appropriate diagnostic workups. Notably, DeepSeek-R1 showed the highest accuracy and completeness in generating diagnostic hypotheses, highlighting its potential utility in providing initial diagnoses based on patient history and laboratory findings. This aligns with previous findings where LLMs have been recognized for their strengths in diagnostic reasoning and knowledge retrieval.²⁸

Regarding the diagnostic output, DeepSeek-R1 can formulate personalized diagnoses for individual cases, utilizing a highly condensed clinical history. For instance, in a case involving a patient with a history of total gastrectomy for gastric cancer, along with laboratory results indicating MCV 115 fL, MCH 37.4 pg, pancytopenia (low WBC/PLT), and an elevated reticulocyte percentage (10.42%), which suggests ineffective erythropoiesis, DeepSeek-R1 accurately identified megaloblastic anemia. However, in terms of differential diagnosis, the model exhibited minor biases. For example, in the case of suspected myeloma, clinicians would expect a differential diagnosis that includes plasma cell tumors and plasma cell leukemia. In contrast, DeepSeek-R1 may suggest conditions like myelodysplastic syndromes and metastatic osteomas. This highlights the need for clinical examiners to exercise caution and verify the veracity of information provided by DeepSeek-R1, as it has occasionally yielded unreliable or inaccurate outputs.²⁹

At the diagnostic workup level, DeepSeek-R1 performed well in recommending appropriate tests for most conditions. In most cases, the model prescribed more specific tests based on the diagnosis, which would help further refine the diagnosis or determine the type of condition. However, there were instances where DeepSeek-R1 recommended a broader array of tests, typically due to insufficient detail in the case analysis provided, which suggests that when using DeepSeek-R1 for diagnostic workup suggestions, it is crucial to provide a thorough and accurate medical history to allow for more personalized test recommendations.

Summary

In summary, DeepSeek-R1 demonstrates the capability to generate diagnostic hypotheses based on patient symptoms, history, and laboratory findings. However, its ability to personalize diagnoses, while promising, still requires refinement. The cases analyzed in this study primarily represent classic and commonly encountered diseases, which are widely used in clinical laboratory teaching to support pattern recognition training. These cases—though structured—are based on anonymized real-world clinical scenarios, as documented in authoritative textbooks. They are typically designed to provide students with a clear understanding of specific disease patterns, facilitating the development of a pattern recognition framework for differential diagnosis. From this perspective, DeepSeek-R1’s performance in pattern recognition and diagnostic workup is commendable, as it has access to an extensive knowledge base, thereby minimizing diagnostic omissions.

Limitations and Challenges

DeepSeek-R1 has the potential for medical advances but has potential limitations. The accuracy of the text generated depends on the training of the model, which may lead to misinformation or misleading interpretations, although this can be corrected by training with multiple dialogue training, which also requires some discernment on the part of the user.^30,31 Furthermore, DeepSeek’s training data may not fully encompass the latest medical advancements, as the current cut-off date for DeepSeek’s progress is October 2023. Like other LLMs, such as ChatGPT, DeepSeek is susceptible to hallucinations, omissions, and incorrect responses, which may lead to contradictory or misleading conclusions. These challenges underscore the importance of verifying the generated content before clinical application. Ethical considerations, such as data privacy and security, are paramount in the broader context of LLM usage for scientific research and clinical decision-making.^32,33 Anonymizing patient data to train LLMs is essential to comply with privacy regulations and protect patient confidentiality.³⁴ Moreover, researchers and clinicians must ensure that biomedical data is securely collected, stored, and utilized.

From a methodological perspective, this study has several limitations. First, the dataset comprised 100 cases derived from structured, textbook-based clinical case materials used in medical laboratory education in China. Although these cases are based on anonymized real-world scenarios, their curated nature may not fully capture the complexity and heterogeneity of actual clinical practice. Second, all three physician evaluators were from the same institution, which may introduce institutional bias in the assessment of DeepSeek-R1’s outputs. This limitation has been acknowledged, and future studies should include a larger and more diverse panel of evaluators from multiple institutions and specialties to improve objectivity and enable inter-rater reliability analysis. Lastly, this study did not include a direct comparison with other state-of-the-art large language models, such as GPT-4o. Future work should incorporate comparative benchmarking and real-world clinical data to better contextualize DeepSeek-R1’s performance within the broader landscape of AI-driven diagnostic tool.

As LLMs continue to evolve, they may soon outperform human reasoning in certain areas, as evidenced by recent studies suggesting that LLMs can achieve superhuman performance on human tests of reasoning, as discussed in The Lancet.³⁵ However, it is critical to remember that clinical decision-making requires comprehensive judgment, which can only be exercised by trained healthcare professionals. Thus, LLMs should be viewed as tools to support, rather than replace, clinical practice.

The accuracy and reliability of the content generated by LLMs are vital, as inaccuracies could have serious consequences, particularly for patients or trainees without sufficient medical expertise.³⁶ Therefore, researchers must rigorously validate the safety and efficacy of LLMs before their application in clinical diagnosis. This study demonstrates that DeepSeek-R1, in its current iteration, is a promising tool for clinical decision-making, although it requires further refinement to fully realize its potential.

Conclusions

This study demonstrates that DeepSeek-R1 has the potential to serve as an effective tool for clinical decision support in laboratory medicine. While its performance in generating initial diagnoses and suggesting diagnostic workups was promising, there are limitations in its ability to personalize differential diagnoses and handle specific clinical nuances. Beyond expanding training data with more diverse and recent case distributions, future development could benefit from incorporating structured medical ontologies—such as SNOMED CT and UMLS—to enhance semantic comprehension and domain-specific accuracy. Additionally, introducing physician-in-the-loop training mechanisms—such as feedback-guided fine-tuning—may help the model better align with expert decision-making processes.

While DeepSeek-R1 should not be viewed as a replacement for human clinical judgment, it may serve as a valuable adjunct, especially in resource-limited settings. Further comparative studies against established models like GPT-4o and real-world validation trials will be essential to define its role in clinical workflows and ensure safe, effective integration into healthcare systems.