The Application of ChatGPT in Medicine: A Scoping Review and Bibliometric Analysis

Introduction

ChatGPT (Chat Generative Pre-trained Transformer), an advanced AI-driven language model, has been developed by OpenAI. It is grounded in the Transformer neural network architecture, particularly the GPT-3.5 variant. This model undergoes extensive pre-training, assimilating statistical laws and discernible patterns from vast datasets. As a result, it exhibits the capability to autonomously generate responses to queries, participate in interactive dialogues with users, and compose various forms of text, including emails and academic papers.¹ Following its public release on November 30, 2022, ChatGPT has rapidly garnered immense popularity, evidenced by millions of user registrations in a brief timeframe and exceeding 100 million active users within two months. This growth trajectory establishes ChatGPT as the fastest-growing consumer application in history, underscoring its exceptional appeal.² Mohammad Fraiwan’s survey elucidates ChatGPT’s extensive applications across diverse domains such as automation technology, computer science, journalism, ethics, and medicine. Concurrently, comparable language models, notably Google’s Bard and Meta’s LLaMA, have emerged, seeking to establish their presence in the market. The proliferating adoption of such models, including ChatGPT, is poised to exert a profound influence across various sectors. Consequently, ChatGPT has captivated global scholarly interest, stimulating discourse on its current utility and prospective research trajectories.

ChatGPT is emerging as a transformative force in various sectors, with a notable impact in the realm of medicine. The burgeoning corpus of research on ChatGPT’s medical applications is expanding. Survey findings reveal that current studies in this area predominantly concentrate on (1) Medical Education, such as assessing ChatGPT’s efficacy in medical examinations,^3,4 facilitating case-based learning,⁵ and enhancing communication skills;⁶ (2) Clinical practice, including optimizing clinical decisions,⁷ generating reports⁸ and providing personalized guidance to patients.⁹ (3) Scientific Research, For example, helping researchers to collect and analyze medical literature,¹⁰ writing abstracts¹¹ and exploring the accuracy of information generated by ChatGPT. Despite the considerable advantages and substantial contributions of ChatGPT to medical advancements, concerns linger regarding the reliability of information sources, ethical implications in medicine, and the potential for academic misconduct.

The majority of current ChatGPT literature in the medical field consists of preprints, reviews, or letters to the editor that cover various aspects of medicine. However, it is unclear what these researches specifically examine. Current reviews on ChatGPT in the medical field are relatively homogeneous. Blanco-Gonzalez’s review described the advantages of ChatGPT in drug discovery, while Aydın and Karaarslan’s review focused on its benefits in knowledge compilation and representation.¹² Malik Sallam¹³ also reviewed the use of ChatGPT in medicine, analyzing the advantages and disadvantages of its use in three areas: medical education, research and practice, but included a high proportion of preprints and lacked analysis of the external characteristics of the literature. Daniel Gödded¹⁴ used SWOT to review ChatGPT in the medical literature, which analyzed its application advantages and limitations from the perspective of medicine as a whole, but the search database was limited to PubMed and to the theoretical level of medicine, without further analysis of the use of ChatGPT in clinical practice. Therefore, the current reviews were limited to specific fields; however, there is no review that explores the current state of ChatGPT research in medicine from a global perspective. In addition, as an emerging technology, research on ChatGPT was scattered and fragmented without a clear theoretical structure. Therefore, it is important for researchers to synthesize the existing literature to gain a comprehensive understanding of the current state of ChatGPT research and future trends.

A scoping review is an established methodology for exploring the design and execution of research within a specific thematic area.¹⁵ This approach is instrumental in disseminating knowledge by delineating the extent, characteristics, and lacunae within existing literature, whilst ensuring the robustness and rigorous quality of the research.¹⁶ Consequently, it is extensively utilized to synthesize evidence pertaining to a given subject, especially in the context of burgeoning fields or in assessing the research trajectory of a particular topic.¹⁷ In light of the aforementioned rationale and our research objectives, a scoping review emerges as the most suitable methodology for our study. Bibliometrics, a method of quantitative analysis employing mathematical and statistical techniques, is utilized to scrutinize various facets of literature, encompassing the volume of publications, authorship, affiliations, publication year, and key terms.¹⁸ The primary aim of this approach is to elucidate the current research landscape and discern emerging trends within a study domain. Of late, ChatGPT has piqued significant interest in the medical sector, leading to an uptick in related research endeavors. Nonetheless, there exists a notable deficiency in comprehensive reports delineating the current state and prospective developments of ChatGPT research in this realm. To bridge this gap, we have undertaken bibliometric analysis, offering insights that could contribute to the ongoing evolution of this field.

Methods

The scoping review used an approach based on the 2005 Arksey and O’Malley¹⁹ framework. The review was conducted in two steps: a scoping review and bibliometric analysis. First, the framework was adopted for a scoping review. Then, a bibliometric analysis was conducted to analyze the key topic domains, and future study trends using retrieved scoping review results. No quality evaluation was carried out because the aim was to broaden the scope of included literature, consistent with the overall objectives and methodology of scope evaluation, and critical evaluation was not a necessary part of the scope evaluation framework. At the same time, in order to more accurately reflect the research trends and applications of ChatGPT in related professional fields. Priority was therefore given to databases focused on medicine, health, engineering, and computer science.

Step1 Scoping Review

Stage1 Identifying the Research Question

The overall research objective of this study was to draw up the current status and trends of ChatGPT research in the medical field.

To accomplish the purpose of this study, the following research questions were identified:

1. What are the key characteristics of publications, such as country and type of publications on the application of ChatGPT in medicine?
2. Regarding the use of ChatGPT in medicine, the purpose of these researches in the literature?
3. To which secondary medical disciplines does the existing literature on ChatGPT predominantly pertain?
4. What are the advantages and disadvantages of using ChatGPT in medicine?

Stage2 Identifying Relevant Studies

Six electronic databases, including PubMed, CINAHL, Embase, Web of Science, IEEE Xplore, and ACM Digital Library were systematically searched from November 30, 2022, to August 16, 2023. The search strategies combined the subject terms and free words, consisting of terms of ChatGPT, medicine, and nursing. The search strategy of each database is shown in Supplementary Material S1. Relevant references of articles and reviews included in this review were manually searched to ensure that all relevant primary studies were contained.

Stage3 Study Selection

The review considered the studies in line with the following inclusion and exclusion criteria. Inclusion criteria: (a) Studies published in English; (b) literature publication year from November 30, 2022, to August 16, 2023; (c) Studies relevant to ChatGPT or aiming to apply it in the medicine field; (d) All peer-reviewed studies, with no restrictions on the type of literature (eg, papers, conference abstracts).

Exclusion criteria: (a) Preprints; (b) Studies involving animals; (c) Literature without full content.

Stage4 Charting the Data

Data were systematically collated from each paper utilizing a structured Microsoft Excel data-extraction sheet. Subsequently, formal data extraction was meticulously conducted by two researchers. The primary information extracted encompassed various dimensions: (a) characteristics of the articles, including author, country of origin, and journal name; (b) types of studies, comprising editorials, cross-sectional, longitudinal, qualitative, experimental studies, reviews, among others; (c) central theme of the articles, such as clinical practice, medical education, or scientific research; (d) secondary disciplines employed in the studies; and (e) advantages and disadvantages noted.

Stage5 Collating, Summarizing, and Reporting the Results

Frequencies and percentages were utilized to describe key characteristics of included publications.

We performed qualitative analyses of the results of the scoping review. The qualitative analysis, for its part, consisted of using content analysis to categorize the study objectives, to group them by category.²⁰

Step2 Bibliometric Analysis

VOSviewer version 1.6.19 was performed to draw a network of keywords co-occurrences, and a network of co-authors. In these visualization images, researchers or keywords are depicted as dots, the size of each dot corresponds to the frequency of co-occurrences, and the connecting lines between nodes indicate interrelationships. The thicker the lines, the more connections. Additionally, NVivo version 12 played a pivotal role in analyzing the term frequency within the titles of the included literature, thereby enriching our understanding of the research topic.

Results

A total of 3426 articles were retrieved from the six databases and 1328 duplicates were removed. A total of 247 articles were screened after careful selection of titles and abstracts, as well as a thorough evaluation of the full text. This included one article that was found in a separate search. Detailed information on these included articles can be found in Supplementary Material S2. Figure 1 presents the process of the article screening and eligibility.

Figure 1 PRISMA flowchart.

Key Characteristics and Bibliometric Properties of the Included Literature

Table 1 presents the characteristics of publications. Of the 247 publications, The United States (n = 77, 31.17%) contributed the most articles, followed by China (n = 24, 9.71%), India (n = 16, 6.48%), and the United Kingdom (n = 13, 5.26%). Most publications (n = 113, 45.75%) pertained to clinical practice, followed by medical education (n = 67, 27.16%) and scientific research (n = 67, 27.13%). Publications that were difficult to classify were “other” (n =27, 10.93%). About 41.30% of the retrieved documents were editorial/ letter to editor (n=107), whereas 11.74% were experimental studies and 7.69% employed review methods. The top three journals were ANNALS OF BIOMEDICAL ENGINEERING, CUREUS, and AESTHETIC SURGERY JOURNAL, accounting for 21.25% of the total.

Table 1 Summary of Key Characteristics of Included Publications

Network of Researchers and Co-Authors

Of 247 articles, 948 authors were included, ranging from one author to 5 authors, According to Price’s law,²¹ the core author’s publication volume Mp = 0.749 × (NPmax)1/2, the calculation Mp ≈ 1.67, that was, the author who published more than 2 papers was the core author, and there were a total of 64 authors, Figure 2C and 2D presents the network of co-authors with a large set of connected authors consisting of 36 items. In this network, a total of 3 clusters occurred (cluster 1 = 13 authors, cluster 2 = 12 authors, and cluster 3 = 11 authors). These three clusters generated co-authorship advantages dominated by Abouammoh, Noura A, Al-Eyadhy, Ayman, Abdulmajeed, and Naif.

Figure 2 Authors and keywords networks analysis using the density visualization module of VOSviewer software. (A and B), networks of keywords; (C and D), network of authors.

Network of Keywords

In this study, a total of 610 keywords were identified, with “artificial intelligence”, “ChatGPT”, “chatbot”, and “medical education” emerging as the most frequently occurring terms (Figure 2A and 2B). By setting a threshold of at least three occurrences, 597 keywords were selected for constructing the keyword network. This network was comprised of seven distinct clusters (as depicted in Figure 2A and 2B): Cluster 1 (in red) and Cluster 2 (in green) predominantly focused on the application of ChatGPT in medicine. Cluster 3 (in blue) concentrated on clinical practice, medical education, scientific research, and the role of OpenAI in medicine. Cluster 4 (in yellow) was mainly related to secondary disciplines within medicine, while Cluster 5 (in purple) focused on applications in medical learning and neurosurgery. Cluster 6 (in cyan) centered on urology applications, and Cluster 7 (in orange) addressed ethical considerations, privacy, and security concerns.

Secondary Disciplines

Out of the 247 publications, 148 mentioned specific disciplines. According to Figures 3 and 4, the top 5 disciplines in order were Radiology (n=12), Oncology (n=9), Surgical (n=9), Orthopedics (n=8), Dentistry (n=7), Dermatology (n=7), Nursing (n=7), Plastic Surgery (n=6), and Urology (n=6). The Figure 4 illustrates the initial exploration of ChatGPT by various disciplines, particularly the disciplinary studies involving image analysis.

Figure 3 Word cloud analysis of secondary discipline.

Figure 4 Specific number of secondary disciplines.

The Application Fields of These Studies

The literature included in this review were original research, review articles, case reports and letters to editors. According to Table 1 and Figure 2A and 2B, it can be seen that ChatGPT is mainly involved in three aspects of the medical field: clinical practice, medical education, and scientific research, such as assisting in diagnosis, providing expertise, and writing abstracts.

ChatGPT in the Clinical Practice

Clinical practice focuses on the following four areas, (1) Clinical decision making: (a) Generating disease differential diagnosis. (b) Optimizing the clinical decision-making process. (2) Disease Information and Medical Consultation: (a) Providing reliable disease information, including drug use analysis, personalized drug recommendations. (b) Identify disease problems and treatment options. (c) Provide comprehensive and reliable medical information. (d) Understand performance assessment in terms of clinical problems and drug-related advice. (e) Optimize radiation dose for age to enhance safety. (3) Medical Documentation and Reporting: (a) Generate and streamline radiology reports. (b) Assist in selecting imaging tests to enhance report quality. (c) Assist in writing case reports. (4) Clinical Discussion and Predictive Modelling: (a) Summarize core elements of clinical discussion. (b) Solve patient problems. (c) Provide predictive modeling of deaths.

ChatGPT in Medical Education

Current research has assessed the applicability of ChatGPT in medical education and has divided its use into two broad segments: assessing its performance in various medical examinations and its ability to provide information support to students and healthcare professionals. Increasingly, scholars are recognizing ChatGPT as a tool for improving the effectiveness, appeal and personalization of medical education systems. This includes its potential role in curriculum development, innovation in teaching methods, and the creation of new assessment and evaluation frameworks. In addition, ChatGPT provides opportunities to enhance students’ knowledge, competencies and capabilities. For example, it can simulate actual differential diagnoses, provide comprehensive and streamlined learning resources for designing lesson plans, act as a virtual patient or professor for interactive clinical case studies, help students review and generate practice questions, and create a coherent framework for integrating existing and new knowledge based on the curriculum. Thus, these findings suggest that the field of medical education is looking forward to the use of ChatGPT.

ChatGPT in Scientific Research

ChatGPT has been active in academia and in the field of manuscript writing due to its unique “learning” capabilities and has been explored as a tool for generating scholarly content, both in scientific research and in scholarly creation (eg, collating and proofreading materials and forming first drafts). Existing research has mainly focused on the use of ChatGPT for aggregating and synthesizing complex datasets, such as generating concise abstracts, creating systematic evaluations, and essay writing. Secondly, ChatGPT also helps researchers discover new connections and hypotheses in the data and drive scientific innovation. Although ChatGPT has the potential to increase efficiency and academic creativity in the research field, and a small amount of literature has been produced through ChatGPT-based abstracts, articles, titles, and references. However, academic institutions and researchers continue to express concerns about it.

The Pros and Cons of ChatGPT

The application of ChatGPT in the medical field presents several advantages: (1) Large Data Processing: ChatGPT’s capability to process vast amounts of data results in more detailed and realistic text generation. (2) Pre-Training Generation: It can create text from extensive datasets, tailored for specific tasks, thereby enhancing text quality. (3) Complex Task Handling: ChatGPT is adept at language translation, answering queries, and summarizing text. (4) Context-Based Text Creation: It produces responses grounded in the provided input knowledge. (5) Real-Time Processing: Offering 24/7 online service availability. (6) Efficiency Improvement: The pre-training aspect facilitates swift responses. These benefits are integral to its practical applications in healthcare. Conversely, an investigation into the ethical dimensions uncovers certain drawbacks. As Wang²² elucidated, several ethical issues were associated with ChatGPT’s usage in healthcare. To further this understanding, we synthesizethe perspectives and attitudes presented in the literature. Out of 247 documents, 149 expressed views on ethics, comprising a total of 373 items. The breakdown of these viewpoints is as follows (refer to Figure 5): Legal ethics (63 items), Humanistic ethics (13 items), Algorithmic ethics (113 points), Information ethics (81 items), Uncategorized biases (36 items), General ethical concerns (54 items), and Miscellaneous issues like misuse, energy consumption, and hallucinations (16 items). Legal ethics issues stem from patient data collection and potential privacy breaches. Humanistic ethics emphasize honesty and the physician-patient relationship. Algorithmic ethics cover concerns such as bias, transparency deficiencies, inappropriate data use, and excessive reliance on data without human intervention. Information ethics involve data bias, inaccuracy, and knowledge limitations. Figure 5 indicates that algorithmic and information ethics are predominant concerns, with the most significant being the risk of incorrect data leading to unreliable information or distrust in ChatGPT-generated content due to its inherent limitations.

Figure 5 Number of ethical categories.

Discussion

In this review, we reviewed published articles (from 30 November 2022 to 16 August 2023) that used ChatGPT within the medicine. Through an extensive literature search, we obtained a large number of research results revealing the exploration of ChatGPT by medical scholars worldwide. Analysis of characteristics, key topic areas using scope review nested bibliometric analysis revealed that current research on ChatGPT in medicine focuses on three main areas: clinical practice, medical education, and scientific research, in addition to outlining the pros and cons, and to the best of our knowledge, this is the first study to utilize such an approach.

It is perhaps not a coincidence that existing research has focused on three areas: clinical practice, education, and scientific research. The integration of advanced science and technology is not only in keeping with the times but can also go a long way towards solving existing problems in the relevant fields. In clinical practice, according to the World Health Statistics Report 2023 released by WHO,²³ the global health workforce is still in a state of shortage and suffers from unequal distribution, inefficiency, lack of support and protection, and shortages at the national and subnational levels. The rational application of ChatGPT may be an effective approach. Clinical application studies currently exist in various processes of clinical work, such as diagnosis,^24,25 decision-making instruments,²⁶ and generating reports,²⁷ and some of these studies have shown that ChatGPT is effective in improving the efficiency of nursing care,^28,29 assisting in diagnosis,³⁰ and so on, but it is important to note that these studies have not been practically deployed in the clinical setting and practice.

In the field of contemporary medical education, the use of advanced technologies such as ChatGPT is gradually changing traditional teaching and learning methods. It is important to note that the audience for education is not limited to students, but also to educators themselves. For students, ChatGPT can serve as a powerful learning tool, providing personalized learning assistance,³¹ and instant feedback to deepen understanding of complex medical concepts. At the same time, the technology simulates clinical situations and helps students practice diagnostic and therapeutic skills in a safe environment,³² which is critical to improving their clinical decision-making and hands-on skills. For educators, the use of technology products such as ChatGPT can greatly improve the efficiency and quality of instruction.³³ Teachers can use these tools to access the latest medical research, update course content, and even use these technologies for classroom interactions and student assessments to better accommodate students’ learning needs and styles. In addition, ChatGPT can also be used as an educational research aid to help teachers analyze student learning data to optimize teaching methods and strategies.

In the field of scientific research, especially medical research, human original thinking, critical analysis and the pursuit of scientific rigor together form the cornerstone of authoritative scientific research. These elements not only drive progress in the field of medicine, but also form the unique “evidence base” of the medical community. Against this backdrop, it is important to weigh the advantages and limitations of introducing AI technologies such as ChatGPT into research activities to ensure that the quality and authority of research are not compromised. While ChatGPT has demonstrated significant advantages in processing big data, accelerating literature search, and assisting in experimental design,³⁴ it still lacks human capabilities in innovative thinking and in-depth critical analyses. AI’s analyses and recommendations may rely to some extent on existing data and prior knowledge, which may limit the possibilities for innovation.

However, benefits are often accompanied by risks. ChatGPT, surpassing basic chatbots in performance metrics, has been tested across diverse knowledge domains. Its versatility stems from deep learning techniques, including the Transformer architecture and pre-training, enabling applications in Q&A, text summarization, language translation, dialogue generation, text completion, and content auditing. Concerns about data bias and academic integrity arise due to opaque training data sources and processes. Some literature suggests that ChatGPT-generated references may be fictitious.^35–38 Currently, the medical community’s views on ChatGPT are mixed, with criticisms mainly focused on tasks requiring creativity or highly customized responses. For example, Molligoda Arachchige³⁹ points out that ChatGPT cannot quite capture complex nuanced details like an expert, and that the limitations of high-quality data will make it difficult to provide the right kind of personalized service. The absence of professional instructions for ChatGPT results in varied user experiences and outcomes. Despite high expectations, the acquisition and application of knowledge by ChatGPT are distinct processes. The medical field’s specificity and privacy issues necessitate focusing on ethical considerations like patient privacy³⁹ and doctor-patient relationships in current literature.

Users expect seamless interactions, yet ChatGPT’s limitations may not fulfill these expectations, highlighting a gap between expectations and reality. While these limitations cannot be fully resolved, they can be mitigated through program improvements, data updates, chatbot training, and user experience standardization. Addressing plagiarism and academic misconduct requires establishing robust ethical standards that balance AI’s value with individual interests. Authors using ChatGPT in academic writing should transparently disclose its usage, allowing for clear identification of authorship. With proper oversight, ChatGPT can assist healthcare professionals and students in task completion efficiently.¹

Despite the current challenges and ethical issues associated with the use of ChatGPT in the medical field, there is still some potential for ChatGPT in the medical field. More empirical studies are needed to assess the exact impact of ChatGPT in the medical field. This is because the available literature so far has focused on the views or perspectives about ChatGPT. It is also worth noting that ChatGPT was not specifically developed for the medical field and further work is needed to enhance its depth of medical knowledge. In addition, it is important to develop appropriate ethical guidelines, restrictions in its use to improve accuracy, originality, bias, and misuse, and to overcome privacy, academic integrity, and ethics related issues.

Limitations

It is worth noting that although our review was designed to provide an indication of the use of ChatGPT in the medical field, the vast majority of the papers reviewed were editorials/letters to the editor, which may have affected the quality of the evidence used in this review, but an assessment of quality was not within the remit of the scoping study. As of the date of conducting the review, there may be little original research on the use of ChatGPT in medicine, which may limit the validity of ChatGPT over time and the ability to provide a comprehensive description of potential problems. Despite the different databases we took to search for research evidence, some literature was missed, and the inclusion of only English language literature may have exacerbated this limitation.

Conclusion

ChatGPT is an advanced language model with numerous advantages and applications in the healthcare and medical fields. It can assist healthcare professionals in various tasks, including research, diagnosis, patient monitoring, and medical education. However, the use of ChatGPT also presents ethical considerations and limitations, such as trustworthiness, plagiarism, copyright infringement, and bias. Therefore, it is crucial to thoroughly evaluate and address potential limitations and ethical considerations before implementing ChatGPT in the healthcare and medical sectors. Future research could focus on developing methods to mitigate these limitations while utilizing the benefits of ChatGPT.