Development and Application of a Reference Manual for Diagnosis and Rational Use of Antimicrobial Agents for Outpatient Primary Care (Digestive System Part): A Delphi Study

Introduction

According to the World Health Organization (WHO)¹ and the Global Burden of Disease (GBD) study published in the Lancet,² approximately 2.05 billion people in the world suffer from digestive diseases, with a mortality rate as high as 31.8%. This highlights the urgent need for effective diagnosis and treatment of digestive system disorders. However, it has been reported that the diagnosis of digestive disorders is often delayed (6.4 months on average after clinical onset) and incorrectly diagnosed.³ This might be attributed to the large number of digestive disease sub-types and the absence of reliable diagnostic markers in the early stages of the diseases.⁴

Incorrect treatment of diseases is also common in primary care outpatient settings.⁵ In our previous study in China, primary care physicians prescribed antimicrobial medications for digestive diseases purely based on patients’ wishes or their arbitrary experiences: up to 89.9% of prescriptions for digestive diseases were irrationally used or abused.⁶ In the context of the digestive system, antibacterial agents are primarily employed to address bacterial infections, while viral infections do not necessitate their use. For instance, when dealing with infectious diarrhea caused by a virus, antimicrobial therapy is not required.⁷ Antimicrobial agent abuse is a significant cause of concern for antimicrobial resistance.⁸ Taking Helicobacter pylori infection as an example, it could lead to various conditions, such as chronic gastritis, duodenitis, and stomach ulcers. It is important to note that the irrational use of antimicrobial medications will exacerbate this infection by fostering resistance, ultimately leading to a reduced eradication rate of Helicobacter pylori and prolonging the infection.⁹

According to Health Statistics Yearbooks of China¹⁰ and Guizhou¹¹ (a typical poor mountainous province in China) over the last three years, primary care staff are in short supply (only 0.39 medical and nursing personnel per 1000 population). The vast majority (85%) of them have only a junior college education or below with insufficient knowledge and expertise.

Currently, although there are numerous clinical practice guidelines and guiding principles available for health workers to use, they are generally somewhat broad and not specially tailored for primary care settings. A PubMed search with the Title/Abstract “primary care” and “China” yielded 1016 studies, and another search with “primary care”, “digestive”, and “China” yielded only 10 studies, until July 18, 2023. It may be inferred that digestive diseases in primary care in China has not received enough attention. This study is necessary, because a valid and reliable reference manual that is easy to understand may promote primary healthcare providers to correctly diagnose and rationally use antimicrobial agents. In this case, the Delphi technique is useful because it helps discover issues in clinical practices that a group of experts believes to be important. It is widely utilized and accepted in medical research.¹²

Our research team had previously established a reference manual for the diagnosis and rational use of antimicrobial agents for respiratory illnesses, which is currently well used in outpatient primary care practices in China.¹³ The present study targets diseases of the digestive system. The objective of this study is to establish a reference manual for outpatient primary care, which is concise and easy-to-understand, for the diagnosis of common digestive diseases and the rational use of antimicrobial agents, through a collective consensus of professionals.

Materials and Methods

Study Design

In this study, the Reference Manual for Diagnosis and Rational Use of Antimicrobial Agents for Outpatient Primary Care (Digestive System Part) (hereinafter referred to as the “reference manual”) was established. This manual is divided into two independent portions: the diagnosis of common digestive diseases (hereinafter referred to as “disease diagnosis”) and the rational use of antimicrobial agents (hereinafter referred to as “rationality of antimicrobial use”).

The methodology for the manual establishment encompassed two processes: the development of a draft manual and the validation of the manual. The development process was based on a literature review and expert discussion. The Delphi technique was systematically and rigorously performed to validate the reference manual, following the Delphi guidelines for the health field,¹⁴ with a specific focus on primary health care.¹⁵ Furthermore, the manual was preliminarily applied among a sample of primary care physicians. A diagram illustrating the methods of this study is presented in Figure 1.

Figure 1 Diagram illustrating the methods of this study.

Development of the Reference Manual

A review of authoritative medical textbooks, guidelines, and related studies included: 1) Diagnostics and Internal Medicine published by People’s Medical Publishing House of China, 2) Guiding Principle of Clinical Use of Antibiotics by the National Health Commission of the People’s Republic of China,⁷ summarized in Supplementary Materials 1, 3) Guidelines for use of antibiotics¹⁶ by the United States Centers for Disease Control and Prevention (CDC), and 4) our previous research.^6,17 They all served as the foundation for the development of the initial draft of the manual. However, current guidelines are evidence-based, with a methodology that locates the best research evidence through systematic literature reviews, to address clinical questions.¹⁸ As we mentioned above, there is not much evidence from primary care, as work burnout is prevalent among primary care physicians, who indicate that there is no spare time for research.¹⁹ Currently, there are no guidelines specifically applicable to primary care settings. In the following step, a meeting of experts from the government and hospitals (the Health Commissions of Guizhou, Guangxi, and Yunnan provinces, the Second Affiliated Hospital of Kunming Medical University, the Affiliated Hospital of Guangxi University of Traditional Chinese Medicine, and the People’s Hospital of Guizhou Province) thoroughly discussed issues in the diagnosis and treatment of digestive diseases in outpatient primary care practices that had been neglected in the literature. They discovered that current international or national guidelines are often specialized in nature, which can be challenging for primary care providers to comprehend. Therefore, there is a need for a concise and easily understandable manual for users in primary care. Subsequently, an initial draft of the manual was prepared, which would be later distributed to Delphi panelists. A pilot test was conducted among the research members prior to the implementation of the Delphi working process, as advised by the Delphi guidelines.¹⁴

Validation of the Reference Manual

Selection of the Panelists

The Delphi technique entails persons with a deep understanding of the topic under research, defined as “panelists” in the current study. Separate eligibility criteria were determined for the two portions of the reference manual. A purposive sampling was employed. Each panelist was purposefully selected based on their knowledge or experience related to our research topic.

The following eligibility criteria were used for the disease diagnosis portion: 1) having experience working or assisting in primary care practices; 2) possessing over 10 years of experience in clinical diagnosis of digestive system diseases; 3) being interested in this study and willing to participate; 4) specialists holding an associate senior title or higher.

The following eligibility criteria were used to select panelists for the rationality of antimicrobial use portion: 1) having experience working or assisting in primary care practices; 2) possessing over 10 years of experience in clinical prescription or pharmaceutical research on digestive diseases; 3) being interested in this study and willing to participate; 4) specialists holding an associate senior title or higher.

The panelists involved in this study were all specialists with experience in primary care settings. China is currently developing and strengthening an integrated healthcare system that facilitates collaboration between primary care and higher hospitals.²⁰ This requires the transfer of specialists to primary care institutions to provide guidance and support to primary care.

Measurement of the Panelists’ Responses

In the initial draft of the manual, the common digestive system disorders among outpatients in primary care, including diseases from the mouth to the anus,²¹ were investigated and enumerated according to the International Classification of Diseases (ICD-10) code, which is familiar in primary care settings in China. In the disease diagnosis portion of the manual, items were formed by standardizing the diagnosis of the above-mentioned diseases in terms of symptoms, signs, and ancillary tests. In the rationality of antimicrobial use portion, the items were evaluated for 13 categories of digestive diseases, assessing the rationality of antimicrobial use in each case. Common categories of antimicrobial agents used in outpatient primary care practices were listed, including Penicillins, Cephalosporins, Macrolides, Quinolones, Lincosamides, Nitroimidazoles, and Aminoglycosides.^6,22 We used broad categories of antimicrobial agents instead of subcategories or single molecules because primary care institutions in China are often confined to the use of the unrestricted class of antimicrobial agents. This is a result of the implementation of antimicrobial stewardship policies in China, which prioritize safety, minimization of the risk of antimicrobial resistance, best practice prescribing, and public health interests. Similar antimicrobial restrictions have been established in many other countries.²³ According to the National Catalog for Clinical Application of Antimicrobial Agents by the National Health Commission of the People’s Republic of China, antimicrobial agents are classified into three classes, unrestricted, restricted, and special. The catalog is detailed in Supplementary Material 2.

All items in both portions of the draft manual were evaluated by the Delphi panelists based on clinical guidelines, their diagnosis or prescription experience, and the features of primary care settings. A Likert scale was used for the quantitative evaluation, ranging from 4 to 1: score 4 (very high diagnostic value, indicating preferred medication, optimal), score 3 (high diagnostic value, indicating antimicrobials can be used or substituted, acceptable but not optimal), score 2 (average diagnostic value, indicating no need to use, e.g., prescribing antimicrobials to prevent viral infections), score 1 (low diagnostic value, indicating incorrect spectrum of antimicrobials based on pharmacological actions, antibacterial spectrum, adverse drug reactions, and a table titled “Principles of empirical antibiotic treatment for various bacterial infections” from the guidelines^7,16 [see Supplementary Table 1]. For instance, when diagnosing acute tonsillitis and acute pharyngitis, the main pathogens are Group A hemolytic streptococcus, with a few being Group C or G hemolytic streptococcus. In such cases, if a particular panelist prefers Penicillins for empirical treatment with antimicrobials, Penicillins would be given a score of 4 (indicating preferred medication, optimal); if he or she uses Cephalosporins as a possible alternative, Cephalosporins would be given a score of 3 (indicating antimicrobials can be used or substituted, acceptable but not optimal). In addition, in order to encourage the panelists to donate as many comments as possible, each rating was followed by a space where they could offer their comments.

Iterative Working Process of the Delphi

The Delphi is an iterative working process that aims to combine opinions into a group consensus. It is important to pre-define the term “consensus”,¹⁴ which in the context of this study refers to the group response reaching stability without further changes.

From August 2020 to February 2021, the Delphi process was conducted through paper-based or mail-based communications. In the first round of the Delphi, after the responses had been returned, the research team synthesized the responses in a summary feedback form (statistical evaluation of the group response) without including any information from the researchers.²⁴ In the second round, the panelists had the opportunity to change their opinions based on the feedback from the previous round that was provided anonymously to the panelists. Our working team continued to present the analytical results of the previous round to the panelists and solicit feedback, until a consensus was reached.²⁵ After discussion among the researchers, this report was finally formed.

Statistical Evaluation of the Group Response

EpiData software (version 3.1) was used to double input panelist scores from paper responses, and SPSS software (version 23, IBM Corp) was used for statistical analysis. Each item of the manual was described by the median for central tendency and the coefficient of variation (CV) for dispersion.

The response of the group was assessed by the level of authority and commitment of the panelists, as well as the degree of agreement among them. First, the level of authority of the panelists was measured by the authority coefficient (Cr), self-reported by the panelists.²⁶ It is the average of the familiarity with the research topic (Cs) and the judgment basis (Ca). The calculation formula is as follows:

It is generally believed that Cr ≥ 0.7 indicates a high level of authority.²⁶ Second, the level of commitment of the panelists was measured based on the response rate. Sumsion (1998) suggested a minimum response rate of 70% for each round.²⁷ Third, the degree of agreement among the panelists was expressed by the consensus rate, and the Kendall W was used for the test. The consensus rate is the percentage of the panelists that have come to an agreement. If the result of the Kendall W-test was significant (P < 0.05), the panelists’ assessments were considered to have a high degree of consensus.

Application of the Reference Manual

The Digestive System Part of the reference manual reported in this study, was combined with other parts developed by our research team for various systemic disorders to form a brochure. This brochure was then distributed to a sample of primary care physicians for their use and evaluation.

A cluster sampling method was used to select samples from 39 primary care institutions. The sampling frame was the list of primary care institutions from the database of the Information Center of the Guizhou Provincial Health Commission, with the eligibility criteria of 1) using the Health Information System of the Information Center of the Guizhou Provincial Health Commission, and 2) having outpatient physicians who had worked in primary care institutions for more than one year.

After the sample of physicians had used the reference manual for more than three months, we used an electronic questionnaire (see Supplementary Material 3) to investigate their manual utilization patterns, satisfaction, and suggestions. The results were presented using percentages.

Results

Characteristics of the Panelists

A total of 30 experts were confirmed eligible and invited to participate. Since one panelist withdrew before the second round, 29 panelists committed to this Delphi working process. The comments of the withdrawal panelist were analyzed and carefully considered in accordance with the Delphi guidelines.¹⁵ Of those who participated, 10 were male and 19 were female. There were 12 clinical doctors, including specialists in the fields of diagnostics or digestive diseases, 16 clinical pharmacists, including specialists in the fields of antimicrobial therapy, digestive diseases, or prescription review. Additionally, one pharmacological laboratory scientist was involved. All had more than 10 years of experience working in their professional fields. More importantly, 4 panelists had ≥ 5 years of primary care experience, 12 panelists had 3~4 years, and 13 panelists had 1~2 years. The characteristics of the panelists are shown in Table 1.

Table 1 Characteristics of the Panelists

Level of Authority of the Panelists

The level of authority of the panelists was calculated and presented in Table 2. The authority coefficients for either portion of the manual were greater than 0.70.

Table 2 Authority Coefficients for Either Portion of the Manual

Level of Commitment

It took two rounds to reach a collective consensus in this study. Totally, 44 free-text comments were contributed by the panel. Table 3 displays the response rates of the Delphi panel.

Table 3 Response Rates for Round 1 and 2

Degree of Agreement

In the second round, the consensus rates for either portion of the disease diagnosis or the rationality of antimicrobial use in the manual were more than 65% and 70%, with Kendall W being 0.60 (P < 0.001) and 0.71 (P < 0.001), respectively. After two rounds of evaluation, the stability of the results led the Delphi work to a consensus.

Reference Manual

The cover, table of contents, and example pages of the reference manual are illustrated in Figure 2.

Figure 2 Cover, table of contents, and example pages of the reference manual. Notes: (A) Cover of the reference manual; (B) Table of contents of the reference manual; (C) Example page of the disease diagnosis portion of the reference manual providing diagnostic indicators for common digestive diseases; (D) Example page of the rationality of antimicrobial use portion of the reference manual providing recommendations for the selection of antimicrobial agents in outpatient primary care practices for common digestive system disorders.

Disease Diagnosis Portion

In Round 1 and 2, a total of 220 diagnostic items were assessed for 29 common digestive system diseases in outpatient primary care. Items that received the lowest diagnostic value assessment (scored 1 on the Likert scale of 1 to 4) from all panelists were removed. The remaining 200 items are presented in the reference manual. Due to the limited space of the manuscript, 167 items that received the highest diagnostic value assessment (scored a median of 4 on the Likert scale of 1 to 4) in the collective consensus were listed as diagnostic indicators in Table 4.

Table 4 Diagnostic Indicators of Common Digestive Diseases in Outpatient Primary Care: Consensus Among Panelists

Rationality of Antimicrobial Use Portion

The recommendations for the selection of antimicrobial agents in outpatient primary care practices for 13 categories of common digestive system disorders are given in Table 5.

Table 5 Recommendations for the Selection of Antimicrobial Agents for 13 Categories of Common Digestive Disorders in Outpatient Primary Care: Consensus Among Panelists

Application of the Reference Manual

A total of 160 physicians, who had used the reference manual for at least three months, participated in our survey, providing feedback into their utilization patterns, satisfaction levels, and suggestions. The findings indicated that more than 90% of them used the manual 2–3 times per week, 88.13% were satisfied with it, more than 95% considered it useful in improving their knowledge and expertise, and 98.13% would recommend it to other outpatient physicians. It is worth noting that 100% of physicians believed this manual was applicable to primary care settings.

Discussion

Findings of the Study

The Delphi technique is widely used and accepted in medical research to establish group consensus and contribute to successful decision-making, particularly for topics that the traditional evidence-based medicine paradigm could not or does not apply to solve.^14,15 The selection of the panel is considered to be crucial for a Delphi study.¹⁵ This study involved 29 panelists with more than 10 years of work experience who have extensive clinical knowledge, expertise, or research in the fields of diagnostics or antimicrobial therapy for common digestive system diseases, and more importantly, experience in primary care work or assistance. The results of high authority coefficients in this study imply that the panelists are authoritative. The purposive sampling with the eligibility criterion of having experience in primary care work or assistance practices suggests that they were selected for our endeavor to improve primary care practices. Furthermore, in both rounds of the Delphi working process, high response rates, that is active commitment, ensured the study’s validity.²⁸ Finally, both the results of consensus rates and Kendall W-tests meet the criteria by the Delphi guidelines.^14,15 Although the consensus rates in this study are not as high as over 80%, they met the stopping criterion—stability of responses—as specified by the present Delphi procedure. It has been explained that an artificially boosted consensus could distort the findings of the study, a phenomenon called “consensus bias”.¹⁵ In fact, for all items in the current study, more than half of the panelists reached a consensus, which we did not consider a low consensus rate, hence we did not remove any items because of panelist disagreement. Collectively, all of the statistical evaluations of the group response discussed above reveal the validity and reliability of the present Delphi study.

According to the WHO, 8 out of the 12 bacteria that are the most threatening to human health infect the human digestive system.²⁹ Digestive system diseases are widespread and occur often in primary care settings. Through the collective wisdom of the panelists, our study established a reference manual for digestive disorders for outpatient primary care. In the disease diagnosis portion of the manual, a total of 200 diagnostic items (standardized by symptoms, signs, and auxiliary tests) are provided for 29 common digestive disorders for primary care settings. Among them, 167 items with the highest diagnostic value (received a median score of 4) were the diagnostic indicators recommended by the reference manual. Based on the panel’s experience in primary care, they specifically added seven indicators that received the highest diagnostic value in the second round of the Delphi work: 1) the auxiliary test for chronic gastritis, “gastroscopy, mucosal congestion, swelling, and erosion; shrinking”; 2–3) the auxiliary test of “HP test” for duodenitis and gastric ulcer, respectively; 4) the sign of “murphy signs” for cholecystolithiasis; 5–6) the signs for acute cholecystitis, “abdominal muscle tension and rebound pain; murphy signs” and “knock pain in gallbladder area”; and 7) the auxiliary test of “vomit/stool OB” for hemorrhage of digestive tract. Regarding the portion on the rationality of antimicrobial use in the manual, recommendations on the rational use of antimicrobials for 13 categories of digestive diseases were formulated on the basis of the group consensus. It is clearly stated that antimicrobial agents are not necessary for the treatment of other diseases of teeth and supporting structures, gastro-esophageal reflux diseases, other non-infectious gastroenteritis, and colitis, cholelithiasis, and other diseases of the digestive system. Antimicrobial agents are acceptable when treating diseases of the pulp and periapical tissues, gingivitis and periodontal diseases, stomatitis and related damages, gastric ulcers, gastritis and duodenitis, acute appendicitis, abscesses in the anal and rectal regions, and cholecystitis. However, Penicillins and Cephalosporins are preferred, whereas broad-spectrum and ultra-broad-spectrum antimicrobial medicines are avoided. As mentioned above, primary care institutions are confined to the use of the unrestricted class of antimicrobial agents in China. Taking Penicillins as an example, the specific antimicrobials available in primary care settings include Penicillin, Oxacillin, Amoxicillin, Ampicillin, Piperacillin, Amoxicillin/Clavulanic acid, Procaine benzylpenicillin, Penicillin V, Cloxacillin, and Carbenicillin (see Supplementary Material 2). To exercise caution, our reference manual only provides recommendations on the rational use of broad categories of antimicrobials, since medication decisions need to be based on the patient’s symptoms, signs, ancillary test results, as well as treatment history. It is still essential that primary care providers refer to clinical guidelines for the selection of subcategories or single molecule drugs to perform customized medication.

Public Health Significance for Primary Care

A Lancet work has reported that subpar clinical quality and diagnostic mistakes persist in primary care in China, with patients’ conditions being correctly diagnosed only 26% of the time.³⁰ Considering the concern over high rates of inappropriate antimicrobial prescriptions, the Chinese government has been developing policies and legislation to limit antimicrobial drug abuse. However, their effectiveness in primary care settings is limited.³¹ Among the primary care physicians sampled for the preliminary application of the manual in this study, the majority of them are older and busy, with little motivation to participate in systematic learning. Existing guidelines for diagnosing and treating digestive diseases are not sufficiently concise and not easy to understand, which they may find difficult to learn. The primary care physicians surveyed expressed satisfaction with the reference manual we developed, based on their usage. It is believed that this manual will aid them in rapid learning to increase disease diagnosis and treatment accuracy.

The recommendations in the Reference Manual for Diagnosis and Rational Use of Antimicrobial Agents for Outpatient Primary Care (Digestive System Part) are laid out in a simple-to-read format that is easy to understand. As illustrated in Figure 2, in the disease diagnosis portion of the manual, the items are presented in an easy-to-understand descending order of diagnostic values as 4, 3, and 2 (items with a diagnostic value score of 1 were excluded), so that primary care physicians can prioritize higher-valued diagnostic indicators. In the portion of the rationality of antimicrobial use of the manual, easy symbols including P (preferred medication, optimal), A (acceptable, an alternative), N (no need to use), and W (wrong spectrum) were given. Primary care physicians and pharmacists can quickly review and select the appropriate antimicrobials for a specific disease. This reference manual has public health significance for primary care. It helps primary care providers improve the correct diagnosis of diseases and rational use of antimicrobial agents by filling the research gaps in current clinical guidelines that do not address primary care.

Limitations

We cannot guarantee that the study we presented is free of bias, even though our working process followed the guidelines of the Delphi technique to reduce the possibility of systematic bias. The Delphi is a “heuristic device” for reaching a consensus among a group of professionals, which does not promise complete correctness.¹⁴ Biases may arise from two sources in this study. Firstly, none of the panelists involved in the disease diagnosis portion had a doctoral degree. However, these panelists are all specialists with over 10 years of experience and possess an excellent understanding of the challenges of primary care practices in China, despite the lack of doctorates. It is worth mentioning that China ranks second-to-last in the number of doctors per 1000 people, according to statistics.³² Health workers in China carry a substantial clinical workload. In particular, specialists carry a greater clinical workload, but acquire a wealth of practical experience. Secondly, the objectivity of the assessment of the panel’s authority may be compromised due to the absence of objective metrics.

The issues brought up by primary care physicians surveyed will receive ongoing attention and improvement from our research team. Moreover, the manual will be further applied among a broader group of primary care providers. In future research, with the support of this reference manual, our plan is to develop an artificial intelligence-based real-time early warning system for identifying irrational antimicrobial prescriptions, and to carry out intervention experiments in primary care settings, in hopes of achieving an effective reduction of antimicrobial resistance.

Conclusions

On the basis of a collective consensus of professionals, a Reference Manual for Diagnosis and Rational Use of Antimicrobial Agents for Outpatient Primary Care (Digestive System Part) was developed and validated. When applied to primary care physicians, the manual received a high level of satisfaction and recognition for its primary care applicability. It can offer recommendations to primary care physicians and pharmacists, assisting them in rapid learning to improve diagnostic and treatment accuracy for digestive diseases.