Derivation and Validation of a Predictive Scoring Model of Infections Due to Acinetobacter baumannii in Patients with Hospital Acquired Pneumonia by Gram-Negative Bacilli

Introduction

For patients with severe HAP, it has been a huge challenge for physicians to prescribe the most appropriate, timely empirical antimicrobial therapy before bacterial culture results are obtained. Especially caused by Acinetobacter baumannii (ABA), due to its severe antimicrobial resistance,^1–3 available antibiotics are very limited and the prognosis of patients is often poor.⁴ Carbapenems are considered to be one of the preferred and the most commonly used initial antibiotics for severe HAP patients. However, several large-scale epidemiological investigations in China^5,6 have shown that one of the most common pathogenic bacteria causing HAP is ABA, which has high level of resistance rate (up to 70%) to carbapenems.⁷ Thus, carbapenems may exist the risk of initial ineffective therapy for ABA-HAP patients, and early empirical antimicrobial therapy should consider a combination regimen based on polymyxin and sulbactam.⁸ Therefore, prior to bacterial culture results, risk factors for ABA-HAP need to be identified in order to guide initial empirical antimicrobial therapy.⁹ In this study, we aimed to develop a clinically predictive scoring model that could be easily applied to help physicians quantify the possibility of patients with ABA-HAP among GNB-HAP patients, and to prospectively validate this scoring model in different groups suffered from GNB-HAP.

Materials and Methods

Study Design

In the first stage of this study, all patients suffered from GNB-HAP who were hospitalized in our hospital between January 1, 2019 and June 30, 2019 were included as subjects. In this retrospective cohort study, independent risk factors associated with ABA-HAP were identified and a predictive scoring model for clinical application was developed. In the second stage, the scoring model was further prospectively validated within GNB-HAP patients admitted to our center during July 1, 2019 to December 31, 2019. This study was approved by the Institutional Review Board of Tang Du Hospital, Fourth Military Medical University of the Chinese People’s Liberation Army (No. TDLL-KY-202101-07).

Inclusion Criteria

1. Age of over 18 years old; 2. all patients met the diagnostic criteria of HAP; 3. the respiratory specimens collected were in accordance with the quality control standards; 4. the result of bacterial culture was GNB; 5. the case data was complete; 6. if the same patient had multiple bacterial culture results, the first culture result were used as a reference.

Exclusion Criteria

1. Patients with bacterial culture results containing ABA and non-ABA were excluded. 2. Patients with ABA colonization were excluded.

Data Collection

The collection date of the first bacterial culture result from respiratory tract specimen was used as baseline, and the different clinical variables of the patients were collected: age, gender, smoking history, drinking history, seasonal distribution, days from the collection date of the first positive results of respiratory tract specimens to the date admitted to hospital, days from the specimen collection date to admission to ICU, days from the specimen collection date to endotracheal intubation, days from the specimen collection date to invasive ventilation, days from the specimen collection date to general anesthesia surgery, type of general anesthesia surgery, ICU admission within the last 3 months, invasive procedures prior to sputum culture (deep vein catheterization, gastric tube intubation, indwelling urinary catheter, blood purification, thoracic drainage, cranial drainage, gastroscopy, bronchoscopy, etc.), a history of cardiopulmonary resuscitation, underlying diseases such as hypertension, diabetes, lung disease, liver and kidney disease, coronary heart disease, heart failure, tumor, craniocerebral trauma, encephalitis, prostate hyperplasia and so on, immunocompromised, risk for aspiration, shock, transferred from other hospitals, dosage of budesonide inhalation, dosage of antacid, blood routine tests, plasma albumin, plasma globulin, procalcitonin, c-reactive protein, and so on, on the day or within the last 3 days of the specimen collection date.

Definitions

Patients with the sputum culture of Acinetobacter baumannii alone were defined as the ABA-HAP group; patients without the sputum culture of Acinetobacter baumannii were defined as non-ABA-HAP group who were infected with a single GNB or a combination of two or more GNB.

Transferred from other hospitals refers to a patient who has been treated in another hospital for more than 2 days and is transferred to our hospital for further treatment due to critical illness and deterioration.

Patients with long-term use of glucocorticoid or short-term high-dose glucocorticoid shock therapy (more than 1 mg/kg/day × 14 days of prednisone or other equivalent glucocorticoid) or long-term use of immunosuppressant, systemic tumor metastasis, radiotherapy and chemotherapy, organ transplantation, HIV/AIDS, agranulocytosis, severe malnutrition, cachexia, and major surgery are considered immunocompromised.

Patients with epiglottic dysfunction caused by long-term indwelling of gastric tube or invasive respiratory support, long-term bed rest due to consciousness disorder and paralysis caused by cerebrovascular disease or other reasons, difficulty in swallowing and choking on drinking water due to various reasons are considered to be at risk for aspiration.

Blood purification refers to intermittent hemodialysis, continuous renal replacement therapy, hemoperfusion, plasma exchange and other treatments.

Encephalitis includes viral encephalitis, autoimmune encephalitis, purulent meningitis and intracranial infection secondary to craniotomy operations due to craniocerebral trauma, intracranial space-occupying lesion, cerebral apoplexy, etc.

Statistical Analysis

Qualitative variables were compared using the Pearson Chi-square test or Fisher’s exact test, as appropriate and the Wilcoxon rank sum test for comparison of two independent samples were used after the continuous variables were converted to the ordinal categorical variables. All analyses were performed with a bilateral alpha risk of 5%. Variables with a P < 0.05 in univariate analysis were then included in the forward stepwise multivariable logistic regression analysis. A scoring model was then developed by assigning points to each independent risk factor confirmed by logistic regression model. The points were transformed by the odds ratio [log(OR) × 5] of the independent risk factors of ABA-HAP and rounded to the nearest integer according to the method used in the previous literature^10,11 and the final predictive score is the sum of the scores assigned for each independent risk factor. The model discrimination was determined by area under the receiver operating characteristic (ROC) curve.¹² The sensitivity, specificity, positive likelihood ratio and negative likelihood ratio of the predictive model at different cutoff values were assessed using standard definitions and methods. The best cutoff value of the predictive scoring model was determined according to the Youden Index.¹³ The patients were divided into low risk, moderate risk and high risk group of ABA-HAP according to different cutoff values, The scoring model obtained was then tested in the VC. The software SPSS version 23 (SPSS Inc., Chicago, USA) was used for data analysis.

Results

In the first stage of the study, a total of 395 patients met the inclusion criteria. Of these patients, 81 patients were in the ABA-HAP group and 314 patients were in the non-ABA-HAP group. Patient characteristics and univariate analysis between ABA-HAP and non-ABA-HAP group were shown in Table 1. All potential risk factors with a P < 0.05 in univariate analysis were included in the multivariate logistic regression analysis, and 10 independent risk factors for predicting ABA-HAP were identified, which included transferred from other hospitals, blood purification, risk for aspiration, immunocompromised, pulmonary interstitial fibrosis, pleural effusion, heart failure, encephalitis, increased monocyte count, increased neutrophils count (Table 2). The scores assigned to each independent risk factor were shown in Table 2.

Table 1 Patient Characteristics and Univariate Analysis Between ABA-HAP and Non-ABA-HAP Group

Table 2 Independent Risk Factors for Predicting ABA-HAP

The diagnostic efficiency of the predictive scoring model at different cutoff values was shown in Table 3. The Hosmer-Lemeshow goodness-of-fit statistic of the model was 0.433 (P < 0.05), AUROC in the DC was 0.845 (95% CI, 0.796 ~ 0.895), which is shown in Figure 1A, and the scores distribution of different risk groups in the DC are shown in Figure 1B. Youden index showed that the best cutoff value of this model was 10 points, and the sensitivity and specificity of the predictive scoring model were 0.778 and 0.793, respectively. When the cutoff value was 8 points, the sensitivity of the predictive model was more than 90%, which could reduce the rate of missed diagnosis; When the cutoff value was 12 points, the specificity of the predictive model was more than 90%, which could reduce the rate of misdiagnosis. According to different scores, patients could be divided into ABA-HAP low-risk group (predictive score < 8 points), moderate-risk group (12 points > predictive score ≥ 8 points) and high-risk group (predictive score ≥ 12 points). The incidence of ABA-HAP in low-risk, moderate-risk and high-risk group was 4.8% (7 out of 147), 16.1% (29 out of 180) and 66.2% (45 out of 68), respectively, and the difference was statistically significant (P < 0.001), as shown in Figure 2.

Table 3 The Diagnostic Efficiency of the Predictive Scoring Model at Different Cutoff Values

Figure 1 The performance of the predictive scoring model in the derivation cohort. (A) ROC curve of the derivation cohort; (B) scores distribution of the different risk groups in the derivation cohort.

Figure 2 The incidence of ABA-HAP in the low-risk, moderate-risk and high-risk groups.

In the second stage of the study, a total of 362 patients met the inclusion criteria. Of these patients, 101 patients were in the ABA-HAP group and 261 patients were in the non-ABA-HAP group. AUROC was 0.807 (95% CI, 0.759 ~ 0.856) in VC, as shown in Figure 3A, and the scores distribution of different risk groups are shown in Figure 3B. The incidence of ABA-HAP in the low, moderate and high risk group was 4.5% (4 out of 89), 20.8% (31 out of 149) and 53.2% (66 out of 124), respectively, and the difference was statistically significant (P < 0.001), as shown in Figure 2.

Figure 3 The performance of the predictive scoring model in the validation cohort. (A) ROC curve of the validation cohort; (B) scores distribution of the different risk groups in the validation cohort.

Discussion

In this study, we developed and validated a scoring model for predicting ABA-HAP in patients suffered from GNB-HAP using clinical variables readily available in practice. According to different scores, our predictive model can effectively divide GNB-HAP patients into ABA-HAP low-risk group, moderate-risk group and high-risk group without waiting for the culture results. Moreover, our predictive model has been proved to be of good diagnostic value and the AUROC is 0.845 and 0.807 in DC and VC, respectively. The incidence of ABA-HAP in the high-risk group was significantly higher than that in the moderate-risk and low-risk group (P < 0.001) either in DC or VC. Therefore, our predictive model can help front-line clinicians make decisions on initial empirical antimicrobial therapy, implement specific interventions to reduce patient mortality, and improve prognosis.

Currently, the guidelines for the treatment of HAP/VAP in China, the United States and Europe^1,14,15 all advocate rapid and appropriate empirical antimicrobial therapy. However, when prescribing empirical antibiotics, clinicians do not know the pathogen of the pneumonia. Epidemiological monitoring is instructive and meaningful to empirical antimicrobial therapy, but for a single individual, the underlying diseases, clinical characteristics, working and living environment of each patient are different, and the pathogenic bacteria are also different, and the antibiotics that can be chosen for different pathogenic bacteria are also significantly different. Especially in the first 24 to 72 hours of infection, the results of bacterial culture are usually not available and furthermore, the positive rate of bacterial culture is low,¹⁶ all of which are not conducive to patients receiving early appropriate empirical antimicrobial therapy. A retrospective cohort study of 175 US hospitals found that MDRAB pneumonia was significantly associated with higher mortality, and that high mortality was significantly associated with inappropriate empirical antimicrobial therapy.⁴ Although the risk factors for ABA infection (eg, invasive mechanical ventilation, critical condition, prior culture result of ABA, etc.) are generally recognized,^17,18 there is currently no universally accepted and easily used prediction tool to evaluate the possibility of ABA-HAP in a single patient. Our model can overcome the limitations of long culture time, low positive rate of bacterial culture and the probability of ABA-HAP unquantified. First of all, the patient was diagnosed as hospital-acquired bacterial pneumonia based on clinical manifestations and imaging results. Before giving empirical antimicrobial treatment, Gram-negative bacilli (GNB) were identified by smear staining of qualified respiratory specimens, and the results were usually obtained within half an hour. Then, with our predictive scoring model, patients could be accurately divided into low, moderate and high risk group of ABA-HAP. Patients in the high-risk group may be empirically given a combination antimicrobial regimen based on polymyxin and sulbactam, while patients in the low-risk group could be avoided from being exposed to many unnecessary antibiotics. Moreover, our model can select different cutoff value according to the patient’s condition, which makes it more flexible and practical for the selection of empirical antimicrobial therapy. For patients with severe HAP, sensitivity of the predictive model should be increased to reduce the rate of missed diagnosis, the cutoff value can be selected as 8 points, and the sensitivity can be as high as over 90%, that is to say, when the predictive score is ≥ 8 points, it is necessary to cover ABA for early empirical antimicrobial therapy. For non-severe patients, it is necessary to increase the specificity and reduce the misdiagnosis rate, the cutoff value can be selected as 12 points, that is to say, when the predictive score is ≥ 12 points, empirical antibiotics against ABA is required.

With the development of rapid diagnostic techniques for pathogens of infectious diseases, some techniques have been applied in clinical practice,¹⁹ such as whole genome sequencing (WGS), second-generation sequencing technology, PCR, Real-time PCR (RT-PCR), and quantitative loop-mediated isothermal amplification (LAMP) assay, etc. Although it has shown some advantages, such as rapid identification of specific pathogens, detection of known drug resistance genes and so on. However, these rapid diagnostic techniques based on gene base sequence have obvious limitations: high false positive rate, interference by airway colonization bacteria and dead bacteria, great influence of primer sequence mutation on detection, high cost, requirement of advanced experimental equipment and professional technicians, etc. However, our model can overcome these shortcomings as shown above by using clinically readily available variables, without excessive cost, with no need for advanced laboratory equipment, good diagnostic accuracy, flexible selection of different cutoff values according to the patient’s condition and so on. Because our model takes the results of bacterial culture as the gold standard, and for each patient, we actively identify infection or colonization, there is little interference by colonized bacteria or dead bacteria. Moreover, our model can be used as a reference to the results of rapid diagnostic technology to improve the accuracy of diagnosis.

In order to promote the rational use of antibiotics, WHO recommends that health Care institutions create tools and implement policies based on real-world data to increase the possibility of patients receiving early and appropriate empirical antimicrobial therapy.²⁰ At present, there are many different predictive scoring models, and most of them have some limitations in independent risk factors. A French scoring system,²¹ which included the history of travel abroad in the past six months, is clearly not suitable for China. Some scoring models^21–23 included the results of previous sputum culture or colonization of drug-resistant bacteria, which were obviously not applicable to patients who visited the hospital for the first time or patients who had no bacterial culture results previously. Some scoring models²⁴ included previous use of antibiotics, which is obviously not suitable for patients transferred from other hospitals, because the medical record system databases between hospitals are not interconnected. Some scoring systems^25,26 included Charlson comorbidity score, APACHE II (Acute Physiology and Chronic Health Evaluation II) and SOFA score (Sequential Organ Failure Assessment scores),²⁷ Clinicians still need to calculate these scores in the process of evaluation, which increases the complexity of variables. Independent risk factors included in our model can be easily obtained and judged by clinical symptoms and examination results of patients before the date of specimen collection.

Although our model has some advantages and has good discrimination in clinical application, it still has certain limitations. First of all, our study is a single-center study, and the predictive scoring model has not been externally verified, but our hospital is a large-scale comprehensive tertiary hospital with 3000 beds in Northwest China, rather than a specialized hospital, because China currently implements a three-level referral system, so inpatients in our hospital are representative to some extent, nevertheless, a multi-center validation study is needed next. Secondly, our study was a retrospective study in the first stage, and the relevant data came from patients’ medical records, so some important confounding variables might be ignored. In addition, our gold standard was the result of bacterial culture, although we have actively identified the infection or colonization in each patient, the possibility of colonization could not be ruled out completely. Despite these limitations, we attempted to collect all patients over a period of time, rather than randomly select some cases, in order to avoid the impact of selection bias.

Conclusions

In conclusion, this simple and prospectively validated predictive scoring model can effectively help clinicians estimate the probability of occurrence of ABA-HAP and accurately classify patients into the low, moderate and high risk group of ABA-HAP. Moreover, our model can choose different cutoff values according to the patient’s condition. It is more flexible and practical for clinicians to select empirical antimicrobial therapy. Subsequently, the probability of patients receiving early and appropriate empiric antimicrobial therapy increases, thereby the mortality of patients decreases.