A Deep Learning Model for the Diagnosis and Discrimination of Gram-Positive and Gram-Negative Bacterial Pneumonia for Children Using Chest Radiography Images and Clinical Information

Introduction

Pneumonia is the most significant cause of death in children in developing countries.^1,2 The most common causes of pneumonia are viral and bacterial pathogens. Based on the Global Burden of Disease data, in 2016, approximately 64% of children under the age of five with pneumonia died of bacterial pneumonia.³ Several types of bacteria can cause pneumonia, including Streptococcus pneumoniae, Klebsiella pneumoniae, Haemophilus influenzae, and Pseudomonas aeruginosa.^4,5 Rapid and accurate identification of pneumonia pathogens, such as gram-positive and gram-negative bacteria, is essential for antibiotic selection. Appropriate drug treatment and limiting the unreasonable use of antibiotics play an important role in improving the survival or cure rates of bacterial pneumonia.⁶ This may improve outcomes and slow the development of antibiotic resistance. Early and accurate identification of bacterial pneumonia pathogens presents a significant challenge for successful treatment.⁷ Symptoms of pneumonia caused by gram-positive and gram-negative bacteria usually overlap, and the etiological diagnosis is mainly based on clinical and diagnostic imaging findings. Accurate classification of pneumonia based on imaging tests alone can be challenging, especially when different types of pneumonia have similar imaging features. Chest radiography (CXR) is a standard diagnostic tool for pneumonia; however, the sensitivity and specificity are not high,⁸ and the interpretation of chest X-ray characteristics varies among clinicians with different levels of experience and ultimately depends on the expertise of the radiologist or clinical physician.⁹ The identification of pneumonia pathogens relies heavily on isolation and culture techniques, which include sputum smears, alveolar lavage, bronchial brushings, and antigen and virus testing. As a standard diagnostic tool for the detection of pathogens, the isolation and culturing process is complex and time-consuming, has many detection restrictions, and has low sensitivity and specificity.⁹ Blood or pleural cultures are highly specific for the diagnosis of bacterial infections, but are less sensitive and take a long time, usually several days.¹⁰ Emerging microbial detection techniques, such as matrix-assisted laser desorption ionization time-of-flight mass spectrometry, can be used to rapidly identify pathogenic microorganisms, such as bacteria, fungi, mycobacteria, and viruses, as well as their drug resistance. However, the technical cost is high and it can only be carried out in qualified units.¹¹ Therefore, early identification of the pathogenic bacteria responsible for pneumonia is particularly important.

Artificial intelligence (AI) can effectively improve the accuracy and speed of pneumonia diagnosis.^12–14 Existing research on the application of AI technology in the field of pneumonia is still mainly focused on the detection and segmentation of pneumonia lesions and differential diagnosis of pneumonia. A deep learning algorithm-based chest X-ray diagnosis method, CheXNeXt, has an accuracy of diagnosis for 11 common chest diseases, which is comparable to or even better than that of practicing radiologists.¹⁵ Another study presented a deep-learning model for lung-thoracic segmentation based on clinical pediatric CXR of 1135 patients, which achieved an area under the receiver operating characteristic curve (AUC) of 0.95 and a sensitivity of 88.7%. Compared with this algorithm, radiology residents showed lower sensitivity but higher specificity.¹⁶ However, most of the current studies are based on the differential diagnosis between viral pneumonia, especially COVID-19, and other types of pneumonia, such as interstitial pneumonia and community-acquired pneumonia.^17,18 There are only a few studies on the differential diagnosis of bacterial pneumonia. One study used a computerized tomography image dataset in children (AUC: 0.75, accuracy: 0.58).¹⁹ However, this study was only based on imaging data and did not combine imaging examination and other indicators. We hypothesize that deep learning could be a valuable tool for the differential diagnosis of gram-positive and gram-negative pneumonia, based on previous research demonstrating the effectiveness of machine learning algorithms in medical diagnosis, which is of great importance to guide the choice of empirical anti-infective therapy and reduce antibiotic resistance, thus reducing pneumonia-associated mortality and sequelae in children. AI technology has been proven to improve diagnostic rates by fusing multi-modal information, which can overcome the problem of sensitivity and specificity that could not be balanced by previous technology.^20,21 Compared with the single mode (X-ray only), the AUC of the existing multi-modal X-ray diagnosis in COVID-19 and non-COVID-19 pneumonia increased from 0.89 to 0.93.²² Therefore, whether multi-modal information including X-ray data can improve the differential diagnosis of gram-positive or gram-negative bacteria requires further investigation.

Therefore, this study aimed to develop a learning model to identify gram-positive and gram-negative bacterial pneumonia in children based on X-ray data with clinical data to effectively improve the accuracy of diagnosis of pneumonia in children as well as to shorten the time to diagnosis, which may be useful for guiding empirical antibiotic treatment.

Materials and Methods

Study Sample

This study included 842 patients with community-acquired pneumonia aged <14 years who were admitted to our institution between January 2016 and June 2021. Precise clinical diagnosis, radiological diagnosis, and clear etiological examination results were available for all patients. Sputum and pleural effusion cultures were used to test the etiological results of bacterial pneumonia. A chemiluminescence immunoassay quantitative diagnostic kit (Chongqing Bypass Medical Instrument Co. Ltd.; Chongqing, China) was used. The inclusion and exclusion criteria for pneumonia cases were first screened on the data platform of our institution. We selected patients diagnosed with pneumonia by chest X-ray examination and who had undergone etiological examination (n=1055). Patients with mixed (n=116), viral (n=55), and fungal (n=42) infections were excluded, leaving 447 cases of gram-positive bacterial pneumonia and 395 cases of gram-negativebacterial pneumonia (Figure 1). Patient consent was waived by the ethical review committee of Southwest Hospital, Army Medical University, as the study was a retrospective analysis of de-identified data. This study was conducted in accordance with the Declaration of Helsinki, and all patient data were kept confidential. The waiver of consent was reviewed and approved by the institutional review board of Southwest Hospital, Army Medical University (approval number: KY2020277).

Figure 1 Patient selection flowchart.

Processing of Data

We conducted a series of preprocessing of the clinical and imaging data used in the experiment to meet the requirements of model training and evaluation. For image data, the size of each image was adjusted to a unified 256×256-pixel resolution, and its intensity value was linearly normalized into the interval [0,1]. In the case system, we collated 8 indicators of clinical signs, including cough and sputum, 5 patient baseline characteristics, 4 previous history, and 27 indicators of laboratory tests. A total of 44 clinical indicators were identified. The clinical data were categorized into categorical and numerical data. The categorical data were converted into numerical data using dummy variable coding. These steps produced 842 matched cases for the final experimental dataset. To test the algorithm, we randomly selected 690 (80%) cases as the training set and 152 (20%) as the verification set.

Model Selection and Parameter Setting

To evaluate whether the fusion of imaging and non-imaging features can improve the classification results of childhood pneumonia, we established models based on imaging and clinical features, respectively. The CNN model consists of three convolution layers and three fully connected layers, each of which uses ReLU as the activation function, and the final output uses sigmoid as the activation function. The model for clinical features had four fully connected layers. Each of the first three layers is followed by BatchNorm1D and ReLU activation function construction, and the output of the last layer uses sigmoid as the activation function. In addition, we compared the most commonly used models for image and clinical feature classifications. We selected logistic regression (LR), support vector machines (SVM), and CatBoost as machine learning models for clinical data and ResNet, ResNeXt and MobileNetV3 as deep learning models for image data for the following reasons: LR and SVM are popular traditional machine learning algorithms that have been extensively used in medical research because of their interpretability and ability to handle high-dimensional data. CatBoost, on the other hand, is a gradient-boosting algorithm that has shown promising results in various applications, including medical clinical data analysis. ResNet, ResNeXt, and MobileNetV3 are deep learning models that have demonstrated exceptional performance in various image classification tasks. These models are known for their ability to handle high-dimensional data and learn complex representations of images. The feature fusion method proposed in this study concatenates image features extracted by a convolutional network using fully connected layers and clinical features. After the fusion of the image and clinical features, the feature space was learned and projected into the classification space to complete the classification task. The evaluation indicators of our classification algorithm were Precision, Recall, AUC, and F1.

Experimental Environment

The experiment was based on python3.7 language, in which the hardware environment used was RAM32G CPU: Intel(R) Xeon(R) Gold 6271C CPU @ 2.60GHz, NVIDIA V100 32G high-performance graphics card. Deep learning was prepared based on the libraries paddle 2.3.2, and machine learning was prepared based on the libraries CatBoost 1.0.6 and scikit-learn 0.22.

Statistics Analysis

Before building the model, we evaluated the differences in the clinical factors between gram-positive and gram-negative bacterial pneumonia. Student’s t-test or Kruskal–Wallis H-test was used for continuous variables, and the χ² test or Fisher’s exact test was used for categorical variables. Statistical analyses were performed using Statistical Package for the Social Sciences version 26.0 (IBM SPSS Statistics; Armonk, New York, USA). The level of statistical significance was set at P<0.05.

Results

A total of 842 children were included in this study, including 447 cases of gram-positive and 395 cases of gram-negative bacterial pneumonia. Table 1 lists the characteristics of the patients in the training and test sets. There were significant differences in age, height, and body mass index between the gram-positive and gram-negative bacteria groups (P<0.05). Among the clinical symptoms, cough, phlegm in the throat, foaming at the mouth, fever, groaning, nasal congestion, cyanosis, and poor mental state showed significant differences (P<0.05). In the laboratory data, the levels of C-reactive protein, direct bilirubin, γ-glutamyltransferase, and alanine aminotransferase in the gram-positive group were significantly higher than those in the gram-negative group (P<0.05).

Table 1 Clinical Characteristics of Children (n=842)

The training and verification sets were divided based on a ratio of 8:2, wherein the final training set contained 690 cases. Among them, 366 were gram-positive and 324 were gram-negative. In total, 152 cases were verified. There were 81 gram-positive and 71 gram-negative patients. We achieved the most basic image binary classification in convolution and full connection. CNN-based refers to a simple CNN network structure. First, the binary task of gram-positive and gram-negative bacterial pneumonia based on image data was established. Then, the binary task of clinical data based on LR, SVM, CatBoost, and full-connection network was established. Finally, the clinical and image features were fused and a multi-modal binary classification task based on deep learning was established. The model fusion strategy is shown in Figure 2. We used four evaluation indicators (accuracy (ACC), AUC, Recall, and F1) to evaluate the diagnosis of bacterial pneumonia in the training and verification sets of the model. Table 2 lists the comprehensive prediction performance of each classifier in the test set. Among the four evaluation indexes, CatBoost had the best performance, and its AUC was significantly higher than that of the LR, SVM, and fully connected network models. In the classification task based on clinical data, we used LR, SVM, CatBoost, and four fully connected networks. SVM had the best accuracy (0.756), CatBoost had the best recall rate and AUC (0.738 and 0.852, respectively), and SVM had the best AUC. After a comprehensive evaluation based on the F1 value, the best was CatBoost (0.778).

Table 2 Performance Comparison of Machine Learning Algorithm Models Based on Clinical Data

Figure 2 Fusion strategies using deep learning.

In Table 3, we comprehensively compare the enhancement effects of the clinical feature fusion image models. The input image size resize of its unified model was 256×256, the learning rate was 1e-3, the epoch was 30, the batch size was 8, the stochastic gradient descent (SGD) optimizer was used, and the mean squared error (MSE) loss function was used. Because the CNN-based network does not converge after adding clinical features, it is not considered in statistics. Comparing the results of the seven models, ResNet18, ResNet34, ResNet50, ResNet101, ResNeXt50, ResNeXt101, and MobileNetV3, the comprehensive index AUC and F1 increased by an average of 5.6% and 10.2%, respectively. This shows that introducing clinical data can significantly improve the model performance. Finally, the best performance of the Resnet101 model accuracy was 0.75, the recall rate was 0.84, the AUC was 0.803, and F1 was 0.782. In Table 4, we employed several common methods to optimize the model performance, including data augmentation techniques such as horizontal flipping, vertical flipping, 90° rotation, and image resizing. Furthermore, we utilized a pre-trained model based on ImageNet to initialize our model, which helped to accelerate the model’s convergence. The results show that not all optimizations are effective; data enhancement is a relatively general optimization scheme.

Table 3 Deep Learning Model Based on Imaging Data Alone and Its Performance Based on the Fusion of Imaging and Clinical Data

Table 4 Optimization of the Resnet101 Model

Discussion

Using gram-positive and gram-negative bacterial pneumonia data, we proved that the multi-modal fusion model based on X-ray images and non-image features is significantly better than image-only and clinical data-only models in predicting the types of bacterial pneumonia. When classifying based on image features, adding clinical information can further improve the performance of the model analysis. On average, the F1 and AUC improved by 5.6% and 10.2%, respectively. Therefore, our results show that the fusion of image data and clinical data can significantly improve the prediction performance of deep learning methods for automatic diagnosis. Conversely, owing to the limited amount of data in this study, the deep learning model offered a manageable advantage. However, the best Resnet101 model still performed as well as the best machine-learning model.^23–27 The optimization scheme of deep learning during image training cannot be directly used in the multi-modal decision fusion of clinical and image data. Data enhancement is a more suitable model optimization scheme. In this study, we focused on a binary classification problem, in which each image was classified as gram-positive or gram-negative. It should be noted that our current results are not particularly ideal, compared with traditional imaging, where it is difficult for radiologists to distinguish the types of bacterial pneumonia using only CXR, which may be related to the pathological manifestations of bacterial pneumonia.²⁸ This deep learning model can provide a reference for continuous optimization in the future.²⁹

Although computerized tomography has a better advantage in diagnosing pneumonia, CXR is the best choice for diagnosing pneumonia in children, given the child’s level of cooperation and radiation exposure.³⁰ The diagnosis of pneumonia is multi-modal. Doctors need to combine imaging findings with data on various clinical risk factors to make a diagnosis and use antibiotics empirically.³¹ Deep learning methods for the automatic classification of pneumonia usually use only image or non-image clinical data, and do not use both. Previous studies have classified childhood bacteria, viral pneumonia, and malaria based solely on clinical data, achieving an overall sensitivity and specificity for bacterial diagnosis of 96% and 86%, respectively.³² Some studies have also realized the classification of gram-positive, gram-negative, and atypical bacterial pneumonia based only on image data, with a sensitivity of 0.57 and specificity of 0.78.¹⁹ In this study, by integrating a deep learning algorithm based on imaging and clinical data, the classification accuracy of gram-negative pneumonia was 0.75, and the recall rate was 0.84. We also investigated the effect of reducing the partitioning of training sets on the classification performance of multi-modal models including clinical and imaging data. Specifically, we reduced the training set from an 8:2 partition to a 6:4 partition, resulting in a decrease in the number of training set samples. In the experiments, the data augmentation techniques were used to complement the training data. The imaging data consisted of images with a size of 256×256, and the base size was set to lr1e-4. We performed ablation experiments to assess the effect of reducing the training set size. Additionally, we assessed the accuracy of a CNN model containing images. The clinical data was used to build the model and compared the performance of the multi-modal model with that of a model constructed solely from clinical data using the catBoost algorithm. We found that reducing the partition of the training set from 8:2 to 6:4 led to a decrease in the AUC of the model’s classification performance. The catboost algorithm was more stable with a smaller amount of data (Supplementary Table 1). The accuracy of the CNN model containing images also decreased significantly (Supplementary Table 2). We observed that the model constructed from clinical data was more stable when the data changed compared with the deep learning model constructed from multi-modal data. This suggests that the classification performance of the multi-modal model mainly relies on image features. Consequently, subsequent research should focus on improving the importance of clinical information features in the model. Reducing the partition of the training set can have a significant impact on the classification performance of multi-modal models. The study also suggests that clinical information features should be given more importance in the development of multi-modal models. Future research should aim to improve the stability and accuracy of such models by incorporating clinical data in a more effective manner.

The current study has several limitations. First, only a single-center database was used. Second, the study population was from China, and patients from other countries were not considered. Third, we excluded patients with low immunity or immunodeficiency to avoid including patients with mixed infections. All patients were isolated and samples cultured to determine the pathogen species; however, the inclusion of potentially co-infected patients could not be entirely excluded. Randomized clinical trials are still needed to test the multi-modal classification algorithm model on a larger scale. Although our research has demonstrated the advantages of multi-modal information in convolutional neural networks, the limited availability of data remains a challenge for achieving optimal results with multi-modal models. In order to further enhance the performance of the model, we hope the following areas for future research: (1) training predictive models on a larger and more diverse dataset that includes both images and clinical data and (2) focusing on optimizing the model structure for clinical data and developing more effective methods for integrating information from multiple modalities.

Conclusion

The multi-modal decision of integrating deep learning-based image data into clinical data has good performance in the classification of pediatric bacterial pneumonia, which may preliminarily guide clinical empirical antibiotic therapy. This algorithm is expected to be useful as a clinical diagnostic assistant in the real world.