Multi-Clinical Factors Combined with an Artificial Intelligence Algorithm Diagnosis Model for HIV-Infected People with Bloodstream Infection

Introduction

With the advent of the era of highly active antiretroviral therapy (HAART), the disease process in patients with acquired immunodeficiency syndrome (AIDS) can be effectively suppressed.¹ However, opportunistic infections, mainly bloodstream infections (BSI), are still the leading cause of death.² BSI refers to a systemic infectious disease caused by invading various pathogens and toxins into the blood, causing damage to human organs such as heart valves and joints.³ It can lead to shock, multiple organ failure, and even death in severe cases.⁴ Even after an acceptable immune status is achieved, people with HIV remain vulnerable because their mortality falls to comparable levels with the general population only after 6 to 10 years of immune recovery and HIV-RNA suppression.⁵ Several factors predispose people with HIV to invasive bacterial and fungal infections. In particular, cell-mediated immune alterations, B cell dysfunction, consequent serum opsonin deficiency, and qualitative and quantitative defects in neutrophils HIV-Infected individuals are not entirely immune until CD4+T-cell counts increase to 750 cells/μL.⁶ Therefore, in a long immune course, if BSI is not detected and controlled in time, it will bring the risk of high mortality.

BSI leads to intensive care unit (ICU) admission in people with HIV more often than Pneumocystis jirovecii pneumonia.⁷ Non-typhoidal Salmonella, Streptococcus pneumoniae, Escherichia coli, and Staphylococcus aureus were the most important pathogens of BSI.^8,9 Fungal and mycobacterial infections are less common but have a considerable clinical and economic impact.¹⁰ Among pathogens causing BSI, Mycobacterium species, Cryptococcus neoformans, and recurrent nontyphoidal Salmonella constitute AIDS-defining conditions.¹¹ Presently, the diagnosis golden standard of AIDS-BSI in clinic is blood culture, but the early symptoms are generally relatively insidious, and clinical intervention can not be carried out at an earlier stage. In addition, the research on AIDS complicated with BSI mainly focuses on the distribution and drug resistance of pathogens, and there are few reports on the risk factors and diagnosis models of HIV-infected people with BSI.^12–14 Changes in many clinical indicators are associated with the occurrence and development of BSI in HIV-infected people. Studies have shown that the main risk factors for Streptococcus pneumoniae are male sex, intravenous drug use, smoking, detectable HIV-RNA, and low CD4+T cell count.¹⁵ Correlation analysis and risk factor analysis of clinical indicators of HIV-infected people with BSI can screen the risk factors reflecting the condition. By monitoring changes in routine clinical indicators of people living with AIDS, doctors can be alerted to the patient’s disease progression. Constructing a diagnosis model by risk factors can provide a new auxiliary diagnosis and treatment strategy for the clinical management of HIV-infected people with BSI.

This study retrospectively analyzed the data of 40 clinical factors of 498 (171 with BSI positive, 327 with BSI negative) HIV-infected people. In addition to exploring the distribution of bacteria in 171 people with BSI, the risk factors of BSI were also discussed. On this basis, various risk prediction models were established by combining artificial intelligence (AI) algorithms in the training set and validated in the testing set. Finally, we report an AI model with high sensitivity and specificity for diagnosing HIV-infected people with BSI formed by combining ten clinical risk factors, gender, age and an AI algorithm (svmRadial). We aim to explore the clinical value of the model for diagnosing HIV-infected people with BSI (Figure 1). This study may provide a basis for clinical auxiliary diagnosis and early empirical management.

Figure 1 Flowchart of the diagnosis model of HIV-infected people with BSI.

Materials and Methods

Sample Size Estimation

The sample size was calculated by MedCalc software. With a sample size ratio of 1:1 in the negative and positive groups and a power of 0.9, a minimum of 52 were required to achieve the expected performance (AUC=0.85). Considering the dropout rate of about 10%, the sample size of the cohort was finally set as follows: 327 cases in the negative group and 171 cases in the positive group.

Study Design and Participants

A total of 498 HIV-infected people admitted to Wenzhou Central Hospital from September 2014 to July 2021 were enrolled. According to whether they were complicated by BSI, they were divided into the BSI-positive group (n=171) and the BSI-negative group (n=327). This study used the hospital information management system to collect clinical data, such as gender, age, height, weight, smoking history and drinking history, total of 40 clinic factors. At the same time, the clinical characteristics, laboratory and imaging examination results and clinical diagnosis of the two groups were collected. The diagnostic results of all patients were in line with the diagnostic criteria of the Chinese Guidelines for the Diagnosis and Treatment of AIDS (2021 Edition), and the BSI diagnosis was in line with the diagnostic criteria of sepsis and bacteremia in Practical Internal Medicine (13th Edition).

This study retrospectively analyzed the data of 40 clinical factors of 498 HIV-infected people. In addition to exploring the distribution of bacteria in 171 people with BSI, the risk factors of BSI were also discussed. On this basis, various risk prediction models were established by combining 8 artificial intelligence (AI) algorithms in the training set and validating the diagnosis performance in the testing set. The model with the best diagnostic performance was selected as the final diagnostic model.

Ethical Consideration

All procedures performed in the study involving human participants were in accordance with the Declaration of Helsinki and the ethical standards of the Ethics Committee of Wenzhou Central Hospital. This study has been approved by the Ethics Committee of Wenzhou Central Hospital (L2022-02-009). We confirmed that all data was anonymized and maintained with confidentiality; therefore, the requirement for informed consent was waived due to the retrospective design.

Construction of AI Prediction Model

A 10-fold cross-validation strategy was used to split the dataset into training (n=399) and testing sets (n=99) for a ratio of 8:2. The training set was used to construct a binary model for predicting case grouping. Eight AI algorithms were used to construct detection models. The classification algorithms included “Average Neural Network (avNNet)”, “Stochastic Gradient Boosting (gbm)”, “Generalized Linear Model (glmnet)”, “Mixture and Flexible Discriminant Analysis (mda)”, “Penalized Logistic Regression (plr)”, “Support Vector Machines with Radial Basis Function Kernel (svmRadial)”, “NaiveBayesian Model (naive_bayes)” and “Random Forest (ranger)”. The prediction models of each algorithm were constructed, and the 5-fold cross-validation method was used to obtain the best model of each algorithm. According to the ROC curve, the optimal threshold was selected at the maximum youden index. The optimal cut-off was the threshold that maximizes the distance to the identity (diagonal) line. According to the optimal models created by each algorithm, the AUC, Kappa, accuracy, sensitivity and specificity were used to evaluate the performance of the models in the testing set, and the algorithm model with the most robust comprehensive performance was selected as the final model.

Statistics

SPSS 22.0 software was used for the statistical analysis of all data. The Kolmogorov-Smirnov normality test was used for continuous variables. A t-test was used to compare groups in measurement data with normal distribution. The Mann–Whitney U-test was used to perform group comparison in non-normal distribution data. Count data were expressed as percentages (%), and the χ² test was used to compare the two groups. Logistic regression analysis was used to screen significant clinical factors associated with BSI. ROC analysis was used to evaluate the prediction performance of the single risk factor. P < 0.05 was considered statistically significant.

Results

The Clinical Features of HIV-Infected People with BSI

A total of 498 HIV-infected people admitted to Wenzhou Central Hospital from September 2014 to July 2021 were selected and divided into the BSI-positive group (n=171) and BSI-negative group (n=327) according to whether they were complicated with BSI. The details of 498 individuals are as shown in Table 1. Among 171 BSI people, 87.13% (149/171) were males, and 12.87% (22/171) were females, with a average age of 44±15.009. Among the 327 people without BSI, there were 87.16% (285/327) males and 12.84% (42/327) females, with a average age of 41±14.339. There was no significant difference in age and gender between the two groups (P > 0.05).

Table 1 The Correlation of the Clinical Features and BSI of Individuals

Among 171 HIV-infected people with BSI, 62 cases were complicated with oral fungal infection, 46 cases with pulmonary disease, 37 cases with pneumocystis pneumonia, 35 subjects with cytomegalovirus infection, 29 cases with bacterial pneumonia, 17 people with tuberculosis, 16 cases with fungal pneumonia, 14 cases with cryptococcal pneumonia, 13 cases with cryptococcal meningitis, 12 cases with syphilis, and 12 cases with gastrointestinal hemorrhage. There were 8 cases of Epstein-Barr virus infection, 6 cases of intestinal infection, 5 cases of skin infection, 4 cases of herpes zoster, 2 cases of toxoplasma infection, 1 case of intestinal obstruction, 1 case of tuberculous meningitis, and 1 case of Kaposi’s sarcoma.

The Distribution of the Pathogenic Bacteria in HIV-Infected People with BSI

Among 171 HIV-infected people with BSI, 169 were positive for one kind of bacteria, and two patients were positive for all kinds of bacterium. A total of 173 strains of pathogenic bacteria were isolated and cultured. The detection rate of fungi was the highest (119 strains, 68.79%), mainly Cyanobacteria marneffei. Thirty-one strains (17.92%) of gram-positive bacteria were detected, primarily coagulase-negative Staphylococcus. Fourteen strains (8.09%) of gram-negative bacteria were detected, mainly Klebsiella pneumoniae subsp. pneumoniae. Nine strains of mycobacteria (5.20%) were detected, especially Mycobacterium tuberculosis, as shown in Table 2.

Table 2 The Distribution of Pathogenic Bacteria in HIV-Infected People with BSI (N=171)

The Risk Factor Analysis of HIV-Infected People with BSI

The univariate analysis was conducted among 40 clinical factors, as shown in Table 1. The results showed that 25 factors had significant differences between BSI negative group and BSI positive group (P < 0.05). The multivariate analysis was conducted among 25 clinical factors. The results showed ten factors of low level of CD4+T cell, hemoglobin (HB) and platelets (PLT), high level of blood urea nitrogen (BUN) and lactate dehydrogenase (LDH), without ART treatment, splenomegaly, shock, nodular shadow, and strip shadow are the independent risk factors for AIDS with BSI (P < 0.05), as shown in Table 3. The people without ART treatment had a higher risk of AIDS with BSI than those who received ART, and the odds ratio (OR) reached 3.63 (95% CI: 2.345–5.626). Compared with people without shock, people with shock had a higher risk of living with HIV and BSI, and the OR reached 3.846 (95% CI: 1.211–12.217). People with the characteristics of computed tomography (CT) features of stripe shadow and nodular shadow are more likely to develop HIV-infected with BSI. The OR was 2.173 (95% CI: 1.242–3.802) and 3.924 (95% CI: 2.176–7.074), respectively.

Table 3 Multivariate Analysis of BSI in HIV-Infected People

The ROC Analysis of HIV-Infected People with BSI

To evaluate the ability of these ten independent risk factors to diagnose HIV-infected people with BSI, ROC analysis was performed on these ten risk factors. The results showed that the AUCs of the ten factors were all larger than 0.5, but only HB and CD4+T were larger than 0.7, as shown in Table 4. This indicates that these nine factors’ diagnostic power is insufficient to diagnose HIV-infected people with BSI accurately.

Table 4 The ROC Analysis of the Risk Factors of HIV-Infected People with BSI

The Construction of the AI Prediction Model for HIV-Infected People with BSI

To make up for the lack of diagnostic ability of the single risk factor, a multi-factor diagnostic model was constructed by combining 12 factors (ten risk factors, age and gender) with the AI algorithm. The people were divided into a training set (n=399) and a testing set (n=99) for the proportion of 8:2. The distribution of the people in the training set and testing set showed that the differences were not significant (P>0.05), as shown in Table 5. According to the optimal model constructed by each algorithm, the ROC analysis method is used to evaluate the performance of the model in the testing set, and the algorithm model with most significant AUC is selected as the final model. The ROC curves of the optimal model constructed by the eight algorithms in the training and testing set are shown in Figure 2A and B. The results show the model built by the “svmRadial” algorithm has the highest AUC of 0.916 in the testing set, and the model is selected as the final HIV-infected people with BSI diagnosis model (Figure 2C). The ROC curve was drawn with the predicted value in the testing set, and the best diagnostic cut-off value was set to 0.376 according to the Youden-index. When the predictive value of the diagnostic model is less than 0.376, it is considered that the patient is not BSI. When the model’s predictive value is more than 0.376, it is regarded as the AIDS patient with BSI, and the evaluation indicators to obtain the diagnosis efficiency of the model are shown in Figure 2D. The results show the accuracy, sensitivity, and specificity of the HIV-infected people with BSI diagnosis model are 0.844, 0.824, and 0.855, respectively.

Table 5 The Baseline Information of Training Set and Validation Set

Figure 2 Evaluation of the diagnosis power of the 12 factors combined AI Algorithm in HIV-infected people with BSI. (A). ROC curves of the 12-factor-based optimal models constructed by the eight algorithms in the training and (B). Testing datasets. (C). ROC curves of the 12-factor-based “svmRadial” model in testing datasets, AUC=0.916. (D). The evaluation indicators of the 12-factor-based“svmRadial”model diagnosis efficacy.

Discussion

BSI is a common and severe complication of HIV-infected people. According to research statistics, the incidence of BSI in people with advanced AIDS is about 20%, and the mortality rate has reached 16%.¹⁶ HIV-infected people with BSI often have atypical clinical manifestations and a long laboratory blood culture cycle, making it difficult to diagnose early, delaying treatment, and increasing the risk of death. This study analyzed the clinical characteristics and pathogen distribution characteristics of HIV-infected people with BSI, further explored the risk factors of BSI, and constructed a diagnosis model with high sensitivity and specificity to provide a reference for early clinical diagnosis and treatment.

The study results showed that HIV-infected people with BSI were often complicated with multiple organ involvement, among which the lung was the most common, followed by the oral cavity. A total of 126 (73.68%) people were complicated with infectious pulmonary lesions (including bacterial pneumonia, fungal pneumonia, tuberculosis and pneumocystis pneumonia), and 62 (36.26%) people were complicated with oral fungal infection. The digestive tract, meninges, skin, and soft tissues are less complicated by infection. Still, they are also more commonly affected organs, similar to previous reports’ research results.¹⁷ More than half of the people had a high fever, cough and sputum, fatigue and poor appetite, lymphadenopathy, anemia and hypoproteinemia, suggesting that the clinical characteristics of HIV-infected people with BSI are complex and diverse but lack specificity. Therefore, when HIV-infected people have multi-system and multi-organ involvement, even if the clinical symptoms are atypical, it is necessary to be alert to the possibility of BSI, and the blood culture should be sent in time to confirm the diagnosis results.

The pathogens of HIV-infected people with BSI are diverse, and the species of pathogens in different regions are different. Jemal et al found that BSI in HIV-infected people is mainly caused by Staphylococcus aureus and Klebsiella pneumoniae in northern Ethiopia.¹⁸ Franceschini et al showed that Enterobacteriaceae, coagulase-negative Staphylococcus, and Staphylococcus aureus were the primary pathogens in the Modena region of Italy, which was different from the results of this study.¹⁹ In this study, the detection rate of pathogenic bacteria from high to low was fungi, Gram-positive bacteria, Gram-negative bacteria, and mycobacteria. Stratified analysis showed that the primary pathogens were Talaromyces marneffei, coagulase-negative Staphylococcus, Klebsiella pneumoniae subsp., and Mycobacterium tuberculosis. Fungi are the primary pathogens of HIV-infected people with BSI in this area. Wenzhou is a southern coastal city, and the warm and humid climate is conducive to the growth of Talaromyces marneffei and the exposure and release of its spores. It is suggested that clinical attention should be paid to the prevalence of pathogens in HIV-infected people with BSI. If fungi are the primary pathogens, early empirical treatment may consider covering fungi appropriately. Coagulase-negative Staphylococcus and Klebsiella pneumoniae subspecies pneumoniae are common opportunistic pathogens identified in the human body’s skin, upper respiratory tract, and intestinal tract. HIV-infected people with low immune function and some injection drug use and interventional examination are also present. Pathogens can enter the body through the skin, respiratory tract, nosocomial infection, and other ways, increasing the probability of BSI in HIV-infected people. In this study, nine strains of mycobacteria were detected, of which eight strains were Mycobacterium tuberculosis, slightly lower than the conclusion of studies in other areas of China.²⁰ It is considered that the culture cycle of mycobacteria is long, the identification is complex, and there may be missed detection. It is reminded that the laboratory should standardize the identification process to improve the detection rate.

This study found that low levels of CD4+T lymphocyte, HB and PLT, high level of BUN and LDH, without ART treatment, splenomegaly, shock, and CT features of nodular shadow and strip shadow were independent risk factors for BSI. Cheng et al confirmed that CD4+T lymphocyte count was significantly correlated with the occurrence of BSI, which was consistent with the conclusion of this study.²¹ CD4+T lymphocyte count represents the level of immune function of people. The lower the CD4+T lymphocyte count, the worse the ability of the body to limit the infection of pathogenic bacteria, which makes it easier for pathogenic bacteria to spread to surrounding tissues and invade the bloodstream and cause BSI. In the process of disease progression in HIV-infected people, HIV infection has an inhibitory effect on bone marrow hematopoietic function, which can cause a decrease of hemoglobin and platelet, resulting in a further decline of immune function and, thus, more susceptibility to BSI.²² At the same time, when BSI developed, pathogenic bacteria can invade bone marrow and blood vessels, reduce erythropoiesis and increase destruction, cause vascular endothelial damage, promote platelet aggregation, and form micro-thrombus, and then cause a more significant decrease in hemoglobin and platelet. It is suggested that clinicians should pay close attention to the dynamic changes of hemoglobin and platelet in people with suspected BSI for auxiliary diagnosis. Several studies have shown that the incidence of BSI is significantly reduced in people initiating antiviral therapy, and this study also confirmed that the initiation of antiviral treatment can reduce the incidence of BSI.²³ Effective antiviral therapy can inhibit the replication of HIV, increase the level of CD4+T lymphocytes, and promote the reconstruction of immune function. Undoubtedly, the recovery of immune function can reduce the risk of BSI. Therefore, antiretroviral therapy should be started as soon as possible for eligible HIV-infected people to reduce the incidence of BSI. Research results showed that people with BSI had a significantly higher proportion of splenomegaly than people without BSI, and Dong et al also drew a similar conclusion.²⁴ When BSI pathogens invade the bone marrow and cause hematopoietic disorders, the spleen may be hyperplastic and enlarged. Pathogens can trigger the body’s immune response, and part of the antibodies produced by the body comes from the spleen. When stimulated by antigens, the germinal center can be enlarged, the phagocytic function can be enhanced, and the spleen may be enlarged.²⁵

More and more models have been applied to the early diagnosis of the disease.²⁶ Still, there are few reports on AI diagnostic models based on multiple clinical indicators for AIDS with BSI. Combining multiple clinical indicators and AI algorithms can compensate for a single biomarker’s poor diagnostic performance and comprehensively analyze the patient’s situation from various dimensions. This study established an AI diagnostic model based on 12 factors of HIV-infected people with BSI. At the same time, the risk factors screened are routine clinical indicators with more practical application value. In this study, the combination of the 12 factors and the “svmRadial” model can identify the HIV-infected people with BSI from the HIV-infected people without BSI with an AUC of 0.916 and a sensitivity and specificity of 0.824 and 0.855, respectively. The model showed excellent performance in the diagnosis of HIV-infected people with BSI. Internal and external validation showed that the prediction model had high clinical application value. However, this study is a retrospective, single-center study with a limited number of cases and inevitable case selection bias. Further prospective and multi-center studies are needed to provide more accurate diagnosis models for clinical practice.

Conclusion

The clinical characteristics of HIV-infected people with BSI are atypical, and the pathogens in this area are mainly fungi. The multi-factor-based AI model established in this study has reliable diagnosis value, which can provide a reference for early clinical diagnosis and treatment. This study may provide a basis for clinical auxiliary diagnosis and early empirical management.