Development and Validation of a Diagnostic Nomogram for Pneumocystis jirovecii Pneumonia in Non-HIV-Infected Pneumonia Patients Undergoing Oral Glucocorticoid Treatment

Introduction

Pneumocystis jirovecii pneumonia (PJP) is one of the most common opportunistic infection with non-specific symptoms.¹ PJP commonly affects HIV-positive patient populations, but it is still an important cause of pneumonia in non-HIV immunocompromised patients.² The most significant risk factors for PJP are glucocorticoid therapy³ and cell-mediated immune deficiency,^4,5 and long-term use of high doses of glucocorticoids has been proven to cause severe immunosuppression and severe infection.⁶ Additionally, existing data suggest that 80% of PJP patients have previously received glucocorticoids.³ Several studies have indicated that PJP in HIV-free patients characterized by fulminant course has a prognosis of higher inflammatory response intensity responses and mortality rate (about 30–60%) than in HIV patients.^7–9 Given the high mortality of PJP patients in HIV-negative populations, there is a need to develop effective predictive tools for the diagnosis of PJP in patients with pneumonia treated with glucocorticoids.

The main clinical manifestations of PJP are low-grade fever, progressive dyspnea, and dry cough. The more severe signs are butterfly pattern of bilateral perihilar opacity on chest radiography and bilateral ground glass opacities on high-resolution computed tomography (HRCT).¹⁰ There is no specific serological marker for PJP. Elevated lactate dehydrogenase (LDH) reflects the extent of lung injury and is therefore informative in determining disease severity, but not etiology.¹¹ The measurement of (1,3)-β-d-glucan (BG) can also be helpful in diagnosis of PJP. The BG level of patients with PJP infection is higher than that of patients with Pneumocystis colonization, which is a useful marker for discriminating between PJP and Pneumocystis colonization.¹² However, BG is a cell wall component of many fungal organisms and is not specific for PJP.⁵ Moreover, the pathogen causing PJP is difficult to isolate and not amenable to culture. Thus, the typical diagnostic approaches that rely on visualization of the organism in respiratory samples¹³ is challenging in HIV-negative patients with low pulmonary fungal load.^8,14 Modern detection methods are based on PCR, and conventional PCR, real-time PCR assays, and next-generation sequencing technology can detect the DNA of Pneumocystis jirovecii in various samples and provide rapid diagnoses of PJP.¹⁵ However, due to the threshold has not been established, it is difficult to distinguish PJP from Pneumocystis colonization even by real-time PCR.⁵ These molecular techniques have not been sufficiently standardized⁸ and are not available in most hospitals in China. In addition, the collection of pulmonary specimens requires invasive techniques and may be associated with a risk of complications, especially in patients with respiratory problems.¹⁶ Substantial delay in PJP treatment is associated with high mortality, indicating the importance of rapid and reliable diagnostic tools for PJP in HIV-negative patients undergoing oral glucocorticoid treatment.

Currently, a prediction model has been established to calculate the probability of PJP. However, this prediction model is limited to hematology patients with acute respiratory failure.¹⁷ Nomogram is a graphical model that enables the user to calculate the overall probability of a particular clinical outcome for an individual patient.¹⁸ Herein, we developed and validated a nomogram based on clinical data for the timely diagnosis of PJP in patients receiving oral glucocorticoids. This study may contribute to the timely diagnosis and prevention of PJP, thereby reducing the mortality of PJP patients.

Materials and Methods

Study Design and Patients

We retrospectively and consecutively recruited pneumonia patients who were hospitalized between January 1, 2013, to December 31, 2019, at 6 secondary or tertiary academic hospitals in China (China–Japan Friendship Hospital, Shijiazhuang People’s Hospital, The Second Hospital of Hebei Medical University, Beijing Luhe Hospital, Weifang NO.2 People’s Hospital and Tianjin First Central Hospital) to construct the development dataset. We selected another group of pneumonia patients who were hospitalized in the intensive care units (ICU, including respiratory ICU, emergency ICU, and surgical ICU) of the Sichuan Provincial People’s Hospital during 2019 as the validation group. The diagnosis of pneumonia was based on the guidelines of The American Thoracic Society and Infectious Diseases Society of America.¹⁹ Information related to underlying diseases, such as nephritic syndrome or connective tissue diseases, were extracted from the patients’ electronic medical records.

Subjects were selected based on the following inclusion criteria: 1) currently on oral prednisone >10 mg/d or equivalent dose ≥3 weeks; 2) pneumonia was diagnosed with community-acquired pneumonia on admission; 3) age above 16 years. The exclusion criteria were as follows: 1) no computerized tomography (CT) scan in the defined period (7 days before admission or in the first 3 days of hospitalization), no routine blood examination, or no blood gas analysis in the first 3 days of hospitalization; 2) total duration of oral glucocorticoid treatment was unclear; 3) inability to provide consent for procedures; 4) patients with severe heart failure; 5) patients with HIV; 6) patients diagnosed with hospital-acquired pneumonia; 7) patients without pathogen tests during hospitalization; 8) patients who had been repeatedly hospitalized in the previous 3 months; 9) patients had received PJP prophylaxis. A team of experienced physicians reviewed and extracted these data. Two physicians checked all data independently. The physicians in charge of clinical characteristics’ extraction were blinded to microbiological results.

Ethics Statement

The present study was approved by the Ethics Committee of the Sichuan Provincial People’s Hospital (20210392) and the Ethics Committee of China-Japanese Friendship Hospital (no. 2015-86). Due to the use of de-identified and anonymized research data, requirements for informed written consent were waived. The study was conducted in accordance with the Declaration of Helsinki.

Data Collection and Processing

Based on previous research, a total of 50 potential predictive variables, including demographics, symptoms, lab results, chest CT scan features and medication history were collected.^17,20–24 The cutoff of continuous duration for glucocorticoid therapy (no less than 1 year) was selected based on previous research.^22,25 Although indicators such as β-d-glucan and KL-6 may be helpful in the diagnosis of PJP,^26,27 they were not included because they are expensive to measure and these data were incomplete in this retrospective study. Variables such as demographics, clinical symptoms, and the medication history were extracted from patient’s admission records. Laboratory findings on admission or in the first 3 days of hospitalization were recorded. Reports of CT scans taken 7 days before admission or in the first 3 days of hospitalization were also collected (Tables 1 and 2).

Table 1 Demographics and Clinical Characteristics Among Patients in the Development Dataset

Table 2 Laboratory, Radiology Findings and Previous Treatment Among Patients in the Development Group

A definitive diagnosis of PJP was made by demonstrating the existence of pathogenic organisms in lung tissue or respiratory tract secretions during hospitalization.²⁸ Patients without such evidence but considered “PJP-probable” were excluded. Bronchoalveolar lavage fluid (BALF) or sputum samples were collected for etiology identification. Methods of pathogen detection in the development group were performed according to Li’s research²⁵ as follows: Bronchoscopic examinations were performed and BALF or sputum samples were collected in non-HIV-infected pneumonia patients, BALF specimens were aliquoted and immediately transported to laboratories. Bacterial cultures were incubated at 35°C in 5–10% carbon dioxide for 48 h. If Nocardia was suspected, the incubation time was prolonged. Fungal cultures were incubated at 27°C for 5 days under ambient conditions. Species were identified using matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (Brooks Instrument, Germany) or a BACTEC 9102 culture instrument (BD Biosciences, USA). Respiratory viral and atypical pathogens were detected by PCRs (Shanghai Zhijiang Biological Technology, China). The Platelia Aspergillus test was used for galactomannan detection (Bio-Rad Laboratories, Marnes-la-Coquette, France). In the validation group, since the Sichuan Provincial People’s Hospital did not have P. JiroVECII PCR test during the entire study period, patients whose BALF or induced sputum had been analyzed for pathogen detection by metagenomic next-generation sequencing (mNGS) were recruited, and these samples were sequenced on a MGISEQ-2000 platform.

Statistical Analysis

Continuous variables and categorical variables were presented as mean ± SD and frequency (%), respectively. Variables with more than 40% of missing data were excluded. Age and lymphocyte count were kept as continuous variables in the logistic regression model by default to prevent over fitting in our relatively small training samples.²⁹ LASSO regression was used for the selection of potentially significant variables in our study.³⁰ The coefficient of LASSO modeling was determined by 10-fold cross-validation, and lambda was selected by 1-SE criterion. The variables identified by LASSO regression analysis were all placed into a binary logistic regression to construct a diagnostic model. Based on the logistic regression analysis, a nomogram was constructed to provide an estimate of the probability of PJP in the development group. Internal validation was performed by a bootstrap method with 500 re-sampling. The discrimination performance of the nomogram was assessed by AUC. The threshold for significance was a P value <0.05. The max Youden index was employed to determine the optimal cut-off point. The calibration curve plotted the predicted probability versus the actual probability, while the goodness-of-fit was assessed by Hosmer–Lemeshow test (P > 0.05 indicated good fitness). The clinical utility of this model for decision-making was assessed by decision curve analysis (DCA). DCA plotted net benefit (NB) at whole range of probability thresholds.³¹ All analyses were performed using Empower Stats software (R) (www.empowerstats.com, X& Y Solutions, Inc. Boston, MA), R software (version 3.2.0), and SPSS 22.0 software (IBM Corp., Armonk, NY).

Results

Characteristics of the Dataset

A total of 553 cases were eligible for analysis, of which 434 patients were assigned to the development group, and 119 patients were assigned to the validation group. In the development group, 110 patients were diagnosed with PJP by PCR tests, and in the validation group, 40 patients were diagnosed with PJP by mNGS tests on BALF. The prevalence of PJP was 25.3% in the development group and 33.6% in the validation group (Figure 1). Tables 1 and 2 present the demographic and clinical characteristics, laboratory results, radiology findings and previous treatment of patients in the development group. The participants had a mean age of 57.28 years (± 14.78), and approximately 51% were male. In the development group, the most common concomitant symptoms were fever (79.26%), followed by dyspnea (64.29%), respiratory failure (51.61%), and Dry cough (19.35%). The most frequent underlying disease was interstitial lung disease (ILD, 47.47%), followed by hypertension (33.18%), and diabetes mellitus (23.73%). Ground glass opacity (GGO, 63.36%) was the most common chest CT abnormity, followed by patchy shadowing (47.00%) and consolidation (46.31%). The mean duration of glucocorticoid therapy was 21.81 (0.70–480.00) months. Supplementary Table 1 shows the pathogen test results of the patients in the development group, and 59.4% (258/434) of the cases were infected with at least one of these pathogens. Regardless of other co-infections, the most common pathogen in all patients was cytomegalovirus, with a prevalence of 20.5%.

Figure 1 Study flowchart.

Diagnostic Indicator Selection

A total of 50 variables were included in the LASSO regression. By using LASSO regression, 8 indicators of PJP were selected (Supplementary Figure 1). As shown in Table 3, these indicators included age, fever, dry cough, respiratory failure, lymphocyte count, GGO on CT, whether patients received oral glucocorticoids treatment for more than 12 months, and whether immunosuppressive agents were taken or not. These indicators were all included in the nomogram. Although lymphocyte count was not statistically significant (P > 0.05), it remained in the final model according to the principle that the “best” model is selected based on clinical importance²¹ and the smallest Akaike information criterion (AIC) value.³² Supplementary Table 2 shows the characteristics of the clinical variables that were ultimately included in the model for patients in the validation group. Supplementary Table 3 shows the comparison of clinical characteristics between the development and validation groups.

Table 3 Binary Logistic Regression Model Using the Development Dataset for Predicting PJP

Diagnostic Nomogram for the Probability of PJP

Based on the final regression analysis, a nomogram for predicting the probability of PJP was constructed (Figure 2). The value of each variable was given a score on the point scale axis. The total score was readily calculated by adding each individual score and then referencing the total score to the lower total subscale, in which the probability of PJP is estimated.

Figure 2 Diagnostic nomogram for predicting PJP in non-HIV-infected pneumonia patients receiving oral glucocorticoids. The value of each variable was given a score on the dot scale axis. The total score was easily calculated by summing each individual score and then referencing the total score to the lower total score table, where the probability of PJP is estimated.

Performance of the Nomogram

The nomogram showed robust discrimination, with an AUC of 0.82 (95% CI 0.77–0.86) in the development group and an AUC of 0.87 (95% CI 0.80–0.94) in the validation group, both showing acceptable calibration (Figure 3A and C). As shown in Figure 3B, the internal verification AUC of the development group was 0.811. Calibration plots are shown in Figure 3D and E. The Brier score was 0.15 in the development group and 0.14 in the validation group. The Hosmer–Lemeshow test was not significant (P > 0.05), demonstrating a favourable integrating degree. The prediction accuracies of our model varied according to different probability thresholds which used to define a positive result (Supplementary Table 4). The optimum cutoff of probability threshold was 30% based on maximum Youden’s index. The Decision curve analysis demonstrated that the nomogram was clinically useful. The corresponding net benefit was about 0.45 when the cutoff of probability threshold was 30% in the development group (Figure 4).

Figure 3 ROC curves and calibration plots. (A–C) represent the ROC curves in the development group, the 500 times bootstrap in the development group, and the external validation group, respectively; (D and E) represent the calibration plots showing the relationship between the predicted probabilities based on the nomogram (X-axis) and the actual values (Y-axis) in the development group (D), and in the validation group (E).

Figure 4 Decision curve analysis for the diagnostic nomogram in the development group. The y-axis represents the net benefit, and the X-axis represents the threshold probability. In our study, the threshold probability of the decision curve is 30% and the corresponding net benefit is 0.45. It indicates that the nomogram improves the benefit compared with the measures that treat all patients and treat none patient when threshold probability is >30%.

Discussion

The corticosteroids (inhaled and oral) are commonly prescribed in the management of asthma and COPD patients, long-term use of glucocorticoids at high doses may result in severe immunosuppression and serious infections.²⁵ In recent years, PJP has become more frequent in non-HIV immunosuppressed populations, and glucocorticoid therapy increased the incidence of PJP in this population Guidelines regarding PJP prophylaxis for hematopoietic cell transplantation recipients and solid organ transplantation have been developed.^33,34 However, PJP prophylaxis for patients treated with glucocorticoids remains controversial.³⁵ Recent studies have found an unexpectedly high proportion of PJP pulmonary samples is observed in critically ill COVID-19 patients.³⁶ An important risk factor for PJP development in COVID-19 patients is the use of corticosteroids for asthma exacerbations and/or the presence of COVID-19.³⁷ Considering the high mortality of the disease and potential adverse effects of prophylactic drugs, timely and accurate diagnosis is crucial for PJP patients.³⁸ The nomogram is an easy-to-use graphical model that enables users to calculate the overall probability of a specific clinical outcome for an individual patient.³⁹ In the present study, we developed and validated a diagnostic nomogram that may provide a useful, convenient and applicable tool to assess the probability of PJP in HIV-negative pneumonia patients being treated with oral glucocorticoids. There were significant differences in some clinical indicators (such as age) between the two groups. However, the prediction effect of the model is still well, which indicates that the model has preferable stability. Moreover, decision curve analysis showed that our model consistently achieved greater net benefits across almost all clinical thresholds. In hospitals where molecular diagnosis is not available, physicians can use nomogram to predict the probability of PJP infection in non-HIV patients with pneumonia treated with glucocorticoids, thereby preventing or treating PJP as early as possible. For a population in which PJP prevalence is about 25%, this PJP diagnostic nomogram appeared to be a reliable screening tool, ruling out PJP with a negative predictive value of 95.24% for a probability threshold of 10%, and the positive predictive value was 100% for a probability threshold of 90%. Thus, this diagnostic nomogram can aid in PJP decision-making, regardless of the presence of other co-infections.

PJP patients without HIV infection were typically presented with fulminant respiratory failure, fever, and dry cough,^40,41 and these traditional indicators were included in this model. The typical CT scans of PJP demonstrate extensive ground-glass opacities, especially in the upper lobes. Unusual radiographic patters include cysts, solitary or multiple nodules, which are less characteristic than GGO.⁴² Absolute peripheral lymphopenia and application of immunosuppressive agents are powerful predictors of PJP in most clinical settings.^21,35,43 In this study, lymphocyte count and ground-glass opacity (GGO) were used as predictors of PJP by LASSO analysis, the principle that the “best” model is selected based on clinical importance²¹ and the smallest AIC value.³² Age was also as a predictor of PJP, and our data showed that PJP was less likely to occur in elderly patients, which was consistent with previous study.¹⁷ This may be due to the fact that older patients rarely receive high-dose chemotherapy or immunosuppressants, as well as treatments that put them at high risk for PJP, such as organ transplantation. Yet, a study involving rheumatoid arthritis patients receiving TNF-ɑ inhibitors indicated that older patients would more likely to developed PJP.⁴⁴ This difference may be because only a few patients were treated with infliximab in our study. Therefore, caution is warranted when applying the nomogram to elderly patients. The median duration of glucocorticoid therapy varies from 2 to 24 months before the development of PJP.^43,45 Many studies have indicated that PJP tends to occur within 1 year of glucocorticoid therapy,^22,43,46 and consistent results were also obtained in this study. Data regarding the association between risk of PJP and threshold daily dose, cumulative dose, pulse treatment, and rapid reduction of glucocorticoids are conflicting.^21,35,43,47 For example, though patients with SLE, sarcoidosis, or giant cell arteritis typically require higher glucocorticoid dosing and longer medication duration, the incidence of PJP in these patients remains low, while patients suffering from granulomatosis with polyangiitis, patients with microscopic polyangiitis, or those with dermatomyositis seem more susceptible to PJP.²³ To generalize the application of our model, we did not put underlie diseases into the diagnostic model. More research is needed to clarify the association between glucocorticoid therapy and the development of PJP in different clinical settings.

Our research had several limitations. First, in the validation group, PJP cases were confirmed by mNGS of BAL, while in the development group, PJP patients were diagnosed by microscopic examination of respiratory samples demonstrating pneumocystis cystic or trophic forms or pneumocystis DNA identified using PCR. Different laboratory methods might cause measurement bias. Secondly, our research was retrospective, the demographics, clinical symptoms, and medication history were extracted from patient’s admission records. Retrospective studies may be biased in terms of the data collected, such as physical examination data and the normal ranges in lab tests. Furthermore, the medication history such as the duration of glucocorticoid treatment may come from the recollection of patients, and thus subject to recall bias. Thirdly, whether the patients had taken PJP prophylactic treatments had not been recorded, nor the location of “GGO”. This lack of information might have affected the accuracy of our research. Furthermore, the detection methods of PJP are based on PCR and have higher utility in non-HIV patients with less sensitive staining. However, PCR cannot distinguish between colonization and infection. Viral pneumonia may share similar clinical characteristics with PJP, and both viral pneumonia and Pneumocystis jirovecii colonization are common in glucocorticoid users.^13,25 This co-infection rate may result in some viral pneumonia patients with Pneumocystis jirovecii colonization in the respiratory track being misdiagnosed as PJP by conventional PCR tests, since these tests can detect low levels of Pneumocystis jirovecii DNA but cannot differentiate active pneumonia from colonization. In the future, quantitative PCR, which can help to differentiate infection from colonization, may help improve the diagnostic nomogram. Finally, the performance of the nomogram may vary with PJP prevalence. Generally, the final diagnosis of PJP is based on the clinical data and imaging results of the risk patients. In view of the limitations of this study, our nomogram is a useful supporting tool in decision-making, not a diagnostic procedure.

Conclusion

Overall, our nomogram provided a useful tool to predict the probability of PJP in HIV-negative pneumonia patients undergoing oral glucocorticoid treatment with good discrimination and calibration. This nomogram may help doctors avoid delays in treating Pneumocystis jirovecii in patients with high scores and avoid potential prophylaxis toxicity in patients with low scores. Clinical benefits of using this nomogram deserve more assessment in the future.