A Paired Comparison of Plasma and Bronchoalveolar Lavage Fluid for Metagenomic Next-Generation Sequencing in Critically Ill Patients with Suspected Severe Pneumonia

Introduction

Severe pneumonia is one of the most common causes of infectious diseases among patients in the intensive care unit (ICU), and this can lead to various complications and high mortality.^1–3 Timely and accurate pathogen diagnoses are crucial for appropriate antimicrobial therapy and improved clinical outcomes. However, the low detection rate and time-consuming process (3–5 days) of conventional methods, including culture and serological tests, are the problems faced by clinicians.⁴ Pathogens have been detected in only 38% of patients with radiographic evidence of pneumonia.¹ In a study reported by Leven et al,⁵ even the combination of culture-based identification, targeted molecular methods, and serology tests, yielded the potential pathogen detection rate of only 59% in patients with lower respiratory tract (LRT) infection. For critically ill patients with severe pneumonia, the complexities of illness and prior antibiotic exposure have further complicated the effective detection of pathogens.

Metagenomic next-generation sequencing (mNGS) is a potential diagnostic tool for severe pneumonia^6–9 because of its broad range of detection and a shorter turn around time than culture methods.¹⁰ Invasive clinical specimens, such as LRT aspirates, bronchoalveolar lavage fluid (BALF), bronchial needle brushing, and even lung biopsy tissues,^7,11–13 are generally necessary to perform mNGS for patients with severe pneumonia. In recent years, plasma mNGS has been developed to diagnose a wide range of clinical infectious diseases^14–17 and is expected to be useful for predicting impending bloodstream infections to guide pre-emptive therapy.¹⁸ Unlike BALF, the plasma sample is more accessible and hence, the procedure is less invasive. However, clinical studies investigating the performance of plasma mNGS in patients with severe pneumonia are limited and controversial.^19–23 Studies have reported that plasma mNGS has moderate sensitivity and high specificity for invasive mold infections in pulmonary patients²³ and is highly consistent with BALF mNGS in patients with severe pneumonia.¹⁹ Yang et al²⁴ also found a novel link between plasma mNGS and systemic inflammation in patients with severe pneumonia. However, two other studies suggested that BALF mNGS is more sensitive than plasma mNGS in patients with pneumonia, and concordance between paired BALF and plasma was not sufficient.^21,22 Notably, for critically ill patients with pneumonia, antibiotic administration before sampling is an unavoidable and complex bias for the mNGS assessment. One study suggested that although plasma is a poorer indicator than the respiratory sample, plasma mNGS could be an alternative for detecting pneumonia pathogens when respiratory specimens are unavailable.²⁵ The necessity of simultaneously testing both BALF and blood samples thus remains unclear.

This study aimed to evaluate the clinical diagnostic utility of simultaneous plasma and BALF samples using mNGS in patients with severe pneumonia and determine the difference between the two samples for the mNGS test in clinical practice.

Materials and Methods

Study Design and Specimen Processing

All patients with suspected severe pneumonia admitted to the medical ICU at China–Japan Friendship Hospital from April 2019 to December 2020 were retrospectively investigated. Patients with suspected pneumonia, including community-acquired pneumonia²⁶ and hospital-acquired pneumonia,²⁷ were enrolled after meeting the following inclusion criteria: older than 18 years, meeting the severe pneumonia diagnostic criteria of the Infectious Diseases Society of America/American Thoracic Society consensus guidelines,²⁸ and with available BALF and plasma mNGS results. Patients were excluded if the BALF and plasma were not acquired simultaneously for the mNGS or the patient had been discharged before the mNGS results were obtained. Septic shock²⁹ and immunosuppression were defined according to a previous study.³⁰ For all enrolled patients, demographic and clinical data were collected, including comorbidities, laboratory test results, microbiological testing, days of hospital/ICU stay at the time of specimen acquisition, Median Acute Physiology and Chronic Health Evaluation (APACHE) II score, Sequential Organ Failure Assessment (SOFA) score, ICU treatment data, and patient outcomes.

This study was approved by the Institutional Review Board of the China–-Japan Friendship Hospital (2019–180-K121-1) and performed in accordance with the Declaration of Helsinki. The need for informed consent was waived due to the retrospective nature of the study and because the data were anonymously analyzed.

Conventional Microbiological Tests

The conventional microbiological tests (CMTs) included bacterial or fungal smear and culture, acid-fast stain, and real-time polymerase chain reaction (PCR) to detect cytomegalovirus (CMV), Epstein–Barr virus (EBV), influenza virus, respiratory syncytial virus, Pneumocystis jirovecii, Legionella, Mycoplasma, and Chlamydia. PCR was performed at the clinician’s discretion. Previous studies have reported that the CMTs of BALF samples (smear, culture, and PCR) were consistent with the standard clinical procedure.^31,32

mNGS

DNA sequencing of both the BALF and plasma samples was conducted for each patient, and RNA sequencing of BALF libraries was conducted only for patients with a suspected viral infection. BALF DNA was extracted from all samples using the QIAamp^® UCP Pathogen DNA Kit (Qiagen) following the manufacturer’s instructions. Human DNA was removed using Benzonase (Qiagen) and Tween 20 (Sigma). BALF total RNA was extracted using the QIAamp^® Viral RNA Kit (Qiagen), and ribosomal RNA was removed using the Ribo-Zero rRNA Removal Kit (Illumina, San Diego, CA, USA). Cell-free DNA in plasma was extracted using the QIAamp^® Circulating Nucleic Acid Kit (Qiagen). Extracted DNA was used to generate libraries using the Nextera XT kit (Illumina) according to the manufacturer’s instructions. The library was sequenced on an Illumina NextSeq 550 sequencer using a 75 bp sequencing strategy. For negative controls, we also prepared peripheral blood mononuclear cell samples with 10⁵ cells/mL from healthy donors in parallel with each batch using the same protocol, and sterile deionized water was extracted alongside the specimens to serve as non-template controls.

Fastp was used to remove low-quality reads, adapter contamination, duplicate reads, and those shorter than 50 bp. Low-complexity reads were removed using Kcomplexity with default parameters.³³ The human sequence data were identified and excluded by mapping to a human reference genome (hg19 and hg38) using SNAP v1.0beta.18.³⁴ To construct the microbial genome database, pathogens and their genomes or assemblies were selected following the Kraken 2 criteria for selecting representative assemblies for microorganisms (bacteria, viruses, fungi, protozoa, and other multicellular eukaryotic pathogens) from an open-access Kraken2 database (https://benlangmead.github.io/aws-indexes/k2). Microbial reads were aligned to the database using Burrows–Wheeler Aligner software.³⁵ We defined reads with fewer than four mismatches of reference as “mapped reads.” Reads with multiple locus alignments within the same genus were excluded in secondary analysis. Only reads mapped to the genome within the same species were considered. We defined a bioinformatic parameter, standardized strictly mapped reads number (SDSMRN), using the formula below:

Pathogen and Clinical Composite Diagnosis

Given the lack of standard threshold criteria for interpreting mNGS results, we used the criteria derived and revised from previous studies.^36–40 The mNGS results were considered as positive if one of the following thresholds were met, and a literature evidence of its pulmonary pathogenicity is available: (i) culture/PCR and mNGS identified the same microbe; (ii) at the species level, the relative abundance of bacteria (mycobacteria excluded), fungi (molds excluded), and parasites was greater than 30%, or the coverage rate scored 10-fold greater than that of any other microbes (for fungi, whose coverage rate scored 5-fold higher than that of any other fungus); (iii) positive virus and molds will be considered when the SDSMRN was no less than 3; (iv) for the detection of Mycobacterium spp., Nocardia spp., the pathogen was considered as positive when SDSMRN is greater than 1.

Microorganisms with unclear significance, such as Torque teno virus (TTV) and EBV, were considered “non-pathogenic microbes.” Oral and intestinal microorganisms were identified as colonized or contaminated unless clinical evidence is available, such as aspiration. All the mNGS results were classified as “probable pathogen”, “definite pathogen”, or “non-pathogenic microbe.” Only the microbe that was defined as “probable pathogen” or “definite pathogen” was considered as a true positive mNGS result.

To confirm the clinical composite diagnosis, two experienced infectious disease specialists who were experienced in the ICU independently reviewed the medical records of each patient. First, they determined whether the patients had infectious pneumonia; next, they identified the clinical composite diagnosis based on clinical manifestation, laboratory tests, and chest radiology, microbiological tests (including CMTs and mNGS).^39,41

The impact of mNGS results was assessed according to the data obtained. The clinical impact was classified as no change and treatment change, wherein the initial antimicrobial drugs were added or reduced according to the mNGS results.

Statistical Analysis

The t-test and Wilcoxon rank test were used for the comparative analysis of the continuous variables. Pearson chi-square, Fisher exact, or McNemar tests were used for discrete variables where appropriate. For the sensitivity, specificity, positive predictive value, and negative predictive value, we used the clinical composite diagnosis as the reference standard. Data analyses were performed using the SPSS 26.0 (IBM, Armonk, NY, USA) software and Empower (R) (X&Y solutions, Inc. Boston, MA, USA). P values < 0.05 were considered significant, and all tests were two-tailed.

Results

Patient Characteristics

During the study period, 24 of 81 ICU patients with suspected severe pneumonia who had both BALF and plasma mNGS results were excluded; this was done either because BALF and plasma mNGS were not performed simultaneously or because the patient had been discharged before the mNGS results were obtained. In total, 57 patients (median age: 59.9 years, male: 40, female: 17) were enrolled in this study and were divided into the negative plasma mNGS (n = 35) and positive plasma mNGS (n = 22) groups. The baseline characteristics and outcomes of patients are shown in Table 1. 29/57 (50.9%) received mechanical ventilation. Seventeen of the 57 patients were diagnosed with septic shock, and 31 had immunocompromised conditions. The median APACHE II and SOFA scores were 14.5 and 4, respectively. Days of hospital and ICU stay at the time of specimen acquisition were 7 and 2 d, respectively. Additionally, 49/57 (86%) of the patients had been treated with antibiotics before sampling, and 19 (36%) patients had a positive BALF culture. Patients with positive plasma mNGS had fewer days of hospital stay at the time of specimen acquisition (4.5 vs 11, P = 0.028), had higher positive BALF culture rates (50% vs 22.9%, P = 0.033), and were younger (52.1 vs 64.7, P = 0.001), compared with the patients with negative plasma mNGS. No significant differences in the severity of illness (APACHE II, SOFA, and septic shock) and total 28-day mortality were observed between the negative and positive plasma mNGS groups (Table 1).

Table 1 Characteristics of Patients with Suspected Severe Pneumonia Included in the Study

Comparison of mNGS Results Between BALF and Plasma

In total, 53/57 patients (93%) were diagnosed with severe pneumonia, and the remaining four were diagnosed with idiopathic interstitial pneumonia (n = 3) and secondary organizing pneumonia (n = 1). Fifty-three (100%) true positive results were obtained using BALF mNGS from all 53 patients with severe pneumonia, and two of the four patients with a non-infectious etiology had negative BALF mNGS results. However, 22/53 (41%) of patients with severe pneumonia had true positive plasma mNGS results. The most frequently detected pathogens using BALF mNGS were Pneumocystis jirovecii, Pseudomonas aeruginosa, and Klebsiella pneumonia (Figure 1 and Supplementary Data Table S1). Meanwhile, CMV, P. jirovecii, and K. pneumonia were the most common pathogens observed in plasma mNGS results (Figure 1). All the results of microbiological tests are shown in Supplementary data Table S1.

Figure 1 Pathogen spectrum of mNGS results in patients with severe pneumonia (n = 53). Abbreviations: BALF, bronchoalveolar lavage fluid; mNGS, metagenomic next-generation sequencing.

Among the 53 patients with severe pneumonia, significant differences in the positivity rate of BALF and plasma mNGS were observed (100% vs 41%, P < 0.001; Figure 2A). In 27/53 cases, mNGS identified two or more organisms (polymicrobial infections) without difference in the polymicrobial infection rate between the positive mNGS results of BALF and plasma [25/53 (47%) vs 8/22 (15%), P = 0.479; Figure 2A). In total, 94 and 32 pathogens were detected using BALF and plasma mNGS, respectively (Figure 2B). In 13 cases, more than two co-infecting pathogens were detected using BALF mNGS, with only one such case detected using plasma mNGS (Figure 2C). Although no significant difference was observed, the proportion of bacteria detected using BALF mNGS was higher than that using plasma mNGS (42% vs 32%, P = 0.301; Figure 2D). However, the proportion of virus in positive plasma mNGS results was higher than in BALF mNGS (23% vs.11%, P = 0.173; Figure 2D).

Figure 2 Diagnostic performance of mNGS in patients with severe pneumonia (n = 53). (A) Comparison of positive rates (P < 0.001) between BALF and plasma mNGS. (B) Different spectra of bacteria, fungi, and viruses were detected using BALF and plasma mNGS. No differences in the detected proportions of bacteria, fungi, and viruses were observed between BALF and plasma mNGS (P > 0.05). (C) Cases with various numbers of detected pathogens via mNGS. (D) Distribution of different types of pathogens in panel C. Outer circle, pathogens identified by BALF mNGS (n = 53); inner circle, pathogens identified by plasma mNGS (n = 22). Abbreviations: BALF, bronchoalveolar lavage fluid; mNGS, metagenomic next-generation sequencing.

Diagnostic Performance of BALF and Plasma mNGS

In total, 28/53 patients had positive CMT results, including 19 cases with a positive BALF culture test. No patient had a positive blood culture test. As shown in Table 2, based on the clinical composite diagnosis as a reference, plasma mNGS had a much lower sensitivity (100% vs 42%, P < 0.001), and the diagnostic accuracies of mNGS using BALF and plasma samples were 96% and 46%, respectively. However, in this study, the specificity of plasma mNGS was 100%, whereas that of BALF mNGS was 50%. Among 28 patients with positive CMT results, the positivity rate of plasma mNGS was 43% (12/28). The Kappa value of the two methods was 0.37 (0.12–0.60). The sensitivity of a single plasma mNGS test was not superior to that of CMT (42% vs 53%).

Table 2 Diagnostic Performance of mNGS and CMTs in Patients with Suspected Severe Pneumonia (n = 57)

Clinical Impact of Plasma mNGS

Plasma mNGS reported an additional 12 microorganisms in 11 patients, including EBV, CMV, TTV, human alphaherpesvirus 3, Aspergillus, and human adenovirus, as shown in Table 3. Specifically, four pathogens were defined as definite, three were probable, and five microorganisms were unlikely, based on ICU clinical composite adjudication. In this study, the beneficial effects were categorized as a “treatment change” relative to the initial treatment. Based on the additional organisms that were identified using plasma mNGS, a beneficial impact was obtained for 5/53 (9%) patients (Table 3). A multicenter retrospective study showed that the positive clinical impact of plasma mNGS for the diagnosis of infections is 7.3%.¹⁶

Table 3 Impact of Microorganisms Detected Using Only Plasma mNGS in the Paired Sample (11 Cases)

Examples of cases in which plasma mNGS resulted in an antimicrobial change were as follows: 1) for patient no. 6, a 46-year-old immunocompromised patient with severe pneumonia, voriconazole was administered when Aspergillus was detected using only plasma mNGS, whereas the BALF mNGS and CMT detected Nocardia cyriacigeorgica, P. jirovecii, and CMV; 2) for patient nos. 43 and 55, CMV was identified using only plasma mNGS, and thus ganciclovir was added to the treatment regimen; and 3) for patient nos. 41 and 53, who were diagnosed with human alphaherpesvirus 3 infection based on the plasma mNGS, were treated with acyclovir.

Discussion

This study conducted a paired comparison between BALF and plasma mNGS in terms of their performance in detecting pathogens in ICU patients with suspected severe pneumonia. The reliance on BALF mNGS, but not plasma mNGS, to identify pathogens in patients with severe pneumonia is supported by the results of this study. Plasma mNGS has recently been regarded as a promising and less invasive microbial detection technique for severe pneumonia. A previous study found that the release of pathogen DNA into the bloodstream might be more common than appreciated in pneumonia patients with negative blood cultures,²⁵ and that the DNA burden in body fluids is 160-fold higher than that in plasma.⁴² Additionally, the cost of one sample subjected to mNGS is approximately $400; therefore, the cost of using both BALF and plasma mNGS for patients with severe pneumonia must be considered.

In this study, BALF mNGS was superior for pathogen detection than plasma mNGS, and the diagnostic accuracy was higher than that of plasma mNGS (96% vs 46%) in patients with severe pneumonia. Hill et al²³ also found that plasma mNGS had moderate sensitivity and high specificity for detecting pathogenic molds in hematopoietic cell transplant recipients with pneumonia.

Current knowledge indicates that the microorganisms in plasma might be transmitted from the lungs to the bloodstream of patients. However, in this study, the distribution of detected pathogenic species obviously differed between the results of BALF and plasma mNGS, and only 15/57 (26%) patients with suspected pneumonia had at least one plasma mNGS result that matched that of the BALF mNGS. To the best of our knowledge, only four studies have evaluated the clinical significance of simultaneous BALF and plasma mNGS in patients with pneumonia, but these studies do not have a consistent conclusion. Chen et al¹⁹ reported that the results of plasma mNGS were highly consistent with those of BALF samples. Jiang et al²⁰ only evaluate the utility of plasma mNGS for pneumocystis jirovecii pneumonia. Xie et al²¹ presented a description of the results of both samples in 37 patients. The other remaining study used the results from the culture method, which did not use CMTs as the reference standard, and was consistent with our results, in which BALF mNGS was more sensitive than plasma mNGS.²²

Furthermore, we found that the patients with positive plasma mNGS had shorter days of hospital stay at the time of specimen acquisition and a higher positivity rate of BALF culture than those with negative plasma mNGS. These results infer that the sensitivity of plasma mNGS could be influenced by the number of days of hospital stay at specimen acquisition. Future studies using a larger cohort should be performed to assess the generalizability of these findings.

We can detect new microorganisms that were previously unidentified members of the human microbiome.⁴³ However, blood samples contain molecules originating from virtually every tissue, including the microorganisms that colonize the body. In this study, viruses were the most detected pathogen (12/32) in all plasma mNGS results, whereas bacteria (50/94) were most detected in BALF mNGS. This finding is consistent with the study of Chen et al,²² in which plasma mNGS was found to have advantages in identifying viruses. Fang et al⁴⁴ reported that the plasma mNGS also can be clinically useful for treatment monitoring in patients with severe pneumonia and viremia. Notably, microorganisms, including TTV, EBV, and CMV, were only detected using plasma mNGS and not using BALF mNGS in 11/57 patients. Although plasma mNGS detected additional viral pathogens, these results had no clinical impact. Nevertheless, a beneficial impact was obtained in 45.5% (5/11) of the patients. Further studies should be conducted to determine if the microorganisms detected using only plasma mNGS, especially viruses, are derived from pneumonia or only reflect uncultivable microorganisms present in the blood. A previous study reported that anelloviruses (including TTV) could be identified in 96% of adult human blood samples during steroid-refractory/dependent graft-versus host disease.¹⁷ The clinical significance of these viruses is poorly understood; however, anelloviruses were reported as non-pathogenic endogenous markers of the immune function,⁴⁵ and the quantification of TTV can guide immunosuppression to reduce graft rejection and infection.⁴⁶ We suggest that routine and simultaneous BALF and plasma mNGS is unnecessary, as doing so will not be more beneficial to patients and only increase the hospital expenses. For patients with suspected virus infection, the paired plasma mNGS may be helpful.

This study had some limitations. First, this was a single-center retrospective study with a small cohort. We mostly included patients with prior antibiotic exposure, and some patients had an average of 7 d of hospital stay at the time of specimen acquisition, which might have affected the analysis results. Prior antibiotic exposure is an inevitable question in patients admitted to the ICU with suspected severe pneumonia. We used a paired comparison of simultaneous plasma and BALF samples of the same patient to reduce this bias. Second, the mNGS results from plasma did not include RNA mNGS tests. Third, although we did not perform specific screening, all patients had negative blood cultures, and thus, all positive cultures were derived from BALF samples, and the impact of plasma mNGS could not be further evaluated in patients with bloodstream infections.

Conclusion

Based on the results of our study, we do not recommend performing BALF and plasma mNGS simultaneously. For patients with suspected virus infection, plasma mNGS can be a supplementary test. Prospective studies are required to clarify the optimal indications for plasma mNGS as an additional test for patients with severe pneumonia.