Performance of Xpert MTB/RIF Ultra for the Diagnosis of Pulmonary Tuberculosis Using Bronchoalveolar Lavage Samples in People Living with HIV/AIDS (PLWHA) in China: A Prospective Study

Introduction

According to Global Tuberculosis Control Report 2020,¹ the estimated number of tuberculosis (TB) patients worldwide in 2019 was 10 million, with 820,000 TB/HIV co-infected patients (8.2% of the total patients with TB). In 2019, the estimated number of deaths from TB worldwide was 1.2 million, of those 208,000 deaths were attributed to TB/HIV co-infection. Considering that the risk of Mycobacterium tuberculosis (MTB) infection in people living with HIV/AIDS (PLWHA) is 21 times (16–27 times) higher than other populations in the world, timely diagnosis and treatment is essential in high-risk regions and populations.²

The molecular point-of-care testing Xpert MTB/RIF assay (Xpert) has dramatically improved the diagnosis of TB. The pooled sensitivity and specificity Xpert in the diagnosis of TB was 85% (95% Credible Interval [CI] 82% to 88%) and 98% (95% CI 97% to 98%) in 70 studies with high certainty evidence.³ The pooled sensitivity of Xpert was 98–100% in patients with smear-positive, culture-positive TB.^2,4 However, current diagnostic tests for tuberculosis are usually performed using sputum specimens. Since patients with advanced HIV have little or no sputum, the performance of Xpert is limited in patients with paucibacillary sputum or HIV infection.^2,5

The Xpert MTB/RIF Ultra (Ultra), a new version of the molecular point-of-care test, was designed to improve the sensitivity for MTB DNA detection. Ultra shows a decreased limit of detection (LOD) for MTB detection (15.6 colony-forming unit [CFU]/mL) compared to Xpert (LOD, 112.6 CFU/mL),^3,6 which improved MTB detection in smear-negative sputum samples collected from PLWHA.⁷

Timely diagnosis and treatment of TB is hampered for patients who are unable to provide sputum samples for TB testing, such as people living with HIV/AIDS and children. Using bronchoalveolar lavage (BAL) as an alternative sample type and using Ultra instead of Xpert should be an improved method for MTB detection among people living with HIV. However, studies on the performance of Ultra using bronchoalveolar lavage (BAL) samples collected from PLWHA have rarely been reported. Therefore, the aim of this study was to compare the diagnostic performance of Ultra with four other methods (Xpert, MTB culture, acid-fast bacillus test [AFB] smear, tuberculosis real-time fluorescence quantitative PCR [TBDNA]) using bronchoalveolar lavage (BAL) samples and MTB antigen-specific interferon gamma enzyme-linked immunospot assay (T-SPOT.TB) using blood samples for the diagnosis of TB among people living with HIV.

Methods

Study Design and Participants

The study was conducted at the Third People’s Hospital of Shenzhen, China, which is located in a high TB burden setting. From February 5, 2018 to March 30, 2019, a total of 105 HIV-infected patients with suspected pulmonary tuberculosis were included in this study. We recruited participants according to criteria as follows: 1) aged 18–65 years; 2) tested positive for HIV-1 antibody by ELISA and confirmed by Western blot method; 3) combined pulmonary infection of unknown cause and not previously treated with anti-tuberculosis; 4) CD4+ T-lymphocyte count ≤200/µL; 5) voluntarily signing of the patient’s informed consent and able to guarantee follow-up; 6) no plans to move away from the current trial site during the course of the trial; 7) the overall condition of the subject does not, in the opinion of the investigator, interfere with the evaluation and completion of the trial. We excluded patients in the acute phase of infection and patients with a combination of opportunistic infections other than tuberculosis (as defined by the national guidelines for the treatment of AIDS) at the time of enrolment and whose condition was unstable or with a combination of malignant neoplasms. We also excluded patients of non-Chinese nationality. In the end, 6 participants were excluded from the analysis due to lack of enough specimen (n=4) and absence of follow-up (n=2). Thus, 99 patients were included in the final analysis: 25 definite cases, 9 suspected cases and 65 non-pulmonary TB (PTB).

Bronchoscopy was performed to collect BAL samples, and blood samples were collected at the same time. A questionnaire interview was performed to collect information on clinical symptoms, risk factors, and care-seeking behaviors. Participants were divided into three groups according to the composite reference standard (CRS), which consists of clinical, laboratory and radiological examinations, and follow-up data: 1) confirmed PTB: BAL samples positive for MTB cultures and patients responding well to anti-TB therapy; 2) suspected PTB: BAL samples negative for MTB culture, but clinical symptoms and radiological findings suggestive of PTB, and the patients responding well to empirically administered anti-TB therapy; and 3) non-PTB: negative MTB culture of BAL sample and patients improved without receiving anti-TB treatment. All the patients were followed up for at least 6 months.

Procedures

Demographic information and medical history were obtained on admission, and blood T-SPOT.TB test, T lymphocyte subset tests, plasma HIV RNA load, computed tomography (CT) scan and flexible bronchoscopy were performed at the same time. Bronchoscopy was performed on all patients under conscious sedation in a dedicated ward to collect BAL samples. Fifty milliliters of sterile saline was instilled and aspirated from the lung segments involved. Twenty milliliters BAL were centrifuged at 3000 × g for 15 min at 4°C. The supernatant was discarded and the pellet was resuspended in 10 mL of sterile saline.

Fresh BAL specimens collected from people living with HIV were tested simultaneously with Xpert, Ultra, mycobacterial culture, AFB smears, and TBDNA test. T-SPOT.TB test was performed using blood specimens. Xpert and Ultra assays were performed by adding sample reagents (manufacturer: Cepheid) to collected BAL specimens at a 3:1 dilution, and then 2.0 mL of the resulting mixture was added to one Xpert and one Ultra kit (manufacturer: Cepheid, Xpert MTB/RIF ULTRA assay (package insert), Sunnyvale, CA, 2017). Samples were analysed using a standard four-module GeneXpert instrument with automated assay readouts (invalid [no internal assay control detected]; not detected; or detected [semi-quantitative]). The semi-quantitative levels of Xpert Ultra results were trace, very low, low, or medium. The semi-quantitative levels of Xpert results were very low, low, medium, or high. Staff performing the Ultra tests were blinded to the Xpert results.

For MTB culture, 0.5 mL of the sediment of sputum sample was inoculated into MGIT tubes with enrichment supplement (OADC) and antibiotic mixture (PANTA), and then onto Löwenstein–Jensen (LJ) solid medium (manufacturer: Becton–Dickinson Diagnostic Instrument Systems, USA). The MGIT tubes were placed into the BACTEC MGIT 960 system, which is a fluorescence-based detection instrument [23]. The bacterial growth of the liquid cultures was continuously monitored for 6 weeks or until marked positive by the instrument.

All participants underwent T-SPOT.TB testing in accordance with the manufacturer’s instructions (Dakewe Biotech Co., Ltd., China). Briefly, an 8-mL blood sample was obtained from an HIV-infected patient. The blood sample was centrifuged to enable quantification of peripheral blood mononuclear cells prior to incubation with ESAT-6 and CFP-10 antigens in an enzyme-linked immunosorbent spot assay. T-SPOT.TB testing was performed in the clinical Laboratory of The Third People’s Hospital of Shenzhen by a trained medical laboratory technologist. The results were read manually by a technologist who was blinded to patient’s clinical information and test results.

TB DNA testing was performed using BAL specimens. DNA extraction and amplification of BAL samples were performed according to the manufacturer’s instructions (DaanGene, China). To detect AFB, BAL samples were stained with Ziehl–Neelsen stain (Hunan Tianqi, China) and examined under the oil immersion objective lens of the microscope (×1000). T-lymphocyte subpopulation analysis was conducted using CD³, CD⁴ and CD⁸ monoclonal antibody and was operated in strict accordance with the instructions (Becton Dickinson, USA).

Statistical Analysis

Comparisons of clinical characteristics between culture-positive and culture-negative groups were performed by chi-square Test, rank sum test and Fisher’s exact probability test. Measures of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), area under the curve (AUC), and kappa values, including 95% confidence interval (95% CI), were evaluated in six tests. Sensitivity was calculated as true positive divided by true positive plus false negative, and specificity was calculated as true negative divided by false positive plus true negative. Unpaired and paired categorical variables were compared using the chi-square and McNemar’s test, respectively. Consistency tests were used using the kappa statistic. Statistical analyses were performed using GraphPad Prism (version 6.0), MedCalc version 18.6, and Microsoft Excel. MTB culture and clinical PTB treatment were used as reference standards for the detection of MTB. A P-value <0.05 was considered statistically significant.

Ethics Approval

The survey was approved by the Institutional Review Board of the Third People’s Hospital of Shenzhen and all study components were conducted in accordance with the Declaration of Helsinki. Study participants were informed of the study purpose and were required to fill in an informed consent form. This trial was registered with ClinicalTrials.gov (number ChiCTR1800014792).

Results

Clinical Characteristics of the Patients

A total of 99 patients were included in the final analysis with 25 definite cases, 9 suspected cases and 65 non-pulmonary TB (PTB). Demographic and clinical information are summarized in Tables 1 and 2. Cough was reported by 74 (74.7%) patients, but only 35 (35.4%) had sputum and 7 (28.0%) had definite PTB. There were no significant differences between culture-positive and culture-negative groups in the presence of classic TB symptoms (cough, sputum, fever, night sweats, weight loss) or chest radiographic findings of upper or lower lung infiltrates (with or without cavitation) (Table 2).

Table 1 Demographic and Clinical Characteristics of the Participants

Table 2 Comparisons of Clinical Characteristics of Patients with Culture-Positive PTB and Those with Culture-Negative PTB or Non-PTB

Comparative Performance of the Six TB Tests with Two Standards (MTB Culture and Clinical PTB Treatment)

Using BAL MTB culture as a gold standard, Ultra had a sensitivity of 96.0% (95% CI 92.1–99.9%), and a specificity of 91.9% (95% CI 86.5–97.3%), while Xpert had a sensitivity of 80.0% (95% CI 72.1–87.9%) and a specificity of 95.9% (95% CI 92.0–99.8%). In addition, the sensitivity and specificity of other diagnostic tests were 84.0% (76.8–91.2%) and 93.2% (88.2–98.2%) for T-SPOT.TB; 44.0% (34.2–53.8%) and 95.9% (92.0–99.8%) for TBDNA; 4.0% (0.1–7.8%) and 98.6% (96.3–100.0%) for AFB smear, respectively (Table 3). The areas under the curve (AUC) for the six tests are shown in Table 4 and Figure 1, in which Ultra had the highest AUC (0.957). With the Cohen’s kappa coefficient of 0.82, Ultra also had the highest agreement with MTB culture (Table 5).

Table 3 Comparative Performance of Ultra, Xpert, Culture, T-SPOT.TB, TBDNA and AFB Using Two Standards (Culture and Empiric Treatment of PTB)

Table 4 The Areas Under the Curve (AUC) for the Six Tests of PTB Using Culture as the Gold Standard

Table 5 McNemar’s Test and Measurement of Agreement Kappa Value (Using Culture as Gold Standard, n=99)

Figure 1 Receiver operating characteristic (ROC) curves of sensitivity and specificity of different testing techniques, using culture as reference.

If the final clinical diagnosis and treatment of PTB was used as reference standard, sensitivity of Ultra, Xpert, culture, T-SPOT.TB, TBDNA, AFB smear was 85.3%, 67.6%, 73.5%, 64.7%, 38.2%, and 8.8%, respectively (Table 3). The specificity of culture and Xpert was 100% for both assays. The specificity was 98.5% for Ultra, TBDNA, and AFB smear, and 93.8% for T-SPOT.TB. The AUC for Ultra (0.92) was marginally better than that of Xpert (0.84) and Culture (0.86) (P<0.001) (Table 6, Figure 2). Moreover, the similar results were observed for agreement (Table 7).

Table 6 The Areas Under the Curve (AUC) for the Six Tests of PTB Using Empiric Treatment of PTB as the Reference Standard

Table 7 McNemar’s Test and Measure of Agreement Kappa Value (Using Empiric Treatment as Reference Standard, n=99)

Figure 2 Receiver operating characteristic (ROC) curves of sensitivity and specificity of different testing techniques, using empiric treatment of PTB as reference.

A Difference-in-Differences Analysis of Ultra and Xpert

When using the two different standards, the sensitivity of Ultra was statistically significantly higher than that of Xpert. The specificity of Ultra was lower than that of Xpert based on AUC and kappa but it was not significant. According to the McNemar’s test, both Ultra and Xpert performed as a substitute for culture. In addition, there was statistically significant difference between Xpert and Ultra (P=0.016) (Table 8).

Table 8 Comparative Accuracy Between Ultra and Xpert in HIV-Infected Patients

Diagnostic Performance of Ultra and Xpert Regarding CD4 Count Status

The presentation of active TB is influenced by the degree of immunodeficiency [73,74]. In patients with CD4 counts <200 cells/mm³, the radiographic findings of PTB are markedly different, with infiltration not favoring the upper lobes and cavitation uncommon [73,75,76]. Our study found that the sensitivity of Ultra and Xpert was 100% and 85.7%, respectively, in the patients with CD4 counts >200 cells/mm³ (Table 9). However, in patients with CD4 counts <200 cells/mm³, the sensitivity was 94.0% for Ultra, and 77.8% for Xpert (Table 9). Moreover, compared to patients with CD4>200 cells/mm³, the kappa value of Ultra and Xpert was lower than those with CD4<200 cells/mm³ (0.172 vs 0.141) (p<0.05). The results indicated that HIV-infected patients with CD4 <200 cells/mm³ had significantly reduced the accuracy of Ultra and Xpert for the diagnosis of PTB.

Table 9 Comparative Accuracy for Detection of Tuberculosis Using Culture as a Gold Standard in HIV-Infected Patients

Discussion

According to findings in our study, diagnosis of PTB based on clinical symptoms and excreted sputum may be challenging in people living with HIV/AIDS. First, in our study, 72% of clinically diagnosed, culture-positive patients were not able to provide a sputum sample. In addition, the classical clinical features of PTB were not statistically different from these of non-PTB in our study population, such as classical TB symptoms (cough, sputum, fever, night sweats, and weight loss) and common TB chest radiographic manifestations.

In agreement with previous reports,^1,6,7 our data suggest that Ultra is more sensitive than Xpert (96% versus 80%) when considering all culture-positive BAL specimens. This is also consistent with other study conducted among children infected with HIV.⁸ The reason for the higher sensitivity is that Ultra incorporates two different multi-copy amplification targets (IS6110, −16 copies/cell and IS1810, −5 copies/cell) and a larger DNA reaction chamber,¹ which significantly decreased the LOD from 112.6 CFU/mL using Xpert to 15.6 CFU/mL.⁶

Results from other study⁹ using BAL specimens in patients with suspected PTB reported a sensitivity of 83.4% (64/77), which is similar to our results in HIV-infected patients with CD4>200 cells/mm³ (85.7%, 6/7) and higher than that in whom with CD4<200 cells/mm³ (77.8%, 14/18). The degrees of immunodeficiency in patients with PTB could help to explain the differences. When CD4 counts decreasing to less than 200 cells/mm³, it indicates that the body is in a low immunity level and multiple opportunistic infections or tumors may occur. In patients with CD4 counts >200 cells/mm³, HIV co-infected TB is generally similar to TB among individuals without HIV. In most patients, the disease is confined to the lungs and the common chest imaging presentation is an upper lobe infiltrate with or without cavitation. Piersimoni et al¹⁰ used 269 respiratory specimens (including 58 BAL samples) from the patients with unknown HIV status in a low TB low-epidemic area. The sensitivity of Ultra and Xpert was 87% and 75%, respectively, lower than that of Ultra and Xpert in our study (96% and 80%, respectively). However, since they did not report in detail whether BAL samples were used to evaluate the performance of ultra, we could not conclude whether their lower sensitivity is due to the use of BAL samples in our study.

Although Ultra and Xpert results were available for all participants within 1 week of the visit, these delays may have increased the propensity of clinicians to select same-day empirical treatment for high-risk patients in a high TB prevalence setting. Due to the false-negative rates for MTB culture, empiric anti-TB treatment may be considered as a reference standard for TB testing. When empiric TB treatment was used as a standard, the sensitivity of the diagnostic TB tests decreased (85.3% for Ultra, 67.6% for Xpert, and 73.5% for Culture) compared to that of using culture as the gold standard. However, Ultra still had the highest sensitivity in diagnostic TB testing. A study conducted by Kendall et al¹¹ found that patients who were Xpert negative and had negative cultures had higher prevalence of HIV infection, which may have contributed to the decreased sensitivity of the diagnostic TB test.

When using culture as the gold standard, Dorman⁷ reported a lower specificity for Ultra than Xpert in HIV-infected patients, ie, 96% for Ultra versus 99% for Xpert. In BAL specimens obtained from HIV-positive patients, the specificity of Ultra (91.9%) was lower than that (95.9%) of Xpert if we used culture as the gold standard, but the difference was not statistically significant.

The performance of Ultra in our study was significantly better than the other diagnostic tests for TB as evidenced by the respective ROC curves with AUOC values (AUC for Ultra 0.957 vs for Xpert 0.875, for T-SPOT.TB 0.886, for TBDNA 0.700, for AFB 0.553). Wu et al¹² study showed that the AUC for culture, Xpert and Ultra were 0.608, 0.650, and 0.723, respectively, in all extra-pulmonary specimens. In contrast to the extra-pulmonary study results, similar results were seen in our study. According to Cohen’s kappa, the results of Ultra, Xpert, and T-SPOT.TB (0.84, 0.78, and 0.76, respectively) were in good agreement with the results of culture, which suggests that Ultra is superior to the other two tests. When using empiric treatment as the standard, Ultra was in a better agreement with clinical TB diagnosis compared to culture.

Our results of the diagnostic accuracy study show that the performance of Ultra and the other five tests was different in different CD4 counts levels among people living with HIV. According to a previous report,¹³ CD4 count <200 cells/mm³ was reported as the cutoff value for typical HIV co-infection with opportunistic infections. Our study demonstrates a decrease in sensitivity and specificity of Ultra when compared to all culture-positive groups in patients with CD4>200 cells/mm3 (78%, 86% vs 94%, 96%, respectively). The kappa value slightly decreased, particularly for Xpert test, which was only 0.73 for patients with CD4<200 cells/mm³, suggesting that the performance and accuracy of Ultra and Xpert in diagnosing PTB in patients with immunodeficiency.

Our research also has some limitations. Due to the limited accessibility of patient sputum, we were unable to compare the performance of different tests in the diagnosis of PTB using two different respiratory specimens (BAL and sputum). BAL sample is collected using bronchoscopy, which is an invasive test. Side effects such as bilateral lung infections and the spread of Mycobacterium tuberculosis may occur. However, a study conducted by Pan et al found that BAL should be chosen over expectoration for the evaluation of Xpert.⁹ This would presumably imply that the performance of Ultra using BAL samples can be better than that using sputum. In addition, the small number of study participants prevented us from assessing the performance of the Ultra test for diagnostic MTB on BAL samples in HIV-infected patients with different immune status. Moreover, we did not include HIV-negative patients. The study by Nicol showed that the sensitivity of Ultra in HIV-infected children was the same as in HIV-negative children. Several studies have found that the Xpert is more sensitive in HIV-negative participants than in HIV-infected individuals.^5,7,14

In conclusion, our study shows a higher sensitivity of Ultra compared to the other five tests in the diagnosis of PTB using BAL specimens, especially from HIV-infected patients with CD4<200 cells/mm³. This suggests a potential benefit of Ultra and BAL in improving early detection of PTB among people infected with HIV.