Diagnostic Performance of GeneChip for the Rapid Detection of Drug-Resistant Tuberculosis in Different Subgroups of Patients

Introduction

Tuberculosis (TB) is a chronic communicable disease that remains a significant public health concern worldwide. Globally, an estimated 10.0 million (range: 8.9–11.0 million) people fell ill with TB in 2019, and an estimated 1.4 million TB deaths occurred.¹ Treatment of drug-susceptible TB is based on the principle of combined chemotherapy for several months by using the first-line drugs isoniazid (INH), rifampin (RIF), pyrazinamide (PZA), ethambutol (EMB), and/or streptomycin (SM).² Multidrug resistance (MDR) is defined as Mycobacterium tuberculosis (M.tb), which is resistant to at least RIF and INH, the two most potent anti-TB drugs.³ MDR-TB results from either infection with already drug-resistant organisms or may develop during the treatment course, which poses the greatest obstacle to success in the “end TB” strategy.¹ The wide application of anti-TB drugs, especially when these drugs are misused or mismanaged, exacerbates the occurrence of drug-resistance.⁴ Although historically MDR-TB was predominantly caused by mismanagement of treatment, the direct transmission of the MDR-TB pathogen has been identified as a severe problem worldwide.^5,6

The culture-based drug-susceptibility test (DST) is the gold-standard assay for testing drug resistance, but it takes several weeks to obtain results,⁷ and access to the necessary laboratory facilities in countries with a heavy disease burden is often scarce.⁸ A traditional phenotypic DST for M.tb does not meet the goals of the early diagnosis and targeted regime of TB treatment. Therefore, highly sensitive, specific, easy-to-apply, time-saving, and cost-effective methods are urgently required to promote the diagnosis of MDR-TB and improve treatment outcomes.^9–11

Several molecular techniques have been applied to identify M.tb isolates and drug resistance, including real-time polymerase chain reaction (RT-PCR), line probe assays (LPAs), DNA sequencing, and oligonucleotide or DNA microarrays.^12,13 The GeneChip MDR kit (CapitalBio, Beijing, China) was designed to identify common mutations related to RIF and INH resistance in the rpoB, katG, and inhA genes based on molecular analyses from multiple PCRs and reverse hybridization. By uncovering mutations in these genes, GeneChip can detect M.tb and its MDR form from sputum samples. Most importantly, the entire procedure of the GeneChip assay only takes 6 hours.^14,15 A systematic review consisting of 159 studies revealed that the pooled sensitivity of GeneChip microarrays was 0.89 (95% CI: 0.86–0.91) for detecting drug resistance to RIF, 0.79 (95% CI: 0.75–0.82) for detecting drug resistance to INH, and 0.79 (95% CI: 0.73–0.84) for detecting MDR-TB. The specificity of identifying the above three types of drug resistance was >97%.¹⁶ GeneChip has been regarded as a more cost-effective method of diagnosing MDR-TB over the conventional DST.¹⁷ However, there is a lack of data to assess its value in different subgroups of DR-TB patients.

Thus, we performed a molecular epidemiological study to evaluate the diagnostic performance of GeneChip in detecting drug resistance to RIF and INH, the two most effective drugs against M.tb, in overall and subgroup TB patients in a Chinese population.

Materials and Methods

Study Sites and Data Collection

We recruited newly registered sputum smear-positive pulmonary TB patients from January 2011 to September 2020 in Lianyungang City, Jiangsu Province, China. We used a questionnaire to collect the patient’s characteristics, including age, sex, treatment history, treatment outcomes, and place of residence. Two sputum samples were collected from each patient and transported to corresponding prefectural or municipal laboratories within three days after collection. Each sample was divided into two aliquots for drug resistance detection, where one was used for a culture-based traditional DST and another was used for GeneChip.

Sputum Culture and Traditional DST

Sputum samples were cultured on Löwenstein-Jensen (L-J) medium at 37°C. We identified the M.tb isolates by p-nitrobenzoic acid and thiophene carboxylic acid hydrazine resistance tests. Species other than M.tb were excluded from the current analysis. We then detected the drug susceptibility of M.tb with L-J medium supplemented with RIF and INH. The DST was performed according to the proportion method recommended by the World Health organization (WHO) and International Union against Tuberculosis and Lung Disease (IUATLD). For internal quality assurance, a standard H37Rv strain was included with each new batch of L-J medium, and a DST was also performed when readings were performed after 4 and 6 weeks. External quality control for culture and the DST was conducted by the provincial TB reference laboratory, which participates in the annual proficiency review of the DST organized by the Hong Kong Supranational Tuberculosis Reference Laboratory and has passed each audit since 2010. The anti-TB drug concentration was 0.2 μg/mL for INH and 40 μg/mL for RIF. An isolate was confirmed to be a drug-resistant strain if there was more than 1% colony growth on the drug-containing medium compared with the control. Otherwise, it was declared sensitive to the specific drug.

GeneChip Assay

The GeneChip assay (CapitalBio) was performed according to the manufacturer’s instructions.^14,18 In brief, sputum specimens were centrifuged to pellet the bacteria and then liquefied with 10% NaOH. The pellet obtained after discarding the supernatant was resuspended in 0.9% saline and centrifuged again. The pellet was resuspended in Tris-EDTA buffer and transferred to an extraction tube. The materials and reagents of DNA extraction were supplied in the CapitalBio Universal Kit (CapitalBio). Total DNA was isolated and the extraction tube was incubated at 95°C for 5 min, centrifuged briefly, and then stored at −20°C until use. Oligonucleotide probes were printed onto OPAldehyde Slide^TM aldehyde-activated slides at a concentration of 10 μM in DNA Spotting Solution using a SmartArrayer-48 microarrayer (CapitalBio) and were covalently immobilized on slides via an amino group at their 5ʹ ends to create the gene chips.

Uracil DNA glycosylase and dUTP were used to prevent carry-over contamination during amplification. Multiplex asymmetric PCR was performed in two amplification rounds. After the initial activation step at 37°C for 10 min, DNA was denatured at 94°C for 10 min, followed by the first round of exponential amplification with 35 cycles at 94°C for 30 s, 60°C for 30 s and 72°C for 40 s; the second round of linear amplification with 10 cycles at 94°C for 30 s and 72°C for 60 s; and a final extension step at 72°C for 5 min. GeneChip hybridization was performed in a BioMixer II hybridization oven (CapitalBio). After hybridization, slides were washed and dried by spinning. A confocal LuxScan-10K laser scanner (CapitalBio) was applied to analyze microarrays on the slides. Fluorescent intensities were quantified by using the M.tb Drug Resistance Detection Array Test System (CapitalBio). The drug resistance pattern for RIF and INH can be found in the manufacturer’s instructions.¹⁴ All technicians were trained by the National TB Reference Laboratory and confirmed by proficiency testing.

Statistical Analysis

Normally distributed data are described as the mean ± standard deviation. One-way analysis of variance (ANOVA) was used to compare continuous variables between groups. Categorized variables are expressed as percentages; these variables were analyzed by the chi-square test. We used the R package “confusionMatrix” to calculate the kappa value, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and agreement rate (AR) of GeneChip testing. We also applied the Classification and Regression Tree (CART)¹⁹ to explore factors related to the performance of GeneChip by the R package “party”. Differences of independent kappa between groups were analyzed by the Fleiss method using an R package of “multiagree”.²⁰ Differences in sensitivity, specificity, PPV, and NPV between subgroups were compared by the chi-square test. The significance level was set at 0.05. All analyses were performed using R software for Windows version 4.0.3 (https://www.r-project.org/).

Ethics Statement

This study was approved by the Ethics Committee of Nanjing Medical University. After informed consent was obtained from all participants, questionnaires were used to collect demographic data. All procedures performed involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the Declaration of Helsinki.

Results

General Characteristics

We collected 5946 sputum smear-positive TB samples in the study area. After excluding 700 sputum culture-negative samples, 49 culture medium-contaminated samples, 516 nontuberculous mycobacteria (NTM) infection samples, and 540 samples without drug-resistant results by both DST and GeneChip, 4141 samples remained for analysis (Figure 1). The age of the study subjects ranged from 12 years to 94 years (50.7 ±20.3 years). There were 3219 (77.7%) males, 3648 (88.1%) newly treated cases, 110 (2.7%) cases with treatment failure outcomes, and 2929 (70.3%) cases with patients living in rural areas.

Figure 1 Flow chart of TB cases included in this study. Abbreviations: TB, tuberculosis; M.tb, Mycobacterium tuberculosis; NTM, nontuberculous mycobacterium; MDR, multidrug resistance.

There was a significantly higher prevalence of RIF resistance in females (9.10% vs 6.09%), young cases (8.43% vs 5.06%), previously treated cases (19.88% vs 4.98%), treatment failure cases (20.91% vs 6.37%) and urban cases (9.90% vs 5.46%) (P <0.05). Similar results were found in INH resistance and MDR. Females (12.58% vs 9.91%), previously treated cases (25.76% vs 8.44%), treatment failure cases (21.82% vs 10.20%) and urban cases (13.28% vs 9.35%) had a higher INH resistance (P <0.05). Females (6.62% vs 4.41%), young cases (6.09% vs 3.70%), previously treated cases (15.21% vs 3.51%), treatment failure cases (13.64% vs 4.67%) and urban cases (7.67% vs 3.76%) had a higher MDR prevalence (P <0.05) (Table 1). The characteristics (sex, age, treatment history, treatment failure, and place of residence) of TB patients were significantly associated with the diagnostic performance of GeneChip (P <0.05), except for drug resistance to INH within sex-specific or age-specific groups (Table 1).

Table 1 Findings from the GeneChip and Traditional DST Stratified by Subgroup Patients

CART Analysis for the Performance of GeneChip

We used the CART to determine the most “important” variables affecting the diagnostic value of GeneChip. CART is a powerful and popular predictive machine learning technique that is used for both classification and regression. A response class is predicted in each terminal node of the tree (or each rectangular section in the partition) from all observations in this node, either the average response value in regression or the most frequent response class in classification trees.¹⁹ In the terminal nodes of the tree, the scale of the bar chart represents the relative probability of “false negative (FN)”, “false positive (FP)”, “true negative (TN)” and “true positive (TP)” responses to the performance of GeneChip in differing combinations of characteristics in TB cases.

Five variables that were significant in the diagnosis of RIF resistance were evaluated in the CART analysis (Figure 2). Among previously treated cases, the age group was selected as the variable of second splitting discrimination. For cases aged <55 years, the next best predictor of the performance of GeneChip was the place of residence. For cases aged ≥55 years and living in urban areas, the next best predictor of the performance of GeneChip was sex. Among newly treated cases, the place of residence was assigned as second splitting discrimination. When the residence was rural, the next best predictor of the performance of GeneChip was treatment failure. For the node with new, rural, and successfully treated cases, the age group was selected as an additional significant variable. Therefore, a total of nine subgroups of cases were produced by five predictive variables selected in this CART analysis of GeneChip for the diagnosis of RIF resistance.

Figure 2 CART analysis of GeneChip performance in the diagnosis of rifampin resistance. Abbreviations: CART, Classification and Regression Trees; FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Three variables that were significant in INH resistance were evaluated in the CART analysis (Figure 3). Among previously treated cases, residence was selected as the variable of the second split for discrimination. Additionally, among newly treated cases, the place of residence was assigned as the variable of second splitting discrimination. Four subgroups of cases were produced by two predictive variables selected in this CART analysis of GeneChip for the diagnosis of INH resistance.

Figure 3 CART analysis of GeneChip performance in the diagnosis of isoniazid resistance. Abbreviations: CART, Classification and Regression Trees; FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Five variables that were significant in the MDR diagnosis were evaluated in the CART analysis (Figure 4). Among previously treated cases, age group and sex were selected as the variables of second and third splitting discrimination, respectively. For newly treated cases, the place of residence was selected as the variable of second splitting discrimination. Finally, six subgroups of cases were produced by four predictive variables selected in this CART analysis of GeneChip for the diagnosis of MDR.

Figure 4 CART analysis of GeneChip performance in the diagnosis of MDR. Abbreviations: CART, Classification and Regression Trees; FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Diagnostic Performance of GeneChip in Subgroup Patients

For RIF resistance, the overall sensitivity, specificity, PPV, NPV, AR and kappa values were 0.81 (95% CI: 0.76–0.86), 0.98 (95% CI: 0.97–0.98), 0.72 (95% CI: 0.67–0.77), 0.99 (95% CI: 0.98–0.99), 0.97 (95% CI: 0.96–0.97) and 0.75 (95% CI: 0.70–0.79), respectively (Table 2). The PPV (younger cases: 0.78, 95% CI: 0.71–0.84 vs older cases: 0.63, 95% CI: 0.54–0.71; treatment failure cases: 0.92, 95% CI: 0.73–0.99 vs cases without treatment failure: 0.70, 95% CI: 0.65–0.75) and kappa (younger cases: 0.79, 95% CI: 0.75–0.84 vs older cases: 0.67, 95% CI: 0.59–0.74; treatment failure cases: 0.92, 95% CI: 0.83–1.00 vs cases without treatment failure: 0.73, 95% CI: 0.69–0.77) of GeneChip were significantly higher in younger and treatment failure cases. Newly treated cases had significantly higher specificity (newly treated cases: 0.98, 95% CI: 0.97–0.98 vs previously treated cases: 0.94, 95% CI: 0.92–0.96), NPV (newly treated cases: 0.99, 95% CI: 0.98–0.99 vs previously treated cases: 0.96, 95% CI: 0.93–0.97), and AR (newly treated cases: 0.97, 95% CI: 0.97–0.98 vs previously treated cases: 0.92, 95% CI: 0.89–0.94). No significant difference between subgroups was observed for sex or place of residence (Table 2).

Table 2 Performance Evaluation of GeneChip for Rifampin and Isoniazid Resistance in Tuberculosis Cases with Different Characteristics

For INH resistance, the overall sensitivity, specificity, PPV, NPV, AR and kappa were 0.74 (95% CI: 0.70–0.78), 0.97 (95% CI: 0.96–0.98), 0.75 (95% CI: 0.70–0.79), 0.97 (95% CI: 0.96–0.97), 0.95 (95% CI: 0.94–0.95) and 0.71 (95% CI: 0.68–0.75), respectively. Newly treated cases had significantly higher NPV (newly treated cases: 0.98, 95% CI: 0.97–0.98 vs previously treated cases: 0.90, 95% CI: 0.87–0.93) and AR (newly treated cases: 0.95, 95% CI: 0.95–0.96 vs previously treated cases: 0.89, 95% CI: 0.86–0.91) and lower PPV (newly treated cases: 0.72, 95% CI: 0.66–0.76 vs previously treated cases: 0.85, 95% CI: 0.76–0.91). The NPV (treatment failure cases: 0.90, 95% CI: 0.82–0.95 vs cases without treatment failure: 0.97, 95% CI: 0.97–0.98) and AR (treatment failure cases: 0.90, 95% CI: 0.83–0.95 vs cases without treatment failure: 0.95, 95% CI: 0.94–0.95) were significantly lower among patients with treatment failure outcomes. In patients living in urban areas, the sensitivity (urban cases: 0.69, 95% CI: 0.61–0.76 vs rural cases: 0.77, 95% CI: 0.72–0.82) and NPV (urban cases: 0.95, 95% CI: 0.94–0.97 vs rural cases: 0.98, 95% CI: 0.97–0.98) were significantly lower, but the PPV (urban cases: 0.82, 95% CI: 0.74–0.88 vs rural cases: 0.72, 95% CI: 0.66–0.77) was higher (Table 2).

Diagnostic Performance of GeneChip for MDR-TB in Subgroup Patients

The overall sensitivity, specificity, PPV, NPV, AR and kappa for detecting MDR-TB were 0.70 (95% CI: 0.63–0.70), 0.99 (95% CI: 0.98–0.99), 0.78 (95% CI: 0.72–0.84), 0.98 (95% CI: 0.98–0.99), 0.98 (95% CI: 0.97–0.98) and 0.72 (95% CI: 0.67–0.78), respectively. The NPV (male cases: 0.99, 95% CI: 0.98–0.99 vs female cases: 0.98, 95% CI: 0.96–0.99; newly treated cases: 0.99, 95% CI: 0.98–0.99 vs previously treated cases: 0.94, 95% CI: 0.91–0.96) and AR (male cases: 0.98, 95% CI: 0.97–0.98 vs female cases: 0.97, 95% CI: 0.95–0.98; newly treated cases: 0.98, 95% CI: 0.98–0.99 vs previously treated cases: 0.92, 95% CI: 0.89–0.94) were significantly higher in males and newly treated cases, but a significantly lower NPV (urban cases: 0.97, 95% CI: 0.96–0.98 vs rural cases: 0.99, 95% CI: 0.98–0.99) and AR (urban cases: 0.96, 95% CI: 0.95–0.99 vs rural cases: 0.98, 95% CI: 0.98–0.99) was observed in urban patients. The specificity value (newly treated cases: 0.99, 95% CI: 0.99–1.00 vs previously treated cases: 0.96, 95% CI: 0.94–0.98) was significantly higher in newly treated cases. No significant difference was observed in the performance of GeneChip for detecting MDR between patients with different treatment outcomes (Table 2).

RIF Resistance, INH Resistance, and MDR-TB

Patients sensitive to RIF detected by GeneChip had a significantly low probability of MDR (Prob =0.008, 95% CI: 0.006–0.012, P<0.001). RIF resistance was correlated with MDR (kappa=0.636, 95% CI: 0.584–0.688, P<0.001), which was regarded as “substantial agreement”. Patients with INH resistance detected by GeneChip had a significant probability of MDR (Prob =0.360, 95% CI: 0.314–0.407, P<0.001). Patients without INH resistance detected by GeneChip had a significantly low probability of MDR (Prob =0.013, 95% CI: 0.010–0.017, P<0.001) showing “moderate agreement” (kappa =0.452, 95% CI: 0.395–0.510, P<0.001). Patients with MDR detected by GeneChip had a significantly higher probability of MDR detected by DST (Prob =0.783, 95% CI: 0.716–0.841, P<0.001) than patients without MDR (Prob =0.016, 95% CI: 0.012–0.020, P<0.001). Patients with MDR detected by GeneChip were strongly correlated with MDR (kappa =0.723, 95% CI: 0.670–0.777, P<0.001), which was regarded as “substantial agreement” (Table 3). Additionally, younger patients with MDR detected by GeneChip had a significantly higher kappa value (0.77) than older patients (0.65) (Table 2).

Table 3 Performance of GeneChip Detecting RIF Resistance, INH Resistance, and MDR Compared with DST

Discussion

In this comparative study, we explored the diagnostic performance of GeneChip in detecting RIF resistance, INH resistance, and MDR-TB in overall and subgroup TB patients. Characteristics including sex, age, treatment history, treatment outcomes, and place of residence were found to be significantly associated with the performance of GeneChip. We also established a CART approach to identify predictors of GeneChip performance.

Globally, there were an estimated 465,000 (range: 400,000–535,000) incident cases of MDR/RIF-resistant TB (RR-TB) in 2019, and 78% of RR-TB cases were estimated to be MDR-TB. Nearly 50% of global cases were in India (27%), China (14%), and the Russian Federation (8%).¹ Drug-resistant TB has shifted the schema of global TB control and has been a major public health concern. Therefore, the possibility of drug resistance should be considered. The CapitalBio DNA microarray, which incorporates specific nucleotides at given positions of the rpoB, inhA, and katG genes, has been developed to detect M.tb isolates and MDR forms in sputum specimens and proven to have notable sensitivity and specificity.^16,21,22

A study in Zhejiang, China, reported that GeneChip had a sensitivity of 86.08% and a specificity of 97.70% for RIF resistance and a sensitivity of 79.36% and a specificity of 98.71% for INH resistance.²³ For MDR-TB, the sensitivity was 78.01%, and the specificity was 98.86%.²³ Zhu et al explored the performance of GeneChip on 1747 sputum specimens and showed a sensitivity of 83.30% and a specificity of 97.93% for RIF resistance and a sensitivity of 63.81% and a specificity of 97.57% for INH resistance.²¹ Another study in China reported that GeneChip performed better in newly treated cases in detecting RIF resistance, INH resistance, and MDR-TB.²⁴ However, few studies have evaluated the performance of GeneChip in subgroups of patients with different characteristics.

Previous studies found that RIF-resistant TB was more prevalent among relapse and treatment failure cases, but had a lower frequency among older patients.²⁵ Compared with rural areas, urban areas have a higher prevalence of MDR-TB.^26,27 Therefore, the characteristics of cases should not be ignored when evaluating the performance of GeneChip.^24,28 In the current study, we observed that sex, age, treatment history, treatment failure outcome, and place of residence were significantly associated with the diagnostic value of GeneChip.

Prediction, classification, and the assessment of drug resistance are fundamental tasks in TB clinical treatment management. For the classification of GeneChip performance, it is possible to predict an estimate of the class probabilities from the relative frequencies of each class in the terminal nodes of CART.¹⁹ We observed a high concordance between GeneChip and DST among TB patients with different characteristics, indicating that GeneChip can be a potential alternative tool for rapid MDR-TB detection. It may be used as an ancillary test for drug resistance for urgent clinical needs.^29,30

Although MDR-TB is defined as resistance to at least RIF and INH, the critical determinant for treatment failure is RIF resistance. Detection of RIF resistance has thus been proposed as a proxy for MDR-TB diagnosis,³¹ especially in areas with a high burden of drug resistance.³² Areas with additional circulating drug-resistant strains should use comprehensive DST before prescribing a standard shorter-term regimen.³³ The WHO recommends that TB patients with confirmed resistance to RIF can be treated as MDR until INH resistance is confirmed.³⁴

The Xpert M.tb/RIF assay (Cepheid, Sunnyvale, CA, USA) is an automated, single-cartridge-based nucleic acid amplification test.³⁵ It has excellent performance with lower biosafety requirements and more straightforward contamination control. However, the major drawback of real-time PCR-based methods is that the instrumentation and reagent costs at present are significantly higher than those of array-based technologies.¹² Compared with Xpert, GeneChip is relatively less expensive and more cost-effective.¹⁷ Although GeneChip has a lower sensitivity than the Xpert M.tb assay, their specificities were both >0.97.¹⁶ Therefore, GeneChip still has value in clinical application, especially in areas lacking resources.

A major strength of this study is that we recruited all smear-positive TB cases in four counties and one downtown area of the study area, which guaranteed good representativeness of the samples. In addition, data were collected within ten consecutive years, and the sample size was relatively large for molecular epidemiological study. This study provided novel insight into the performance of GeneChip in differing characteristic TB cases and potentially worth expanding to clinical settings in China.

However, several limitations of this study should not be ignored. First, we did not routinely test each patient for HIV, which might affect the accurate estimation of the diagnostic value of GeneChip. Considering the low prevalence of HIV in the study area, its effect was believed to be low. Second, recall bias of the treatment history may cause misclassification of newly and previously treated cases. Third, low-level but probably clinically relevant RIF-resistant TB linked to some specific rpoB mutations, which are not covered by GeneChip, is easily missed, resulting in the risk of a false negative.^30,36,37

In conclusion, GeneChip is a rapid diagnostic method that can detect drug resistance to RIF and INH among different subgroups of TB patients. We observed a high concordance between GeneChip and DST among patients with different characteristics, indicating that GeneChip can be a potential alternative tool for rapid MDR-TB detection.