Moving Past Just Confidence Intervals: Why We Need Stronger Heterogeneity Assessment in Lung Cancer Meta-Analyses? [Letter]

Dear editor

We read with great interest the systematic review and meta-analysis by Qiao et al,¹ entitled “Chronic Obstructive Pulmonary Disease and Lung Cancer: A Meta-Analysis of Risk Association”. We commend the authors for tackling this important topic. However, several methodological concerns may limit the reliability and interpretability of its findings.

Our primary concern is about studies included in the meta-analysis for assessing the relative risk (RR) of lung cancer:

1. The inclusion of Powell et al² is problematic due to its case–control design comparing patients with lung cancer to individuals without cancer. This methodology precludes the accurate derivation of incidence rates and thus overestimates the relative risk of lung cancer in COPD patients. The observed higher prevalence of COPD among cancer cases (54.7% vs. 20.5% in controls) reflects exposure prevalence rather than outcome incidence, and the association may be inflated by ascertainment bias, where COPD is often diagnosed concurrently with lung cancer investigations.
2. Indeed, the included study by Lee et al³ was unsuitable as this investigation exclusively enrolled patients with a confirmed COPD diagnosis. Consequently, it provides no data on the baseline risk difference between individuals with and without COPD.

Secondly, regarding the assessment of the RR of lung cancer associated with inhaled cortico-steroids (ICS) use among COPD patients, the inclusion of the Kiri study, while employing a nested case–control design, was not appropriate for a direct “ICS vs. non-ICS” comparison. Its control group consisted of short-acting bronchodilator users, rather than a “non-ICS” group.

Third, reliance solely on the 95% confidence interval (CI) is insufficient to assess heterogeneity. While the CI describes the precision of the mean effect size, it does not address how much the true effect size might vary across studies. The prediction interval (PI) is the appropriate metric to assess heterogeneity, as it quantifies the range of true effect sizes that may be observed in future clinical settings.^4,5 As explained by Borenstein et al,⁴ “I² is not an absolute measure of heterogeneity”. Based on the data presented in Figure 3 of the manuscript,¹ we calculated the 95% PI for the RR of lung cancer associated with ICS use, which ranged from 0.57 to 2.34. As this interval includes 1, it demonstrates no statistically significant difference in risk (Supplementary Figure 1). However, given that the included studies were a mix of prospective and retrospective designs, the calculation of a relative risk may not be appropriate. Therefore, we re-extracted the data and calculated the risk difference instead, which ranged from -0.036 to 0.028 (Supplementary Figure 2). The inclusion of 0 within this interval indicates that no statistically significant difference in risk could be ascertained.

These results suggest that future studies may find no protective effect of ICS use against lung cancer development in COPD patients, or even a potential increased risk.

Fourth, the data presented in Table 1 appear to contain several inconsistencies. Specifically, the case and control data for the studies by Parimon,⁶ Kiri,⁷ and Liu⁸ seem to have been inverted. Moreover, the data reported by Suissa et al⁹ do not correspond to those in the original article. In addition, the reference numbers cited in Table 1 do not align with those used in the main text and should be verified for consistency.

Our conclusion is different from the authors’ conclusion, which was “The use of ICS was associated with a slightly increased risk in lung cancer development, and male individuals exhibited a higher risk compared to females”.¹