Back to Journals » Cancer Management and Research » Volume 13

Development and Internal Validation of a Nomogram Used to Predict Chemotherapy-Induced Neutropenia in Non-Small Cell Lung Cancer Patients: A Retrospective Cohort Study

Authors Zou W, Xu NL

Received 26 January 2021

Accepted for publication 3 March 2021

Published 26 March 2021 Volume 2021:13 Pages 2797—2804


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Antonella D'Anneo

Wei Zou,1 Neng-Luan Xu2

1Shengli Clinical Medical College of Fujian Medical University, Fujian Medical University, Fuzhou, Fujian, People’s Republic of China; 2Department of Pulmonary and Critical Care Medicine, Fujian Provincial Hospital, Fuzhou, Fujian, People’s Republic of China

Correspondence: Neng-Luan Xu
Department of Pulmonary and Critical Care Medicine, Fujian Provincial Hospital, No. 134 Dong Street, Fuzhou, 350001 Fujian, People’s Republic of China
Tel +86 13805088136
Fax +86 0591-87557768
Email [email protected]

Purpose: This study was designed to develop a nomogram for predicting neutropenia caused by chemotherapy in non-small cell lung cancer (NSCLC) patients.
Patients and Methods: Information was collected from 376 patients between November 2017 and November 2020. The endpoint was chemotherapy-induced neutropenia (absolute neutrophil count < 2× 109/L). Logistic regression was performed to appraise the prognostic value of each potential predictor. Risk predictors from the final predictive model were used to generate a nomogram. C-index and calibration curve as well as decision curve analysis (DCA) was applied to evaluate model performance.
Results: The multivariate regression model ultimately included three predictors: previous radiotherapy, the current cycle of chemotherapy and neutrophil counts before current chemotherapy. A nomogram was developed and displayed better discrimination (with C-index of 0.875 in the development group and 0.907 in the validation group). Favorable consistency was shown between predicted probability and observed probability in the calibration curves. DCA illustrated that when the threshold probability was 8%-90%, the predictive model provided a net benefit relative to the intervention-all or the intervention-none strategy, indicating that the nomogram had favorable potential clinical utility.
Conclusion: This nomogram will be an available tool to quantify the risk of neutropenia after chemotherapy in patients who suffer from NSCLC and deserves further external validation.

Keywords: chemotherapy, neutropenia, nomograms, non-small cell lung cancer


Chemotherapy plays a significant part in methods to treat non-small cell lung cancer,1 and neutropenia is a major adverse event following chemotherapy. A review reported that chemotherapy-induced neutropenia is correlated with the chemotherapy drug dose reductions as well as delays that can compromise therapeutic efficacy and severe infections.2 Furthermore, neutropenia-related complications often give rise to additional health care costs.3 To settle these problems, many studies endeavored to find measures for reducing the occurrence of chemotherapy-induced neutropenia. Relevant researches have concluded that the prophylactic utilization of colony-stimulating factor (CSF) can reduce the occurrence rate of infection-related hospitalization, severity, risk and duration of neutropenia.4–7 However, because of the associated costs, CSF is recommended for high-risk patients with neutropenia-related complications.8,9 Hence, numerous studies have sought to determine risk factors that can identify high-risk patients, to maximize the benefits and decrease medical costs by using more economical and effective prevention strategies.

There have been a great number of prediction models of chemotherapy-induced neutropenia, but most of them are for solid tumors. Moreover, to enhance the cost-effectiveness of colony-stimulating factor, many models were constructed to predict severe (absolute neutrophil count <1×109/L) and febrile (temperature≥38°C) neutropenia.10–12 These models excluded other patients who also experienced chemotherapy-induced neutropenia, like patients who had neutropenia-related infections while did not meet the standards mentioned above. Besides, to develop more useful prevention strategies, dose individualization, prophylactic antibiotics and other measures are also being investigated besides prophylactic CSF.13–15 Therefore, these predictive models have some limitations in the application.

For constructing a model that was appropriate for other prevention measures and detected neutropenia in time, a clinical nomogram that can predict the risk of neutropenia after chemotherapy for patients who have non-small cell lung cancer was developed and subsequently validated internally. The outcome was defined as the neutrophil count is less than 2×109/L in this research, which was different from most extant models.

Patients and Methods

Participants and Data Collection

This study developed an NSCLC dataset and the data were collected between November 2017 and November 2020 at Fujian Provincial Hospital in China. Inclusion criteria were: 1) All participants were pathologically confirmed to suffer from NSCLC. 2) The patients who had to accomplish at least 1 cycle of chemotherapy. 3) PS (performance status) ≤2. 4) Blood routine was performed before chemotherapy. 5) Blood routine was performed at least once a week for three consecutive weeks after chemotherapy. Exclusion criteria were: 1) Neutropenia occurred in prior chemotherapy. 2) Colony-stimulating factor (CSF) was used before chemotherapy. 3) Patients with a history of primary hematological diseases. Potential predictive factors were filtrated based on the literature and clinical reasoning. So we collected demographic characteristics, clinical characteristics and laboratory results included age, gender, body surface area, histological type, initial stage, bone metastases, number of drugs in the current chemotherapy, prior history of radiotherapy, previous history of surgery, the current cycle of chemotherapy, and neutrophil counts before current chemotherapy. All data came from electronic medical records without missing data. The ethics committee of Fujian Provincial Hospital approved this retrospective cohort study (approval number: K2021-01-010) and approved the waiver of informed consent due to the retrospective nature of the study. Furthermore, the ethics committee ensured all patient data confidentiality and its compliance with the Declaration of Helsinki.

Development and Internal Validation Population

A total of 376 patients were enrolled in total and were separated randomly into development (n=264) and validation (n=112) population using a 70% versus 30% split by R software version 4.0.3.

Assessment of Neutropenia

Blood routine examination data, including neutrophil count (cells×109/L), were collected after chemotherapy and at least once per week for three weeks. Neutropenia (neutrophil count < 2×109/L) has been used as an indicator for evaluating toxic effects of chemotherapy in many studies16–20 according to WHO HANDBOOK FOR REPORTING RESULTS OF CANCER TREATMENT. Following the WHO criteria21 for myelosuppression, neutropenia signified the neutrophil count is less than 2×109/L. For patients with multiple episodes of neutropenia, the data from the first occurrence of neutropenia were collected. For patients who had never experienced neutropenia during chemotherapy, the date from the most recent chemotherapy was collected.

Statistical Analysis

Continuous variables were presented as median (interquartile range), and Mann–Whitney U-test was applied in the inter-group comparisons. In addition, categorical variables were expressed as frequency and proportion, and Fisher’s exact test or chi-squared test was performed in the inter-group comparisons. Univariate logistic regression analysis was performed to analyze the relationship between neutropenia as a dependent variable and each of potential predictors as an independent variable for selecting diagnostic factors in the development cohort. Potential predictors with P-value <0.1 were chosen for multivariate analysis which was performed with forward stepwise logistic regression on, to construct multivariate diagnostic model. The statistical analysis mentioned above was performed by using IBM SPSS Statistics version 20.0.A nomogram was constructed to predict the probability of neutropenia caused by chemotherapy in patients with NSCLC by risk predictors in the final multivariate logistic regression model, and ROC curve was plotted to estimate the ability of the predictive model. C-index, calibration curve as well as decision curve analysis was applied to estimate discrimination, as well as calibration and clinical utility. R software version 4.0.3 was used for statistical analysis in this section.


Participant Characteristics

This study included data from 376 patients. The development group consisted of 264 patients. The median age was 61.5 years (range,25 to 80), 72.7% were male and 41.7% had stage 4 lung cancer. The internal validation group consists of 112 patients. The median age was 61 years (range,27 to 78), 72.3% were male and 47.3% had stage 4 lung cancer. A total of 125 episodes of neutropenia were observed in total in the development group and 55 episodes in the validation group throughout the research period. Characteristics are depicted in Table 1, all characteristics were no statistically significant differences between development group and validation group (all P>0.05). There was no missing data during the data collection.

Table 1 Characteristics in Development Group and Validation Group

Construction of Diagnostic Model in the Development Group

The factors with p<0.10 were gender, previous radiotherapy, number of drugs in the current chemotherapy, the current cycle of chemotherapy, and neutrophil counts before current chemotherapy in the univariate analysis. These variables were entered into a multivariate logistic regression analysis. After stepwise forward elimination, the diagnostic model ultimately included three predictors: previous radiotherapy, the current cycle of chemotherapy and neutrophil counts before current chemotherapy. The procedure is presented in detail in Table 2.

Table 2 Univariate and Multivariate Logistic Regression Analysis for Neutropenia in the Development Group

Nomogram Construction, Validation and Assessment

On the basis of the multivariate analysis, a predictive model was finally presented as a nomogram (Figure 1). For individual patient, each value of these predictors has a corresponding score on the uppermost rule. A total score was obtained by adding every single score together. Finally, the risk probability of neutropenia was found on the lowest rule. The ROC curves of the predictive model are depicted in Figure 2A and B. The C-index was 0.875 (0.8337–0.9167, 95% CI) in the development group (Figure 2A) and 0.907 (0.8555–0.9585, 95% CI) in the validation group (Figure 2B), which meant excellent discrimination. For the development cohort, when 51.2% was chosen as the cut-off threshold, sensitivity as well as specificity values was 80% and 82%, respectively. Sensitivity and specificity values were 80% and 86% when 46.9% was the cut-off threshold in the validation cohort. In addition, calibration curves were drawn to assess calibration. Favorable consistency was shown between predicted probability and observed probability in both the development group and the validation group (Figure 3A and B). Decision curve analysis was applied to quantify the clinical utility by utilizing data from the development cohort (Figure 4). DCA demonstrated that when the threshold probability was 8%–90%, the nomogram provided a net benefit relative to the intervention-all or the intervention-none strategy, indicating that the nomogram had favorable potential clinical utility.

Figure 1 Nomogram for predicting chemotherapy-induced neutropenia. The nomogram included three predictors: previous radiotherapy, the current cycle of chemotherapy and neutrophil counts before current chemotherapy. Each value of these predictors has a corresponding score on the rule listed at the top of the figure. A total score was calculated by adding every single score together. Finally, the risk probability of neutropenia was found on rule listed on the bottom of the figure.

Figure 2 (A) ROC curve of the constructed nomogram in the development group. (B) ROC curve of the constructed nomogram in the validation group.

Figure 3 (A) Calibration curve in the development cohort. (B) Calibration curve in the validation cohort. The x-axis and y-axis indicate the predicted and observed probability of chemotherapy-induced neutropenia, respectively. The long-dashed line represents a perfect prediction through the use of an ideal model. The short-dashed line and the black line indicate the performance of the apparent and bias-corrected model, respectively.

Figure 4 The DCA of the nomogram predicting the occurrence of neutropenia. The x-axis indicates the threshold probability as well as the y-axis means net benefit. The gray line named “All” considers all patients will experience neutropenia whereas the horizontal black line named “None” considers all patients will not experience neutropenia. The dashed line named “Model” represents the net benefit by using the nomogram.


Neutropenia is a common and severe complication of chemotherapy. It can give rise to additional health care costs and life-threatening infections. An online survey of cancer patients who had experienced chemotherapy-induced myelosuppression (CIM) showed that 88% of participants held an idea that CIM has a moderate or significant influence on their lives.22 The data emphasized that the real-world influence of CIM on patients is huge. So a predictive model that can distinguish patients who probably suffer from neutropenia after a specific cycle of chemotherapy was constructed. This model is appropriate for the following applications: 1) More and more prevention measures are under study, they need corresponding models to test and improve the efficacy and cost-effectiveness. 2) It highly recommends that high-risk patients need re-examined blood routine frequently to discover and dispose of neutropenia timely, because neutropenia often occurs after hospital discharge according to our data. 3) Clinicians can communicate with patients in advance and promote shared decision-making.

Most models are not specific for NSCLC, so this research developed a predictive model for it and the outcome was defined as the neutrophil count is less than 2×109/L. Previous radiotherapy, the current cycle of chemotherapy and neutrophil counts before current chemotherapy were included by multivariate logistic regression analysis. According to the result, earlier cycles of chemotherapy are at a higher risk of neutropenia, which corresponds to previous studies.2 The risk reduction in subsequent cycles probably due to the lessening of neutrophil counts during chemotherapy, which makes clinicians take action to halt the decline, such as dose reduction. Radiotherapy is a major treatment for NSCLC, and neutropenia is a major complication in patients receiving radiotherapy.23,24 The history of previous radiotherapy was significant in the univariate and multivariate analysis in this study. Finally, a predictive model was constructed, which had excellent discrimination, calibration and clinical utility. To make it easier for clinicians, this study developed a simplified nomogram25,26 Based on simple and available information, clinicians can distinguish patients who probably experience neutropenia.

Despite these advantages, several limitations should be noted. Firstly, it was a single-center retrospective study with small sample sizes. The results may be influenced by selective bias. Secondly, although the C-index was found to be 0.907 in the internal validation of the predictive model, the nomogram still requires further verification by more independent external cohorts. Finally, the research only offered a conjecture about the relationship between predictors and outcome. Therefore, further studies are needed to answer mechanistic questions. To sum up, prospective and multicenter clinical trials with large sample sizes should be carried out in the future.


In short, this study constructed a predictive model which was presented as nomogram to quantify the risk of neutropenia in patients with NSCLC after chemotherapy. This nomogram will be available for clinical practice as it enables a simple calculation of individual probability, and clinicians can find a new angle on treatment strategy decisions by using it. Although needing further verification, this predictive model still is a potentially highly valuable tool for clinicians and researchers.


We would like to show gratitude to all staff members in the Respiratory Medicine Department of the Fujian Provincial Hospital.


The authors report no conflicts of interest in this work.


1. Zappa C, Mousa SA. Non-small cell lung cancer: current treatment and future advances. Transl Lung Cancer Res. 2016;5(3):288–300. doi:10.21037/tlcr.2016.06.07

2. Crawford J, Dale DC, Lyman GH. Chemotherapy-induced neutropenia: risks, consequences, and new directions for its management. Cancer. 2004;100(2):228–237. doi:10.1002/cncr.11882

3. Stokes ME, Muehlenbein CE, Marciniak MD, et al. Neutropenia-related costs in patients treated with first-line chemotherapy for advanced non-small cell lung cancer. J Manag Care Pharm. 2009;15(8):669–682. doi:10.18553/jmcp.2009.15.8.669

4. Crawford J, Ozer H, Stoller R, et al. Reduction by granulocyte colony-stimulating factor of fever and neutropenia induced by chemotherapy in patients with small-cell lung cancer. N Engl J Med. 1991;325(3):164–170. doi:10.1056/NEJM199107183250305

5. Trillet-Lenoir V, Green J, Manegold C, et al. Recombinant granulocyte colony stimulating factor reduces the infectious complications of cytotoxic chemotherapy. Eur J Cancer. 1993;29(3):319–324. doi:10.1016/0959-8049(93)90376-Q

6. Li L, Ma S, Wu M, Tan X, Zhong S, Lang J. The prophylactic effects of long-acting granulocyte colony-stimulating factor for febrile neutropenia in newly diagnosed patients with epithelial ovarian cancer: a randomised controlled study. BMJ Support Palliat Care. 2019;9(4):373–380. doi:10.1136/bmjspcare-2019-001862

7. Dale DC. Colony-stimulating factors for the management of neutropenia in cancer patients. Drugs. 2002;62(Suppl 1):1–15. doi:10.2165/00003495-200262001-00001

8. Lyman GH, Kuderer NM. The economics of the colony-stimulating factors in the prevention and treatment of febrile neutropenia. Crit Rev Oncol Hematol. 2004;50(2):129–146. doi:10.1016/j.critrevonc.2004.01.001

9. Cosler LE, Calhoun EA, Agboola O, Lyman GH. Effects of indirect and additional direct costs on the risk threshold for prophylaxis with colony-stimulating factors in patients at risk for severe neutropenia from cancer chemotherapy. Pharmacotherapy. 2004;24(4):488–494. doi:10.1592/phco.24.5.488.33360

10. Bozcuk H, Yıldız M, Artaç M, et al. A prospectively validated nomogram for predicting the risk of chemotherapy-induced febrile neutropenia: a multicenter study. Support Care Cancer. 2015;23(6):1759–1767. doi:10.1007/s00520-014-2531-6

11. Lyman GH, Lyman CH, Agboola O. Risk models for predicting chemotherapy-induced neutropenia. Oncologist. 2005;10(6):427–437. doi:10.1634/theoncologist.10-6-427

12. Cao X, Ganti AK, Stinchcombe T, et al. Predicting risk of chemotherapy-induced severe neutropenia: a pooled analysis in individual patients data with advanced lung cancer. Lung Cancer. 2020;141:14–20. doi:10.1016/j.lungcan.2020.01.004

13. Gjafa E, Ng K, Grunewald T, et al. Neutropenic sepsis rates in patients receiving bleomycin, etoposide and cisplatin chemotherapy using olanzapine and reduced doses of dexamethasone compared to a standard antiemetic regimen. BJU Int. 2020;127:205–211. doi:10.1111/bju.15175

14. Kouranos V, Dimopoulos G, Vassias A, Syrigos KN. Chemotherapy-induced neutropenia in lung cancer patients: the role of antibiotic prophylaxis. Cancer Lett. 2011;313(1):9–14. doi:10.1016/j.canlet.2011.08.024

15. Lombard A, Mistry H, Aarons L, Ogungbenro K. Dose individualisation in oncology using chemotherapy-induced neutropenia: example of docetaxel in non-small cell lung cancer patients. Br J Clin Pharmacol. 2020. doi:10.1111/bcp.14614

16. Atasilp C, Chansriwong P, Sirachainan E, et al. Effect of drug metabolizing enzymes and transporters in Thai colorectal cancer patients treated with irinotecan-based chemotherapy. Sci Rep. 2020;10(1):13486. doi:10.1038/s41598-020-70351-0

17. Choi JH, Choi J, Chung SM, et al. The clinical efficacy and safety of four-weekly docetaxel as first-line therapy in elderly lung cancer patients with squamous cell carcinoma. Tuberc Respir Dis (Seoul). 2019;82(3):211–216. doi:10.4046/trd.2018.0019

18. Garg MB, Lincz LF, Adler K, Scorgie FE, Ackland SP, Sakoff JA. Predicting 5-fluorouracil toxicity in colorectal cancer patients from peripheral blood cell telomere length: a multivariate analysis. Br J Cancer. 2012;107(9):1525–1533. doi:10.1038/bjc.2012.421

19. Hashimoto H, Oshika Y, Obara K, Tanaka Y, Shimizu E. [A dialysis patient with advanced lung adenocarcinoma who was safely given biweekly gemcitabine therapy]. Gan Kagaku Ryoho. 2010;37(8):1553–1556. Japanese.

20. Nishio M, Ohyanagi F, Taguch F, et al. Phase I study of combination chemotherapy with gemcitabine and irinotecan for non-small cell lung cancer. Lung Cancer. 2005;48(1):115–119. doi:10.1016/j.lungcan.2004.09.006

21. World Health Organization. WHO Handbook for Reporting Results of Cancer Treatment. Geneva: World Health Organization; 1979.

22. Epstein RS, Aapro MS, Basu Roy UK, et al. Patient burden and real-world management of chemotherapy-induced myelosuppression: results from an online survey of patients with solid tumors. Adv Ther. 2020;37(8):3606–3618. doi:10.1007/s12325-020-01419-6

23. Videtic G, Donington J, Giuliani M, et al. Stereotactic body radiation therapy for early-stage non-small cell lung cancer: executive summary of an ASTRO evidence-based guideline. Pract Radiat Oncol. 2017;7(5):295–301. doi:10.1016/j.prro.2017.04.014

24. Guckenberger M, Andratschke N, Dieckmann K, et al. ESTRO ACROP consensus guideline on implementation and practice of stereotactic body radiotherapy for peripherally located early stage non-small cell lung cancer. Radiother Oncol. 2017;124(1):11–17. doi:10.1016/j.radonc.2017.05.012

25. Kattan MW, Scardino PT. Evidence for the usefulness of nomograms. Nat Clin Pract Urol. 2007;4(12):638–639. doi:10.1038/ncpuro0968

26. Vickers AJ, Cronin AM. Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). Urology. 2010;76(6):1298–1301. doi:10.1016/j.urology.2010.06.019

Creative Commons License This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.

Download Article [PDF]