A Novel Nomogram for Predicting Morbidity Risk in Patients with Secondary Malignant Neoplasm of Bone and Bone Marrow: An Analysis Based on the Large MIMIC-III Clinical Database

Introduction

Bone and bone marrow are the third most frequent sites of tumor metastases, following the lungs and liver.¹ Once diagnosed, bone metastases can rarely be cured, indicating a short-term prognosis in cancer patients. Breast and prostate cancers are the most frequent origins of skeletal metastases (over 70%).² Bone metastases are much more common than primary bone cancers. The median survival times of patients with bone metastases of melanoma, and of lung, bladder, renal, prostate, breast, and thyroid cancers, are low, being 6 months, 6 months, 6–9 months, 12 months, 12–53 months, 19–24 months, and 48 months, respectively.³ Bone and bone marrow metastases are also associated with high morbidity rates, including pathologic fractures, severe pain, impaired mobility, hypercalcemia, and bone marrow aplasia.²

Effective methods for diagnosing bone and bone marrow metastases have not yet been developed, especially for patients in general intensive care units (ICUs). The development of a severity assessment model is essential to stratify patients at risk of mortality.⁴ Although several critical-care scoring tools are currently available for clinical use, no effective bedside prognostic scoring models can predict mortality in patients with bone and bone marrow metastases.

A nomogram is a commonly used tool based on several key variables and parameters that can mathematically predict patient prognosis, including disease progression and death.⁵ The ability to calculate the probability of a clinical event based on several critical factors⁶ can give rise to a powerful and easy-to-use method for predicting outcomes in individual patients.⁷

The main objective of the present study was to identify factors independently predictive of survival in patients with secondary malignant neoplasms of bone and bone marrow. The patient cohort and the factors were selected from the Medical Information Mart for Intensive Care III (MIMIC-III) database, and a prognostic nomogram for improving prediction of overall survival (OS) of these patients was developed.

Materials and Methods

Data Source

All the data were extracted from the MIMIC-III database (version 1.4), which contains information about over 58,000 patients hospitalized at the Beth Israel Deaconess Medical Center in Boston, MA, including 38,645 adult and 7875 neonatal patients.⁸ Information on patients in this database was anonymous, thus informed consent was not required. The research personnel participated in courses from the US National Institutes of Health (NIH) and were authorized to access the database (No: 40269496).

Study Population

Data were extracted with the Structured Query Language (SQL) program in PostgreSQL (version 13.5), and the database was screened for patients with secondary malignant neoplasm of bone and bone marrow using the International Classification of Diseases, ninth edition (ICD-9), code 1985. Patients aged <18 years and those who died within 24 h following admission to an ICU were excluded. The primary outcome was all-cause death rates 30 days after hospital admission.

Information was extracted from the MIMIC-III database using hadm_id and icustay_id. Demographic and clinical factors included age, gender, ethnicity, marital status, insurance, comorbidities, vital signs, laboratory parameters, severity scoring system, and survival information. Comorbidities, as assessed using the Elixhauser Comorbidity Index, included hypertension, obesity, diabetes, chronic pulmonary, renal failure, liver disease, heart disease, fluid electrolyte disorders, alcohol abuse, and anemia. Vital signs included mean heart rate, blood pressure (MBP), respiratory rate, temperature, and percutaneous oxygen saturation (SpO₂) during the first 24 h of ICU stay. Laboratory parameters included anion gap (AG); hematocrit; bicarbonate, creatinine, chloride, glucose, hemoglobin, potassium, sodium, and blood urea nitrogen (BUN) concentrations; white blood cell (WBC) and platelet counts; international normalized ratio (INR); prothrombin time (PT); and partial prothrombin time (PTT). Severity scoring systems included angus score, Simplified Acute Physiology Score II (SAPSII), Sequential Organ Failure Assessment (SOFA) score, acute physiology score III (APSIII), and Oxford Acute Severity of Illness Score (OASIS).

Statistical Analysis

Parameters with >20% missing values were not included in the study. Values missing for other parameters were filled in using multiple imputation with the “mice” package of R open-source software.

The dataset was randomly divided into a training cohort (70%) and a validation cohort (30%). The training cohort was utilized to construct the nomogram, and the validation cohort was used for external validation. Categorical variables were reported as frequency and percentage and compared using the chi-square or Fisher’s exact test. Continuous variables were reported as mean- and standard-deviation or as median and interquartile-range, with the Shapiro–Wilk test utilized to determine whether the data were normally distributed.

Variables associated with 30-day all-cause mortality rate were initially determined using univariate Cox regression analysis. Factors differing significantly in univariate analyses (P < 0.05) were included in a multivariable Cox regression model with forward stepwise selection, with the results of the multivariate analysis visualized using a nomogram. The predictive accuracy of the nomogram was evaluated using Harrell’s concordance index (C-index) and the area under the receiver operating characteristic curve (AUC).⁹ The consistency between predicted probabilities and actual outcomes was assessed using a calibration curve.¹⁰ Model accuracy was compared by determining net reclassification improvement (NRI), with the effectiveness of improvements determined by calculating integrated discrimination improvement (IDI).¹¹ The clinical value of the predicted models was determined by decision curve analysis (DCA).¹²

Statistical analyses were performed using R software (version 3.6.1, CRAN) and SPSS software (version 24.0, Chicago, IL), with P < 0.05 considered statistically significant.

Results

Baseline Characteristics of Patients

This study enrolled 872 patients with secondary malignant neoplasm in bone and bone marrow, 610 in the training cohort and 262 in the validation cohort. The training cohort included 262 (42.6%) women and 350 (57.4%) men, of median age 63.8 years (IQR = 55.8–73.4 years), whereas the validation cohort included 120 (45.8%) women and 142 (54.2%) men of median age 63.7 years (IQR = 56.0–73.5 years). Most of the patients in the training and validation cohorts were white (>70%) and married (>59%), and had medicare (>47%) or private (37%) insurance.

The baseline clinicopathological characteristics of the training and validation cohorts did not differ significantly (Table 1). The median length of hospital stay was 8 days (IQR = 5.54–13 days) in the training cohort and 8.5 days (IQR = 4.89–14.4 days) in the validation cohort, whereas the median length of ICU stay was 2 days in both cohorts. The 30-, 60-, and 90-day mortality rates in the training cohort were 37.2% (n = 227), 48.6% (n = 297), and 56.7% (n = 346), respectively. The 30-day period was selected for further analyses.

Table 1 Baseline Demographic and Laboratory Characteristics of Patients with Secondary Malignant Neoplasm of Bone and Bone Marrow in MIMIC-III Database

Nomogram Construction

Univariate analyses showed that significant predictors of 30-day mortality were age, angus score, SAPSII, SOFA, OASIS, APSII, comorbidities with liver disease, heart disease, coagulopathy, fluid electrolyte disorder, AG, bicarbonate, chloride, glucose, hemoglobin, potassium, sodium, PTT, INR, PT, WBC, heart rate, respiratory rate, temperature, and SpO₂ (Table 2). All of these factors were included for the multivariate Cox regression analyses, which found that factors predictive of improved 30-day survival included temperature (hazard ratio [HR] = 0.68, P = 0) and SpO₂ (HR = 0.94, P = 0.006), whereas risk factors included SOFA (HR = 1.08, P = 0.042), OASIS (HR = 1.03, P = 0.022), comorbidities with coagulopathy (HR = 1.71, P = 0.001), WBC count (HR = 1.01, P = 0.024), heart rate (HR = 1.01, P = 0.013), and respiratory rate (HR = 1.06, P = 0.003) (Table 3). A nomogram was established based on the significant variables identified in multivariate analyses (Figure 1). The nomogram showed that temperature had the greatest impact on prognosis, followed by respiratory rate, WBC, SpO₂, OASIS, SOFA, heart rate, SAPSII, APSIII, and coagulopathy.

Table 2 Univariate Cox Regression Analysis Based on First 24 h Data in the Training Set

Table 3 Multivariate Cox Regression Analysis Based on First 24 h Data in the Training Set

Figure 1 Nomogram predicting 30-day mortality. The point of each variable was summed to obtain a total score corresponding to the predicted probability of 30-day survival, shown at the bottom of the nomogram.

Nomogram Validation

C-index analysis was performed to confirm the validity nomogram. The C-index values of the training cohort and validation cohorts were 0.82 (95% confidence interval [CI] = 0.85–0.79) and 0.81 (95% CI = 0.87–0.76), respectively, with these high C-index values for 30-day survival indicating that the model had good discriminative ability. A comparison of the predictive abilities of the nomogram and the SOFA scoring system showed that the AUC values of the nomogram in the training and validation cohorts were 0.821 (95% CI = 0.787–0.855) and 0.811 (95% CI = 0.755–0.866), respectively, with both being significantly higher than the AUC values of the SOFA system (Figure 2). The optimal cutoff point in the training cohort was 0.331, with a sensitivity of 0.718 and a specificity of 0.786. In the validation cohort, the optimal cutoff point was 0.390, with a sensitivity of 0.780 and a specificity of 0.714. The calibration curve showed good consistency with the 45-degree ideal line (Figure 3). Compared with the SOFA system, the NRI values of the nomogram in the training and validation cohorts were 0.751 (95% CI = 0.634–0.975) and 0.725 (95% CI = 0.669–1.17), respectively, whereas the corresponding IDI values in the training and validation cohorts were 0.129 (95% CI = 0.101–0.157) and 0.235 (95% CI = 0.179–0.289), respectively, indicating that this nomogram shows better recognition ability than the SOFA system. The DCA curves show that the nomogram has large net benefits in predicting 30-day survival (Figure 4).

Figure 2 ROC curves of the training and validation cohorts, generated to validate the discrimination of the nomogram. The SOFA scoring system was used for comparison. Abbreviations: ROC, receiver operating characteristic; SOFA, Sequential Organ Failure Assessment.

Figure 3 Calibration plots showing the relationship between the predicted probabilities based on the nomogram and the actual 30-day survival rates of the training and validation cohorts.

Figure 4 DCA curves of the training and validation cohorts. The abscissa is the threshold probability, and the ordinate is the net benefit rate. The horizontal line indicates that all samples were negative and were not treated, with a net benefit of 0. The oblique line indicates that all samples were positive. The red line shows the net benefit of SOFA score, and the blue line shows the net benefit of the nomogram. Abbreviation: SOFA, Sequential Organ Failure Assessment.

Discussion

This study revealed that temperature, SpO₂, SOFA, OASIS, comorbidities with coagulopathy, WBC count, heart rate, and respiratory rate were independent risk factors for 30-day survival in patients with secondary malignant neoplasms of bone and bone marrow. These indicators were used to construct a nomogram to estimate the 30-day mortality rate after hospitalization. The nomogram model was validated using multiple methods, including AUC, calibration curve, IDI, NRI, and DCA. The optimal values were evaluated according to the Youden index, and the sensitivity and specificity were determined.

Disease severity and clinical outcome for adult patients are frequently predicted using scoring systems, such as SAPS II, APACHE III (APS III), SOFA, and OASIS, with these systems often used for investigational purpose. APACHE III and SAPS II scores on admission were shown to be related to patient mortality. For example, univariate analyses showed that APACHE III and SAPS II scores were predictors of poor outcomes in lung cancer patients admitted to the medical ICU,¹³ and that APACHE II score was an independent factor of survival in patients with hematological malignancies.¹⁴ In multivariate analyses, however, APACHE III and SAPS II scores were not significantly predictive of survival in patients undergoing hematopoietic cell transplantation.^15,16 SOFA score, the most frequently used scoring system in clinical practice, was shown to be effective in assessing the prognosis of cancer patients in the ICU. In contrast to APACHE III and SAPS II scores, higher SOFA scores,^17–19 and higher logistic organ dysfunction (LOD) scores²⁰ were associated with an increased risk of mortality in patients with both solid and hematological malignancies. The OASIS scoring system, established using a machine learning algorithm based on the variables of the APACHE II and including 10 easily determined indicators,²¹ was shown to be predictive of the prognosis of critically ill patients.²² OASIS scores and SOFA scores did not differ significantly in predicting the short-term prognosis of patients in the ICU.²³ The present study shows that the SOFA and OASIS scoring systems are significant predictors of 30-day mortality in patients with bone metastases. The developed nomogram includes both indicators, with the nomogram model being more predictive of mortality than the SOFA score.

The present study also found that WBC count was a significant independent risk prognostic indicator for patients with bone metastases. An increase in WBC count is indicative of many types of cancer, especially bone marrow-related cancers, such as acute myeloid leukemia and chronic myeloid leukemia.²⁴ Evaluation of vital signs showed that heart rate, respiratory rate, body temperature, and SpO₂ were independent risk factors for patients with secondary malignant neoplasms of bone and bone marrow, with lower body temperature being associated with increased risk. The finding, that coagulopathy was also a risk factor, was unsurprising, as hypothermia can induce coagulopathy due to the reversible impairment of platelet aggregation and the resulting impairment of platelet plug formation.^25,26 One meta-analysis showed that even mild hypothermia increased blood loss by approximately 20%.²⁷ Among critically ill adult patients with cancer, mechanical ventilation for respiratory failure was found to be the most consistent predictor of poor prognosis, with mortality rates >40%.^17,28,29 SpO₂ reflects the supply of oxygen to the body and the degree of hypoxia, serving as a factor associated with critical illness.³⁰ The present nomogram showed that higher respiratory rate and lower SpO₂ level correlated with poor prognosis.

Although the model predicting a 30-day mortality rate in patients with bone and bone marrow metastases developed in the current study showed good performance and fitness, this study had several limitations. First, the data were extracted from a public database of a single center in the United States, which could result in population bias and thus limit the applicability of the model. Second, the model included laboratory parameters collected after hospital admission, and patients were not followed up. Third, the database is relatively old, suggesting the need for validation using external data from our own center.

Conclusions

The present study describes the first prognostic nomogram for predicting 30-day mortality rates of patients with secondary malignant neoplasms of bone and bone marrow. This nomogram can be easily applied in clinical practice.