Nomogram for the Prediction of Biochemical Incomplete Response in Papillary Thyroid Cancer Patients

Introduction

Thyroid cancer is a common endocrine malignancy, with papillary thyroid carcinoma (PTC) being the most frequently observed histologic subtype.^1–4 Previous studies have shown favorable prognoses associated with PTC;^5,6 however, the incidence of locoregional recurrence is significant in PTC patients during the follow-up period. The risk of recurrence was initially predicted using the staging system, which includes patients’ disease characteristics at diagnosis.^7–9

In the 2015 American Thyroid Association (ATA) guideline, the dynamic risk stratification (DRS) system was proposed to evaluate individual responses to initial therapy in PTC patients.¹⁰ This tool is equivalent to the initially used staging system, as recurrence risk in the initial staging system can be re-stratified based on the patients’ response to initial therapy, allowing for the guidance of follow-up management strategies.

In the DRS system, patients are classified into four response categories according to their imaging and laboratory test results: excellent response (ER), indeterminate response (IR), biochemical incomplete response (BIR), and structural incomplete response (SIR). Although the disease spontaneously evolves to “no evidence of disease” with additional therapy in a majority of these patients with BIR, BIR may be clinically significant, as 19–27% of patients experience persistently abnormal high thyroglobulin (Tg) levels and 8–17% of them experience structural recurrence in the 5–10-year follow-up period after initial therapy.¹⁰ Although the management guidelines for patients with SIR suggest the extirpation of recurrence lesions, there are no distinct guidelines for the management of patients with BIR.

Accordingly, clinician-related variations exist in management strategies for patients showing a BIR, including active surveillance with frequent laboratory and imaging studies, additional radioactive iodine (RAI) ablation therapy, or sustained thyroid-stimulating hormone (TSH) suppression therapy that may not be necessary.¹¹ Moreover, patients with BIR may be exposed to unwarranted imaging studies to identify possible recurrence lesions, leading to unnecessary biopsy or surgery and eventually increasing morbidity and worsening patients’ quality of life. However, previous studies have reported only individual risk factors for BIR; there is no comprehensive prediction model including risk factors for the estimation of BIR risk. Therefore, the development of a reliable prediction model for the estimation of BIR risk on DRS could aid in the prediction of prognoses and provide reliable information on the clinical course and surveillance recommendations for PTC patients.

Therefore, in this study, we aimed to develop a nomogram for the prediction of BIR in the DRS of PTC patients without structural recurrence, and to investigate its validity through internal and external validations.

Patients and Methods

Study Cohorts

We retrospectively reviewed the patients’ electronic medical records for clinicopathologic data and classified them into the training and validation cohorts based on two independent hospital datasets. In the training cohort, the data of 1104 PTC patients aged >18 years who were treated with total thyroidectomy (TT) in St. Vincent’s Hospital between January 2012 and December 2017 were examined. In the validation cohort, the data of 776 PTC patients aged >18 years who were treated with TT in Seoul St. Mary’s Hospital between May 2013 and December 2015 were collected.

Ethical Statement

This study was approved by the Institutional Review Board (IRB) of St. Vincent’s Hospital (IRB No. XC20RADE0141). All procedures involving human participants were conducted in accordance with the ethical standards of the Institutional Research Committee and the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. All participants provided written informed consent for storing medical information in the hospital database and use this information for research purposes.

Data Collection and Patient Inclusion

Data on the baseline clinicopathologic characteristics outlined in Table 1 were obtained. Central lymph node dissection (CLND) was performed for prophylaxis or treatment at the surgeon’s discretion. When CLND was performed prophylactically, the ipsilateral or bilateral central neck compartment in the same side as the PTC was dissected, even in the absence of clinical suspicions of lymph node (LN) metastasis. When CLND was performed therapeutically, the ipsilateral or bilateral central neck compartment was dissected only when metastatic LNs were preoperatively identified, as confirmed by biopsy or suspicions on imaging study. Modified radical neck dissection (MRND) was performed therapeutically only when the presence of metastatic LNs in the lateral neck compartment (levels II, III, IV, V) was confirmed by biopsy or preoperative imaging studies. RAI ablation after surgery was performed at the clinician’s or multidisciplinary team’s discretion.

Table 1 Baseline Characteristics of Patients

All patients were clinically evaluated and underwent laboratory tests, including those for the serum free thyroxine, triiodothyronine, TSH, Tg, and serum antithyroglobulin antibody (anti-TgAb) levels, every 3–6 months within 2 years of initial treatment. Imaging studies, including neck ultrasonography, chest radiography, and neck computed tomography (CT), were routinely performed. Additional imaging studies, including chest CT, diagnostic whole-body iodine scan, or positron emission tomography with fluorodeoxyglucose/CT, were performed if needed. The serum TSH levels were considered stimulated at >30 IU/mL following 4 weeks of levothyroxine hormone withdrawal, whereas the serum anti-TgAb levels were considered abnormal at ≥40 IU/mL. The serum Tg and anti-TgAb levels were measured using the Siemens Immulite^® 2000 thyroglobulin and antithyroglobulin assay (Siemens Healthineers, Erlangen, Germany), with the same method employed in the two hospitals.

Patients were excluded because of the following: presence of distant metastasis at diagnosis, absence of adequate clinicopathologic data for analysis, abnormal anti-TgAb levels at diagnosis or in the follow-up period, and experience of structural recurrence before DRS. In accordance with the exclusion criteria, three patients with distant metastasis at diagnosis, 33 without adequate clinicopathologic data for the analysis, 42 with abnormal anti-TgAb levels at diagnosis or in the follow-up period, and 21 with structural recurrence before DRS were excluded in the training cohort. In the validation cohort, 59 patients without adequate clinicopathologic data, 15 with abnormal anti-TgAb levels at diagnosis or in the follow-up period, and two with structural recurrence before DRS were excluded. Finally, 1005 and 700 patients were assigned to the training and validation cohorts, respectively.

The DRS results were evaluated according to the stimulated or unstimulated serum Tg levels, as obtained at 6–24 months after the completion of the initial therapy. Among patients who underwent TT with RAI ablation, the DRS results were determined as follows: ER, unstimulated Tg level <0.2 ng/mL or stimulated Tg level <1 ng/mL and negative findings on imaging study; IR, unstimulated Tg level 0.2–1 ng/mL or stimulated Tg level 1–10 ng/mL and negative findings on imaging study; and BIR, unstimulated Tg level >1 ng/mL or stimulated Tg level >10 ng/mL and negative findings on imaging study. Among patients who underwent TT without RAI ablation, DRS results were determined as follows: ER, unstimulated Tg level <0.2 ng/mL or stimulated Tg level <2 ng/mL and negative findings on imaging study; IR, unstimulated Tg level 0.2–5 ng/mL or stimulated Tg level 2–10 ng/mL and negative findings on imaging study; and BIR, unstimulated Tg level >5 ng/mL or stimulated Tg level >10 ng/mL and negative findings on imaging study.

Statistical Analyses

Descriptive characteristics are summarized as numbers and percentages for categorical variables and as means±standard deviations for continuous variables. For comparing the clinicopathologic characteristics between the groups, the Student’s t-test or Mann–Whitney U-test was used for continuous variables, and the chi-squared or Fisher’s exact test was used for categorical variables to assess the differences in the distribution of the variables.

Regarding the construction of the nomogram, data on the training cohort were used for nomogram model development. First, univariate and multivariate logistic regression analyses with the backward stepwise procedure, based on the Akaike information criterion, were performed to determine significant predictors of BIR among the variables in the training cohort. Subsequently, these significant predictors were selected for the development and construction of a nomogram for BIR prediction. To construct the nomogram, each regression beta-coefficient of variables in the multivariate logistic regression was proportionally converted to a 0–100-point scale. The variable with the highest beta-coefficient value was assigned 100 points, and the others were assigned 0–100 points according to the beta-coefficient value relative to the highest one. After the assignment of scores to each variable, the individual scores for each patient in the training and validation cohorts were calculated, which were subsequently used to estimate the individual BIR risk. To assess the predictive accuracy and discrimination of the nomogram in the training cohort, receiver operating characteristic (ROC) curves with areas under the curve (AUC) were calculated using the individual scores. Next, the internal validity of the prediction model was tested with 1000-bootstrap resampling to reduce the effect of overfitting bias. The degree of calibration was assessed with calibration plots (1000-bootstrap resampling) and the Hosmer–Lemeshow (HL) goodness-of-fit test. In the HL goodness-of-fit test, the level of model calibration assessment was considered adequate at values of p>0.05. For external validation, the nomogram performance was assessed in the validation cohort using the same methods as those employed in the training cohort.

All statistical analyses were performed using SPSS Statistics for Windows, version 12.0 (SPSS Inc., Chicago, IL, USA) or R version 4.0 (R Foundation for Statistical Computing, Vienna, Austria). Odds ratios (ORs) were presented with their 95% confidence intervals (CIs). All statistical tests were two-sided, and the significance level was set at p<0.05.

Results

Baseline Characteristics

Overall, 1705 (1005, training cohort; 700, validation cohort) patients were included. In the training cohort, 79 patients (7.86%) had BIR on DRS and 926 (92.14%) had no BIR. In the validation cohort, 41 patients (5.85%) had BIR on DRS and 659 (94.15%) had no BIR. The clinicopathologic data on the study cohort are presented in Table 1.

In the training cohort, the BIR group had significantly more male patients; a larger tumor size; higher rates of extrathyroidal extension (ETE), lymphovascular invasion (LVI), RAI, and MRND; and increased incidence of central and lateral LN metastases, number of metastatic LNs, number of excised LNs, and metastatic/excised LN ratio than the non-BIR group. The same findings were observed in the validation cohort; in addition, the BIR group showed a shorter interval between the initial treatment and DRS, and increased Tg levels in the DRS.

Between the training cohort and validation cohort, significant differences were observed in the following parameters: age, sex, tumor size, tumor number, histologic subtype, ETE, LVI, multifocality, bilaterality, extent of LN excision, site of LN metastasis, number of metastatic LNs, number of excised LNs, metastatic/excised LN ratio, RAI ablation treatment, interval between surgery and RAI ablation, RAI ablation treatment dose, stimulated Tg and TSH levels in the DRS, and DRS result.

Determining the Predictors of BIR and Nomogram Construction

To determine the predictors of BIR, univariate and multivariate logistic regression analyses were performed (Table 2). The multivariate analysis demonstrated that age ≥55 years (OR: 1.686, 95% CI: 1.018–2.794, p=0.042), male sex (OR: 1.851, 95% CI: 1.037–3.303, p=0.037), site of LN metastasis (central: OR: 2.316, 95% CI: 1.120–4.791, p=0.024; lateral: OR: 3.470, 95% CI: 1.416–8.500, p=0.007), presence of ETE (microscopic: OR: 1.804, 95% CI: 1.049–3.105, p=0.033; gross: OR: 2.547, 95% CI: 1.059–6.127, p=0.037), and presence of LVI (OR: 2.146, 95% CI: 1.076–4.282, p=0.03) were significantly associated with BIR.

Table 2 Univariate and Multivariate Logistic Regression Analyses for the Prediction of BIR Risk Factors in the Training Cohort

In addition to the abovementioned five significant predictors, the tumor size was included to construct the nomogram for the prediction of BIR risk, as it is a well-known prognostic factor for PTC, although statistical significance was not achieved in our multivariate analysis. Finally, six predictors (age, sex, tumor size, site of LN metastasis, ETE, and LVI) were used to construct a BIR risk prediction nomogram (Figure 1).

Figure 1 Nomogram for biochemical incomplete response risk prediction in papillary thyroid cancer patients.

Internal and External Validation of the Nomogram for BIR Prediction

In the training cohort, the ROC curve analysis showed good predictive performance and discrimination, with an AUC of 0.765 (95% CI: 0.712–0.818) (Figure 2A). After bootstrapping for internal validation, the bootstrap-corrected AUC of the prediction model was 0.76 (95% CI: 0.709–0.816) with 1000 resampling. The calibration plot showed good agreement between the nomogram’s prediction and the actual observation (Figure 2B), and the HL test did not suggest a lack of fit (p=0.1613).

Figure 2 (A) Receiver operating characteristic curve analysis of the nomogram in the training cohort. (B) Calibration plot of the nomogram in the training cohort with bootstrapping. Abbreviation: AUC, area under the curve.

Subsequently, to overcome the limitations of internal validation, we conducted an external validation to determine the generalizability of the prediction model using a validation cohort. In the external validation performed in the validation cohort, the AUC was 0.828 (95% CI: 0.768–0.889) in the ROC curve analysis, and the calibration plot showed a good calibration level (Figure 3A and B). The HL test did not suggest a lack of fit (p=0.2161). Although the training and validation cohorts had different baseline characteristics, the performance of the model remained good.

Figure 3 (A) Receiver operating characteristic curve analysis of the nomogram in the validation cohort. (B) Calibration plot of the nomogram in the validation cohort with bootstrapping. Abbreviation: AUC, area under the curve.

Discussion

In this study, we developed and validated a nomogram for BIR risk prediction on DRS in PTC patients without structural recurrence using the following predictors in the multivariate analysis: age ≥ 55 years, male sex, central and lateral LN metastasis, ETE, and LVI. Additionally, the nomogram was successfully validated through internal and external validations using an independent cohort dataset. The clinical and pathologic risk factors in our model could be easily corrected, minimizing the degree of variation in the collection of patients’ data and improving the general applicability of the prediction model.

Per the ATA guideline, a BIR on DRS is observed in 11–19%, 21–22%, and 16–18% of low-risk, intermediate-risk, and high-risk patients, respectively.^12,13 In previous studies, the definition of BIR was based on several factors, such as whether the patient underwent RAI ablation treatment or whether Tg production was stimulated on DRS. The 2015 ATA guideline suggests the use of the Tg levels for defining BIR on DRS in PTC patients who underwent RAI ablation: stimulated Tg >10 ng/mL or unstimulated Tg >1 ng/mL. Subsequently, several previous studies have suggested the following cut-off points for defining BIR in PTC patients who have not undergone RAI ablation treatment: stimulated Tg >10 ng/mL or unstimulated Tg >5 ng/mL.^14,15 However, this definition has not been fully validated, and our study results should be interpreted with caution, as patients who had not undergone RAI ablation were included in the training (64.6%) and validation (31.4%) cohorts.

The prognoses of patients with BIR on DRS are generally favorable; 56–68% of patients with BIR are finally classified to have “no evidence of the disease,” 19–27% experience a persistent abnormal Tg value without structural recurrence, and 8–17% develop structural recurrence over a 5–10-year follow-up period.^12,13,16 Interestingly, previous studies have not reported the disease-specific death rates in patients with BIR over a 10-year follow-up period, implying that the clinical significance of BIR in disease-specific mortality is relatively low. However, the clinical impact of BIR cannot be ignored concerning the disease recurrence, which may lead clinicians to drive excessive efforts toward the identification and treatment of potential recurrence. Tuttle et al reported that patients with BIR on DRS are exposed to subsequent aggressive TSH suppression, frequent imaging, and additional treatments, including empirical RAI ablation or surgical exploration.¹¹ In these patient, surgical exploration may be associated with complications, including parathyroid gland and recurrent laryngeal nerve injury.^17–19 Therefore, along with efforts aimed at reductions in the excessive management of patients with BIR, clinicians should consider the possibility of frequent evaluation and active treatment strategy for the identification and treatment of potential recurrence in patients with a high predicted risk of BIR on DRS. Our prediction model may contribute to BIR risk prediction in PTC patients and aid in the formulation of reliable and individualized management strategies.

Risk factors for BIR on DRS in PTC patients have been investigated previously. In a study on 554 PTC patients, Danilovic et al reported that the presence of gross ETE was a significant risk factor for BIR,⁷ whereas Mendoza et al demonstrated the following risk factors: male sex, extent of LN metastasis, ETE, and multifocality.²⁰ The total RAI treatment dose and presence of BRAF V600E mutations have also been identified as significant risk factors for BIR in PTC patients.^4,21 However, these studies only reported the possible risk factors for BIR and have not suggested a comprehensive prediction model for calculating the BIR risk in individual PTC patients. The risk factors identified in our study correspond to those noted in previous studies, and our model can predict patients’ risk for BIR on DRS considering the relative weight of individual risk factors. Although the clinicopathologic characteristics noted in the training and validation cohorts were different, the consistent performance of the prediction model in the two cohorts implied its general applicability.

Our study had several limitations. First, the BRAF V600E mutation status of patients was not evaluated, which was associated with poor prognoses in PTC.^22,23 Although Kowalska et al reported that the BRAF V600E mutation status was not significantly correlated with the response to initial therapy,²⁴ the inclusion of BRAF mutation status as a predictor may enhance the performance of our prediction model. However, we were unable to include this parameter because a large number of patients had missing data. Second, data were retrospectively collected, indicating a potential selection bias. Third, the number of BIR events in our study was relatively small, which may have negatively affected the power of the prediction model. Fourth, although the size of the LN metastasis was considered an important risk factor for recurrence in the 2015 ATA guideline, this was not investigated in our study. Finally, our study only used the serum Tg levels and not the anti-TgAb levels for defining BIR on DRS, which was inconsistent with the definition of BIR in the 2015 ATA guideline. However, we employed the ambiguous cut-off range for the anti-TgAb levels to define BIR on DRS, which was only referred to as the “rising level” in the 2015 ATA guideline. Although previous studies have reported on the association between the increasing anti-TgAb levels and PTC recurrence,^25–29 they have not suggested a distinct cut-off range for increasing the anti-TgAb values. Despite these several limitations, our study established and validated the nomogram with good performance for predicting BIR risk on DRS in PTC patients. A more properly designed future study would help improve the reliability of our prediction model.

Conclusion

In conclusion, this study presented a nomogram that could effectively predict BIR risk in the DRS of PTC patients without structural recurrence. Further studies are needed to evaluate the practicality of our nomogram in real-world clinical situations.