A Comparison Between the Online Prediction Models CancerMath and PREDICT as Prognostic Tools in Thai Breast Cancer Patients

Introduction

Breast cancer is a heterogenous disease in terms of various biological and clinical behaviors.^1,2 The prognosis of early breast cancer has significantly improved with adjuvant treatments. The combinations of optimal chemoendocrine regimens have resulted in 57% and 45% reductions in 15-year mortality for women under the age of 50 and those aged 50–69, respectively.³ However, among breast cancer patients who meet the eligibility criteria for adjuvant systemic therapy, it has been argued that up to 60% of these patients experienced loss of quality of life due to toxicity with little or no survival benefit.⁴ The combined impact on quality of life as well as a rise in financial costs mean the avoidance of unnecessary treatment is a crucial goal for patients, oncologists, and policymakers. As a result, many models relating to risk prediction have been developed in order to aid oncologists in selecting patients who would benefit overall from systemic therapy.

At present, a number of prognostic tools such as multigene assays or free web-based prognostic calculators are available to aid decision-making in administering adjuvant systemic therapy. In more economically-developed nations, commercial gene expression-based prognostic tests such as Oncotype Dx and Mammaprint are widely implemented but these tools are inaccessible for many countries due to their expensive costs.⁵ Therefore, online prognostic calculators are valuable in clinical practice. There are several free web-based prognostic calculators to predict the survival of early breast cancer such as Adjuvant! Online,⁶ CancerMath,⁷ and PREDICT.⁸ These have been validated in Western populations and deemed to be calibrated and accurate in discrimination.^9–11

CancerMath was proposed in 2009 (http://www.lifemath.net/cancer/breastcancer/outcome/index.php). It predicts the risk of mortality for the first 15 years after diagnosis, any reduction in life expectancy, and any impact of adjuvant treatments. PREDICT, another prognostic model, was established in 2010 (https://breast.predict.nhs.uk). The newest version of PREDICT provides predicted 5-, 10-, and 15-year overall survival (OS) with additional benefits of each adjuvant therapy. There have been a small number of studies to confirm the prognostic ability of these programs in Asian populations.^12,13 Despite widespread use of both CancerMath and PREDICT, data comparing the validity between these two models are limited and controversial. Lass E and colleagues demonstrated that CancerMath and PREDICT had similar outcomes in American and Canadian breast cancer patients who were estrogen receptor (ER)-positive and human epidermal growth factor receptor 2 (HER2)-negative.¹⁴ On the other hand, a study comparing the two models in the United Kingdom illustrated that PREDICT outperformed CancerMath in terms of discriminatory accuracy, calibration, and clinical utility.¹⁵

Genetic backgrounds, socio-economic status, and cultures in Asia are all different factors that play distinct roles in the prognosis and treatment of breast cancer among Asians and people in America and Europe. Thus, the aim of this study was to validate the CancerMath and PREDICT models in Thai breast cancer patients.

Materials and Methods

Study Population

Breast cancer patients who underwent surgical treatment for stage I to III breast cancer, according to the American Joint Committee on Cancer Staging Manual seventh edition, from 2003 to 2011 at the Division of Head Neck and Breast Surgery, Department of Surgery, Siriraj Hospital, Mahidol University, Thailand were recruited. No informed consent was required because this study collected existing data in a manner that the subjects could not be identified. This study was approved by the Siriraj Institutional Review Board (certificate of approval number Si 732/2018) and conducted in accordance with the Declaration of Helsinki. The inclusion criteria of patients were age between 25 and 85 years old with unilateral breast cancer with tumor size T1-T3, available axillary nodal status, no distant metastasis, available ER/progesterone receptor (PR)/HER2/and Ki-67 status, and those who had received complete surgical treatment (Mastectomy or breast-conserving surgery followed by radiotherapy). Exclusion criteria were those who were receiving neoadjuvant therapy whose tumor size, tumor grade, or adjuvant treatment status were unknown. Patients who were lost to follow-up after surgery were also excluded. Patient data was reviewed from their medical records. The ER and PR status were defined as positive when there was 1% to 100% of tumor nuclei positive.¹⁶ HER2 immunohistochemistry was interpreted as follows: score 0 and 1+ =negative; score 2+ =equivocal; and score 3+ =positive. HER2 equivocal by immunohistochemistry was further determined by dual in situ hybridization (DISH). The ratio of HER2/CEP17 of ≥2 was defined as positive HER2.¹⁷ Follow-up time was calculated from the date of surgery to the date of death or date of last follow-up. The date of last data collection was December 1st, 2018. The final dataset for validation of 5-year and 10-year survival was composed of 545 and 397 patients, respectively.

Predicted 5- and 10-Year OS by CancerMath

For Breast Cancer Treatment Outcome Calculator using the CancerMath model, information for each patient was manually entered. This information included age at diagnosis, tumor diameter (cm), number of positive lymph nodes, ER/PR/HER2 status (positive, negative, or unknown), histologic type (ductal, lobular, others, or unknown), grade (1, 2, 3, undifferentiated, or unknown), hormonal therapy (none, tamoxifen, aromatase inhibitor (AI), tamoxifen followed by AI, ovarian ablation, or ovarian ablation plus tamoxifen), and chemotherapy (none, first-generation, second-generation, or third-generation regimens). For chemotherapy, first-generation regimens consisted of cyclophosphamide/methotrexate/5-fluorouracil (CMF) and doxorubicin/cyclophosphamide (AC). Second-generation regimens included AC followed by taxanes, docetaxel/cyclophosphamide (DC), 5-fluorouracil/epirubicin/cyclophosphamide (FEC), 5-fluorouracil/doxorubicin/cyclophosphamide (FAC), and cyclophosphamide/doxorubicin/5-fluorouracil (CAF). Docetaxel/doxorubicin/cyclophosphamide (TAC) and FEC followed by docetaxel were categorized as third-generation regimens. In every entry, the program predicted survival curves up to 15 years after diagnosis for 5 conditions: OS, OS with therapy, breast cancer-specific survival (BCSS), BCSS with therapy, and non-cancer related survival. Predicted 5 and 10-year OS with therapy were extracted for data analysis.

Predicted 5- and 10-Year OS by PREDICT

For the PREDICT model version 2.1, the data used to calculate predicted OS included age at diagnosis (between 25 and 85), postmenopausal status (yes, no, or unknown), ER status (positive or negative), HER2 status (positive, negative, or unknown), Ki-67 status (>10%, ≤10%, or unknown), invasive tumor size (mm), tumor grade (1, 2, or 3), detection method (screening, symptom, or unknown), number of positive nodes, hormone therapy (no or yes), chemotherapy (none, second generation, or third generation), trastuzumab (no or yes), and bisphosphonates (no or yes). Categorization of chemotherapy generation was different from CancerMath. There was no first-generation regimen included. Anthracycline-based regimens such as FEC were defined as second-generation chemotherapy and chemotherapy regimens that contained taxanes were categorized as third-generation chemotherapy. Because of this difference, first-generation regimens were registered as second-generation regimens. The calculator predicted 5-, 10-, and 15-year OS for 5 scenarios as follows; without adjuvant treatment, additional benefit of adjuvant hormone therapy, additional benefit of adjuvant chemotherapy, additional benefit of trastuzumab, and additional benefit of bisphosphonates. Predicted OS for 5 and 10 years with additional benefit of adjuvant treatment were collected.

Statistical Analysis

Kaplan-Meier analysis was used to estimate the observed 5 and 10-year OS in the entire study population and within subgroups. The CancerMath and PREDICT 5- and 10-year OS were calculated by individual predicted survival probabilities. The t-test and p-values of <0.05 from the comparison between model-predicted OS and patients’ actual OS represented a significant difference between predicted and observed survival. Moreover, the multivariable regression coefficients were applied to estimate the relation between parameters. The difference between actual and predicted survival and the ability of the models was analyzed using receiver-operating characteristic (ROC) analysis. The area under the ROC curve (AUC) demonstrated the discriminatory performance of the model. An AUC of 0.5 demonstrated no discriminative performance and an AUC of 1.0 illustrated perfect discrimination. Analyses were performed using SPSS for Windows version 22.0 (SPSS Inc., Chicago, IL, USA).

Results

Patients’ Characteristics

A total of 615 patients were recruited. The median age at diagnosis was 50 years (27–85 years). Most of the patients had invasive ductal carcinoma. The uncommon histological types were 4 lobular carcinomas (0.7%), 18 mucinous carcinomas (2.9%), and 11 other carcinomas (1.8%). Among 584 patients (95.0%) who received systemic therapy, 407 patients (66.2%) received chemotherapy and 414 patients (67.3%) were treated with endocrine therapy. Most of the patients were treated with first-generation chemotherapy regimens (n=258, 42.0%) and tamoxifen (n=195, 31.7%) as endocrine therapy (Table 1). The median follow-up time was 128.7 months (2.5–189.3 months).

Table 1 Baseline Characteristics of 615 Breast Cancer Patients

Comparison of Observed and Model Predicted 5- and 10-Year OS

Overall, CancerMath and PREDICT models underestimated both 5- and 10-year OS. The predicted 5-year OS by CancerMath and PREDICT were 88.9% and 83.1%, respectively, whereas the observed 5-year OS was 90.8% (p<0.001). The predicted 10-year OS via CancerMath and PREDICT were 78.3% and 72.0%, respectively, whereas the observed 10-year OS was 82.6% (p<0.001). Additionally, CancerMath had a higher accuracy in prediction than PREDICT in all subgroups for both 5- and 10-year OS, especially for breast cancer patients with T3 tumors, metastases in 4 or more lymph nodes, or who were classified as stage III (Table 2).

Table 2 Observed and Model-Predicted 5-Year OS Stratified by Patients’ Characteristics

CancerMath demonstrated a more noticeable underprediction of OS in patients older than 60 years (mean difference −6.3% for 5-year OS and −2.1% for 10-year OS), tumor size smaller than 2 cm (mean difference −4.3% for 5-year OS and −7.2% for 10-year OS), and ER or PR positive with HER2-negative subtype (mean difference −4.4% for 5-year OS and −6.6% for 10-year OS). Furthermore, the underestimation of OS also occurred in patients who received adjuvant treatment with third-generation chemotherapy (mean difference −11.3% for 5-year OS and −24.4% for 10-year OS) or hormonal therapy containing AI (mean difference −6.6% for 5-year OS and −11.1% for 10-year OS).

In contrast, CancerMath significantly overpredicted survival in patients with N3 (≥10 positive nodes) (mean difference 15.1% for 5-year OS and 11.8% for 10-year OS) and patients receiving ovarian ablation plus tamoxifen (mean difference 8.3% for 5-year OS and 7.2% for 10-year OS). The model seemed to overestimate short-term OS in triple-negative breast cancer, even though it was not appeared in predicted long-term OS as shown in Tables 2 and 3.

Table 3 Observed and Model-Predicted 10-Year OS Stratified by Patients’ Characteristics

According to multiple regression analysis, the factors that revealed significantly large standardized coefficients were age >60 year (0.39 at 5-year OS and 0.43 at 10-year OS), N3 status (0.21 at 5-and 10-year OS), and stage III (0.28 at 5-year OS and 0.35 at 10-year OS).

Performance of CancerMath and PREDICT in Thai Breast Cancer Patients

ROC analysis illustrated that both of CancerMath and PREDICT models were moderate for discrimination performance. The AUC for 5-year OS in CancerMath and PREDICT were 0.74 (95% CI; 0.65–0.82) and 0.78 (95% CI; 0.71–0.85), respectively. The AUC for 10-year OS in CancerMath and PREDICT were 0.75 (95% CI; 0.68–0.82) and 0.78 (95% CI; 0.71–0.84), respectively (Figure 1).

Figure 1 ROC curves for 5- and 10-year overall breast cancer survival. (A) AUC for 5-year OS by CancerMath and PREDICT were 0.74 (95% CI 0.65–0.82) and 0.78 (95% CI 0.71–0.85), respectively. (B) AUC for 10-year OS by CancerMath and PREDICT were 0.75 (95% CI 0.68–0.82) and 0.78 (95% CI0.71–0.84), respectively.

Discussion

Accurate prognostic tools are crucial for individualized treatment of breast cancer. This study validated two models (CancerMath and PREDICT) in Thai breast cancer patients and the results showed that both models underpredicted 5- and 10-year OS in the overall dataset. Between the two, CancerMath predicted outcomes more accurately than PREDICT in all subgroups. Both CancerMath and PREDICT performed fair discrimination between survivors and non-survivors at 5- and 10-years after diagnosis.

To our knowledge, this is the first study to validate the outcomes between CancerMath and PREDICT in Thai breast cancer patients. In Asian populations, a previous study by Wong et al demonstrated the validity of using PREDICT in most subgroups of patients recruited from University Malaya Medical Centre. The model-predicted 5- and 10-year OS of 86.3% and 77.5%, respectively, whereas the actual 5- and 10-year OS were 87.6% and 74.2%.¹² For the PREDICT model, our results and the previous study were discordant even though the calculated OS was similar. The better observed OS in our study may be due to the inclusion of fewer patients with poor prognostic factors such as locally advanced stage and HER2 positive breast cancer.

There are a number of factors that may have resulted in the PREDICT model underestimating survival in this study. First, only the ER status was collected in PREDICT and therefore PREDICT could not account for patients with negative ER but positive PR (n=37, 6.0%). Second, PREDICT excluded first-generation chemotherapy which was used in 46 patients (7.5%), thus potentially resulted in an inaccurately calculated OS. Finally, 602 patients (97.9%) had no Ki-67 status which also contributed to inaccurate calculation of predicted survival.

The study by Miao et al that validated CancerMath in the Singapore Malaysia Hospital-Based Breast Cancer Registry revealed that the model slightly overestimated survival. The CancerMath predicted and observed OS probabilities were 87.3% and 83.4% at 5 years while they were 75.3% and 70.4% at 10 years after diagnosis. In addition, an overestimated OS in patients with N3 (≥10 positive nodes) was also found which corroborates the results of the current study. In the study by Miao, CancerMath also showed fair discrimination with an AUC for 5-year and 10-year OS of 0.77 (95% CI; 0.75–0.79) and 0.74 (95% CI; 0.71–0.76) respectively.¹³

In this study, the greatest overpredicted OS of CancerMath was found in the subgroup of patients with N3. In this subgroup, 37.2% (n=19) were HER2 positive breast cancer but only 3.9% (n=2) were treated with trastuzumab as adjuvant treatment. Among 31.4% (n=16) of postmenopausal patients with positive ER or PR status, 7 patients (13.7%) received tamoxifen instead of AI. Furthermore, 84.3% (n=43) of this subgroup received radiation therapy. Owing to undertreatment, the actual survival of this subgroup was lower than those calculated by CancerMath. Additionally, CancerMath had only been validated among patients with a number of positive nodes of less than 10 while 8.3% (n=51) of the patients in our dataset had more than 10 positive nodes. Finally, many studies claimed that Asian breast cancer patients tend to manifest with characteristics consistent with poorer prognosis than their Western counterparts, such as more advanced stage, younger age, and more aggressive subtypes including HER2 overexpression and basal-like subtype;^18,19 thus, the predicted OS from models based on Western populations might overestimate OS in Asian patients.

Women with added ovarian ablation to tamoxifen was another group that had overestimated OS using CancerMath. This phenomenon may have been caused by two conditions. First, in this cohort, 151 premenopausal patients with hormonal receptor positive breast cancer who had a risk of recurrence were supposed to get tamoxifen plus ovarian ablation to permit adjuvant chemotherapy. However, none of these patients ended up receiving ovarian ablation. Second, there were a small cohort of patients in ovarian ablation plus tamoxifen subgroup (n=18). This implies that a greater emphasis on intensive treatment policies for locally advanced breast cancer and premenopausal women with positive ER or PR must be considered in Thailand.

The most underestimated OS using CancerMath occurred in patients who received third-generation chemotherapy. This may be explained by the small patient cohort (n=12) and the absence of any mortalities amongst this subgroup. CancerMath also underestimated OS in patients using AI-containing regimens, which happens to be the group of patients aged >60 years who are assumed to be the group to receive AI. The authors hypothesized that the high rate of discontinuation of AI in Western populations (32.4%-36.2%)^20,21 and the low rate of full adherence (49%) in patients may have contributed to this outcome.²² A few studies based on Asian populations revealed that the rate of interruption and non-adherence to AI was 14.9% and 15.4%-22.7% in adjuvant hormonal therapy, respectively.^23,24

Several limitations of this study should be considered. This study was a retrospective study and hence, there may be selection bias. This study was also conducted in a single center with a small number of patients; thus, a multicenter study looking at using these models in Thai populations may address this limitation.

Conclusion

Both CancerMath and PREDICT models underestimated OS in Thai breast cancer patients and CancerMath outperformed PREDICT in predictive performance in all subgroups for both 5- and 10-year OS. The clinical application of CancerMath should not be implemented to patients with N3 and aged over 60. Therefore, a novel prognostic model for Thai breast cancer patients is urgently required.