Construction of Clinical Predictive Models for Heart Failure Detection Using Six Different Machine Learning Algorithms: Identification of Key Clinical Prognostic Features

Introduction

Heart failure (HF) is a multifaceted and life-threatening syndrome characterized by high morbidity and mortality rates, impaired functional capacity and quality of life, and substantial economic burden, affecting over 64 million individuals worldwide.¹ There are four principal categories of heart failure: systolic heart failure, diastolic heart failure, right heart failure and acute heart failure. The aetiology of systolic heart failure is primarily the result of a reduction in the contractility of the cardiac muscle, which impairs the heart’s ability to effectively pump blood to the peripheral circulation.² In this form of heart failure, the left ventricular ejection fraction (LVEF) is typically less than 40%. Diastolic heart failure is characterised by impaired filling of the heart during the diastolic phase. In contrast to systolic heart failure, patients with diastolic heart failure may exhibit normal or only mildly reduced LVEF (≥ 40–50% LVEF).³ Additionally, there are specific forms of heart failure, such as right heart failure, which primarily affects the right ventricular function.⁴ The ventricular ejection fraction (RVEF) can be utilized as a diagnostic indicator in such cases. Acute heart failure, on the other hand, is primarily diagnosed based on the presence of myocardial infarction or arrhythmia, along with a notable decline in cardiac function as observed through cardiac ultrasound.⁵ While several biomarkers that correlate with the prediction of HF occurrence have been identified, the left ventricular ejection fraction (LVEF) remains the principal criterion for diagnosis, prognosis, and treatment selection in HF.^2–4 As evidenced by these factors, the development of HF predictive models can not only improve early detection and intervention of HF but also enhance patient prognosis and quality of life while concurrently alleviating the strain on the healthcare system.⁶ However, predictive models that utilize markedly distinct clinical features or clinical data screening for HF patients are currently absent.

Machine learning is an artificial intelligence technique that is trained on a substantial corpus of data with the objective of identifying patterns and relationships within the data, thereby enabling the generation of predictions or decisions.⁷ The utilisation of a substantial corpus of patient data, encompassing clinical symptoms, laboratory test results, imaging data, and other pertinent information, enables medical practitioners to diagnose heart failure with greater precision and anticipate the disease’s progression and the patient’s prognosis.^7,8 The utilization of machine learning has progressively emerged as a crucial approach for improving clinical strategies in medical research.⁸ Clinical predictive models have been shown to effectively predict the risk of mortality in patients that were diagnosed with HF and admitted to the intensive care unit (ICU). In this context, extreme gradient boosting (XGBoost) models have been analyzed using Shapley Additive exPlanations (SHAP), enabling a comprehensive examination of the prognostic factors associated with HF.⁷ The SHAP method is an invaluable tool for elucidating the predictions of machine learning models and is instrumental in analysing the relationships inherent to complex models.⁷ Notwithstanding the varied etiologies of HF, the performance of the natural language processing (NLP) model and the predictors of adverse outcomes within one year were consistent across LVEF categories, suggesting the model’s broad applicability.⁹

It is worth noting that the Australian Heart Failure (AUS-HF) model utilizes routine data gathered from peer-to-peer interactions, facilitating real-time risk assessment that is easy to implement in clinical settings and aids in decision-making concerning the intensity of post-discharge follow-up.¹⁰ The AUS-HF model is a specific model designed for the diagnosis and prognosis of heart failure.¹⁰ It is typically based on the patient’s clinical characteristics, laboratory test results, imaging data, and patient history information. A previous study has shown that chronic heart failure patient-reported outcomes (CHF-PRO) possess strong predictive ability concerning patient outcomes, significantly improving the predictive model’s efficacy.¹¹ The CHF-PRO is a tool designed to assess the health status and quality of life of patients with chronic heart failure (CHF) across multiple domains, including physical, psychological, and social functioning.¹¹ This is achieved through patient self-reporting. Historically, data analysis from various sources has been performed in isolation, utilizing statistical and machine learning techniques. However, the integration of multiple omics and clinical data is essential for the progression of biomedical research and precision medicine. Data integration introduces novel computational challenges while also exacerbating issues related to uniomics research. Additionally, the proficient and effective analysis of integrated biomedical data from various sources requires the creation of specialized computational techniques.⁸ As a result, mortality risk scores generated using machine learning methods is more precise than current scores. Thus, in the modelling process, particular attention was paid to the inclusion of left ventricular ejection fraction (LVEF) as a pivotal input variable, given that LVEF is markedly diminished in patients with systolic heart failure.¹²

Additionally, indicators reflecting the degree of myocardial damage, such as myocardial troponin levels, were included, as myocardial infarction and other causes of systolic heart failure are frequently accompanied by myocardial cell damage. A low LVEF (less than 40%) typically indicates a significant impairment in the systolic function of the heart, which may be associated with the extensive loss of cardiomyocytes resulting from myocardial infarction and cardiomyopathy. In patients with HF exhibiting relatively high LVEFs (greater than 40%), the presence of diastolic dysfunction or other complex pathophysiological processes may be a concern. The aforementioned indicators assist in the identification of characteristics indicative of a poor prognosis in patients with heart failure. It is expected to enhance the evaluation of patients with HF, along with other situations where risk prediction is challenging.¹³ A substantial body of evidence attests to the pivotal role played by numerous clinical predictive models in the fields of medical research and practice.^14,15 In instances where conventional clinical data prove inadequate for providing sufficient predictive information, the incorporation of biomarkers into a predictive model may enhance its overall accuracy.

The current study aimed to create an HF risk prediction model utilizing six machine learning algorithms, emphasizing the differences in clinical characteristics based on LVEF values below and above 40. After excluding features with excessive missing values, the samples were randomly partitioned into training and validation sets. Six machine learning algorithms including logistic regression (LR), support vector machine (SVM), linear discriminant analysis (LDA), random forest (RF), naive bayes (NB), and K-nearest neighbor (KNN), were utilized to construct the predictive models. The optimal model was then selected based on an assessment of various metrics, including the area under the curve (AUC), accuracy, specificity, recall, and F1 scores. Furthermore, key clinical features were identified based on a hierarchy of feature significance. The study, thus provide a substantial basis for clinical assessment and a valuable tool for the prompt identification of HF and the formulation of intervention strategies.

This study employed a range of machine learning algorithms to construct a prediction model for heart failure patients with a poor prognosis. The optimal model was selected based on multiple performance indicators to investigate the clinical characteristics of significant differences between patients with a left ventricular ejection fraction (LVEF) of less than 40% and those with a LVEF of 40% or greater. This offers insights into the clinical characteristics that differentiate patients with disparate left ventricular ejection fractions (LVEF). As a result, the prediction model is better positioned to account for these differences, predict heart failure (HF) risk with greater precision, and provide a foundation for subsequent disease diagnosis, treatment, and prognosis evaluation.

Materials and Methods

Data Collection and Patient Stratification for HF

The data for this study were obtained from a sample of 160 patients suffering from heart failure (HF) with a total of 279 clinical features, informed consent was obtained from the patients, and all the studies were conducted in accordance with the ethical standards set by the Institutional Review Board of Shaanxi Provincial People’s Hospital, and approval was obtained from the Ethics Committee of Shaanxi Provincial People’s Hospital. On the day of 14th March, 2023 under the approval number of SPPH-LLBG-17-3.2. Following the data acquisition, patients were categorized according to the follow-up LVEF assessed one year subsequently. Cases lacking follow-up LVEF were excluded. Subsequently, the patients were classified into two groups based on a cutoff value of 40%: LVEF≤40% and LVEF >40%. Classification according to EF (LVEF≤40% and LVEF >40%) improved the predictive accuracy and relevance of the model. A low LVEF (≤ 40%) usually indicates significantly impaired cardiac systolic function, which may be associated with massive loss of cardiomyocytes due to myocardial infarction, cardiomyopathy, etc., whereas patients with HF who have a relatively high LVEF (> 40%) may have cardiac diastolic dysfunction or other complex pathophysiological processes.

Development and Validation of Machine Learning Models

In order to screen out important clinical features related to heart failure, a t-test was performed using the rstatix package (v 0.7.2) to examine the relationship between different clinical features and LVEF in HF patients, thus comparing the two groups (LVEF≤ 40% and LVEF > 40%) with a significance level of P <0.05. The data were re-evaluated for clinical features exhibiting significant differences to eliminate variables lacking numerical values. Finally, the model was constructed the clinical feature data remaining after this data cleaning. Subsequently, six machine learning algorithms from the caret package (v 6.0–94)¹⁶ were utilized to predict the factors that were associated with the LVEF in HF patients. The samples were randomly allocated to the training set (60%) and the testing set (40%) using the createDataPartition function. The machine learning models in the training were developed using KNN, LDA, LR, NB, SVM, and RF. After the models were successfully established, the receiver operating characteristic (ROC) curves were generated using the pROC package (v1.18.0).¹⁷ The accuracy, precision, recall, and F1 score of each model were computed based on the ROC curve of the results. The AUC value was calculated using the “roc” function of the “pROC” package (v1.18.0)¹³ Based on the projected and actual values, it was calculated by the following formulae:

Where FN stands for false negative rate; FP stands for false positive rate; TN stands for true negative rate; TP stands for true positive rate.

To evaluate the reliability of the results derived from the training set, six machine learning models were then constructed and assessed using identical methodology on the testing set. The optimal machine learning model was chosen for further analyses based on the computed performance metrics.

Ranking of Clinical Features Based on Importance

Importance ranking of clinical features refers to the assessment and ranking of various clinical features associated with a disease based on certain criteria or methods to determine which features have a more important impact on the diagnosis, treatment and prognosis of the disease. When the optimal machine learning model determined from prior analyses was used in the current study, the clinical features that demonstrated significant disparities between the groups were ranked according to their importance, which helps to identify clinical features that play a key role in the differences groups in LVEF. Consequently, clinical features with an importance value exceeding 0.8 were recognized as essential clinical features for this study.

Correlation Analysis

In order to understand the correlation between key clinical characteristic variables and the remaining significantly different clinical characteristic variables. Spearman correlation analysis was employed to examine the relationship between identified key clinical features and other clinically significant features. The correlation coefficients were calculated and subsequently visualized using a heatmap through the ggplot2 package (v3.4.4).¹⁸

Statistical Analysis

This study predominantly employed the R package (v4.2.2) for statistical analysis. Additionally, the t-test was employed to assess differences between groups (P <0.05).

Results

Baseline Features

160 patients with HF after excluding cases with null LEVF at one year follow-up. Following data exclusion, the study included a total of 102 patients, comprising 71 in the LVEF≤40% group and 31 in the LVEF >40% group. Statistically significant differences in clinical features were identified between the two groups (P value < 0.05), specifically regarding the following: angiotensin-converting enzyme inhibitors (ACEI) dose (P = 0.001), survival time (P = 0.046), left ventricular end systolic diameter (LVESD) (P = 0.001), left ventricular end systolic volume (LVESV) (P = 0.002), left ventricular end diastolic diameter (LVEDD) (P <0.001), left ventricular end diastolic volume (LVEDV) (P = 0.002), mean hemoglobin concentration (P = 0.020), atypical lymphocyte percentage (P = 0.010), albumin (P = 0.047), ionized calcium (P = 0.001), red blood cells (P = 0.026), and blood calcium (P = 0.012) (Table 1).

Table 1 T-Test of Identifying Important Risk Factors

The LR Model Was Screened as the Optimal Machine Learning Model

In order to found factors that can significantly influence the rise in ejection fraction scores, models were further constructed using machine learning algorithms. Further data cleaning was performed for significantly different clinical features that were screened using t-tests. After excluding clinical features with significant missing data (albumin, ionized calcium, atypical lymphocyte percentage, red blood cells, LVESD, LVESV, LVEDD, LVEDV), four clinical features (blood calcium, ACEI dose, mean hemoglobin concentration, and survival time) were selected for the construction of the predictive model. Following data cleaning, 42 samples were retained, comprising 29 in the LVEF≤40% group and 13 in the LVEF>% 40 group.

Utilizing the four clinical features derived from the aforementioned results, six machine learning algorithms were employed to develop risk prediction models within the training set. The area under the ROC curve (AUC) for the risk prediction models derived from the six machine learning algorithms exceeded 0.7 in both the training and testing sets, with the SVM algorithm attaining the highest AUC (training set AUC = 0.98, testing set AUC = 0.93) (Figure 1a and b).

Figure 1 ROC curves plotted based on six machine learning algorithms (a) test set (b) training set.

Furthermore, all performance metrics were assessed, revealing that the LR model emerged as the optimal machine learning algorithm for modeling, exhibiting an accuracy of 0.64, precision of 0.45, recall of 0.72, F1 score of 0.51, AUC of 0.81 in the training dataset and AUC of 0.91 in the testing set (Table 2), this suggested that the model had good generalisation ability.

Table 2 Evaluation of Six Machine Learning Models in the Training Set

Three Clinical Features Were Identified

The LR model used a logistic regression modelling algorithm ranked the significance of clinical features, identifying blood calcium, ACEI dose, mean hemoglobin level, and survival time as the most critical predictors. Survival time was a key indicator for assessing treatment efficacy and patient prognosis. Differences in survival time between groups of patients may reflect the effectiveness of treatment, disease progression, or patient biology. Among these, the clinical features with importance exceeding 0.8 were selected as the key clinical features for this study, and included blood calcium, ACEI dose, and mean hemoglobin level (Figure 2).

Figure 2 Clinical feature ranking based on LR model.

Correlation Analysis Between Key Clinical Features and Other Significant Clinical Features

Correlation analysis was performed for the key clinical features and other significant clinical features. A robust correlation was noted between blood calcium and ionized calcium (cor = 0.99, P = 3.84×10⁻¹⁴). In addition, significant correlations were identified between ACEI dosage and several left ventricular parameters, including LVESD (cor = 0.72, P = 7.94×10⁻⁴), LVESV (cor = 0.57, P = 1.40×10⁻²), LVEDD (cor = 0.68, P = 1.77×10⁻³), and LVEDV (cor = 0.58, P = 1.24×10⁻²). However, no significant correlation between mean hemoglobin levels and other clinical factors was observed in this study. Nonetheless, a correlation with additional clinical characteristics cannot be dismissed, necessitating further comprehensive investigation (Figure 3 and Table 3).

Table 3 Correlations Between Clinical Features

Figure 3 Results of correlation analysis of key clinical features with other significant clinical features.

Discussion

The field of medical research and clinical practice is witnessing a notable shift towards digitalisation and precision. The advent of machine learning algorithms has ushered in a new era of efficient methods for disease diagnosis, treatment and prognosis assessment. In this study, the logistic regression model was identified as the optimal algorithm, which is of great significance. This is because the study found that the characteristics of blood calcium, ACEI dose and so on are of outstanding importance, and therefore have great guiding value in disease diagnosis and treatment. The results of this study provide a more targeted indicator for clinical assessment of patients’ HF (heart failure) risk, which can more accurately capture the potential risk of patients developing HF.

A significant number of HF prediction models and algorithms have emerged in recent years. In a previous retrospective cohort study utilizing a comprehensive public ICU database, researchers formulated and validated four machine learning algorithms to predict the mortality of patients diagnosed with HF.⁷ Herein, the XGBoost model demonstrated enhanced performance relative to the LR, RF, and SVM models. As a result, an interpretable XGBoost prediction model was developed that demonstrated superior efficacy in assessing the mortality risk of patients with HF. Another study established and validated predictive models for all-cause mortality and hospitalization resulting from HF in patients receiving maintenance hemodialysis (MHD).¹⁵ Herein, the patients faced a significantly elevated risk of developing cardiovascular disease (CVD), exhibiting a 20-fold increased probability relative to the general population. In another study, a superior model for predicting HF incidents using standard electronic health records data was established, offering a promising direction for additional research into the prediction of other intricate conditions.¹⁹ Accordingly, the LR model was found to exhibit the highest efficacy in predicting HF. As a result, LR has been deemed the gold-standard for clinical prediction. This is further confirmed by additional studies wherein the LR model was found to be more effective than the regression tree model in accurately predicting in-hospital mortality for patients with HF.²⁰

The present study identified blood calcium, ACEI dosage, and mean hemoglobin level as the most critical clinical features, as determined by the optimal machine learning model. An increasing amount of evidence suggests that a disruption in calcium homeostasis correlates with a heightened risk of short-term mortality in patients with HF.²¹ In fact, about one-third of HF patients exhibit hypocalcemia, which is associated with an unfavorable prognosis. Scientific literature has established that elevated serum calcium levels correlate with an increased risk of HF with preserved ejection fraction (HFpEF) in patients with type 2 diabetes.²² Moreover, it has been concurrently recorded that reduced serum magnesium levels and increased serum phosphorus and calcium concentrations are each independently associated with a heightened risk of developing HF.²³ Herein, the correlation analysis of key clinical features and other notable differences demonstrated a significant relationship between blood calcium and ionic calcium. A previous study demonstrated that elevated doses of ACE inhibitors had no significant impact on all-cause mortality, cardiovascular mortality, or hospitalization rates.

However, the reduced systolic Ca transient noted in HF may be partially ascribed to a decrease in sarcoplasmic reticulum Ca²⁺ levels. This decline can be ascribed to reduced SERCA activity, heightened leaky RyRs, or increased NCX activity.²⁴ Alterations in the process of electrical coupling regarding both local and global calcium signals constitute a crucial mechanism that underlies contractile depression and the propensity for arrhythmia.²⁵ However, it was noted that ACEI improved functional capacity and increased the risk of hypotension, while simultaneously decreasing the incidence of cough.²⁶ In patients with HF with reduced ejection fraction (HFrEF), elevated doses of ACE inhibitors and Angiotensin II Receptor Blockers (ARBs) exhibited a modest decrease in the composite endpoints of all-cause mortality and heart failure hospitalization relative to lower doses. Nonetheless, no substantial impact was noted on drug discontinuation rates.²⁷ Furthermore, the optimal dosage of ACEI/ARB attained in elderly individuals with HFrEF correlates with prolonged survival.²⁸ A decrease in hemoglobin levels can aggravate organ damage associated with HF, and anemia may also play a role in the onset of HF. Iron deficiency also comprises a common factor leading to reduced hemoglobin levels in patients with HF.²⁹ Other results indicate a significant correlation between HbA1C and an elevated risk of both HFpEF and HFrEF, with a similar degree of association.³⁰ Even slight increases in hemoglobin levels have been associated with a heightened annual incidence of new-onset HF.³¹ Thus, iron therapy, as a principal treatment for anemia, was linked to enhancements in quality of life (QOL) and objective measures of congestive heart failure (CHF), without any associated adverse effects.³²

The present study utilized machine learning algorithms to create and validate six predictive models, with the LR model demonstrating superior performance. This is mainly due to the fact that compared with other models, LR model has the characteristics of high computational efficiency, stable model training and strong interpretability.³³ This algorithm, utilizing the clinical characteristics of notable differences among various ejection fraction HF subgroups, demonstrated that blood calcium, ACEI dosage, and average hemoglobin level were significant predictors of HF risk. These findings highlight the particular importance of factors such as blood calcium as predictors of heart failure risk in specific ejection fraction subgroups. It should be noted, however, that the study is not without limitations. Firstly, the sample size and morbidity types are not sufficiently comprehensive, which limits the generalisability of the constructed prediction model. Consequently, the accuracy and reliability of the model may be significantly reduced when applied to a wider group of HF patients. Secondly, in the process of data processing, this study constructed a prediction model by eliminating features with significant missing values. It is possible that some potentially valuable information may be lost.

In future research, it would be beneficial to increase the number of samples from different types. Inclusion of a more diverse range of HF patient samples, comprising patients from different regions, age groups and with different aetiologies, would enable a more comprehensive reflection of the actual situation of the HF patient population. Furthermore, it would be beneficial to investigate more sophisticated techniques for handling missing values. This approach could enhance data integrity while retaining a greater proportion of the sample information, thereby enhancing the reliability of the prediction model. Additionally, the incorporation of disease-specific biomarkers into the model could facilitate more effective clinical practice. By employing these strategies, it may be possible to more effectively identify potentially crucial features related to HF within the data, ultimately leading to more accurate HF risk prediction. Nonetheless, the study’s sample size was limited, and in the absence of particular clinical characteristics, certain factors that could potentially affect the ejection score of HF patients were excluded. In future studies, more samples can be collected to further validate the model established in this research.

Conclusion

By utilizing the optimal model LR, blood calcium, ACEI dosage, and average hemoglobin level were determined as effective predictors of HF risk in the present study. The monitoring of these indicators enables the identification of HF patients with poor prognosis risk at an early stage, thereby facilitating the development of more targeted treatment strategies.