Predictions of Small Intracranial Aneurysms’ Rupture Risk with Ensemble Machine Learning Model (Super Learner): A Retrospective Study in Two Tertiary Hospitals in China

Introduction

Intracranial aneurysms (IAs) affect approximately 2–3% of the general population.¹ When ruptured, they can result in subarachnoid hemorrhage (SAH), a devastating condition associated with high rates of mortality² and long-term disability.³ Timely identification of aneurysms at high risk of rupture is therefore critical for guiding treatment decisions and improving patient outcomes.

Aneurysm size is one of the most widely studied predictors of rupture risk,^4,5 with smaller aneurysms generally considered less likely to rupture. For instance, rupture rates as low as 0%, <0.5%, and 1% have been reported for aneurysms measuring 1 mm, 3 mm, and 7 mm, respectively.⁶ However, clinical experience and retrospective studies have demonstrated that some small aneurysms do rupture, often with catastrophic consequences.^7,8 This highlights a key limitation of relying on aneurysm size alone for risk stratification.

To address this limitation, several clinical risk assessment tools—such as the PHASES score—have been developed,⁹ incorporating factors like aneurysm size, location, and patient history. While useful, these tools lack the granularity needed for individualized risk prediction and often fail to account for complex morphological and hemodynamic features.

Advancements in computational fluid dynamics (CFD) have enabled the quantification of hemodynamic parameters within aneurysms, such as wall shear stress and flow instability.^10,11 These features offer valuable insight into the biological processes underlying aneurysm rupture, but their integration into clinical practice remains limited, partly due to a lack of validated predictive models.

In recent years, machine learning (ML) has emerged as a powerful tool for medical prediction tasks, capable of uncovering nonlinear patterns and high-dimensional relationships from complex datasets.^12,13 Prior studies have demonstrated the potential of ML-based models to outperform conventional statistical approaches in predicting aneurysm rupture.¹⁴ However, most existing ML models rely on single algorithms, which may suffer from inconsistent performance across datasets and lack robustness in clinical application.

To address this gap, we propose a Super Learner ensemble model that integrates multiple machine learning algorithms using cross-validation to optimize prediction performance.^15–17 This approach aims to leverage the strengths of various base learners while mitigating individual model limitations. Furthermore, we emphasize external validation and model interpretability to ensure both robustness and clinical applicability.

In this study, we developed and externally validated a Super Learner-based model for predicting rupture risk in intracranial aneurysms, incorporating clinical, morphological, and hemodynamic features. The model’s interpretability was further assessed to enhance its translational potential in clinical decision-making.

Methods

Patients Selection

This retrospective study was approved by the Ethics Committee of Shanghai Fourth People’s Hospital and the Institutional Review Board of Suzhou Municipal Hospital. Since the study used anonymized medical records and did not involve direct patient interaction, the need for informed consent was waived by the ethics committees. All data were handled confidentially, and the study complied with the ethical principles of the Declaration of Helsinki. This study retrospectively collected data from all intracranial aneurysm patients treated at the stroke centers of two hospitals between December 2021 and December 2023.

Study participants were selected according to predefined inclusion criteria and excluded based on specified exclusion parameters. Inclusion criteria: (1) The presence of intracranial aneurysms was confirmed through either digital subtraction angiography (DSA) or computed tomography angiography (CTA); (2) a saccular aneurysm. Exclusion criteria: (1) Traumatic, infectious, thrombotic, or dissecting aneurysms; (2) Saccular aneurysms associated with arteriovenous malformations. (3) Incomplete or poor-quality imaging that prevents aneurysm reconstruction and hemodynamic analysis. (4) H max > 7 mm. The study flowchart is shown in Figure 1. In cases of multiple aneurysms, the lesion with the maximum diameter was selected as the index aneurysm for classification purposes. Rupture status of intracranial aneurysms was determined retrospectively from inpatient medical records. Aneurysms were classified as ruptured if imaging (DSA or CTA) confirmed the presence of an intracranial aneurysm accompanied by subarachnoid or intracerebral hemorrhage. Small aneurysms were defined as those with Hmax ≤ 7 mm.

Figure 1 The flowchart of this study (a). Super learner algorithm construction illustrated with 10-fold cross-validation and 5 base models (SVM, KNN, DT, XGBOOST, RF) (b).

CFD Analysis

Patient-specific vascular geometries were generated through 3D reconstruction of clinical DICOM datasets acquired via rotational angiography (3D-DSA). Subsequent mesh optimization, including anatomical refinement and surface regularization, was performed using Geomagic Wrap 2015 (3D Systems, NC, USA) as illustrated in Figure 2. Minor branches were removed during this process, while aneurysms, parent vessels, and key branching vessels were preserved.

Figure 2 The flow chart of the model’s construction and validation (a) The 3D aneurysm’s model (b) The computational fluid dynamics model of the aneurysm (c) The streamline chart of aneurysm (d) The cross validation of features, CV (e) Least Absolute Shrinkage and Selection Operator algorithm was used to select features. (f) Receiver operating characteristic curve of machine learning models.

The reconstructed model was used for hemodynamic simulations. The reconstructed geometry was processed in ANSYS ICEM CFD 16.2 (ANSYS Inc., PA) for computational mesh generation, employing a uniform element size of 0.16 mm throughout the domain.

Numerical simulations were performed in ANSYS CFX 2019 (ANSYS Inc., PA) by using the incompressible Navier-Stokes equations, with the computational setup illustrated in Figure 2C. The blood flow was simulated as an incompressible Newtonian fluid (ρ=1056 kg/m³, μ=0.0035 Pa·s) under laminar regime conditions. Vascular walls were treated as rigid boundaries enforcing no-slip conditions. The inlet boundary condition specified a pulsatile flow rate of 4.6 mL/s, while outlet flow distribution followed Murray’s hemodynamic principle.¹⁸ The twelve quantitative hemodynamic parameters analyzed in this study are detailed in the supplementary materials.

Morphological Parameters Measurement

Diagnostic evaluation included digital subtraction angiography for all study participants, with three-dimensional vascular reconstructions created from rotational angiography data using Syngo Workplace post-processing software (Siemens Healthineers, Germany). The post-processing algorithm converts DICOM-formatted axial DSA images into three-dimensional vascular models exported in stereolithography (STL) file format for computational analysis. Two board-certified neuroradiologists independently quantified the aneurysm and adjacent vascular geometry using dedicated workstation measurement tools. Both neuroradiologists were blinded to the patients’ clinical manifestations and CT findings. Sixteen morphological parameters were analyzed, including inflow angle, tilt angle, vascular angle, HMAX, DMIDDLE, DNECK, DVESSEL, H, S, V, AR, UI, SR, EI, NSI, BNR, and irregular shape. The definitions and measurements of these parameters are provided in Supplementary figure 1.

Construction and Validation of Machine Learning Model

Data from the Cerebrovascular Disease Center of Shanghai Fourth People’s Hospital was randomly split into a training set (n=97) and an internal validation cohort (n=42) in a 7:3 ratio. The Least Absolute Shrinkage and Selection Operator (LASSO) algorithm, combined with 10-fold cross-validation, was used to select modeling variables in the training cohort (Figure 1). Collinearity analysis was performed on the selected variables. Variables with a collinearity value >5 were removed as appropriate based on practical considerations (see supplementary materials for details). Based on the final selected modeling variables, five machine learning models and one Super Learner model were built based on the training cohort data. The models included SVM, RF, XGBoost, KNN, DT, and the Super Learner model. The models’ effectiveness was validated using the internal validation cohort and an external validation cohort (N=86) (Figure 1). The models’ performance was evaluated by the receiver operating characteristic (ROC) curve, area under the curve (AUC), sensitivity, specificity, accuracy, F1 score, positive predictive value (PPV), and negative predictive value (NPV). To ensure model diversity and reduce redundancy among base learners, we evaluated each algorithm’s contribution to the ensemble. Based on cross-validated weights, models with zero coefficients (e.g, XGBoost, KNN) were removed from the final Super Learner ensemble, while models with distinct decision mechanisms and meaningful contribution (eg, random forest and decision tree) were retained.

Missing Data Handling

No missing data were encountered in this study. Imaging data were subject to strict quality control, and cases with incomplete or poor-quality scans were excluded prior to analysis. Clinical variables were retrospectively collected from comprehensive medical records, ensuring completeness. As such, no imputation or exclusion for missing values was required during model development.

Statistical Analysis

Statistical analyses were conducted using IBM SPSS Statistics software (v25.0, Armonk, NY) for descriptive statistics and R programming language (v3.4.3, R Foundation) for advanced predictive modeling.

Data distribution normality was evaluated using the Kolmogorov–Smirnov goodness-of-fit test. Continuous variables following normal distribution were summarized using parametric measures (mean ± SD), whereas non-parametric descriptors (median [IQR]) were employed for skewed distributions.

Nominal variables were summarized using absolute counts and relative frequencies (%). Between-group differences in categorical data were assessed using either Pearson’s χ²-test or likelihood ratio tests, selected based on expected cell frequencies. For parametric data, intergroup comparisons employed independent samples t-tests (two groups) or one-way ANOVA (≥3 groups) with post-hoc Tukey’s testing. If ANOVA revealed significant differences, post hoc analysis was conducted for pairwise comparisons. Non-normally distributed data were analyzed using the independent sample non-parametric test. All statistical tests were two-sided, with a significance level set at p < 0.05. Multicollinearity among the modeling variables was evaluated using the Variance Inflation Factor (VIF), with additional information provided in the supplementary materials. The Super Learner model in this study was developed using an ensemble learning algorithm that integrated multiple weighted base models (SVM, RF, XGBoost, DT, and KNN). Cross-validation was applied to select the optimal ensemble model. The hyperparameters of the five base models are shown in Supplementary Table 1. Super Learner is an ensemble-based algorithm that improves predictive performance by optimally weighting multiple base models. It determines the best model combination strategy through cross-validation. In this study, the Super Learner model incorporated ten-fold cross-validation and five base models (SVM, KNN, DT, XGBoost, and RF). The training dataset was randomly divided into 10 folds using K-fold cross-validation (K = 10). Each base model was trained on K−1 folds and tested on the remaining fold (Figure 1). The predictions from all base models on the test folds were collected, forming a new dataset in which each sample corresponded to multiple base model predictions. The prediction errors of each base model were then calculated. A non-negative least squares algorithm was used to learn the optimal combination weights while ensuring non-negative constraints. The optimal weighted averaging scheme was determined to minimize prediction error (see supplementary Table 2 for details on model parameters and weights).

Super Learner is inherently a black-box model, making direct interpretation challenging. To improve interpretability, we employed the following approach. Each quantitative modeling variable was standardized, and based on the predicted probability of aneurysm rupture in the validation dataset, patients were stratified into four risk groups using interquartile ranges: low risk (0–Q1), low-to-moderate risk (Q1–Q2), moderate-to-high risk (Q2–Q3), and high risk (Q3–Q4). The standardized mean of each quantitative variable was calculated within each risk group to illustrate its trend across different risk levels. For categorical variables, the proportion within each risk group was computed to examine their distribution across different risk levels. This approach provides insight into how individual variables contribute to aneurysm rupture risk prediction and enhances the interpretability of the Super Learner model.

Results

Demographics, Aneurysm, Hemodynamics Parameters

A total of 225 patients were included in this study and were divided into a training cohort (n=97), an internal validation cohort (n=42), and an external validation cohort (n=86). Table 1 presents the baseline characteristics of patients in the training cohort, comparing the ruptured and unruptured aneurysm groups. Significant differences were observed in Hypertension, Irregular shape, H max, H, S, V, AR, UI, SR, EI, NSI, WSSmin, OSI, LSA, and OSI max between the two groups (all P < 0.05). Table 2 provides detailed characteristics of patients, aneurysms, and hemodynamic parameters across the training, internal validation, and external validation cohorts. The training cohort consisted of 97 patients (ruptured: n=21, unruptured: n=76), the internal validation cohort included 42 patients (ruptured: n=13, unruptured: n=29), and the external validation cohort comprised 86 patients (ruptured: n=36, unruptured: n=50). Significant differences were observed in certain aneurysm, patient, and hemodynamic parameters among the three cohorts (P < 0.05).

Table 1 Characteristics of Demography, Aneurysms and Hemodynamic Between Ruptured and Unruptured Patients in the Training Cohort, No.(%)/M[IQR]

Table 2 Characteristics of Patients, Aneurysms, and Hemodynamics in the Training and Validation Datasets

Construction and Validation of mL Model

Figure 3a illustrates the ROC curves of six models (Super learner, Random Forest, XGBOOST, SVM, DT, KNN) in the internal validation cohort. The AUC values for Super learner, RF, XGBOOST, SVM, DT, and KNN were 0.86 (0.72–1), 0.73 (0.59–0.88), 0.89 (0.76–1), 0.89 (0.76–1), 0.89 (0.77–1), and 0.53 (0.32–0.74), respectively. Figure 3b presents the ROC curves of these 6 models in the external validation cohort. The AUC values for Super learner, RF, XGBOOST, SVM, DT, and KNN were 0.94 (0.89–1), 0.83 (0.76–0.91), 0.93 (0.87–0.99), 0.82 (0.73–0.92), 0.84 (0.76–0.93), and 0.51 (0.38–0.63), respectively. The Super learner model demonstrated overall superior performance compared to the other five base models. Table 3 summarizes the AUC (95% CI), sensitivity, specificity, accuracy, F1 score, PPV, and NPV for each model in the training, internal validation, and external validation cohorts.

Table 3 AUC, Sensitivity, Specificity, Accuracy, F1 Score, PPV and NPV of Super ML Model

Figure 3 Receiver operating characteristic curves for Super learner in the internal (a) and external validation dataset (b).

Interpretable Analysis

Figure 4 shows the contribution of each modeling variable to the Super learner in the external validation set. Hypertension, irregular shape, and SR exhibit a monotonically increasing trend with rupture risk. The increase in these factors may significantly raise the risk of aneurysm rupture. WSS min is negatively correlated with aneurysms, and lower WSS min may be an important predictor of rupture.

Figure 4 The interpretable analysis of Super learner model.

Discussion

In this study, five basic machine learning models and a super learner were constructed based on demographics, aneurysm characteristics, and hemodynamic parameters to accurately predict the rupture status of intracranial aneurysms. The models were validated in an independent external validation cohort, showing stable performance. Additionally, interpretability analysis of the super learner confirmed that hypertension, irregular shape, larger SR values, and lower WSSmin increase the rupture risk of intracranial aneurysms.

Previous studies have shown that the size of an aneurysm^19,20 is a significant factor influencing aneurysm rupture. Although smaller aneurysms have a lower rupture risk,^21,22 they still carry a risk of rupture. Ruptured aneurysms are associated with high mortality and morbidity rates. Therefore, the small aneurysm rupture risk prediction model developed in this study holds significant clinical value.

Previous intracranial aneurysm rupture risk prediction models have largely been based on patient baseline data and morphological parameters of aneurysms,¹⁴ which are static and do not reflect the true hemodynamic state within the aneurysm, thus presenting certain limitations.

This study used computational fluid dynamics to simulate the hemodynamic parameters of intracranial aneurysms in their true state, comprehensively analyzing the baseline data, morphology, and hemodynamics of ruptured aneurysms. It found that hypertension, irregular morphology, larger SR values, and lower WSSmin may be significant factors influencing aneurysm rupture. This is consistent with previous research results,^23–26 which show that chronic hypertension leads to aneurysm expansion and a reduction in the wall strength, thereby increasing the rupture risk. Irregular aneurysms are more prone to rupture, possibly due to turbulent flow within the aneurysm. The larger the SR value of the aneurysm, the larger it is relative to the neck, which is often associated with a slightly higher likelihood of intracranial aneurysm rupture. Extremely low WSSmin may lead to increased local inflammation, triggering aneurysm rupture.

Several prior studies have integrated morphology, hemodynamics, and patient characteristics to stratify the rupture risk of intracranial aneurysms. Notably, Detmer et al²⁷ developed a rupture probability model based on patient demographics, aneurysm location and morphology, and CFD-derived hemodynamic variables, achieving an AUC of approximately 0.85. Similarly, Rajabzadeh et al²⁸ proposed the Rupture Resemblance Score (RRS), a data-driven approach that identified high-risk aneurysms by comparing them to known ruptured phenotypes, demonstrating promising clinical utility.

Most existing machine learning studies on aneurysm rupture prediction have constructed multiple independent models,²⁹ assessing their performance using conventional metrics such as sensitivity, specificity, and AUC. While these approaches have shown reasonable accuracy, their predictive performance remains suboptimal. Owing to the distinct assumptions and algorithmic mechanisms underlying each model, no single method consistently achieves optimal performance across all evaluation domains.

In contrast, the Super Learner algorithm offers an ensemble-based framework that combines the strengths of diverse base learners through cross-validated weighting.³⁰ This approach improves model robustness and stability, especially in heterogeneous datasets, and reduces the risk of overfitting by minimizing reliance on any single poorly performing model.³¹ Compared to traditional models like Detmer’s or the RRS, the Super Learner in our study not only demonstrated superior discrimination (external AUC = 0.94), but also benefited from enhanced flexibility and generalizability without relying on rigid feature assumptions.

Additionally, our model incorporated interpretability analyses to reveal critical rupture-associated factors—including hypertension, irregular morphology, elevated SR, and reduced WSSmin—all of which align with previously reported risk features, thereby reinforcing the biological plausibility and reliability of our findings. Taken together, these results support the notion that ensemble-based machine learning approaches such as the Super Learner may provide a valuable complementary tool for aneurysm rupture risk stratification, especially when combined with detailed hemodynamic simulation and clinical data.

The Super Learner model achieved an AUC of 1.00 in the training set, which may suggest potential overfitting. However, several measures were implemented to mitigate this risk. First, stratified 10-fold cross-validation was used during training to improve model robustness. Second, ensemble learning methods such as Super Learner can inherently reduce variance and overfitting by combining diverse base learners. Most importantly, the model demonstrated strong and consistent performance in both internal and external validation cohorts, with an external AUC of 0.94, indicating good generalizability and suggesting that overfitting was limited.

One of the most promising clinical applications of our model lies in the management of small (<7 mm) unruptured intracranial aneurysms, where treatment decisions remain highly controversial. In many cases, clinicians rely heavily on experience and imaging features to estimate rupture risk. However, for small aneurysms lacking overt high-risk characteristics, this assessment can be highly subjective and uncertain. Our model offers a quantitative, individualized risk prediction that may assist in such ambiguous scenarios. When the model indicates a relatively high rupture probability despite the aneurysm’s small size, clinicians may consider early intervention or intensified surveillance. Conversely, low-risk predictions may help avoid unnecessary procedures and reduce patient anxiety. Furthermore, as the model is applied prospectively, newly accumulated cases can be incorporated into ongoing training and refinement, enabling continuous performance improvement and enhanced clinical utility over time.

In this study, the Super learner accurately predicted the rupture status of intracranial aneurysms, with stable validation results, outperforming the other five base models in overall predictive performance and demonstrating higher reliability.

Limitation

Finally, this study has several limitations. First, the sample size is relatively small, with only 150 cases, which limits the generalizability of the findings. A larger sample size is needed to draw more robust conclusions. Second, the Super learner is a black-box model, and further advancements in algorithms are needed to better understand its internal mechanisms.

Conclusion

This study developed a Super learner model based on demographics, aneurysm, and hemodynamic parameters, which accurately predicts the rupture status of intracranial aneurysms and outperforms the other five base models. Additionally, it validated that hypertension, irregular shape, larger SR values, and lower WSSmin increase the risk of intracranial aneurysm rupture.