Using Radiomics and Convolutional Neural Networks for the Prediction of Hematoma Expansion After Intracerebral Hemorrhage

Introduction

Non-traumatic intracerebral hemorrhage (ICH) caused by rupture of arteries, veins, and capillaries, is becoming one of the main neurological diseases leading to death and disability in adults.¹ It has been demonstrated that patients diagnosed with ICH, despite being conscious in their initial presentation, may experience a poor outcome or even fatality in their subsequent stages.² Surgical intervention is usually required for critically ill patients. However, various complications may arise after craniotomy, such as infection, epilepsy, peptic ulcer, deep vein thrombosis et al. These complications may in turn accelerate the deterioration of patients’ health, ultimately leading to death, primarily among elderly patients.^3,4 Prior research has identified a group of independent risk factors that can predict a poor prognosis of ICH, such as the patient’s age, Glasgow coma scale score, hematoma volume, hematoma enlargement (HE), and ventricular penetration.^5–9 Among these factors, HE, which mainly occurs within 24 hours after the initial hemorrhage,^10,11 is the only factor that can be prevented after the patient is admitted to the hospital. Accordingly, exploring radiological signs for HE has become a research focus in recent years for radiologists.^12–16

In 2006, Wada et al discovered the “spot sign”, which is a computed tomographic angiography (CTA) sign for identifying HE and has been further confirmed in SCOREIT and PREDICT research, but the sensitivity is only about 50%.^14–18 Orito et al later identified another radiological sign in the arterial and the delayed phase of CTA called the “leak sign” with a higher sensitivity of 93.3% in predicting HE.¹⁹ However, these two signs rely on CTA examination, which exposes the patient to the risk of iodine allergy and radiation. In addition, many medical institutions, especially community hospitals, lack CTA inspection protocols in emergency departments. Conversely, several non-contrast computed tomography (NCCT) signs, such as “vortex sign”, “black hole sign”, “mixed density sign”, and “low density sign” were also used for predicting HE,^20–23 but the sensitivity of these signs is still relatively low and lack objective diagnostic criteria as well.²⁴ Convolution neural network (CNN) is a new deep learning (DL) method and has been recently used in radiology image classification. With raw data, DL can supervise learning tasks to determine classification features with ideal area under the curve (AUC) in both the training and verification cohorts,^25–29 but several challenging issues still exist, such as inaccurate segmentation resulting from the irregular shape and blurred boundary of hemorrhage in CT scanning, time-consuming and laborious in practical operation owning to hemorrhage labeled by manual work.

In order to provide an automatic prediction method more objectivity and sensitivity for clinical practice by incorporating other clinical databases, we proposed to build a hybrid model to automatically segment hemorrhage and predict HE risk using NCCT image to address the challenges mentioned above. In our proposed framework, we first employed a precise and reliable model to segment the intracerebral hematoma and quantify its volume. Subsequently, we developed a CNN model that took the representative patches extracted from the segmented region to learn rich features. We further stacked an additional fully connected layer above the concatenation of the learned features from the DL models for classification. By combining the features of the CNN model and other clinical data learning, the proposed model generated the final prediction of HE.

Materials and Methods

Ethical Approval

The General Hospital of Ningxia Medical University institutional review board granted has approval for this observational, retrospective study (Registration number: 2022–58). Informed written consent was waived due to the retrospective nature and use of de-identified data. Our study protocol complied with the Declaration of Helsinki.

Patients and Data Acquisition

We first retrospectively collected 771 medical records of adult patients who were clinically diagnosed ICH in our hospital from Jan 2015 to Dec 2020. All patients underwent baseline NCCT scans within 6 hours of symptom onset and were re-examined within 72 hours of onset. The exclusion criteria were as follows: secondary hemorrhage caused by trauma, tumor, aneurysm, arteriovenous malformation, cerebral infarction, transformation, surgical intervention before re-examination of CT, severe artifacts. Finally, a total of 408 patients were included in this study.

All CT examinations were performed using a LightSpeed 64-slice CT scanner manufactured by GE Healthcare. The scanning range extended from the lower edge of the anterior arch of the atlas to the top of the skull with a thickness and spacing of 5 mm. The tube voltage was 120 kVp and the current was 80 mA, while the image matrix size was 512×512. All scans are reconstructed with the Healthcare’s standard and bone reconstruction kernels with the slice thickness of less than 1 mm. Additionally, some parts of the patient’s history, clinical, and laboratory data were recorded from the health information system (HIS) including the patient’s gender and age, blood pressure on admission, activity of daily living scale index (ADL Index), history of hypertension, heart disease, diabetes, platelet count, neutrophil count, percentage of neutrophils, D-dimer, fibrinogen, prothrombin time.

Automated Segmentation of Hemorrhage

We employed the open-source program “ichseg” to preprocess the CT scans, which including skull stripping, computing predictors of ICH. The program utilized these predictors to predict a binary hemorrhage mask from the NCCT scan.³⁰ This R language-based program has been tested and verified on the Minimally Invasive Surgery plus rt-PA in ICH Evacuation (MISTIE) dataset. The authors evaluated the performance of the automatic segmentation model and expert radiologists by using the Pearson correlation and the Dice Similarity Index (DSI) to measure the test data. The DSI value was found to be 0.899, while the correlation between the manually delineated hemorrhage volume and the predicted volume was 0.93. This result indicated that the model showed good agreement with the gold standard of manually delineated hemorrhages.

This program was licensed under GPL-2 (GNU General Public License, version 2), which authorizes users to freely modify the code without permission. We partially modified the code to suit our research and added an interface for automatically calculating the hematoma volume. Next, we exported the relevant CT images from Picture Archiving and Communication Systems (PACS) in DICOM format and converted them to Neuroimaging Informatics Technology Initiative (NIfTI) format. Our modified software automatically segments hemorrhage from NCCT image and output them as a NIfIT format file, the hematoma volume is calculated automatically by counting the number of segment voxels multiplying them by the voxel size (provided in mL). We reconstructed all the original NCCT images and segmentation images with the same pixel matrix size (299 x 299 voxels) as the input layer of the pre-trained Inception_v3 model by matching the original residual network.

Feature Extraction

We labeled the supervised data set with a binary label (negative HE or positive HE) by comparing the hematoma volume between the baseline NCCT scan and the re-examination scan. Previous research has defined HE as a volumetric increase of more than 6 mL or 33% from the initial CT scan to the re-examination CT scan within 72 hours of onset.^3,31

We utilized the “pyradiomics” package in Python to extract high-throughput image features from the hemorrhage images for radiomics feature extraction. The selected features were chosen according to the definitions provided by the Image Biomarker Standardization initiative (IBSI) reporting guidelines, which has been presented a description previously.³²

Inception_V3 were used as the basic model for extracting characterization features. This module has been pre trained on a large-scale, well annotated ImageNet database. To extract a representative 2-dimensional (2D) images.³³ We automatically selected the axial slice maximum hematoma area image and 3 upper and 3 lower slices beside the maximum hematoma area image from the NCCT image in per patient. Based on the selected comprehensive 2D image, a data augmentation effect beyond artificial data augmentation such as flipping or rotation, and a robust prediction by combining probabilities from multiple slices to reduce the risk of overfitting and might help to boost the performance of a classifier. In the training process, the convolutional residual network was trained with image inputs as a “warm-up training”, to learn the relevant imaging features. The network was optimized using Adam optimizer (beta1 = 0.9, beta2 = 0.999, initial learning rate = 1e-04), with L2 penalty of 1e-02, the batch size was set to 128. The convolutional kernels were convolved over the image to obtain the local features and global features of the images at the same time. The weights of the convolutional kernels were adapted by back-propagation in every training process. Therefore, the networks were adjusted to the characteristics of the input data of first cohort. The maximum training epoch was set to 100, and the model was saved when the maximum validation accuracy was achieved from the testing set. This CNN was taken as an image filter, and the features of the regions were generated from the feature maps from the last fully connected layer as CNN features.

Hematoma Expansion Prediction

To improve diagnostic accuracy, we use fusion approaches for fusion of features with the aim of collecting complementary information from radiomics, clinical data, and CNN features with the aim of improving diagnostic accuracy, as shown in Figure 1.

Figure 1 Fully automated hybrid model for HE prediction. In Model 2, we use feature-level fusion approaches for fusion of features with the AIM of collecting complementary information from radiomics, clinical data.

Following the fusion pipeline, we chose K-highest score (K_Best) for the feature selection. This alternative could select a subset of features that is most relevant to the problem automatically. The grid search (GS-K_Best) parameters comprehended many values for the complexity parameter and the kernel type for the separability. Then we used support vector machines (SVM) as the classifier to predict HE.

Dividing Training Data and Test Data

We randomly select half of the cases as the first cohort for training CNN, using the default ratio set that 80% of the training set as training data and 20% as test data. The other half of the cases were in the second cohort as the validation cohort, with 100 randomly selected cases as test data and the rest as train data. We used the “sklearn. model.selection. train_test_split” function to randomly separate the training set and test the set.

Statistical Analysis and Metrics

A statistical analysis was conducted to compare the characteristics of the patients. Continuous data that followed a normal distribution were expressed in mean ± standard deviation, while others used the median value. For categorical variables, differences were assessed using the chi-square test or Fisher’s exact test. The Student’s t-test or Mann–Whitney’s U-test was used to determine differences among continuous variables. A significance level of p < 0.05 was adopted for all analyses.

Next, the model’s performance was evaluated using metrics. The ability of the classifier to distinguish between classes was measured using the area under the receiver operating characteristic curve (AUC-ROC). The confusion matrix was used to determine instances in a predicted class while each column indicated instances in an actual class. Precision indicated the ability of the classifier to avoid labeling a negative sample as positive while recall reflected the ability of the classifier to locate all positive samples. The F1-score was calculated using the harmonic mean of precision and recall through the formula F1-score = 2 × (precision × recall) / (precision + recall). The average precision (AP), which is the area under the precision-recall curve, was also used to compare the performance of computational object detection methods regardless of their underlying algorithm.³⁴ To evaluate the contributions of different features to the classification, we examined the top 15 features with high importance in the SVM-based classifier. Statistical analyses were conducted using the Python software (version 3.7.11) with the “scipy” and “Scikit Learn (version 0.24.2)” packages.

Results

Characteristics of the Study

Figure 2 depicts the patient enrollment process. Among 771 patients who met the inclusion criteria, 363 patients were excluded due to the following reasons: (1) trauma, aneurysm, vascular malformation, venous sinus embolism, tumor-induced cerebral hemorrhage (n=137); (2) emergency surgery before CT review (n=83); (3) surgical intervention performed during the 72-hour observation period, regardless of hematoma expansion (n=136); (4) CT images with artifacts (n=7). A total of 408 patients were finally enrolled for analysis. The CNN was trained using the first cohort, consisting of images from 204 case with 163 cases as training data while 41 cases as test data. The second cohort with the remaining 204 cases with 104 cases as the training data and 100 cases as the test data for validation. Table 1 provides detailed clinical characteristics of the patients, and the ADL Index was identified as the only significant difference between the two cohorts.

Table 1 The Characteristics of All Inclusion Participants

Figure 2 The patient enrollment process.

Model Performance

Hematomas were automatically labeled and their volumes calculated in all cases. Figure 3 illustrates an image showing the accurate masking of the hematoma. Table 1 shows the basic characteristics of all the cases included in the study.

Figure 3 Representative image of hematoma by automatically labeled. The effect of the automatic hematoma labeling tool on a certain patient. The red color in the above picture shows the range of the labeled hematoma, and the picture below shows the original axial image of the brain.

We then applied fusion methods to combine clinical data, radiomic features, and CNN features and used SVM as a classifier to predict hemorrhage expansion. In each case, a total of 6266 features were extracted, comprising 107 radiomics features, 15 clinical features, and 6144 CNN module features. The performance of the hybrid model on the test set of the second cohort is presented in Table 2 and Figure 4. Our experiments yielded a best accuracy of 0.95, an AUC of 0.949, a precision of 0.95, a recall of 0.83, and an averaged precision (AP) value of 0.94. Notably, NEG-HE achieved a precision of 0.95, recall of 0.99, and f1-score of 0.97, while POS-HE attained a precision of 0.95, recall of 0.83, and f1-score of 0.89. We included the macro-average and weighted average indices in Table 2 to present the precision/recall/f1-score values more clearly for unbalanced data.

Table 2 Results of Hybrid Module Receiver Operating Characteristic, Precision-Recall

Figure 4 Performance of the hybrid model in the prediction of HE status. (A) The confusion matrix shows how well the model predicts the test set. (B) ROC curve of the model. (C) The precision-recall curve of the model. (D) Top 15 features with high importance in the SVM based classifier.

To investigate the contribution of different features to the classification process, the top 15 features with the highest importance for classification are displayed in Figure 4D. The feature with the highest coefficient is imagenet_2377, and both imagenet_2377 and image_1 are the top specific features in the Inception_V3 pretrained model, indicating a significant role of this convolutional neural network in the prediction of the hybrid model. Three radiomic shape-based measures were also among the top features, illustrating a high correlation between hematoma outcome and their morphological properties. The remaining 9 were radiomic texture features, indicating that the texture features were more discriminative and could provide complementary information to the shape-based features. Besides, the D-Dimer was the only clinical feature that might have some influenced outcome of hematoma.

Discussion

In this study, we first employed an automated model to segment ICH in NCCT images. We then developed a hybrid model that fused radiomics, CNN features, and clinical risk features to predict early HE in patients. Our results demonstrated that our model exhibited advantages in automatic segmentation hemorrhage and prediction HE risk with high sensitivity and accuracy rate.

In recent years, several morphological imaging features on NCCT images that predict early hematoma outcome have been identified. Meta-analyses conducted by Du et al and Dowlatshahi et al investigated the diagnostic performance of the “spot sign” in ICH patients and found that as onset-to-CTA time increased, there was a significant decrease in hematoma expansion in spot-positive patients, with decreasing positive predictive values.^35,36 Some researchers have also proposed using clinical scores or laboratory tests to predict the outcome of hematoma. Morotti et al, for instance, developed a BAT score that can rapidly identify ICH patients at high risk for HE. The BAT score only contained three variables, namely “Blend sign”, “intrahematoma hypodensities”, and baseline NCCT timing.^18,24

Recent advances in artificial intelligence (AI) have shown its potential in clinical applications. Liu et al proposed an SVM model with an AUC of 0.85 to predict HE, but only relied on a bundle of clinical data.³⁷ Similarly, Zhu et al used handcrafted radiomics features and an SVM model to predict HE through manual 2D image segmentation, achieving a maximum AUC of 0.787 in the testing cohort.³⁸ In our study, we used an automated segmentation model to achieve the hemorrhage in NCCT, which helped avoid subjective errors resulting from manual labeling and saved significant time. The model’s output 3D segmentation mask objectively described the hematoma’s morphological characteristics. To enhance our classification accuracy, we concatenated the CNNs connected layer, enabling the model to combine clinical data, radiomics, and convolutional neural network features, thereby providing more meaningful features to the input layer. Our hybrid model had an excellent AUC of 0.949 with high precision, recall, and AP value in both cohorts, outperforming conventional NCCT markers and the BAT score. Moreover, our automated end-to-end model can help clinicians in emergencies by simplifying their decision-making process.

When constructing the hybrid model, we identified 15 most predictive features from the 6266 initially extracted features. The coefficient feature is an inception feature, but its meaning has not been explicitly cleared. Most of the remaining 12 features were radiomics features, with three shape features reflecting the three-dimensional shape heterogeneity of the hematoma. This finding was consistent with the results reported by Barras et al’s morphology classification model, which predicted early hematoma expansion and poor clinical prognosis in ICH patients. These findings indirectly confirmed the reliability of previous studies on the predictive value of irregular hematoma morphology in predicting the risk of hematoma expansion.³⁹ Additionally, five of the top 15 features were gray level co-occurrence matrix (GLCM) features, which were used to measure the statistical relationship between two neighboring voxels.²⁸ This feature quantitatively described the unevenness of density within the hematoma, as reflected in some published subjective signs such as the “black hole sign”, “blend sign”, and “satellite sign”.^40,41

Our study has several limitations. Firstly, the data was collected from a single center. The total number of cases included in this article, especially the number of positive cases, was limited. Secondly, we observed that some patients’ hematomas did not appear to expand within a short period, but surgical intervention was still performed within 72 hours. Although this kind of situation is not uncommon in clinical practice, it was not included in the study. Furthermore, hematoma expansion is not the gold standard for surgical intervention, and current DL models cannot provide a complete guide to clinical practice. Thirdly, in this retrospectively study, we were unable to fully capture all clinical information for our analysis. For example, the blood pressure control data, showing the reduction of increased ABP within first 6 hours after the onset of symptoms, were missed in some medical records. This may affect our analyzed quality.

Conclusions

Our study developed an automatic hybrid model with high sensitivity and accuracy rate using DL to detect early hematoma expansion in ICH patients. This model is an end-to-end method and can supplement clinical decision-making, thereby facilitating personalized treatment for patients with ICH.