Noninvasive Evaluation of Lupus Nephritis Activity Using a Radiomics Machine Learning Model Based on Ultrasound

Introduction

Lupus nephritis (LN) is a common complication of systemic lupus erythematosus (SLE). The severity of renal damage is closely related to the prognosis of SLE. Renal involvement and progressive renal dysfunction are one of the main causes of death in SLE.^1,2 LN causes renal damage by deposition of circulating or in situ immune complexes.³ The goal of LN treatment is to eliminate active inflammation and achieve renal remission and disease quiescence.⁴ Currently, renal biopsy is the gold standard for diagnosis and treatment of LN.⁵ It is also the only way to judge disease activity.^6,7 When the treatment of LN is inadequate or if LN relapses, it is often necessary to repeat renal biopsy to accurately judge pathological changes and formulate corresponding treatment plans.⁸ However, renal biopsy is an invasive diagnosis with the risk of bleeding. Repeated renal biopsy is difficult to be accepted by patients. Therefore, a noninvasive method is urgently needed to accurately determine the activity of LN.

Ultrasound is the most commonly used imaging method to understand the renal morphology, and the ultrasonic performance reflects the chronic pathological changes of the kidney.⁹ However, as LN tends to be more normal in the ultrasonic graphic performance, there is no study to evaluate the activity of LN by imaging examination currently. We assume that changes in LN activity can be detected by advanced computing technologies (such as radiomics). Radiomics obtains multiple quantitative features from single or multiple medical imaging modes, highlighting image features that are not visible to the naked eye.¹⁰ Machine learning technology (ML) shows the ability of high sensitivity, high-dimensional information mining and high-throughput computing, and has more important value in differential diagnosis, the ML part is used to predict the results in the subsequent steps. These biomarkers are combined with various ML techniques to identify imaging features of subtle but complex tissue changes.¹¹ In recent years, ultrasound radiomics has been developed to assess the glomerular lesions or changes in renal function in nephropathy.^12,13 Our aim was to use the ML model based on ultrasound radiomics to noninvasively evaluate LN activity.

Method

This study protocol conformed to the ethical guidelines of the Declaration of Helsinki and was approved by the institutional review committee of the First Affiliated Hospital of Anhui Medical University (PJ2022-11-29). Written informed consent was not required due to the retrospective nature of this study. The personal information of patients was strictly protected. We included 149 LN patients who underwent renal biopsy from the First Affiliated Hospital of Anhui Medical University and Nanchong Central Hospital from January 2019 to June 2022. The inclusion criteria include: 1. LN confirmed by renal biopsy; 2. There were more than 10 glomeruli in the light microscope specimens; 3. Age of more than 18 years old; 4. Those who underwent ultrasound less than three days before renal biopsy. The exclusion criteria were: 1. Acute renal damage and heart valve disease; 2. Renal artery stenosis¹⁴ or urinary tract obstruction; 3. The patients with associated diseases either immune or non-immune.4. There was cyst or tumor in the kidney; 5. Ultrasonic image missing and poor ultrasound image quality. The enrolled patients were randomly divided into training cohort (n=104) and test cohort (n=45) at a ratio of 7:3. The research flow chart is shown in Figure 1.

Figure 1 The flow diagram of the study.

Renal biopsy was performed within three days after renal ultrasound examination by two experienced nephrologists. The right kidney was selected for renal biopsy. LN was classified according to the 2003 International Society of Nephrology/Society of renal pathology (ISN/RPS) standard.¹⁵ Specific histological features (glomerular cell proliferation, glomerular leukocyte exudation, nuclear rupture and cellulose like necrosis, cell crescent, hyaline deposits, interstitial inflammation, glomerulosclerosis, fibrous papule score, tubular atrophy, and interstitial fibrosis) were graded semi-quantitatively as described previously (0 to 3 +, indicating no, mild, moderate, or severe). The sum of the scores of individual active lesions indicate the activity index. The activity index was divided into active and inactive according to a factor >7 and ≤7, respectively.^16,17

Ultrasonic Image Acquisition

Ultrasonography was performed by four sonographers with 5, 6, 8, and 10 years of experience in renal ultrasound examination. The patients underwent ultrasound examination and image acquisition within three days before renal biopsy. The US instruments employed to acquire the images used in this study includes: GE vid E9 (General Electric Co., USA) and Mindray Resona 7 (Shenzhen Mindray Biomedical Electronics Co., Ltd., China) which used a multi frequency (2–5MHZ) convex array probe (C5-2). All measurements were performed during supine position and apnea at the end of inspiration. The ultrasound probe is gently positioned in the right abdomen by oblique projection, and the kidney is visualized as a longitudinal image to obtain the maximum coronal section of the right kidney. The parameter configuration in the acquisition process is based on the optimal display setting of the ultrasound image.

Image Segmentation and Feature Extraction of Ultrasonomics

The region of interest (ROI) of renal was manually segmented using ITK software (3.8.0., http://www.itksnap.org/pmwiki/pmwiki.php?n=Downloads.SNAP3). The ROI segmentation was performed by Reader 1 (with 9 years of renal ultrasound imaging experience) and Reader 2 (with 7 years of renal ultrasound imaging experience). The renal ultrasound images of the 50 patients were randomly selected, and the ROI was separately delineated by Reader 1 and Reader 2. Lastly, the consistency test between groups was performed. Two weeks later, the ultrasound images of 50 patients were randomly selected, and Reader 1 performed the ROI delineation step again, and the intra-group consistency test of the characteristics was performed. When the ICC (intra-class correlation coefficient) values of the inter group and intra group consistency tests are greater than 0.75, it indicates that the extracted features have good consistency. Then, the remaining image segmentation is performed by Reader 1 alone. Only features with good consistency were retained in subsequent analysis.

Construction of Ultrasonomics Features

Radiomics feature extraction is performed using Pyrodiomics (v3.0.1, https://github.com/AIM-Harvard/pyradiomics). It can extract ultrasonic radiomics features and use many engineering hard coded feature algorithms to extract features from ultrasonic images. As shown in Figure 2, the steps of selecting sonomics features and signature construction were as follows: First, the feature of ICC > 0.75 in the training cohort was retained. Second, univariate analysis U-test was used to screen out the characteristics of significant distribution difference between the active group and the inactive group in the training cohort (P < 0.05). Third, the most valuable features were selected using the lasso feature selection algorithm (least absolute shrinkage and selection operator).

Figure 2 The radiomics flow chart of the study.

As the alpha increases, the mean square error (MSE) of the LASSO model decreases first and then increases. Take the lowest alpha of MSE as the best alpha value, and the non-zero coefficient feature is the selected feature. As the alpha value increases, the coefficient of the feature gradually converges, and the characteristic graph shows the change of the characteristic coefficient with the increase of the alpha value. Under the best alpha value, the Spearman correlation analysis is performed between the selected features, display the correlation coefficient between features through thermal diagram (Figure 2).

Then, five machine learning models were trained based on the selected the best features, including logistic regression (LR), multilayer perceptron (MLP), support vector machine (SVM), random forest (RF), and extreme gradient lifting tree (XGBOOST). The radiological characteristics are verified by five-fold cross validation, and the training is repeated five times on the training set. The model AUC of each time is calculated separately, and the average value of five times is taken to obtain the best parameter configuration for training. Details of ML algorithms are presented in the Supplementary Material. The hyperparameters of the above machine learning algorithm are adjusted by grid search CV and 5-fold cross validation in the training data set. In each cycle of the five-fold cross validation, the hyperparameters with the best AUC in the validation data set are retained, and the entire training data set is used for the establishment of the final model. Patients in the test dataset were used to evaluate model performance. For each round of model training, each patient will obtain the predicted probability value of the corresponding prediction tag. Details of adjusting the hyperparameters of different machine learning algorithms are presented in the Supplementary Material.

Statistics

All statistical analyses were performed using SPSS 25.0 (IBM Corp., Armonk, NY, USA) and Python 2.7 (Python software foundation, Beaverton, or, USA) software. Quantitative data with normal distribution are expressed as mean ± standard deviation, while quantitative data with non-normal distribution are expressed as median ± interquartile range. Meanwhile, categorical variables are expressed in numbers and percentages. Chi-square test, independent sample t-test, and Mann Whitney U-test were used for univariate analysis. The area under the receiver operating characteristic curve (AUC) of the five models was compared with sensitivity, specificity, positive predictive value and negative predictive value calculated. Two-sided P values <0.05 were considered to be statistically significant.

Result

A total of 149 patients with LN confirmed by renal biopsy were included in this study, 32 of whom were male participants, while the other 117 were female. Their average age was 37.2 ±12.4 (18–73). Among them, 66 were active, while 83 were inactive. The baseline characteristics of the patients are shown in Table 1.

Table 1 The Baseline Characteristics of the Patients

Feature Extraction, Selection, and Ultrasonic Radiomic Feature Construction

A total of 1550 radiomic features were extracted from ultrasound images. After inter- and intra-observer consistency tests and univariate analysis, 208 radiomic characteristics were found to be significantly different between the two groups. These features were imported into the lasso feature selection algorithm to obtain the most valuable features and to generate 10 useful features (Figure 3). Finally, ultrasonic radiomic features were established using these 10 features.

Figure 3 Ten features were used to establish the radiomic features.

Based on a five-fold cross-validation strategy, we constructed different machine models using different algorithms. The diagnostic efficacy of these models is presented in Table 2. The receiver operating characteristic (ROC) curves of the five models based on the training cohort and the test cohort is shown in Figure 4. For judging the activity of LN, the average AUC of the models was 0.79, of which the MLP model was the best. The MLP ‘AUC of the training group was 0.89, with an accuracy of 81.7%, sensitivity of 83%, specificity of 80.7%, negative predictive value of 85.2%, and positive predictive value of 78%. The AUC, accuracy, sensitivity, specificity, negative predictive value, and positive predictive value in testing group were 0.82, 73.3%, 78.9%, 69.2%, 81.8%, and 65.2%, respectively. As described as Figure 5, the number of the predicted patients of two categories was 39(+) and 46(-) in the training set (Figure 5A), and 15(+) and 18(-) in the testing set (Figure 5B).

Table 2 Performance of the Five Model in the Training and Testing Cohorts

Figure 4 The receiver operating characteristic (ROC) curves of the five models. (A) five model ROC curves in the training cohort. (B) five model ROC curves in the testing cohort.

Figure 5 The confusion matrix of MLP model in the training (A) and testing cohorts (B).

Discussion

The individual histological features of LN activity, such as cell crescent and cellulose-like necrosis, are positively correlated with renal failure.^16,17 Uncontrolled LN activity accelerates the loss of nephrons, increases the risk of early end-stage renal disease and death, accurately judges the activity performance of LN, and guides clinicians in formulating effective treatment plans.¹⁸ However, it is difficult to understand LN activity without renal biopsy.¹⁹ Renal biopsy is an invasive examination that results in both physical and emotional pain to patients. Although there are clinical measures that can be used to assess disease activity and treatment response by measuring the renal function and proteinuria, the clinical measurement of renal function and proteinuria is not a reliable biomarker for histological activity. Therefore, clinicians are looking for new non-invasive biomarkers that can be used to assess LN activity.⁸ We developed five ML models based on the radiomics of ultrasound images to distinguish the LN activity of LN. Among the five ML models, MLP exhibited the best evaluation performance, with an AUC of 0.82 and accuracy of 73.3% in the test set. This model can be used to non-invasively assess LN activity in the clinic. To the best of our knowledge, this is also the first study that applied the ML model analysis of ultrasonic radiomic features in distinguishing LN activity.

Renal ultrasound is a routine method in the outpatient follow-up of renal changes. It is mainly used to detect renal morphological changes, including renal size, parenchymal echogenicity, and cortical differentiation;^20–22 however, the information that can be obtained from the kidney by the human eye is limited, and many details in the image cannot be recognized by the naked eye. Radiomics can automatically extract the microscopic characteristics of ROI in medical images, save the digital encrypted medical images related to tumour pathophysiology, and convert them into high-dimensional data that can be mined to reflect the microscopic structure and biological tissue information of tumours.²³ Since this information can be utilized through quantitative image analysis, the potential of radiomics to improve clinical decision support systems overcomes doubt.²⁴

In recent years, some studies attempted to apply ultrasound radiomics in clinching the diagnosis of nephrotic diseases. For instance, Zhang et al used radiomics based on ultrasound imaging to classify diffuse glomerular lesions. Quantitative characterisation of ultrasound images can be effective for IgA nephropathy and membranous nephropathy. The area under the ROC curve (AUC) of random forest was 0.7639.²⁵ On the other hand, Zhu et al extracted the radiomic features of ultrasound images to distinguish PTRF through different machine learning methods and obtained the highest ROC value of 0.839. They think that radiomics based on ultrasound images can be used to detect pathological changes that cannot be directly observed in conventional ultrasound images.¹³ Furthermore, Bandara et al extracted 10 radiomic features based on a wavelet transform to establish a model. The random forest model can effectively identify health and CKD.¹² At present, there have been no reports on the non-invasive evaluation of LN activity using ultrasound imaging omics. Therefore, the identification of LN activity using ultrasound imaging omics is worth studying.

Our study hypotheses are as follows: (1) the spatial heterogeneity of LN varies between tissue activities and (2) the spatial heterogeneity can be quantified using ultrasonic radiomics. We used 50% cross-validation to train the model in the training set. After the U-test, Spearman, and lasso feature selection, we selected 10 ultrasonic radiomic features to evaluate the activity of LN. These radiomic features were used to build prediction models using various classical classification algorithms (LR, MLP, SVM, RF, and XGBOOST). In our study, the MLP classifier had the best comprehensive performance in the noninvasive diagnosis of LN activity, with an AUC of 0.891 and 0.822 in the training and test groups, respectively. This result indicates that the ultrasound imaging omics model has a strong processing ability for high-throughput data and can identify biomarkers of subclinical changes based on ultrasonic images. In addition, the model we trained had a good diagnostic performance. In particular, the AUC ranges between 0.767 and 0.822 in the test set, indicating that the screened image omics features were also stable for constructing machine learning models.

This study has several limitations. First, the sample size was relatively small. Thus, prospective clinical studies with larger sample sizes are required in the future. Second, multimodal renal ultrasound and basic clinical information might have predicted the LN activity more accurately. This is the direction of our future research.

In conclusion, we extracted the imaging features of LN. The ML classifier based on ultrasound radiomics has a high recognition accuracy for LN activity, which can accurately and noninvasively detect LN activity and can be used as an effective tool in assisting the clinical decision-making process.