Establishment and Validation of a Machine Learning-Based Prediction Model for Termination of Pregnancy via Cesarean Section

Introduction

In 2010, a survey from the World Health Organization (WHO) showed that the cesarean section rate was as high as 46.2% in China, which was more than 2 times that recommended by the United Nations (15%), and the cesarean section without indications accounts for 11.6%.¹ Cesarean section is an important strategy to treat dystocia and reduce serious complications of pregnancy, leading to the decrease of maternal and infant morbidity and maternal mortality. However, if the indications to cesarean section are not strictly controlled, cesarean section without medical indications will cause a series of short-term and long-term maternal and neonatal complications,² including post-partum maternal infection, anesthesia-related complications, thrombosis-related events, postpartum hemorrhage, chronic pelvic pain, endometriosis, pelvic adhesions, neonatal pulmonary hyaline membrane disease, respiratory distress syndrome, and long-term neurodevelopmental abnormalities, which bring heavy burden to the society and economy.

Since the two-child policy was fully liberalized in October 2015, the reproductive desire in women over 35 years has increased in Beijing. As the age of child-bearing women increases, the function of various systems (including the reproductive system) gradually declines, the psychological burden becomes heavier during pregnancy in these women, and therefore the risk of complications increases during pregnancy and delivery. Under this condition, the proportion of high-risk pregnant women increases significantly. In addition, the fertility policy encourages vaginal delivery, but vaginal delivery increases the risk of pregnant women with advanced age, and cesarean section will markedly increase maternal and neonatal complications and cause adverse effects on the maternal physiology and psychology.

Accurate identification of high-risk factors of caesarean section and taking measures to reduce these risks are effective to reduce the rate of cesarean section, and controlling the rate of cesarean section has been an important index for quality control in the midwifery institutions. In recent years, artificial intelligence (AI) has been widely used in the medical field. Machine learning, as a major branch of AI, has the advantages of more stable model construction and more accurate prediction, and has been widely employed in clinical prediction. The machine learning model is also known as “black box”, which is in contrast to the models commonly used in medical research, such as Logistic regression. In the Logistic model, the model coefficient can be referred to explain the model and its prediction. However, the lack of interpretability in the machine learning prediction models limits their clinical controllability. In recent years, with the iteration of algorithms, the “black box” problem has been partially solved. Since 2020, published studies on the clinical machine learning have only shown the predicted outcomes, but the reason for the changes in outcomes with characteristic variables has never been explained. In the present study, the clinical characteristics of pregnant women were reviewed, and machine learning algorithm was employed to screen risk factors and establish predictive models. Visualizing the “black box” (independent variable ranking chart and univariate partial dependence profile) was used to demonstrate the contribution of each characteristic variable to the predictive variable, and the change of predictive variable with the characteristic variable was predicted. With these analyses, R language was used to establish a risk prediction system for cesarean section based on the individual characteristics of pregnant women and the course of pregnancy.

Materials and Methods

Patient Data

Beijing Shijitan Hospital of Capital Medical University is a top-three teaching hospital with midwifery qualifications. The vaginal delivery rate in our hospital has always been well controlled in the midwifery institutions of Beijing. A total of 2552 women received delivery in our hospital between 2020 and 2021. The characteristics of each subject were recorded in the electronic medical record database and checked by the obstetricians, obstetric nurses, and midwives. After data input, data were reviewed by the obstetric quality managers to ensure the accuracy of data input.

The demographics (age, height, weight at delivery, weight gain, etc.), information about pregnancy complications and labor (HDP, delivery analgesia, etc.), and postpartum fetal weight were recorded according to the electronic medical record database. This study was approved by the Ethics Committee of Beijing Shijitan Hospital. This study was under a retrospective design, the data was anonymized, and no identifiable information was obtained. Therefore, the informed consent was waived.

The inclusion criteria were as follows: (1) the gestational weeks were longer than or equal to 28 weeks; (2) there was single live birth; (3) there were no serious organic diseases or serious mental illnesses in the pregnant women. Exclusion criteria were as follows: (1) there was unplanned induction of labor; (2) there was induction of labor due to deformity; (3) Cesarean section was performed due to economic, personal, or social factors. Indications for cesarean section: the presence of fetal distress, active-phase arrest, and other events based on the criteria from the Obstetrics and Gynecology (ninth edition) and relevant Chinese expert consensus and guidelines.^3–5

Methods

This was a retrospective case–control study. The characteristics of subjects were collected as follows: (1) General information: age, number of deliveries (para), and gestational weeks; (2) Mode of delivery: cesarean section and vaginal delivery; (3) the preferential indication to cesarean section. A total of 2552 pregnant women were included in the present study and divided into cesarean section group (n = 702) and vaginal delivery group (n = 1850).

Statistical Analysis

R (Version 4.1.2) and RStudio (Version 1.4.1106) were used for the univariate analysis. Quantitative data with normal distribution are expressed as mean ± standard deviation ( ± s) and compared with t-test between groups. The qualitative data are expressed as frequency or percentage and compared with Chi square test between groups. A value of P < 0.05 was considered statistically significant.

According to the timeline, the total dataset was divided into a training set (2020.1–2021.6) and a validation set (2021.7–2021.12). In the training set, the variables with statistical significance (P < 0.05) in the univariate analysis were independently included in the cross-validation Lasso regression (Lasso.cv) and Boruta to screen the characteristic variables. The intersection variables between Lasso.cv and Boruta analysis were selected as the independent risk factors for caesarean section. Four models, including logistic regression (LR), K-nearest neighbor (KNN), classification and regression tree (CART), and random forest (RF), were established in the training set using K-fold cross validation, hyperparameter optimization, and random oversampling techniques. The best model was selected and the sort graph of feature variables, univariate partial dependency profile, and break down profile were delineated. Finally, the parameters of confusion matrix were calculated in the validation set, and the receiver operating characteristic curve (ROC), precise recall curve (PRC), calibration curve, and clinical decision curve analysis (DCA) were delineated.

For the binary prediction model, the pmsampsize function of RStudio was employed. At the given adjustment, the maximum R² was 0.63241. A total of 32 independent variables were included; the cesarean section rate was 0.2. The final sample size was calculated as 617, which was less than the number of subjects included in the final training set of this study.

Results

The number of caesarean section was 542 (26.765%) in the training set and 160 (30.36%) in the validation set. The baseline characteristics were compared between training set and validation set (Table 1).

Table 1 Comparisons of Baseline Characteristics Between Training Set and Validation Set

The inclusion, modeling, validation, and interpretation of the data are shown in Figure 1. In the training set, the characteristic variables were screened by univariate analysis, Lasso.cv and Boruta (Figure 2), and the independent risk factors of caesarean section included age, height, weight at delivery, weight gain during pregnancy, para, assisted reproduction, abnormal blood glucose, pregnant hypertension, scar uterus, premature rupture of membranes, placenta praevia, abnormal fetal position, thrombocytopenia, floating fetal head, and labor analgesia. Then, the total dataset was divided into training set and validation set based on the timeline. Four models were constructed in the training set using K-fold cross validation, hyperparameter optimization, and random oversampling techniques. As shown in Figure 3, the model with the lowest misclassification rate was RF. In the validation set, the accuracy of confusion matrix parameter of the best model RF was 0.8956357; MCC was 0.753012; AUC-ROC was 0.9790787; AUC-PRC was 0.957888. These indicated that RF model had excellent accuracy, discrimination performance, consistency, and generalization ability.

Figure 1 Flow chart of subject inclusion and analysis.

Figure 2 Screening of characteristic variables using Lasso.cv and Boruta. (A and B) Clinical feature selection using the Lasso.cv regression; (C) clinical feature selection using Boruta.

Figure 3 Construction of 4 models in training set.

Figure 4 shows the receiver operating characteristic curve (ROC), precise recall curve (PRC), calibration curve (calibration curve), and clinical decision curve analysis (DCA) of RF in the validation set. Results indicated that the model performed well in the overall differentiation in the validation set, and the differentiation (prediction of caesarean section) of positive results, consistency, and clinical net benefit were also favorable.

Figure 4 ROC, PRC, calibration curve, and DCA of RF in validation set. (A) ROC for RF prediction model on test set; (B) PRC for RF prediction model on test set; (C) Calibration curve for RF prediction model on test set; (D) DCA for RF prediction model on test set.

As shown in Figures 5–7, a total of 15 characteristic variables were obtained from RF model, and the sort graph of feature variables, univariate partial dependency profile, and break down profile were delineated. The importance ranking could intuitively display the contribution of each variable to the predicted variable. The univariate partial dependency profile was used to analyze the RF model, and reflected the impact of each characteristic in the sample, as well as the positive and negative effects. Simultaneously, when the value of a characteristic variable was higher or lower than the critical value, the predictive variable underwent qualitative transformation. The Break Down profile could be employed to predict any sample.

Figure 5 Sort graph of characteristic variables.

Figure 6 Univariate partial dependence profile.

Figure 7 Break down profile of RF.

Discussion

In this study, we first determined the high-risk factors of caesarean section: age, height, weight at delivery, weight gain, para, assisted reproduction, abnormal blood glucose, pregnant hypertension, scarred uterus, premature rupture of membrane, placenta praevia, abnormal fetal presentation, thrombocytopenia, fetal head floating, and labor analgesia. These factors were further analyzed based on reviewing literatures.

Since the two-child policy was fully liberalized in October 2015, the reproductive desire in women over 35 years has increased in Beijing. Pregnant women of advanced age refer to those ≥35 years at childbirth. Pregnant women of advanced age have the following obstetric characteristics: poor elasticity of the uterus; the incidence of various medical complications and pregnancy-related complications increases with age; in recent years, the rate of cesarean section has increased in pregnant women, including re-pregnant women of advanced age. It has been reported that the risk of cesarean section increases in the pregnant women of advanced age.⁶ In addition, with the trend of late marriage and late childbirth, increasing primiparous women of advanced age refuse vaginal delivery due to some reasons and prefer to cesarean section, which results in a high rate of cesarean section⁷ and also increases the risk during the second pregnancy. In this study, our results showed the rate of cesarean section increased when the pregnant women were 34 years old or older (Figure 6), which was consistent with previous findings.⁸ We help pregnant women understand the process and benefits of vaginal delivery through maternity schools and midwifery clinics, which help pregnant women and their families build confidence in vaginal delivery.

Maternal height and weight at delivery also affected pregnancy outcomes. Pregnant women with short stature often have narrow pelvic cavity, which may cause floating fetal head and abnormal fetal position, eventually leading to the failure of vaginal delivery and the subsequent termination of pregnancy by cesarean section. However, there is evidence showing that the success rate of vaginal delivery is still very high in the pregnant women with short stature alone.⁹ The effect of short stature is clinically significant only when there are concomitant risk factors.

A cohort study on 2394 women with planned pregnancy showed that the pre-pregnancy body mass index (BMI) and weight gain during pregnancy significantly influenced the pregnancy outcomes, including pregnancy-related complications (such as gestational hypertension, gestational diabetes mellitus), and delivery-related complications (such as premature rupture of membrane, macrosomia, and low-birth-weight infants).¹⁰ Pregnancy- and delivery-related complications may increase the risk of cesarean section.^11–15 Our results showed that the probability of Cesarean section increased when the weight gain was 12 kg or larger (Figure 6). Therefore, we speculate that the pre-pregnancy and gestational weights can be used to predict pregnancy outcomes.

Our results indicated that the outcomes of delivery were related to comorbidities and complications during pregnancy and at delivery. As shown in Figure 6, with the aggravation of gestational hypertension, the risk of cesarean section increases, which is consistent with previous findings.¹² Abnormal blood glucose may cause disordered glucose and lipid metabolism and giant fetus, therefore increasing the probability of cesarean section.¹³ Thrombocytopenia during pregnancy may increase the risk of intrapartum bleeding, postpartum bleeding, and even intracranial hemorrhage in pregnant women. Studies have shown that, when the platelet count is lower than 50×10⁹/L, cesarean section can reduce the risk of intracranial hemorrhage.¹⁶ Placenta praevia refers to the partial or complete covering of the internal opening of the cervix by the placenta. The 2020 Canadian guidelines recommend cesarean section in the case of placenta praevia.¹⁷ In the process of vaginal delivery, fetal breech position may cause difficulty in delivering the fetal head, resulting in neonatal injury and asphyxia. Therefore, most pregnant women choose cesarean section to terminate pregnancy. Khadija's research shows fetal breech position is one of the factors predicting caesarean section.¹⁸

Premature rupture of membranes is prone to chorioamnionitis. Intrauterine infection may cause fever, increased fetal heart rate, uterine inertia, and fetal distress, thus increasing the risk of cesarean section.¹⁹ Premature rupture of the membrane leads to the reduction of amniotic fluid, which is harmful for the fetal head rotation at delivery. Under this condition, the umbilical cord is susceptible to be compressed. These may increase the risk of cesarean section.

Women who received assisted reproductive technology, such as in vitro fertilization-embryo transfer, usually refuse vaginal delivery and prefer elective cesarean section because of their precious fetuses, difficulties in pregnancy and complex family, social, and psychological factors. There is evidence showing that most pregnant women with a history of infertility choose cesarean section to terminate pregnancy.⁸ Thus, correcting this perception and promoting successful vaginal delivery in women receiving assisted reproduction is still a challenge in clinical practice and also an important responsibility for midwifery agencies.

Labor analgesia is a milestone in the progression of modern society. The severe and long pain at delivery may cause nervousness in pregnant women and therefore they fear delivery and prefer cesarean section. This is an important reason for cesarean section without medical indications and it also increases the incidence of postpartum depression. With the popularity of labor analgesia in recent years, the use of labor analgesia may reduce the rate of cesarean section.^20,21 The new labor process proposed in 2014 extends the labor time, and the use of labor analgesia minimizes the delivery-related pain in pregnant women, which avoids unnecessary intervention and elevates the vaginal delivery rate. A study indicates that analgesia initiated in the latency is safe and may not affect the fetus and neonate and not increase the rate of cesarean section.^20,22 With the social development and the implementation of two-child policy, the number of pregnant women of advanced age is increasing over year, and therefore the pregnancy-related complications increase in recent years. Studies have indicated that, in pregnant women with gestational hypertension or diabetes mellitus, labor analgesia may stabilize blood pressure and glucose at delivery.²³ Liu et al found that labor analgesia was able to reduce the incidence of postpartum depression.²⁴

In 2010–2020 Annual Report on Quality Control in Midwifery Institutions of Beijing, the rate of successful vaginal delivery in our hospital is well controlled in Beijing. The measures to reduce the rate of cesarean section in our hospital are as follows: (1) The preoperative discussion is standardized, the indications to cesarean section are strictly controlled, and the cesarean section without medical indications is reduced. (2) The intra-pregnancy education is strengthened to improve awareness among pregnant women and their family members about the benefits of vaginal delivery. (3) The competence of midwiferies is improved, and some measures (such as labor analgesia and doula delivery) are employed. (4) The perinatal health care is highlighted to reduce the incidence of complications in pregnant women and neonates. Based on our experience, the mode of delivery was analyzed in our hospital, aiming to develop a prediction model that can accurately identify high-risk factors for cesarean section and help clinicians make decision to reduce the complications of vaginal delivery, control the rate of cesarean section, and ensure the health of mother and neonates.

Another study investigated the mode of delivery in 15,091 pregnant women in Beijing between 2016 and 2019. The results showed that re-pregnancy after cesarean section (OR: 145.565, 95% CI: 121.822–173.935), abnormal fetal position (OR: 57.843, 95% CI: 37.915–88.246), twin pregnancy (OR: 19.748, 95% CI: 13.559–28.762), age of pregnant women (OR: 1.225, 95% CI: 1.200–1.250), gestational weeks (OR: 0.844, 95% CI: 0.823–0.865), and para (OR: 0.211, 95% CI: 0.191–0.232) were factors related to the termination of pregnancy by cesarean section.²⁵ Our study confirmed these factors and also identified other factors that could be used to predict the mode of delivery, and the prediction model was thereafter validated.

Among the independent high-risk predictive factors in this prediction model, scar uterus, labor analgesia, weight at delivery, para, and weight gain during pregnancy contributed significantly to the final outcomes. With the introduction of new birth policy, pregnant women who receive caesarean section at first delivery often choose cesarean section to terminate their second pregnancy due to concern about the risk of uterine scar rupture in vaginal delivery, which endangers the safety of mother and baby. A study on the rate of and indications to caesarean section in Shanghai shows that the leading reason for cesarean section is scar uterus.²⁶ However, studies have reported that vaginal delivery is feasible at the second delivery in 60%−80% of women who receive prior caesarean section,^26–28 and the risk of uterine rupture is less than 1% in these women.^26,29 Therefore, under strict monitoring, pregnant women can receive vaginal delivery after prior cesarean section. Of note, the complete plan should be prepared before vaginal delivery. In case of threatened uterine rupture, labor analgesia can be employed for immediate cesarean section. The control of weight before delivery and weight gain during pregnancy is also an important determinant of vaginal delivery, which depends on high-quality prenatal examination and the awareness of pregnant women on the importance of proper weight gain during pregnancy. The number of vaginal delivery is conducive to reducing cesarean section. Therefore, the indications to the first cesarean section should be strictly controlled to avoid unnecessary caesarean section, thereby reducing the incidence of scar uterus. In short, promoting successful vaginal delivery of pregnant women is a long-term task of midwives.

The screening of characteristic variables is an important part of modeling and plays a crucial role in machine learning. Its goal is to determine the most predictive characteristic subset from a large number of available variables, in order to improve performance and generalization ability of the model. Excessive variables will increase the complexity and overfitting of the model, while less variables will lead to insufficient fitting of the model. In the present study, combination of univariate analysis, Lasso.cv, and Boruta was employed to screen characteristic variables to optimize and simplify the subset.

The low number of positive samples in this dataset was ascribed to classification of imbalanced data. Under this condition, traditional sampling is difficult to obtain accurate results. Thus, K-fold cross validation and random oversampling techniques were used for modeling in the training set modeling, in which the majority of class units remained, and new minority class units were linearly synthesized in the minority class units with close distance, therefore balancing the training set. In the validation set, the original data remained for validation, which may truly reflect the distribution of future samples.

In the classification, a strong classifier with excellent generalization ability is generally established in the training set, and Logistic regression is such a classifier. However, the integrated algorithm aims to construct multiple weaker classifiers and then combine them to form a strong classifier with certain generalization performance. At the algorithm rationale level, more attention is paid to the minority samples, and it is better than a single classifier in the modeling of unbalanced data. In this study, RF was one of integrated learning techniques. Accuracy is no longer appropriate as an indicator for the evaluation of imbalanced classification, as although the classifier achieves high overall accuracy, it is still difficult to guarantee excellent classification performance in the minority classes. Therefore, in our study, the confusion matrix was calculated during the model validation, and finally the accuracy, AUC-ROC, AUC-PRC, and MCC were analyzed, aiming to comprehensively assess the model.

This study attempted to open the “black box” of machine learning by using Interpretable Machine Learning. The contribution degree of each characteristic variable was explained through the sort graph of feature variables. The evolution of predicted variables with changes in characteristic variables was explained through a univariate partial dependency profile; visualization prediction of random individual samples was performed through Break Down profile. These help to better understand the model, solve the problem of lack of interpretability in predictive models, and prompt clinicians to make timely interventions for high-risk patients to reduce risk.

There are still limitations in this study. Firstly, multiple clinical factors were included in this study, but non-clinical factors (such as psychological or socio-economic factors) in pregnant women were excluded. Some studies have investigated the willingness of pregnant women and their family members on the mode of delivery in the third trimester, and results showed that it was difficult to alter the willingness to deliver by cesarean section, which might be related to the strong psychological implication of pregnant women. This also implies the importance of education about the vaginal delivery in the health care during pregnancy.³⁰ Some studies have also shown that the hospital where pregnant women receive delivery also plays a key role in the successful vaginal delivery.³¹ Therefore, in future investigations, the predictive factors should be further expanded, and the impact of these factors on the final mode of delivery should be comprehensively analyzed. Moreover, Time Series Split was employed in this study for external validation. In the future, an independent dataset is required for this model to test its extrapolation and generalization capabilities.

Conclusion

In this study, a total of 2552 subjects were included, including 1850 subjects in the vaginal delivery group and 702 in the caesarean section group. Our results showed risk factors for cesarean section included age, height, weight at delivery, weight gain, para, assisted reproduction, abnormal blood glucose, gestational hypertension, scarred uterus, premature rupture of membrane, placenta praevia, abnormal fetal position, thrombocytopenia, floating fetal head, and labor analgesia. In this study, a machine learning algorithm was employed to predict the outcomes of delivery in 2552 pregnant women, and 81,664 data were analyzed to investigate the factors affecting the outcomes of delivery. In addition, the important characteristic variables were screened. This study aimed to develop a more accurate and powerful prediction model for the delivery outcomes, which may provide reference for management of delivery and selection of mode for delivery.