Practical Models for Predicting Vaginal Intraepithelial Neoplasia in High-Grade Squamous Intraepithelial Lesions Patients within Two years After Conization

Introduction

Persistent infection with HPV represents a significant etiological factor in the development of VaIN and invasive carcinoma. Current research indicates that VaIN exhibits distinct associations with specific HPV genotypes. Molecular epidemiological studies have demonstrated that VaIN II and VaIN III are most frequently associated with the following oncogenic HPV types: HPV 16 (the predominant genotype), followed by HPV 33 and HPV 45.¹ Depending on the depth of the vaginal epithelium, it is divided into grades I, II, and III. VaIN I was considered a benign manifestation of HPV infection and has a high spontaneous firing rate. VaIN II and VaIN III, known as high-level VaIN, may progress as vaginal cancer.² In recent years, with the increase in the use of cervical cancer screening and the use of colposcopy, the incidence of VaIN has increased.³ Studies have found that vaginal cancer and cervical cancer have common risk factors. Among ordinary people, the consistency of HPV infection between the cervix and vagina is as high as 95%. HPV infection causes lesions of the epithelium in the two parts of the vagina and the cervix and then causes cancer, especially HPV16 and HPV18.^4–6 The majority of high-grade vaginal intraepithelial neoplasia (VaIN II/III) cases were associated with HPV infection, with HPV 16 being the predominant genotype. Notably, VaIN III lesions exhibited a higher prevalence of multifocal involvement in HPV 16-positive patients compared to VaIN II. Furthermore, single HPV genotype infection was more common than Multitype Infection in VaIN II/III cases, with a significantly higher incidence of single-type infection observed in VaIN III than in VaIN II. Some studies also suggest that smoking is associated with the occurrence of VaIN⁷.

Although the pathogenesis mechanism is similar, the incidence of VaIN is far lower than that of cervical epithelial tumor changes.⁸ Sillman believes that the incidence of cervical epithelial lesions is 100 times higher than VaIN.

Despite decades of research, there is still uncertainty and debate regarding the most accurate predictor or combination of predictors for treatment failure in high-risk factors of VaIN. Studies believe that older age, low socio-economic status, smoking, previous or accompanying cervical tumors, immunosuppressives, estrogen exposure, and previous pelvic radiotherapy are high-risk factors affecting VaIN.²

In addition, patients treated for cervical cancer or high-grade cervical lesions have an increased risk of VaIN, which is approximately seven times greater than patients without a history of the disease.⁹ The risk of vaginal intraepithelial neoplasia in previous studies summarized in Table S1.

Some analysis reveals that patients aged over 50 years who undergo hysterectomy for CIN have a significantly higher risk of developing VaIN, with age remaining an independent prognostic factor¹⁰ Persistent HPV infection is associated with an increased risk of VaIN in CIN patients after hysterectomy.^5,11,12 Histologically confirmed HSIL at the time of hysterectomy is significantly associated with subsequent development of VaIN.¹³

Therefore, regular follow-up of patients with HSIL has important clinical significance for early detection of VaIN. The existing risk assessment of VaIN after surgery in patients with HSIL is generally based on the work experience of doctors. However, this experience-based approach introduces substantial subjectivity in predicting post-treatment VaIN development, resulting in limited risk stratification accuracy and suboptimal analytical efficiency. Based on work experience, predicting the occurrence of VaIN in patients with HSIL after treatment is highly subjective, and it is challenging to assess the risk and analyze the results efficiently and accurately.

It is reported that 4–18% of patients with HSIL have persistent or recurrent after treatment, most occurring within two years after treatment.^14,15 The occurrence of VaIN after conization in HSIL patients is related to the recurrence of HSIL after conization. This retrospective study analyzed risk factors for VaIN after conization and developed a predictive model for this condition. The aim was to identify reliable risk factors for VaIN development within two years after HSIL conization, provide patients with risk management information, alleviate patient, and facilitate timely treatment.

In recent years, machine learning (ML) methods leveraging electronic medical records (EMRs) have gained significant recognition in clinical practice. The widespread adoption of EMRs across healthcare institutions has dramatically enhanced the accuracy and efficiency of clinical data collection.^16,17 However, the inherent complexity of ML models, particularly their “black-box” nature, poses substantial challenges to model interpretability. To address this key limitation, our study evaluated feature importance to identify the most influential clinical variables. In addition, we applied SHAP values to provide transparent model interpretation. These interpretability methods will generate actionable insights for clinicians, thereby supporting evidence-based treatment decisions. Furthermore, we will implement the predictive model as a user-friendly web application, enabling direct clinical application at the point of care.

Methods

Study Design

We carried out a retrospective study. We collected information for patients with HSIL treated with conization from January, 2013, to December, 2020, at Qilu Hospital of Shandong University. We recorded the development of VaIN within two years after the conization. The inclusion criteria were as follows: women who underwent conization for histologically confirmed HSIL, and the Availability of complete follow-up data. Exclusion criteria included diagnosis of other histological types, no follow-up data, and immunosuppression.

Follow Up

Patients underwent their first follow-up 4–6 months after conization and a second follow-up 10–12 months after conization. Liquid-based cytology and HPV testing were conducted at each follow-up, with cervical biopsy performed for women with abnormal cytology results (eg, atypical squamous cells of indeterminate significance or more severe lesions), positive HPV results, or abnormal colposcopy results. Specimens were collected for HPV testing using Digene or Roche Cobas 4800 kits, which detect various HR-HPV genotypes. A relative light unit/cutoff (RLU/CO) ratio ≥1.0 indicated a positive result for Digene, while the Cobas 4800 kit detected HPV16, HPV18, and 12 other HR-HPV genotypes qualitatively.

VaIN Diagnosis

The colposcopy report is crucial to obtain clinical information in this study. The patient took the bladder lithotomy position, placed the speculum to expose the cervix fully, wiped the secretions with a cotton swab, carefully observed the vulva, vagina, and cervix, put the white cotton ball of acetic acid on the surface of the cervix or suspected surface of vaginal lesions for at least 90 seconds, and observed whether the white epithelial of acetic acid on the vaginal wall and its degree. An experienced colposcopists should carefully evaluate the suspected VaIN lesion. If necessary, a histological biopsy can be selected to diagnose lesions in the vaginal. Finally, pathological diagnosis is the gold standard for diagnosis, and the tissue is diagnosed by two pathologists alone.

Predictors and Endpoints

The study collected the following clinical information: age, menopausal status, pregnancy, parity, contraceptive method, types of cervical transformation zones, preoperative cytology, preoperative HPV, Preoperative VaIN, margins after conization, follow-up cytology after conization, and follow-up HPV after conization. These variables were used as predictors in the analysis. The endpoint of the study was the development of VaIN within two years after HSIL conization. The presence of VaIN after the conization of HSIL was identified by colposcopic biopsy histology. The follow-up endpoint was described as the interval from conization for HSIL to the first appearance of VaIN.

Model Construction and Validation

We randomly split all patients into development and validation cohorts at a ratio 7:3. To optimize variable selection for clinical prediction modeling, we utilized three machine learning approaches: Random Forest (RF), Least Absolute Shrinkage and Selection Operator (LASSO), and Extreme Gradient Boosting (XGBoost). These methods were selected for their complementary strengths in handling complex, high-dimensional medical data while ensuring robustness and interpretability.

RF is an ensemble learning technique that leverages the aggregation of multiple decision trees to enhance feature selection and predictive performance.¹⁸ By employing bootstrap sampling to generate diverse tree subsets and random feature selection at each node split, RF achieves high accuracy and computational efficiency, even with large datasets. A key advantage of RF is its ability to provide an unbiased estimate of generalization error during model construction, along with its robust handling of missing data without performance degradation.^19,20 LASSO regression is particularly well-suited for medical research due to its capacity to handle high-dimensional data and perform automated variable selection. Through L1 regularization, LASSO produces sparse models by assigning zero coefficients to irrelevant variables, thereby preventing overfitting. Unlike “black-box” algorithms, LASSO offers transparent and interpretable outputs, where each variable’s contribution is explicitly quantified through regression coefficients. This transparency aligns with clinical needs, enabling practitioners to understand how individual predictors influence outcomes—a critical requirement for evidence-based decision-making.^21,22 XGBoost, a state-of-the-art machine learning algorithm, excels in modeling complex clinical data by mimicking the iterative refinement process observed in clinical reasoning. Similar to how senior specialists sequentially correct preliminary diagnoses, XGBoost trains decision trees sequentially, with each subsequent tree optimizing errors from its predecessors. Beyond its superior predictive performance, XGBoost enhances clinical interpretability through tools like SHAP values. These values provide visual insights into how each feature contributes to individual predictions, bridging the gap between algorithmic output and clinical workflow integration. This combination of predictive power and interpretability makes XGBoost particularly valuable for translating machine learning models into actionable clinical insights.^23,24

We evaluated our models using discrimination power, which measures the agreement between predicted and actual VaIN disease probability. We chose four widely used machine learning algorithms—Logistic regression (LR), LASSO, SVM, and XGBoost — to comprehensively evaluate our hypotheses. The performance of each model was assessed using a comprehensive set of metrics, including ROC curve analysis and the corresponding AUC value. Additional evaluation parameters included accuracy, F1 score, and discrimination index (sensitivity and specificity). Following model comparison, we utilized the SHAP method to elucidate the underlying prediction mechanism and explain the feature importance within the framework of the best-performing model.

Statistical Analysis

The statistical analysis was performed using R software (version 4.1.0) and the Python Scikit-Learn ML library (version 0.24.1).²³ The chi-squared test was used to compare distributions of potential predictive factors between VaIN patients and controls, while the DeLong test was utilized to evaluate differences between ROCs. All tests were two-tailed, with significance defined as p<0.05.

Results

Patient Characteristics

We collected 5358 patients treated with conization (including CKC and LEEP) due to HSIL, and 99 cases of VaIN occurred after conization. The incidence is 1.84%. Because there are fewer cases of VaIN after conization, we randomly matched patients at a 1:5 ratio, and 495 patients were used as the control group. We collected postoperative follow-up information of 594 patients; 79 patients had no follow-up data, and 14 patients underwent hysterectomy. Finally, the study included 501 Patients. There were 99 cases of VaIN after conization and 402 cases in the control group. The study process is shown in Figure 1.

Figure 1 The research flow chart of this study. (A) Collection of clinical data. (B) Model development and validation.

A total of 501 patients were followed up, with the first follow-up conducted 4–6 months after conization and the second follow-up 10–12 months after conization. We defined HPV16, HPV18, or HR-HPV as positive if the RLU/CO was greater than 1000 before conization. Table 1 summarizes patient characteristics, and factors with p<0.05 were included in modeling analysis based on chi-squared test results in the development cohort.

Table 1 Patients Characteristics

Variable Screening

The RF algorithm generated 500 decision trees, with each tree randomly selecting 10 predictors during each split. The mean descending Gini coefficient (MDG) was used as the variable importance ranking indicator to determine the most influential predictor variables for the risk of VaIN and arrange them in descending order of importance (Figure 2A). In the LASSO regression model, six predictors with non-zero coefficients were identified, and their relative importance rankings are shown in Figure 2B. The XGBoost algorithm offers advantages such as fast computation speed, accurate training results, minimal data constraints, strong generalization ability, and good scalability (Figure 2C). Figure 2D demonstrates that by taking the intersection of features selected by univariate analysis, RF, LASSO, and XGBoost identified five common predictors for constructing the prediction models: age, transformation zone (TZ) type, Preoperative VaIN, post-conization follow-up cytology, and post-conization follow-up HPV status.

Figure 2 The selected features of different methods. (A) RF. (B) Lasso. (C) XGBoost. (D) Variable Wayne diagram screened by four methods.

Prediction Models Development and Validation

Table 2 presents a comparative evaluation of four machine learning algorithm in both development and validation cohorts. In the development cohort, all models demonstrated comparable performance, with marginal differences observed. The XGBoost algorithm achieved the highest AUC (0.905; 95% CI: 0.859–0.950) and sensitivity (0.870), while the LR model exhibited higher performance in specificity (0.883) and F1 score (0.725). In the validation cohort, XGBoost maintained superior discriminative ability, yielding the highest AUC (0.910; 95% CI: 0.854–0.966) and sensitivity (0.867). The LR, SVM, and RF models showed comparable accuracy (0.854), with LR achieving the highest F1 score (0.694) in this cohort. Notably, XGBoost demonstrated consistent predictive performance across both cohorts, particularly in AUC and sensitivity metrics. Given its robust and generalizable performance, we selected XGBoost as the optimal predictive model, with AUC values of 0.905 (development cohort) and 0.910 (validation cohort) (Figure 3). The risk score for each patient and the risk calculation equations of the prediction models are shown in Figure S1, and the calibration curves are illustrated in Figure S2.

Table 2 Prediction Performance of Different Models

Figure 3 The ROC curves and radar chart visualization for the development and validation cohorts. (A) Receiver operating characteristic (ROC) curves. Area under the curve (AUC) and 95% CIs of the Development cohort. (B) Receiver operating characteristic (ROC) curves. Area under the curve (AUC) and 95% CIs of the Validation cohort. (C) The radar chart visualization of prediction performance for the Development cohort. (D) The radar chart visualization of prediction performance for Validation cohort.

XGBoost Combined Model for SHAP

We computed both global and individual Shapley values to interpret the feature importance in the XGBoost model. The SHAP summary bar plot (Figure 4A) highlights the five most influential features, ranked by their mean absolute SHAP values: age (1.54), transformation zone (TZ) type (0.38), Preoperative VaIN (0.21), follow-up HPV (0.09), and follow-up cytology (0.08). Notably, follow-up HPV status emerged as the most significant predictor. The SHAP beeswarm plot (Figure 4B) visualizes the distribution of feature effects, where red and blue hues represent positive and negative impacts, respectively, on the predicted probability of VaIN recurrence within two years post-conization in HSIL patients. Furthermore, the SHAP heatmap (Figure 4C) demonstrates the directional influence and relative magnitude of each feature across all observational cases in the dataset. Finally, the SHAP decision plot (Figure 4D) delineates the cumulative contribution of key features in shaping the model’s final risk prediction for VaIN recurrence.

Figure 4 Calculated SHAP values to explain XGBoost feature importance. (A) The SHAP feature importance bar plot ranks the top five predictors by their mean absolute impact on the model’s output. (B) The SHAP beeswarm plot illustrates the directional effects of each feature, where red and blue denote positive and negative contributions to the predicted probability. (C) The SHAP heatmap visualizes the consistency and magnitude of each feature’s influence across all individual cases in the dataset. (D) The SHAP decision plot shows the impact process of each significant features on the final predicted probability.

Convenient Application for Clinical Utility

A comprehensive web application has been developed to facilitate the implementation of our predictive model in clinical practice, as shown in Figure 5. This user-friendly interface allows healthcare providers to easily assess a patient’s risk of VaIN progression following conization procedures. The application requires clinicians to input data for the five key predictive variables identified by our model. Based on these inputs, the system automatically calculates the patient’s personalized risk score for progression to invasive cancer. To enhance interpretability, the platform generates a visual force plot displaying how each individual factor influences the final prediction. Features displayed in blue serve as protective factors supporting a “non-VaIN” classification, while those highlighted in red indicate risk factors driving the prediction toward a “VaIN” diagnosis. For patients identified with elevated risk levels, the application provides standardized communication guidance to ensure proper follow-up care. The system displays the following recommendation: According to our predictive model, you have a high risk of developing VaIN after conization. Although this result is an estimate based on the model’s calculations, it suggests a significant potential risk. I strongly recommend that you consult a gynecological specialist as soon as possible for further evaluation, accurate diagnosis, and timely management or treatment if necessary. To promote widespread adoption in clinical settings, the application has been made freely accessible online at https://vainpredict.streamlit.app. This web-based platform eliminates barriers to implementation, allowing healthcare providers across various practice settings to incorporate evidence-based risk assessment into their patient care workflows without requiring specialized software installation.

Figure 5 Convenient application for clinical utility. The convenient application of the final XGBoost model with 5 features is available for VaIN prediction. When entering actual values of the 6 features, this application automatically displays the predicted probability. Meanwhile, the force plot for individual child indicates the features that contribute to the decision of “VaIN”: the blue features on the right are the features pushing the prediction towards the “non- VaIN” class, while the red features on the left are pushing the prediction towards the “VaIN” class.

Discussion

This study compared the clinical characteristics of the VaIN group and the control group after HSIL treatment. Based on these predictive factors, we have developed a practical identification model based on preoperative and postoperative factors to identify VaIN after HSIL patients are treated with conization effectively. This almost zero-cost model minimizes the lack of follow-up of HSIL.

ML technology is an effective computer algorithm that can extract relevant information reflecting the disease’s classification and prognosis value from the medical data to achieve intelligent risk prediction and prognosis judgment of the disease.²⁵ We used ML algorithms to validate our models, and our results indicate that these prognostic risk assessment models have potential as assessment tools for patients with VaIN after conization. This model can be used as a supplementary screening program for VaIN after conization for patients with HSIL, which is particularly useful in underdeveloped and remote areas.

VaIN is mainly 33–55 years old, with an average age of 50.⁸ Studies in various countries have revealed some differences due to differences in race. Data from the United States show that VaIN ‘s onset is 51 years old.³ In our study, the age ≥48 years old significantly impacts VaIN after conization (P <0.001). In addition, we have also studied the effects of menopause on the VaIN after conization. In our study, 19.2% (19/99) patients appeared VaIN (P <0.001; OR, 3.435; 95% CI, 1.813, 6.506) after conization. Menopause has become a high-risk factor for VaIN after conization. The incidence of postmenopausal women is 2.09 times that of non-menopausal women. This is primarily due to the increase in age, decline in ovarian function, decreased estrogen secretion, and impaired vaginal immune function, as well as changes in the vaginal micro-ecological environment. Studies have shown a significant difference in the menopause time between VaIN/III and VaINI patients.²⁰ Our results also show that the average age of VaINII/III is significantly higher than that of VaINI.

The cervical transformation zone is an important symbol of the colposcopy examination. The type of cervical transformation zone is related to the patient’s age and menopausal status. Patients in the III-type transformation zone are often dissatisfied with the colposcopy examination, which can easily cause missed diagnoses.²⁶ In addition, patients with HSIL in the III-type transformation zone can quickly cause disease residues during conization. Therefore, doctors should adequately adjust the depth of conization following other inspections to achieve better treatment effects.²⁷ Previous research believes that for patients with HSIL, the III-type transformation zone is a risk factor for recurrence after conization.²⁸ In this study, type III cervical conversion zone is a risk factor for VaIN after conization (P <0.001).

Most patients with HSIL clear HR-HPV after six months of receiving surgical treatment. These patients had a lower risk of developing more severe lesions than those who had not cleared HR-HPV.^29,30 Studies have shown that the sensitivity of persistent HPV infection to predict persistent/recurrent lesions after surgery in patients with HSIL is 81–97%.^29,31 Due to the HSIL in patients with CKC surgery, the high-risk HPV infection rate was 21.2% within one year after surgery.³² This is similar to our study, with an infection rate of 20.7% for HR-HPV after conization. In addition, 61.4% of the patients in the VaIN group were positive for HR-HPV within one year after conization.

Cytology is crucial in follow-up of patients with cervical high-grade intraepithelial lesions after surgery. Women with three consecutive negative cytology results at 6, 12, and 24 months following surgical treatment have lower risk of HSIL or higher disease than those with at least one abnormal result.³³ Positive HPV or cytology test results can be used as risk factors to predict residual or recurrent cervical intraepithelial lesions at 6- and 12-month follow-up after conization.^28,34 In our study, the infection rate of high-risk HPV after conization was 20.7%. In addition, the cytological results within one year after conization were the risk factors for VaIN (P < 0.001; OR,19.304, 95% CI 8.453–44.085). Positive HPV or cytology test results in the one-year follow-up after conization can be used as risk factors for predicting VaIN. Our study strengths include the development of four prognostic assessment models for predicting VaIN occurrence after conization in HSIL patients using a combination of preoperative and postoperative follow-up data.

In underdeveloped regions with limited follow-up resources, preoperative factors can be used to screen HSIL patients at risk for VaIN after conization. However, our preoperative prediction model requires further improvement. Our model is not meant to replace postoperative co-testing screening and colposcopy, but rather to provide individualized recommendations based on patient variability. Based on our model’s predictions, we recommend that patients undergo combined cytology and HPV testing following conization. In particular, patients with a positive co-test screen one year after conization would benefit from model prediction. Patients who were identified as being at high risk and advised to follow up closely and receive prompt treatment. Our research in countries without strict cervical screening is meaningful.

Our study had several limitations. First, follow-up data was only collected for one-year post-conization, with subsequent data obtained according to follow-up rules. Second, being retrospective, the study may have been subject to bias due to missing follow-up data. Third, the prediction model we developed lacks independent external validation.

Conclusions

This study identified preoperative and postoperative risk factors for VaIN development in HSIL patients after conization and established four accurate and practical recognition models for predicting VaIN occurrence. These models revealed important risk factors associated with VaIN. Our identification model can serve as a supplementary screening tool for VaIN in HSIL patients after conization.