Construction and Validation of a Model for Predicting Cervical Intraepithelial Neoplasia Grade II+: A Cross-Sectional Population Study via Machine Learning

Introduction

Cervical cancer remains one of the most common gynecologic malignancies threatening women’s health, ranking second only to breast cancer in global incidence¹ and the leading cause of cancer-related mortality among females worldwide.² It is also the primary cause of cancer-related deaths in developing countries,^3,4 where it accounts for approximately 25% of all female cancers.⁵ In recent decades, cervical cancer has emerged as the second most prevalent malignancy among Chinese women, with a trend toward younger age at diagnosis and increasing incidence rates,⁶ This disease poses a severe threat to patient survival. Therefore, establishing predictive models for high-grade cervical lesions is critical, particularly in resource-limited settings in developing countries.^7,8 Similar to other malignancies, cervical cancer is a chronic, multifactorial disease driven by genetic and environmental interactions.⁹ The pathological progression of cervical lesions follows a hierarchical sequence: Cervical Intraepithelial Neoplasia Grade I (CIN I), progressing to CIN II, then to CIN III, carcinoma in situ (CIS), early invasive carcinoma, and finally invasive carcinoma. The transition from CIN to CIS typically spans 3–8 years.^10,11 Notably, CIN I lesions often regress spontaneously, whereas CIN II+ (including CIN II, CIN III, and invasive cervical cancer) represents a high-risk category requiring intervention.¹² Early detection of precancerous lesions through screening is paramount, as cervical cancer remains the only gynecologic cancer amenable to early diagnosis and curative treatment.^13–16

Due to advancements in cervical cancer screening technology, the incidence and mortality rates of cervical cancer have shown a downward trend in both developed and developing countries.^17,18 Early screening is key to reducing cervical cancer mortality. Thinprep cytologic test(TCT) using liquid-based methods has been widely applied for identifying abnormal cell morphology, but its sensitivity is limited and it is susceptible to sampling errors and subjective interpretation. In recent years, Human papillomavirus (HPV) nucleic acid testing has become an important supplementary method due to its direct detection of the pathogen, although its combined application with other clinical risk factors remains unclear. Current models for predicting cervical lesion risk primarily rely on single testing methods (such as TCT) or limited clinical variables (such as age and HPV infection history), leading to insufficient model accuracy. Some have proposed data-driven models,^18,19 but these are impractical in resource-poor environments. Therefore, it is crucial to determine whether integrating HPV testing with traditional clinical data can enhance model performance, which is significant for optimizing the screening process.

Existing predictive models for CIN II+ primarily rely solely on HPV as a single data source, exhibit suboptimal performance in specific populations, and lack integration with clinical factors. In contrast, our approach integrating TCT, HPV, and clinical data can enhance clinical discriminative performance, improve generalizability, and optimize clinical interpretability. Furthermore, the innovation of our machine learning framework, compared to traditional statistical models or prior machine learning studies, lies in its incorporation of ten machine learning algorithms, which optimizes algorithm selection.

In this study, we selected variables associated with cervical intraepithelial neoplasia grade II or higher (CIN II+) and performed logistic regression analysis to identify risk factors for CIN II+. Additionally, ten machine learning algorithms were employed to construct predictive models for CIN II+ risk, this can enhance their robustness and generalizability. The aim of this study was to construct a predictive model for CINII+, and to compare the predictive performance of machine learning models integrating TCT + HPV testing with clinical data versus TCT combined with traditional clinical data for CIN II+.

Materials and Methods

Research Design

Clinical data and screening results were collected from women aged 35 to 65 years old undergoing free cervical cancer screening at Linqian County Maternal and Child Health Hospital between January 2020 and December 2024, retrieved through the electronic medical record (EMR) system. Inclusion Criteria: 1. Underwent both liquid-based thin-layer cytology (TCT) and HPV testing, and cytology was interpreted using the Bethesda 2014 nomenclature, and HPV genotyping was performed via PCR assay to detect individual high-risk HPV genotypes; 2. Had complete pathological results; 3. Provided complete clinical data (eg, demographics, reproductive history). Exclusion Criteria: 1. Pregnant or breastfeeding women; 2. Incomplete clinical or pathological data; 3. History of endometrial cancer, ovarian cancer, fallopian tube cancer and primary vaginal cancer; 4. Lost-to-follow-up patients.

Our study included a total of 26 variables. 577 cases were divided into a training set and a testing set in a 7:3 ratio. Therefore, the sample size for the training set was 405 cases and the sample size for the testing set was 172 cases.

Data Collection

Clinical data from participants were collected using a self-developed scale, including: HPV status, TCT status, colposcope results, vaccination status, history of cervical cancer screening, history of smoking, symptoms of contact bleeding, number of sexual partners, history of CIN, marital status, gravidity, parity, history of oral contraceptives, work situation, educational level, history of drinking, menopausal status, human immunodeficiency virus status, residential area, number of abortions, history of cervical treatment; family history, age, age at first pregnancy, age of menarche, BMI=weight(kg)/[height(m)]².

Grouping Criteria

Based on the final pathological diagnosis, participants were categorized into two groups: CIN II- group (managed with follow-up, exhibiting regressive potential); CIN II+ group (requiring interventions due to high-risk progression to cervical cancer).

Variables and Model Development

Twenty-six risk factors were defined as independent variables. Univariate logistic regression analysis was performed with cervical lesion types (determined by pathological results) as the dependent variable. The association strength between each independent variable and the outcome was quantified. Variables with P < 0.05 were selected for multivariate logistic regression to explore their joint effects on cervical lesions.

Machine Learning Modeling

Significant variables identified through logistic regression were further modeled using ten machine learning algorithms: logistic regression (LR), support vector machine (SVM), gradient boosting machine(GBM), artificial neural network (ANN), random forest (RF), extreme gradient boosting (XGBoost), k-nearest neighbors (KNN), AdaBoost, LightGBM, and CatBoost.

Model Performance Evaluation

The performance of the model was evaluated around three aspects: discrimination, calibration and clinical utility. In this study, AUC was used to evaluate the model’s discrimination ability. Calibration curves were used to determine the degree of agreement between predicted probabilities and observed outcomes. Decision curve analysis (DCA) was used to assess clinical validity. The same three methods are applied to validate the model using the testing set data. All analyses in this study were performed using R software (version 4.4.1). All tests were two-tailed, and the difference was statistically significant at P <0.05.

Comparison of Cervical Lesion Prediction Models

The performance of paired sample models was evaluated using Receiver Operating Characteristic (ROC) curve analysis, with the Area Under the Curve (AUC), sensitivity, and specificity calculated. All statistical tests were two-tailed, and a P-value < 0.05 was considered statistically significant.

Statistical Methods

All analyses were performed using R software (version 4.4.1). Measurement data are presented as (x ± s). For inter-group comparisons, the Wilcoxon rank-sum test is used. Count data are expressed as n (%), and inter-group comparisons are conducted using the chi-square test or Fisher’s exact test. The differences between the two models were compared in the same sample set. The AUC values that obeyed the normal distribution were tested by paired sample T-test, and the AUC values that did not obey the normal distribution were tested by paired sample nonparametric test. All tests are two-tailed, and a P value of<0.05 is considered statistically significant.

Results

Univariate Analysis of Variable Factors for CINII+

405 cases were included in the training set, divided into CINII-group (n=299) and CINII+group (n=106) based on the degree of cervical lesions. A one-way analysis was conducted on twenty-six variables for both groups, revealing statistically significant differences in TCT, HPV, colposcopy, work status, and age at first pregnancy (P<0.05), while no other variables showed statistical significance (P>0.05), as detailed in Table 1.

Table 1 Univariate Analysis of Risk Factors for CIN II+

Predictive Model Development

For the twenty-six included variables, a stepwise logistic regression method was used to screen model variables. Univariate logistic regression showed that TCT, HPV, colposcopy, work situation and age at first pregnancy were risk factors for CINII+. Multivariate logistic regression of the univariate results identified HPV positivity, TCT indicates HSIL, colposcopy result indicates a high-grade lesion and the first age of pregnancy as independent risk factors for CINII+, as detailed in Table 2. Ten machine learning methods were used to build models using these four variables. Additionally, HPV was excluded, models were established for TCT, colposcopy and age at first pregnancy to evaluate the added value of HPV in cervical lesion screening.

Table 2 Logistic Regression Analysis of Risk Factors for CIN II+

Model Evaluation of TCT+HPV Combined Clinical Data

The training set was evaluated in terms of discrimination, calibration and clinical utility. The AUC value is used to evaluate the discrimination ability of the predictive model by examining the occurrence of CIN II +. The AUC values for ten machine learning models are as follows: 0.804, 95% CI: 0.752–0.855; 0.771, 95% CI: 0.712–0.829; 0.866, 95% CI: 0.826–0.906; 0.839, 95% CI: 0.793–0.844; 0.827, 95% CI: 0.781–0.872; 0.834, 95% CI: 0.788–0.880; 0.914, 95% CI: 0.884–0.944; 0.752, 95% CI: 0.702–0.802; 0.957, 95% CI: 0.937–976; 0.867, and 95% CI: 0.825–0.908. The sensitivities of the ROC curves for the ten machine learning models are respectively 0.547,0.613,0.830,0.821,0.660,0.830,0.840,0.557,0.840,0.764. And the specificities of the ROC curves for the ten machine learning models are respectively 0.936,0.890,0.729,0.696,0.993,0.692,0.826,0.933,0.916,0.819. (see Table 3). The results show that the line graph has good distinguishing and predictive value, capable of accurately identifying individuals with or without CIN II +. The calibration plot of the model demonstrates excellent predictive accuracy between actual and predicted probabilities. The clinical utility of the model was evaluated using the DCA curve. The decision curve indicates that the net benefit of the predictive model is significantly higher than in both extreme scenarios, suggesting that the predictive model has a higher net benefit. The results are shown in Figure 1.

Table 3 Results of Training and Validation Analysis

Figure 1 Training Sets Performance Evaluation for HPV+TCT models.(A) ROC curve (B) calibration curve (C) DCA curve.

The testing set was evaluated in terms of discrimination, calibration and clinical utility. The AUC value is used to evaluate the discrimination ability of the predictive model by examining the occurrence of CIN II +. The AUC values for ten machine learning models are as follows: 0.807, 95% CI: 0.735–0.880; 0.768, 95% CI: 0.683–0.854; 0.817, 95% CI: 0.748–0.887; 0.835, 95% CI: 0.774–0.897; 0.680, 95% CI: 0.598–0.763; 0.823, 95% CI: 0.756–0.891; 0.835, 95% CI: 0.768–0.901; 0.696, 95% CI: 0.617–0.775; 0.771, 95% CI: 0.691–0.852; 0.834, and 95% CI: 0.768–0.899. The sensitivities of the ROC curves for the ten machine learning models are respectively 0.733,0.733,0.800,0.822,0.511,0.800,0.844,0.467,0.733,0.911. And the specificities of the ROC curves for the ten machine learning models are respectively 0.748,0.732,0.677,0.709,0.835,0.717,0.693,0.913,0.732,0.598.(see Table 3). The results show that the line graph has good distinguishing and predictive value, capable of accurately identifying individuals with or without CIN II +. The calibration plot of the model demonstrates excellent predictive accuracy between actual and predicted probabilities. The clinical utility of the model was evaluated using the DCA curve. The decision curve indicates that the net benefit of the predictive model is significantly higher than in both extreme scenarios, suggesting that the predictive model has a higher net benefit. The results are shown in Figure 2.

Figure 2 Testing Sets Performance Evaluation for HPV+TCT models.(A) ROC curve (B) calibration curve (C) DCA curve.

Model Evaluation of TCT Combined Clinical Data

The training set was evaluated in terms of discrimination, calibration and clinical utility. The AUC value is used to evaluate the discrimination ability of the predictive model by examining the occurrence of CIN II +. The AUC values for ten machine learning models are as follows: 0.793, 95% CI: 0.739–0.847; 0.794, 95% CI: 0.740–0.848; 0.807, 95% CI: 0.753–0.861; 0.768, 95% CI: 0.709–0.826; 0.775, 95% CI: 0.727–0.823; 0.800, 95% CI: 0.745–0.856; 0.828, 95% CI: 0.776–0.879; 0.761, 95% CI: 0.706–0.816; 0.885, 95% CI: 0.846–923; 0.809, 95% CI: 0.756–0.861. The sensitivities of the ROC curves for the ten machine learning models are respectively 0.557, 0.585, 0.642, 0.519, 0.566, 0.604,0.632, 0.491, 0.689, 0.613. And the specificities of the ROC curves for the ten machine learning models are respectively 0.926, 0.913, 0.860, 0.943, 0.983, 0.896, 0.933, 0.943, 0.910, 0.906 (see Table 3). The results show that the line graph has good distinguishing and predictive value, capable of accurately identifying individuals with or without CIN II +. The calibration plot of the model demonstrates excellent predictive accuracy between actual and predicted probabilities. The clinical utility of the model was evaluated using the DCA curve. The decision curve indicates that the net benefit of the predictive model is significantly higher than in both extreme scenarios, suggesting that the predictive model has a higher net benefit. The results are shown in Figure 3.

Figure 3 Training Sets Performance Evaluation for TCT models.(A) ROC curve (B) calibration curve (C) DCA curve.

The testing set was evaluated in terms of discrimination, calibration and clinical utility. The AUC value is used to evaluate the discrimination ability of the predictive model by examining the occurrence of CIN II +. The AUC values for ten machine learning models are as follows: 0.796, 95% CI: 0.719–0.873; 0.797, 95% CI: 0.720–0.874; 0.769, 95% CI: 0.687–0.852; 0.772, 95% CI: 0.688–0.857; 0.683, 95% CI: 0.602–0.764; 0.790, 95% CI: 0.705–0.874; 0.737, 95% CI: 0.650–0.825; 0.702, 95% CI: 0.614–0.789; 0.712, 95% CI: 0.614–811; 0.801, and 95% CI: 0.723–0.879. The sensitivities of the ROC curves for the ten machine learning models are respectively 0.778, 0.778, 0.733, 0.733, 0.489, 0.800, 0.600, 0.578, 0.578, 0.600. And the specificities of the ROC curves for the ten machine learning models are respectively 0.693, 0.693, 0.724, 0.732, 0.866, 0.685, 0.780, 0.756, 0.843, 0.874 (see Table 3). The results show that the line graph has good distinguishing and predictive value, capable of accurately identifying individuals with or without CIN II +. The calibration plot of the model demonstrates excellent predictive accuracy between actual and predicted probabilities. The clinical utility of the model was evaluated using the DCA curve. The decision curve indicates that the net benefit of the predictive model is significantly higher than in both extreme scenarios, suggesting that the predictive model has a higher net benefit. The results are shown in Figure 4.

Figure 4 Testing Sets Performance Evaluation for TCT models.(A) ROC curve (B) calibration curve (C) DCA curve.

Model Performance Comparison

Model performance was evaluated by comparing the area under the receiver operating characteristic curve (AUC) using paired samples. Training Set: The AUC of the TCT+HPV+clinical data model (0.84 ± 0.06) was significantly higher than that of the TCT+clinical data model (0.80 ± 0.04, P = 0.006). Testing Set: The AUC of the TCT+HPV+clinical data model[Median (IQR): 0.81 (0.75–0.83)] was significantly higher than that of the TCT+clinical data model [Median (IQR): 0.77 (0.71–0.80), P = 0.035], as shown in Table 4 and Figure 5.

Table 4 Comparison of AUC Values Between Models

Figure 5 Comparison of AUC values between models.**indicates P<0.01;*indicates P<0.05. (A) Comparison of training sets. (B) Comparison of testing sets.

Discussion

Cervical cancer is a globally recognized public health issue. This type of cancer can lead to loss of life, but it is preventable if detected and treated early.²⁰ Previous studies have shown that sexually transmitted infections, hormonal influences, genetics, and participant factors (risky behaviors, drug abuse and alcohol consumption) are risk factors for cervical cancer in developing countries. To date, predicting cervical lesions remains a challenge. Previous research has primarily focused on exploring risk factors for cervical lesions, with fewer studies on predictive models for cervical lesions. Binyue Sheng et al²¹ developed a predictive model for high-grade cervical lesions that includes TCT results, HPV results, acetowhite epithelium, abnormal blood vessels and mosaic. The predictive indicators of this model are not routine examination items, which makes its application very difficult. Our study provides a predictive method to address this issue. Our model integrates routine clinical data, which can be easily obtained from electronic medical records. In addition, we investigated whether HPV adds value to the prediction of high-grade cervical lesions. Our study identified TCT results, HPV results, colposcopy results, and age at first pregnancy as variables for establishing the predictive model. The AUC values of the training sets of the ten machine learning models we developed ranged from 0.752 to 0.957, indicating high accuracy and specificity of our models. ThinPrep cytology test (TCT) is a common cytological screening method for cervical cancer.^22,23 HPV infection is a necessary condition for cervical lesions. HPV strains can be classified as low-risk or high-risk based on their propensity to cause cervical cancer.^24,25 A substantial amount of evidence shows that the rates of HPV infection and positive TCT results increase with the severity of cervical lesions. Previous studies have shown that 96% of epithelial lesions or malignancies (NILM) are negative. Abnormal results include atypical squamous cells of undetermined significance (ASC-US), low-grade squamous intraepithelial lesion (LSIL), high-grade squamous intraepithelial lesion (HSIL), atypical squamous cells—cannot exclude high-grade squamous intraepithelial lesion (ASC-H), and squamous cell carcinoma (SCC).²⁶ Our study found that colposcopy results are an independent risk factor for CIN II+, which is consistent with previous research. Priscila Thais et al²⁷ showed that the size of the lesion detected by colposcopy is associated with the severity of the lesion. Colposcopy involves examining the cervix under magnification to detect suspicious areas of cervical intraepithelial neoplasia (CIN) and invasive cancer in real-time.²⁸ It can also measure the size of cervical lesions and is useful for predicting the likelihood of a negative biopsy result in the future.²⁹ Mantoani et al³⁰ also found that in high-grade cervical lesions, the size of the lesion detected by colposcopy is associated with systemic inflammatory and immune responses.

Our study found that CINII+ group had a larger age of first pregnancy than CINII-group, which may be because that advanced maternal age at the time of first pregnancy may be associated with prolonged exposure to estrogen priming, which accelerates cervical epithelial maturation. Mature epithelial cells exhibit decreased regenerative capacity, affecting immune clearance of HPV infection.³¹

In addition, the core finding of this study is that the predictive performance of the combined TCT + HPV model significantly outperformed the TCT + clinical data model (training set comparison: p = 0.006; testing set comparison: p = 0.035). This confirms that HPV testing is a core variable in cervical lesion prediction models, and its inclusion significantly enhances the model’s ability to identify high-risk populations. Previous studies have shown that cytology test results are generally considered to reflect the current risk of CIN, while HPV status reflects both the current and future risks of CIN.^32,33 The AUC values of the traditional TCT + clinical data model’s training set ranged from 0.761 to 0.885, and the testing set’s AUC values ranged from 0.683 to 0.801. This indicates that even without incorporating HPV data, integrating clinical variables still holds some value. All in all, earlier detection and treatment of smaller, less advanced lesions may lead to improved surgical outcomes and reduced recurrence rates, while the decreasing prevalence of high-risk HPV types in vaccinated populations which may alter the overall burden of CIN.

Our model was constructed using machine learning methods with R, which offers significant advantages over traditional logistic regression models, especially in handling complex data, capturing non-linear relationships, and enhancing predictive performance. The predictive model we developed achieves predictive performance comparable to or even better than that of previous prospective cohorts. This study used a 7:3 ratio to divide the data into training and validation sets, with the testing set used for internal validation. The results showed that the model’s AUC values in internal validation ranged from 0.680 to 0.835, and the AUC values of the TCT-only + clinical data model ranged from 0.683 to 0.801. This indicates that the model still has good discrimination and calibration correction ability, and also shows that the model remains well-calibrated. DCA is a model evaluation method based on continuous potential risk thresholds (x-axis), while the net benefit (y-axis) of using the model for patient risk stratification demonstrates the model’s clinical utility. This study shows high net benefit of the model in clinical application. In resource-limited settings for HPV testing or regions where HPV detection methods remain unavailable, a cervical lesion prediction model tailored to such scenarios can be developed using an integrated modeling strategy that combines the thinprep cytologic test (TCT) with patient clinical characteristics. Such models leverage TCT results as the core data foundation, integrate synchronous clinical information, and enable quantitative risk assessment and prospective prediction of cervical lesions through multi-dimensional information fusion analysis. Ultimately, this approach provides a evidence-based scientific framework for cervical cancer screening and early intervention in contexts where HPV testing is inaccessible. In clinical practice, we can provide lifestyle interventions and targeted health education to patients. Additionally, for patients with a history of CIN II+, the model can integrate dynamic data such as HPV clearance status and follow-up TCT results to predict the risk of recurrence, which is a direction for future research.

This study has certain limitations. First, a major limitation of our research is that it was conducted at a single center, which may affect the generalizability and validity of the model application. Second, external validation of the model is needed to assess its value in clinical practice. Third, our study requires a large amount of prospective data to improve the accuracy of the model’s predictions. To address these, we will conduct multi-center external validation with partner institutions, evaluating the model’s performance in diverse populations to enhance real-world applicability.

Conclusion

In this study, it was found that HPV positivity, TCT indicates HSIL, colposcopy result indicates a high-grade lesion and the first age of pregnancy were risk factors for CIN II +. The TCT+HPV-integrated model outperformed the TCT-only model in predicting CIN II+, supporting the incorporation of HPV testing into routine screening to enhance early diagnostic accuracy. According to the findings, we constructed a prediction model based on the information of routine clinical data that could accurately predict the risk of CIN II + patients, which may provide a reference for clinicians to identify high-risk groups early.