The Validation of a Nomogram for Predicting Recurrence in Patients with Non-Valvular Atrial Fibrillation Post-Ablation

Introduction

Atrial fibrillation (AF) is a common arrhythmia, and its global prevalence is increasing annually, particularly among the elderly.¹ AF significantly increases the risk of severe complications such as stroke and heart failure, which not only reduces patients’ quality of life but also imposes a substantial economic burden on families and society. Although radiofrequency catheter ablation (RFCA) can effectively improve patients’ symptoms and quality of life, post-procedural recurrence remains a significant challenge requiring urgent solutions. The predominant mechanism underlying AF recurrence after RFCA is pulmonary vein (PV) reconnection, due to the transient and non-transmural nature of the ablation lesions. This allows for the re-establishment of electrical conduction from previously isolated PVs. Additional factors include the presence of non-PV triggers and progressive atrial substrate.² 2023 ACC/AHA/ACCP/HRS Guideline highlighted the progressive emphasis on patient selection, the central role of pulmonary vein isolation (PVI) as the cornerstone, and the conditional recommendation for more extensive ablation strategies in persistent AF.³

Studies report widely varying recurrence rates after AF ablation, typically ranging from 20% to 40%.⁴ This high recurrence rate not only compromises the effectiveness of the procedure but may also increase the psychological and economic burden on patients. Therefore, accurately predicting the risk of recurrence after AF ablation has become a major research focus in the cardiovascular field. However, existing prediction models still have limitations and remain to be improved.⁵ Therefore, more precise, and readily applicable models are particularly crucial and urgent.

The aim of our study was to utilize the developed nomogram for evaluating the risk of recurrence in non-valvular atrial fibrillation (NVAF) patients after RFCA and compare its performance with the APPLE, ATLAS and Antwerp scores.

Materials and Methods

Study Population

This retrospective study developed and internally validated an AF recurrence risk model by collecting clinical characteristics, laboratory values and echocardiographic data from 242 patients with NVAF who underwent successful RFCA at our hospital between September 2022 and November 2023. Additionally, an interactive web-based dynamic nomogram was constructed.

Definition and Follow Up

AF was classified as paroxysmal or non-paroxysmal (persistent, long-standing persistent, or permanent). New York Heart Association (NYHA) functional classification was categorized according to the 2022 AHA/ACC/HFSA Heart Failure Guidelines.⁶ Patients were followed per a standardized protocol, with outpatient clinic visits at 3, 6, and 12 months. A 12-lead ECG was obtained at each visit. Additionally, all patients received a handheld single-lead ECG monitor and were instructed to record twice daily and upon symptom occurrence. All transmitted recordings were reviewed by two independent cardiologists. AF recurrence was defined as any atrial arrhythmia (AF, atrial flutter, or atrial tachycardia) lasting >30 seconds after the 3-month blanking period.⁶ Systemic immune-inflammatory index (SII) was calculated as: [neutrophil count (×10⁹/L) × platelet count (×10⁹/L)] / lymphocyte count (×10⁹/L).^7–9 Recurrence risk scores were evaluated as follows: APPLE score¹⁰ (range: 0–5 points; 1 point each for): Age >65 years, persistent AF, impaired eGFR (<60 mL/min/1.73m²), left atrial diameter ≥43 mm, left ventricular ejection fraction (LVEF) <50%; ATLAS score:¹¹ age >60 years (1 point), female sex (4 points), non-paroxysmal AF (2 points), current smoking (7 points), left atrial volume index (LAVI) (1 point per 10 mL/m²); Antwerp score:¹² LAVI >50 mL/m² (1 point), history of paroxysmal AF (1 point), QRS duration >120ms (any morphology; 2 points), heart failure etiology (2 points). Baseline characteristics were retrospectively collected from medical records by physicians blinded to study objectives.

Statistical Analysis

Statistical analyses were performed using SPSS 24.0 (SPSS Inc., Chicago, IL, USA) for descriptive statistics and hypothesis testing. Nomogram development and internal validation were conducted in R version 4.3.3 (R foundation for statistical computing). Continuous variables with normal distribution are presented as mean ± standard deviation (SD), while quantitative data with a skewed distribution are expressed as the median [M (Q25, Q75)]. Categorical variables were expressed as percentages (%). For normally distributed continuous variables, independent samples t-tests were employed for comparison; for non-normally distributed continuous variables, the Mann–Whitney U-test was used for analysis. Comparisons of categorical variables were performed using the χ²-test or Fisher’s exact test, based on the data distribution. Independent predictors for the multivariable model were identified by first screening baseline variables via univariate analysis (p < 0.1), followed by backward elimination in a logistic regression model (retained at p < 0.05). The final nomogram was derived from the coefficients of this model. This dynamic nomogram was deployed as an interactive web application using the DNbuilder function (DynNom R package), enabling real-time clinical implementation. Model performance metrics, including AUC, sensitivity, specificity, accuracy, PPV, NPV, were calculated using the “pROC” package in R software. The DeLong test compared nomogram AUC against each of the three scores. Predictive value assessment included 95% confidence intervals for all scores. Additionally, the IDI using the “PredictABEL” package and the NRI using the “nricens” package were calculated to compare the predictive performance of the nomogram against the scores in R software. The lrm function from the “rms” package was used to fit a logistic regression model. The calibrate function from the “rms” and decision curve function from the “rmda” package were used. A p-value < 0.05 was considered to indicate statistically significant.

Results

1. Baseline characteristic of the patients with NVAF were summarized in Table 1. Comparative analysis of the non-recurrent and recurrent patients with NVAF in the two cohorts were summarized in Table 2. The predictive model was developed by five variables including LAVI: OR 1.055 (95% CI: 1.021–1.090), P = 0.001; RAVI: OR 1.040 (95% CI: 1.008–1.073), P = 0.014; SII: OR 1.015 (95% CI: 1.001–1.030), P = 0.036; NYHA functional classification: OR 2.861 (95% CI: 1.282–6.383), P = 0.010; CHA₂DS₂-VASc score: OR 1.417 (95% CI: 1.077–1.864), P = 0.013. Examples of an online (Dynamic Nomogram) application via the shinyapps.io platform (Figure 1a and b). Baseline features of the recurrence risk scores (Table 3): The differences between non-recurrent and recurrent patients were statistically significant for the three scores in the two cohorts (all P < 0.01). The receiver operating characteristic (ROC) curve and area under curve (AUC) analysis for predicting NVAF post-RFCA recurrence risk with the three scores were revealed in Table 4. The results of the DeLong test showed that there were no statistically significant differences in the AUC values between any pairwise comparisons of the three scores in the two cohorts (all P > 0.05).

Table 1 Training Cohort and Validation Cohort of the Patients with Non-Valvular Atrial Fibrillation

Table 2 Comparative Analysis of the Training and Validation Cohort Patients with Non-Valvular Atrial Fibrillation

Table 3 Comparative Analysis of Recurrence Risk Scores Between Recurrent and Non-Recurrent Patients in the Both Groups

Table 4 ROC Analysis of the Three Scores in Predicting Post-Procedural Recurrence Risk of the NVAF Patients

Figure 1 The risk prediction nomogram and an example.(A) For the NVAF patient with the following pre-ablation characteristics: NYHA class: I–IV, CHA2DS2-VASc score, LAVI, RAVI, SII. The dynamic nomogram predicted a probability of 1-year AF recurrence. (B) For a NVAF patient with the following pre-ablation characteristics: NYHA class: III–IV, CHA2DS2-VASc score: 5, LAVI: 55 mL/m², RAVI: 36 mL/m², SII: 70 (values presented as SII/10). The dynamic nomogram predicted a 77.8% (95% CI: 60.6%-88.9%) probability of 1-year AF recurrence. Abbreviations: NVAF, non-valvular atrial fibrillation; NYHA, New York Heart Association; LAVI, left atrial volume index; RAVI, right atrial volume index; SII, systemic immune-inflammation index; CI, confidence interval; AF, atrial fibrillation.

2. Comparison of ROC and AUC curves for predicting NVAF post-RFCA recurrence risk: nomogram vs recurrence risk scores (Figure 2a and b and Table 5).

Table 5 ROC Analysis of the Nomogram and Recurrence Risk Scores

Figure 2 Comparison of ROC curves and AUC analysis for predicting NVAF post-RFCA recurrence risk: nomogram vs recurrence risk scores (A) The nomogram showed the performance metrics with a sensitivity of 63.9%, specificity of 89.8% and accuracy of 83.6% in the training cohort and sensitivity of 89.4%, specificity of 76.9% and accuracy of 83.7% in the validation cohort. (B) The model achieved significantly higher values with an AUC of 0.837 (95% CI: 0.774–0.899) in the training cohort and 0.895 (95% CI: 0.823–0.968) in the validation cohort (P < 0.05) than those of the APPLE score (0.745/0.731), ATLAS score (0.695/0.684), and Antwerp score (0.690/0.683) in both cohorts, respectively. Abbreviations: ROC, receiver operating characteristic; AUC, area under the curve.

The risk of postoperative recurrence increased when values exceeded these thresholds in the validation cohort, that is, nomogram (0.465), APPLE score (2.5), ATLAS score (10.0), Antwerp score (2.5). The sensitivity and accuracy of the nomogram were higher than the three scores in the two cohorts. The nomogram’s positive predictive value (PPV) and negative predictive value (NPV) were also superior to the three scores. This indicated the nomogram had higher discriminatory power for predicting AF recurrence post-RFCA. The limitations of the individual scores were with a consistently low specificity (65.7%/66.0%) of the APPLE score, low PPV (54.9%/51.5%) and might lead to a risk of excessive intervention in non-recurrent patients. The characteristic of the ATLAS score was including inadequate sensitivity in the training cohort (57.4%), imbalance between sensitivity (65.4%) and specificity (68.1%) in the validation cohort, which led to 34.6% recurrence cases missed diagnosis in the validation cohort. The feature of the Antwerp score was including high specificity (91.5%), but drastically reduced sensitivity (34.6%) and finally led to missed diagnosis rate as high as 65.4% in the validation cohort.

By integrating multi-dimensional parameters and optimizing risk stratification, the nomogram significantly reduced the misjudgment rate of 16.5%-53.6% and 24.0%-34.2%, which compared with the APPLE and ATLAS scores, respectively. The nomogram demonstrated significantly better balance between sensitivity and specificity and superior predictive stability compared to the single-dimension scores.

The model achieved significantly higher values with an AUC of 0.837 in the training cohort and 0.895 in the validation cohort than those of the APPLE, ATLAS and Antwerp scores (vs 0.745/0.731, 0.695/0.684, 0.690/0.683) (all P < 0.05). This demonstrated the superior predictive capability and stability of the nomogram for AF recurrence post-RFCA. DeLong test pairwise comparisons revealed that the AUC of the nomogram was significantly different from the AUCs of the three scores in the two cohorts. In the training cohort vs validation cohort, DeLong test pairwise comparisons revealed that the AUC of the nomogram was significantly different from the AUCs of the APPLE score (Z=2.103, P=0.036 vs Z=2.530, P=0.011), ATLAS score (Z=2.741, P=0.006 vs Z=3.155, P=0.002), and Antwerp score (Z=2.800, P=0.005 vs Z=2.778, P=0.005), respectively.

3. Comparison of calibration and decision curves for predicting NVAF post-RFCA recurrence risk: nomogram vs recurrence risk scores (Figure 3a and b).

Figure 3 Comparison of calibration curves and decision curves for predicting NVAF post-RFCA recurrence risk: nomogram vs recurrence risk scores. (A) The model’s calibration curve demonstrated high agreement with the ideal line (diagonal) in the training and validation cohorts. In contrast, the calibration curves of the APPLE, ATLAS score and Antwerp scores showed varying degrees of deviation from the ideal line. This deviation was particularly noticeable in the medium-to-high recurrence probability range for the three scores, indicating reduced prediction accuracy within this risk range. (B) In the training and validation cohorts, the decision curve of the nomogram lay above those of the APPLE, ATLAS, and Antwerp scores across a broad range of threshold probabilities (0.1–1.0). In the validation cohort, the net benefit value of the nomogram (approximately 0.22) was also higher than those of the APPLE score (0.09), ATLAS score (0.06), Antwerp score (0.08). These results indicated that the nomogram provided superior comprehensive net benefit at different clinical decision thresholds. It offered more valuable decision support for clinically predicting recurrence after RFCA across different datasets.

The model’s calibration curve demonstrated high agreement with the ideal line (diagonal) in the two cohorts, while the three scores showed varying degrees of deviation from the ideal line. This deviation was particularly noticeable in the medium-to-high recurrence probability range for the three scores, indicating reduced prediction accuracy within this risk range.

In the two cohorts, the decision curve of the nomogram lay above those of the three scores across a broad range of threshold probabilities (0.1–1.0). These results indicated that the nomogram provided superior comprehensive net benefit at different clinical decision thresholds. It offered more valuable decision support for clinically predicting recurrence after RFCA across different datasets.

4. Comparative analysis of the integrated discrimination improvement (IDI) and net reclassification index (NRI) between the nomogram and recurrence risk scores. (Table 6).

Table 6 Comparative Analysis of NRI and IDI Between the Nomogram and Recurrence Risk Scores

The nomogram demonstrated superior NRI and IDI across the cohorts. Although the improvement in NRI compared to the APPLE score in the training cohort did not reach statistical significance, the nomogram significantly outperformed the three scores on all metrics in the validation cohort (all P < 0.001). Compared to the ATLAS and Antwerp scores in the training cohort, the nomogram showed significant advantages in NRI and IDI (P < 0.001). Compared to the APPLE score, the NRI was not statistically significant (P = 0.136). However, the IDI was still significantly better than the APPLE score (P < 0.001). In the validation cohort, the nomogram comprehensively outperformed the three scores.

Discussion

This study developed a predictive model integrating LAVI, right atrial volume index (RAVI), SII, NYHA classification, and the CHA₂DS₂-VASc score. Compared to the APPLE, ATLAS and Antwerp scores, the nomogram demonstrated superior predictive capability and stability for post-RFCA AF recurrence, evidenced by its higher AUC, improved PPV and NPV. By incorporating multi-dimensional parameters and optimizing risk stratification, it significantly reduced misclassification rates while achieving a more balanced sensitivity-specificity profile than single-dimensional scores. Furthermore, the nomogram maintained robust predictive stability across diverse datasets, offering enhanced clinical decision support for post-RFCA recurrence risk assessment.

Unlike early prediction scores^7,13,14 and subsequent models,^10,11 most traditional scoring systems inadequately incorporated inflammatory biomarkers or lacked standardized criteria. These limitations hindered their application in precision medicine.^6,15–22 In contrast, the present nomogram integrating atrial structure, SII, and clinical factors, enables more precise individualized assessment of AF recurrence risk. This is supported by established evidence, including the dose-dependent relationship between LAVI and recurrence risk (3.2% increase per 1 mL/m²),²³ the independent prognostic value of RAVI >30 mL/m²,^24,25 and the role of SII in reflecting inflammation-driven atrial remodeling that directly fuels AF recurrence.^7,26,27 The inclusion of NYHA class was motivated by the well-established pathophysiological link between heart failure and AF recurrence risk, as these conditions frequently coexist and mutually exacerbate each other. This rationale is supported by a multicenter study,²⁸ which found a significantly higher AF recurrence rate in patients with versus without advanced heart failure (40% vs 29.6%; P=0.011). Similarly, the CHA₂DS₂-VASc score was incorporated as a clinically relevant comorbidity marker to enhance the model’s predictive value. The APPLE score demonstrated superior overall predictive value for post-ablation recurrence in NVAF patients compared to the ATLAS and Antwerp scores, consistent with prior studies.^8,29,30 This advantage was particularly evidenced by its significantly higher AUC on ROC analysis. In both datasets, the APPLE score achieved moderate discrimination for predicting post-procedure AF recurrence, indicating reasonable reliability and stability. All three scores exhibited stronger predictive performance for negative outcomes (absence of recurrence) than for positive outcomes (recurrence). This discrepancy likely stems from the complex clinical profiles and heterogeneous complications among recurrence patients, resulting in higher negative predictive rates, which aligning with existing literatures.^8,29,30

Our nomogram, constructed from multi-dimensional parameters, demonstrated superior discrimination for predicting AF recurrence compared to existing models. This contrasts with Jud et al’s SUCCESS score approach.³¹ While the APPLE score showed some efficacy in early studies,⁶ its parameter selection inadequately incorporated key pathological factors, limiting predictive power. Tang et al³² analyzed 11 prediction models and found the ATLAS score relatively superior, yet it still demonstrated suboptimal overall performance and clinical utility. The Antwerp score exhibited markedly elevated specificity (0.915) but critically low sensitivity (0.346), reflecting an “overly conservative” screening tendency that risks missing high-recurrence patients. This limitation aligns with reports showing insufficient sensitivity in high-risk populations for scores relying solely on QRS duration and heart failure etiology.^33,34 Crucially, our inclusion of SII directly supports Korantzopoulos et al’s “inflammation-structural remodeling-electrical remodeling” theory of AF recurrence.³⁵ The nomogram maintained strong generalizability, with AUC rising to 0.895 in validation. In contrast, all three comparator scores showed performance degradation during external validation (eg, Antwerp AUC: from 0.690 to 0.683), suggesting over-reliance on paroxysmal AF population characteristics.

Significantly, the nomogram achieved higher predictive accuracy and reliability for recurrence compared to all three scores. Their suboptimal calibration likely stems from differences in parameter selection. Huang et al³⁶ demonstrated that the inflammatory marker platelet-to-lymphocyte ratio (PLR) emerged as an independent predictor of post-RFCA recurrence in AF patients. Notably, supplementing the APPLE score with PLR significantly improved its calibration. Furthermore, the ATLAS score consistently underestimated recurrence probability in moderate-to-high-risk AF patients. Although incorporating LAVI, its broad scoring intervals likely compromised model fit for this parameter. While the Antwerp score effectively predicts left ventricular functional recovery post-ablation in heart failure patients (AUC: 0.86–0.93),^10,12 its association with AF recurrence requires further validation.^33,34

Nevertheless, our nomogram’s holistic integration of multi-dimensional parameters enabled more precise identification of high-risk patients. Decision curve analysis (DCA)³⁷ confirmed its superior clinical utility: across the 0.1–1.0 threshold probability range in the validation cohort, the nomogram yielded significantly higher net benefit than comparator scores. This advantage is particularly relevant given the CHA₂DS₂-VASc score’s documented decision uncertainty for intermediate-risk patients in current guidelines.³⁸ By quantifying atrial remodeling and inflammatory status, the nomogram provides a more robust foundation for individualized anticoagulation strategies while demonstrating superior reclassification ability and comprehensive predictive improvement. These findings align with Ma et al,³⁹ whose model outperformed the APPLE score in predicting AF recurrence after RFCA (n=500) using IDI and NRI metrics.^40,41 The nomogram’s stable discriminative enhancement suggests that synthesizing multi-dimensional variables with optimized weighting effectively mitigates prediction errors inherent in the three scores, which may arise from variable selection bias or linear modeling constraints.

Although the nomogram’s NRI compared to the APPLE score did not reach statistical significance in the training cohort, its significant IDI confirmed meaningful enhancement in overall risk discrimination. This differential performance between discrimination and reclassification metrics may reflect the model’s enhanced precision in identifying high-risk subgroups, which consistent with Pencina et al’s⁴² study wherein IDI demonstrates greater sensitivity to improvements within high-risk populations.

We thus demonstrate that the nomogram effectively integrates multiple variables to predict recurrence risk in patients with NVAF following RFCA. These findings underscore its potential for clinical application in recurrence risk assessment. Critically, our nomogram enables the consolidation of diverse parameters including clinical features, laboratory values, and echocardiographic data into a unified model, establishing a comprehensive evaluation system.

Therefore, our study demonstrates that the nomogram effectively integrates multivariable predictors to assess recurrence risk in NVAF patients post-RFCA. These findings highlight its clinical applicability for recurrence risk stratification. Crucially, the model synthesizes diverse parameters including clinical features, laboratory values, and echocardiographic data into a unified model, establishing a comprehensive risk stratification system.

Limitations

There are several limitations in this study. First, this study is a single-center retrospective investigation with a limited sample size, and there may be a risk of over fitting the performance boundary of the model. The clinical applicability of the nomogram requires further validation through multicenter prospective data, particularly for external validation across diverse populations. Second, the measurement of the SII could be affected by laboratory testing methodologies; establishing standardized detection protocols in the future is necessary to minimize this potential bias. Third, the three scoring systems (APPLE, ATLAS, and Antwerp scores) exhibit lower predictive efficacy, largely attributed to their relatively simplistic parameter selection. This limitation hinders their ability to adequately capture the complex pathophysiological mechanisms underlying AF recurrence. Future research should focus on optimizing model parameters, validating external performance, and exploring individualized treatment strategies.

Conclusions

In summary, the APPLE, ATLAS, and Antwerp scores all demonstrated effectiveness in predicting AF recurrence after RFCA in patients with NVAF. Among these established scoring systems, the APPLE score showed better performance compared to the other two. More importantly, our newly developed nomogram exhibited superior performance compared to all three existing scores, demonstrating a marked improvement in predicting the risk of AF recurrence. While our model represents a promising tool, it is still in the preliminary stage and requires further validation in larger, multi-center, prospective cohorts to confirm its generalizability.