Accuracy and Reproducibility of Coronary Angiography-Derived Fractional Flow Reserve in the Assessment of Coronary Lesion Severity

Introduction

Wire-based fractional flow reserve (FFR) is the gold standard for coronary revascularization in patients with stable ischemic heart disease,^1,2 and its favorable clinical outcomes have been demonstrated in several multicenter randomized trials.^3–5 However, the use of wire-based FFR is limited by its high cost, additional time to perform, and patient discomfort.^6,7 Over the past decade, four CFD-derived FFR techniques have been developed to avoid the need of using an invasive pressure wire: quantitative flow ratio (QFR),⁸ FFRangio,⁹ vessel FFR (vFFR)¹⁰ and coronary angiography-derived FFR (caFFR).¹¹ All these techniques provide a good diagnostic accuracy comparable to that of wire-based FFR. However, the cut-off FFR values for percutaneous revascularization are yet to be fully clarified. Based on data obtained during a 5-year follow-up period, the DEFER study³ set the FFR cut-off for percutaneous revascularization at <0.75, while the subsequent FAME family studies elevated the FFR cut-off to ≤0.80 to avoid undertreatment.^4,5 Moreover, the optimal treatment for coronary stenosis in patients with an FFR between 0.75 and 0.80 (the so-called grey zone)^12–14 remains controversial.

As a novel CFD-derived FFR, caFFR, combined with the FlashAngio software (Rainmed Ltd., Suzhou, China) for analysis and calculation of outcomes, has been shown to have good accuracy and agreement with wire-based FFR.^11,15,16 Nevertheless, no study has assessed the inter-observer reproducibility of caFFR. In addition, the diagnostic accuracy of caFFR for FFR values of coronary vessels in the grey zone has not been validated. Accordingly, the aim of this study was two-fold: first, to evaluate the reproducibility of caFFR between two independent laboratories; and second, to investigate the diagnostic ability of caFFR in detecting functionally significant coronary stenosis defined as FFR ≤0.80, especially in the grey zone (0.75≤caFFR ≤0.80).

Materials and Methods

Ethics Statement

This study was approved by the Institutional Review Board (2020BJYYEC-038-01) of Beijing Hospital, and the methods were consistent with the Declaration of Helsinki and Good Clinical Practice Guidelines of the China Food and Drug Administration. All patients provided informed consent for treatment and for the use of their data for research and publication.

Study Group

Eligible patients were those aged >18 years who underwent coronary angiography (CAG) and subsequent invasive FFR measurements between February 2015 and July 2021, at Beijing Hospital. The exclusion criteria were as follows: previous MI (STEMI), LVEF ≤40%, cardiomyopathies, previous PCI or CABG of interrogated coronary artery, valvular heart diseases, stenosis caused by a myocardial bridge, ostial lesions, poor contrast opacification, severe vascular overlap or distortion of the interrogated vessel, poor angiographic image quality precluding contour detection. For patients with stable angina and silent ischemia, we performed coronary angiography due to some positive non-invasive examinations: coronary CTA, treadmill exercise test, myocardial perfusion tomographic imaging or active request of patients. A caFFR value ≤0.80 in the target vessel (compared to the value for invasive FFR ≤0.80 as a reference standard) was considered significant stenosis. Among the 182 coronary vessels (obtained from the 165 eligible patients), 167 (91.8%, 150 patients) were included in the final analysis. Fifteen eligible patients were excluded for the following reasons: lack of qualified angiographic position and separation by <30° (n = 5); target vessels involving myocardial bridges (n = 4); severe vascular overlap (n = 4); and poor contrast filling on angiography (n = 2).

Coronary Angiography

CAG was performed according to the Judkins method from nine conventional projection views, using a 6-F angiography catheter (via a femoral or transradial approach). Angiographic images were recorded using a Philips Allura Xper FD10 system (Philips Healthcare; 15 frames/s). The contrast agent was injected using a mechanical pump at a rate of 3.5 mL/s. All angiography examinations were performed by skilled and experienced interventional cardiologists.

Wire-Derived FFR Measurements

Coronary FFR measurements were performed using a pressure wire system (Certus, Abbott Vascular, Santa Clara, CA, USA). Intracoronary nitroglycerine (200 μg) was routinely injected before FFR. After calibrating the system, a 0.014-inch pressure wire was positioned using a ≥6-F guide catheter, 1–2 cm downstream from the edge of the target lesion. The location of the pressure wire was recorded on images as a standard procedure. Coronary hyperemia was then induced by administering adenosine triphosphate via the antecubital vein at a rate of 140 µg/kg/min for 150 s. FFR was determined by the ratio Pd/Pa, where Pd represents mean hyperemic coronary pressure distal to the lesions measured by the pressure wire, and Pa represents mean aortic pressure measured by the guiding catheter.¹⁷ Hemodynamic data were recorded for at least 10s in the presence of a stable FFR value. The accepted range of pressure drift was between 0.97 and 1.03, measured when the pressure wire was pulled back near the tip of the guiding catheter.

Offline caFFR Measurement

At least two Digital Imaging and Communications in Medicine angiograms were transferred to the FlashAngio system to obtain a three-dimensional (3-D) reconstruction of the coronary arteries. The operator imputed the mean aortic pressure (MAP) and selected the appropriate cine images of the target vessel. The location of the caFFR was consistent with the position of the FFR pressure wire. Resting flow velocities (V) obtained by the TIMI Frame Count method and MAP (Pa ′) from the FlashAngio software were calculated by an optimized CFD method to compute a pressure drop (∆P) along the generated mesh of the coronary artery as caFFR = (Pa ′−∆P)/ Pa ′.¹¹ At the two laboratories, CoreLab1 (Rainmed Ltd., Suzhou, China) and CoreLab2 (Beijing Hospital), caFFR was measured according to standard operation procedures by two highly trained technicians, one in each laboratory. All caFFR measurements were performed offline in a blinded fashion, using the FlashAngio caFFR system.

Statistical Analysis

Clinical and angiographic data were extracted from the electronic medical record system and cardiac catheterization laboratory at Beijing Hospital. Continuous data are expressed as mean and standard deviation. Categorical variables are expressed as count and percentage. The Student’s t-test or chi-squared test was used to evaluate between-group differences. Pearson’s correlation coefficient between FFR and caFFR was calculated, with Bland-Altman plots used to evaluate agreement between FFR and caFFR. The validity of the caFFR for predicting functionally significant stenosis defined as FFR ≤0.80 was evaluated using sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV). The intraclass correlation coefficient (ICC), Pearson’s correlation coefficient and Bland-Altman plots were used to assess the correlation and agreement of caFFR between the two laboratories, CoreLab1 and CoreLab2. The chi-squared test was used to compare the diagnostic performance characteristics of caFFR in two groups. A p-value <0.05 was considered significant for all analyses.

All statistical analyses were performed using SPSS software (version 24.0; IBM SPSS, Armonk, New, USA) and GraphPad Prism software version 8.0.3 (GraphPad Software Inc., San Diego, CA, USA).

Results

Feasibility and Clinical Characteristics of the Patients

After exclusion, our study group included 150 patients (167 vessels), 67% of whom were men. The mean age of our study sample was 63.3 years. Stable angina was the diagnosis in 52% of patients, and 39% of patients had diabetes mellitus. Target vessels were predominantly located in the left anterior descending artery (76%). Detailed clinical characteristics of the patients are presented in Table 1. The angiography characteristics of the target lesions are reported in Table 2.

Table 1 Baseline Demographic and Clinical Parameters per Patient (n = 150)

Table 2 Angiographic and Lesions Characteristics per Vessel (n = 167)

Inter-Observer Agreement

There was good correlation and agreement between measured caFFR outcomes between the two labs (r = 0.723; 95% confidence interval [CI]: 0.645–0.787; p<0.001), with an ICC value of 0.724 (95% CI: 0.642–0.789; p<0.001). On Bland-Altman analysis of caFFR measurements, there were no systematic differences between CoreLab1 and CoreLab2 (mean difference, 0.0; 95% lower and upper level agreement, –0.12 to 0.13; Figure 1). The sensitivity, specificity, PPV, NPV and overall diagnostic accuracy of caFFR-CoreLab1 were as follows: 92%; 91%; 94%; 89% and 91%, respectively. The diagnostic value of caFFR-CoreLab2 for hemodynamically significant stenosis was also good, with a sensitivity, specificity, PPV, NPV and overall diagnostic accuracy of 83%, 82%, 87%, 77% and 83% (all p > 0.05; Figure 2), respectively. The AUC of ROC for caFFR- CoreLab1 and caFFR-CoreLab2 were both high, 0.975 (95% CI: 0.953–0.997, p < 0.001) and 0.883 (95% CI: 0.831–0.935, p < 0.001). C statistic analysis for patients with FFR≤0.8 showed the difference between the AUC of caFFR-CorLab1 and caFFR-CorLab2 was 0.092 (95% CI: 0.043–0.141, p < 0.05).

Figure 1 Bland-Altman analysis of caFFR by CoreLab1 and CoreLab2 (purple band means 95% CI of mean difference of caFFR between CoreLab1 and CoreLab2, green band means 95% CI of upper limit of agreement, pink band means 95% CI of lower limit of agreement).

Figure 2 Diagnostic characteristics of caFFR between CoreLab1 and CoreLab2.

Diagnostic Accuracy of caFFR

In all study patients, the mean of FFR and caFFR were both 0.79±0.09 (p=0.202; Figure 3). There was a strong correlation between caFFR and FFR (r=0.911; p<0.001). On Bland-Altman analysis, there was no systematic difference between caFFR and FFR on per-vessel basis (mean difference, 0.0; 95% lower and upper level agreement, –0.009 to 0.002; Figure 4). The AUC of ROC for caFFR was 0.975 (95% CI: 0.953–0.997; p< 0.001). The sensitivity, specificity, PPV, NPV and overall diagnostic accuracy of caFFR in the prediction of FFR ≤0.80 were as follows: 92%; 91%; 94%; 89% and 91%, respectively. At the per-vessel level, there were 56 vessels in the grey zone and 111 vessels in the non-grey zone, with no significant difference in the clinical characteristics between the patients in these two groups (Table 1). Using an FFR cut-off value of ≤0.80 as reference, caFFR correctly identified 42 vessels with an FFR <0.75, 48 vessels with an FFR between 0.75 and 0.80, and 62 vessels with an FFR >0.80. Incorrect identification by caFFR included three vessels with an FFR <0.75, eight with an FFR between 0.75 and 0.80, and four with an FFR >0.80 (Figure 5).

Figure 3 Mean of FFR and caFFR (n = 167).

Figure 4 Bland-Altman analysis of caFFR and FFR in all patients (purple band means 95% CI of mean difference of caFFR and FFR in all patients, green band means 95% CI of upper limit of agreement, pink band means 95% CI of lower limit of agreement).

Figure 5 Diagnostic performance of caFFR for grey-zone lesions. The diagnostic accuracy of the caFFR in correctly identifying hemodynamically significant lesions among the different FFR groups is observed, with no difference between the grey zone group and the other two groups (FFR <0.75 and FFR >0.80).

In patients in the grey zone, on Bland-Altman analysis, there was no systematic difference between caFFR and FFR (mean difference, 0.0; 95% lower and upper level agreement, –0.07 to 0.05; Figure 6). There was a medium correlation between caFFR and FFR (r=0.329; p<0.0014) in the prediction of 0.75≤ FFR ≤0.80 and a strong correlation between caFFR and FFR (r=0.774; p<0.001) in the prediction of FFR ≤0.80. The sensitivity, specificity, PPV, NPV and overall diagnostic accuracy of caFFR in the prediction of 0.75≤ FFR ≤0.80 were as follows: 60%, 85%, 66%, 81%, 66%, and 100%, 90%, 90%, 100%, 90%, in the prediction of FFR ≤0.80.

Figure 6 Bland-Altman analysis of caFFR and FFR in patients in grey zone (purple band means 95% CI of mean difference of caFFR and FFR in grey zone, green band means 95% CI of upper limit of agreement, pink band means 95% CI of lower limit of agreement).

In patients in the non-grey zone, there was a strong correlation between caFFR and FFR (r=0.931; p<0.001). On Bland-Altman analysis, there was no systematic difference between caFFR and FFR (mean difference, 0.0; 95% lower and upper level agreement, –0.008 to 0.006; Figure 7). The AUC of ROC for caFFR was 0.927 (95% CI: 0.888–0.967; p< 0.001). The sensitivity, specificity, PPV, NPV and overall diagnostic accuracy of caFFR in the prediction of FFR ≤0.80 were as follows: 98%; 85%; 89%; 98% and 94%, respectively. There was no difference in the diagnostic accuracy of the caFFR between the grey and non-grey zones (chi-squared, 2.899; p=0.09).

Figure 7 Bland-Altman analysis of caFFR and FFR in patients in non-grey zone (purple band means 95% CI of mean difference of caFFR and FFR in non-grey zone, green band means 95% CI of upper limit of agreement, pink band means 95% CI of lower limit of agreement).

Discussion

The two main findings of our study are as follows: first, inter-observer reliability supports caFFR as a feasible and reproducible method, which can be applied to most conventional CAG images. Second, the diagnostic value of caFFR was lower in the grey zone compared to the non-grey zone.

The methodology of CFD simulates the actual pressure distribution in the coronary artery by calculating specific fluid control equations.¹⁸ CFD-derived FFR is based on computed tomography (CT) or transcatheter coronary angiograms and does not require pressure wires or the use of vasodilators. These advantages could increase the application of FFR for the diagnosis of coronary stenosis. Inter-observer agreement of CFD-derived FFR values is essential for the generalizability of the technique in practice. The reproducibility of QFR is reportedly high, with a mean inter-observer error of 0 ± 0.03.¹⁹ We identified similar favorable reproducibility of caFFR, with an inter-observer error of 0 ± 0.06. Inter-observer agreement was likely increased by our use of rigorous standard operating procedures for coronary image acquisition, strictly controlled angiography quality, and the elaborate protocol for caFFR analysis used.

CFD-derived FFR software can automatically create a 3-D reconstruction of the coronary artery tree, with manual adjustment of vessel contours required at times. The quality of the image can influence the precision of the examination and thus, the interpretation of the severity of coronary stenosis and the treatment selected.²⁰ The diagnostic value of CT angiography-derived FFR (FFR_CT) has been previously demonstrated;^21–24 however, it is rather low, ranging from 0.70 to 0.80.²⁵ Moreover, angiography-derived FFR showed a favorable diagnostic performance similar to wire-based FFR, when using an FFR of 0.80 as the cut-off value, the diagnostic accuracy of caFFR was 95.7%,¹¹ similar to those of QFR,⁸ FFRangio⁹ and vFFR.¹⁰ Our study further confirmed the “good” diagnostic performance of caFFR using the FlashAngio system for offline analysis of caFFR and calculation of outcomes in all study patients (r 0.911, AUC 0.975, diagnostic accuracy 91%).

The cut-off FFR value for revascularization treatment of coronary stenosis varies in clinical literature. While an FFR value <0.75 has been used to predict myocardial ischemia,²⁶ the DEFER trial³ demonstrated the safety of deferring revascularization therapy for patients with stable coronary artery disease when the FFR is >0.75. Although an FFR >0.80 excludes 90% of the cases of stable coronary artery disease,²⁷ some patients with an FFR of 0.75–0.80 present with typical exercise-induced angina pectoris.²⁸ Accordingly, the FAME⁴ and FAME-2⁵ trials adopted an FFR value of 0.80 to limit the number of ischemic lesions left untreated and demonstrated that the FFR-guided percutaneous coronary artery intervention improved the clinical outcomes of these patients. However, several clinical trials have shown that revascularization does not result in better clinical outcomes for coronary stenosis in patients with an FFR in the grey zone.^12–14 In the grey zone, it is essential to maintain a high diagnostic accuracy of FFR derivative indicators to generalize their application to practice to avoid ambiguous decisions regarding treatment. A diagnostic accuracy of 71.3% has been reported for QFR²⁹ and 85.5% for FFRangio.³⁰ In our study, we reported a diagnostic accuracy of caFFR for FFR of 0.75–0.80; although this rate of accuracy was somewhat lower (66%) than previously reported rates, the diagnostic accuracy for FFR ≤0.80 usually used as the standard of ischemia was high (90%). We observed that there was no difference in accuracy between the grey and non-grey zone groups in the prediction of FFR ≤0.80 (90% vs 94%, respectively; p=0.09).

The limitations of our study need to be acknowledged in the interpretation of our findings in clinical practice. First, it was a retrospective, single-center study with a small sample size; this limits the generalizability of our findings. Second, the accuracy of angiography-derived FFR in complex lesions, including bifurcation, diffuse or in-stent restenosis vessels remains to be verified in a large cohort study. Third, all measurements of caFFR were performed by highly trained professional technicians; the reproducibility of caFFR should be verified when calculated by different technicians in real-world clinical practice. Finally, although QFR-guided coronary revascularization can improve clinical outcomes at 1 year compared to standard angiography guided revascularization,³¹ our study did not consider the long-term effect of caFFR on lesion selection for percutaneous coronary intervention. This will be an important clinical issue to consider in future studies.

Conclusion

In summary, our findings indicate that caFFR is a feasible and reproducible examination for coronary ischemia (including for lesions in the grey zone) and has good correlation with findings on invasive FFR measurements for detecting coronary ischemia. Future studies are warranted to provide further evidence to support the generalizability of caFFR in practice.