Validation Study of Rome Criteria for Assessing COPD Exacerbation Severity and Predicting Clinical Outcomes: Turkish Thoracic Society COPD Assembly

Introduction

Acute exacerbations are the most important cause of morbidity, mortality and disease-related costs in COPD.¹ Prevention of acute exacerbation of COPD (AE-COPD) is the main treatment goal in the disease. Another important goal is the correct and effective management of exacerbation when it develops. The main requirement to achieve this goal is the definition of objective criteria for the definition and diagnosis of exacerbation. To date, the most important controversy about AE-COPD has been that its definition and diagnosis are nonobjective. There was an approach based on the patient’s perception of symptoms rather than specific laboratory or disease parameters. This has been caused by approaching AE-COPD with a clouded mind. However, objectivity in chronic disease management is essential for physicians, patients, and other components of healthcare, such as costs and social support. Just as the prognostic indices such as CURB-65² or PSI,³ whose effectiveness has been proven in patients with pneumonia, there was a need for a similar index for the management of COPD exacerbations.

Although conceptual advances in the definition of COPD have been recorded in recent years in the GOLD report, deficiencies in the definition and diagnosis of exacerbations were noteworthy.

Due to the lack of objective criteria for the diagnosis of exacerbation, there were different approaches in different parts of the world, even in different centers in the same country. However, in 2021, the use of the Rome classification⁴ was recommended by the authorities on COPD and was included in the GOLD 2024.¹ This new classification proposes the use of six objective criteria. These criteria are dyspnoea (assessed by a visual analogue scale (VAS)), arterial oxygen saturation (SaO₂), respiratory rate (RR), heart rate (HR), serum C-reactive protein (CRP) and, in selected cases, arterial blood gas (ABG). However, as this classification system is based on the Delphi methodology, there is a need to validate the predictive performance under real-world conditions. Validation studies have been conducted in China,⁵ South Korea⁶ and Europe⁷ but validation studies in other countries are needed for global acceptance.

This study aims to evaluate the validity of the Rome criteria in determining the severity and prognosis of AE-COPD in Turkey.

Materials and Method

This multicenter and observational study was designed in accordance with the Declaration of Helsinki. Ethics committee approval was received from Cukurova University Faculty of Medicine (ethic approval number 143/542,024). As the study was a retrospective file review, an informed patient consent form was not required and patients’ identity details were concealed.

Study Design

The study included 750 patients from 15 centers who visited the emergency department (ED) or outpatient clinic (OC) with diagnosis of AE-COPD. A 1-year retrospective file scan was conducted between April-June 2024.

Inclusion criteria were as follows: (I) over 40 years of age, (II) the ratio forced expiratory volume in 1 second (FEV1) to forced vital capacity (FVC) <0.7 in post-bronchodilator spirometry in the last 1 year, (III) acute worsening dyspnea related to COPD, in the last 14 days (visual analogue scale (VAS) that using a 0–10 scale checked by initial nursing assessment in ED or OC), (IV) available information on 6 parameters of ROME criteria to assess the severity of AE-COPD such as respiratory rate (RR), heart rate (HR), pH, arterial partial pressure of carbon dioxide (PaCO2), arterial oxygen saturation (SaO2) at rest and serum C-reactive protein (CRP). Exclusion criteria were as follows: (I) the patient being hospitalized for a reason other than COPD, (III) the patient has only pneumonia without AE-COPD.

Demographic characteristics of the patients (age, gender, body mass index (BMI), smoking history, Charlson Comorbidity Index (CCI), comorbidities, current medical treatment), respiratory symptoms, mMRC score, vital signs, radiological and laboratory parameters, spirometry and blood gas parameters, electrocardiography (ECG) and echocardiography (ECHO) findings and clinical outcome parameters (need for intensive care unit (ICU), noninvasive ventilation (NIV), invasive mechanical ventilation (IMV), length of hospital stay, in-hospital mortality, 30- and 90-days re-admission, post-discharged 30- and 90-days mortality) were recorded.

In this multicentre study, a standard Excel template containing variable names, definitions, and value labels was sent to all centres prior to data collection, along with a compatible code book. Standardised coding was used to ensure consistency between centres. Data were de-identified and each observation was marked with unique identifiers at both the centre and patient levels. Although a standard query protocol was not implemented for deficiencies or inconsistencies identified during data cleaning, written/electronic confirmation was obtained by contacting the relevant centres on a case-by-case basis, and the necessary corrections were made. Missing data were handled in the analyses using listwise deletion; no imputation was performed.

Groups According to Rome Classification

Acute exacerbation was categorized into 3 severity groups according to Rome criteria;⁴ mild, moderate and severe AE-COPD groups.

Mild exacerbation criteria were visual analog scale (VAS) < 5, respiratory rate (RR) < 24/min, heart rate (HR) < 95/min, resting oxygen saturation (SO2) ≥ 92% (or known SO2 variability < 3%) on room air or own long term oxygen therapy (LTOT), and C-reactive protein (CRP) < 10 mg/L.

Moderate exacerbation was defined as at least 3 of the following 5 criteria; VAS ≥ 5, SS ≥ 24/min, HR ≥ 95/min, resting SO2 < 92% (or known SO2 variability greater than 3%) on room air or own LTOT, CRP ≥ 10 mg/L or arterial blood gases showed hypoxemia (paO2 ≤ 60 mmHg) and/or hypercapnia (PaCO2 ≥ 45 mmHg) but no acidosis (pH > 7.35) if available.

Severe exacerbation was the presence of hypercapnia and acidosis in arterial blood gas (PaCO2 > 45 mmHg, pH < 7.35).

Endpoints

The primary endpoints were to evaluate the prognostic performance of Rome criteria in determining the severity of AE-COPD and predicting clinical outcomes (need for ICU), need for NIV/IMV). Secondary outcomes were to evaluate the relationship between the severity of AE-COPD according to the Rome classification and in-hospital mortality, 30- and 90-day mortality, and 30- and 90-day re-admission. The other secondary outcome was to evaluate the factors affecting the severity of AE-COPD.

Statistics

Descriptive statistics were presented as numbers (percentages), mean ± standard deviation (SD), and median (interquartile range (IQR)). The normality of the distribution was assessed using the Kolmogorov–Smirnov test and visual inspection with histograms and Q–Q plots. For comparisons between COPD exacerbation severity groups, ANOVA was used for continuous variables showing a normal distribution. Homogeneity of variance was assessed using the Levene test; when the assumption was not met, Welch ANOVA was preferred. Post-ANOVA pairwise comparisons were performed using Tukey HSD; when the assumption of equal variance was not met, the Games–Howell test was used. For non-normally distributed continuous variables, the Kruskal–Wallis test was applied for multiple group comparisons; when significance was detected, pairwise comparisons were performed using the Mann–Whitney U-test. The chi-square test was used to compare categorical variables. Hospitalisation duration between COPD exacerbation severity groups (moderate-severe) was analysed using the Kaplan–Meier method, and the difference between groups was assessed using the Log rank test. Multivariate logistic regression analysis was performed to identify independent factors associated with COPD exacerbation severity. Variables were included in the model based on those with p < 0.10 in univariate analyses or those considered clinically meaningful. Multicollinearity was assessed using VIF (variables were removed/simplified if VIF >5). Potential interaction terms were controlled. Model fit was assessed using the Hosmer–Lemeshow test, and explanatory power was assessed using Nagelkerke R². Results were presented as odds ratios (OR) and 95% confidence intervals. All analyses were performed using SPSS version 15.0 (IBM, New York, USA), and p < 0.05 was considered the threshold for statistical significance.

Results

Of the 750 patients, 99 (13.2%) were in the mild, 479 (63.9%) were in the moderate, and 172 (22.9%) were in the severe group. Demographic and clinical characteristics of the groups are shown in Table 1. In addition, pulmonary and extrapulmonary comorbidities of the groups of AE-COPD are shown in Supplementary Table-1. The severe group was older (p: 0.001), had more comorbidities according to the CCI (p: 0.007), had a higher rate of active smoking history (p < 0.001), had the lowest FEV1 (p < 0.001) and FEV1/FVC (p: 0.012) (Table 1). The gender, BMI (Table 1), and distribution of comorbidities (Supplementary Table-1) were similar in 3 groups. The rate of inhaler drug use was similar among the groups (p: 0.357) (Supplementary Table-2). However, the frequency of long-acting inhalation therapy (LAIT) use was the lowest and the frequency of short-acting inhaler therapy use was the highest in the severe group (respectively p: 0.007, p: 0.005). Hence, LAIT compatibility was poor in the severe group.

Table 1 Demographic Characteristics of the Groups

The use of only long acting beta agonist (LABA) (p:0.131), only long acting antimuscarinic (LAMA) (p: 0.276), LABA+LAMA (0.82) and inhaler corticosteroid (ICS)+LABA+LAMA (0.773) were similar between the groups (Supplementary Table-2). ICS+LABA use was lowest in the severe group (18.6%) and highest in the moderate group (29.9%) (p: 0.014). Looking at the current guideline recommendations, this was an unexpected situation. Most of the severe group used nebulizers (78%) and continue positive airway pressure (CPAP)/bilevel positive airway pressure (BPAP) (41.9%) at home (respectively p < 0.001, p < 0.001). LTOT use was lowest in the severe group (33.1%) and highest in the moderate group (56%) (p < 0.001) (Supplementary Table-2). Regarding non-inhaler medication use, diuretic (33.1%) and anticoagulant (22.1%) use was higher in the severe group compared to the other groups (respectively p: 0.030, p: 0.024). There was no difference between the groups in the frequency of influenza, pneumococcal and COVID vaccination (respectively p: 0.166, p: 0.202, p: 0.296) (Supplementary Table-2). Considering all these characteristics, it is evident that the severe group possesses a statistically significant difference in terms of functional status and medication.

As shown in Table 1, the number of ED admissions in the last 1 year (p: 0.001) and ward admissions in the last 1 year (p < 0.001) was higher in the moderate and severe groups. The number of ICU admission in the last 1 year was significantly higher in the severe group (p < 0.001).

Table 2 shows the distribution of the parameters used for ROME classification according to the groups. As the severity of exacerbation increased VAS (p < 0.001), RR (p < 0.001), HR (p < 0.001) and PaCO2 (p < 0.001) increased, while SO2 (p < 0.001) and pH (p < 0.001) decreased. However, PaO2 (p: 0.062) was not different between moderate and severe groups.

Table 2 Rome Classification Parameters for Determining AE-COPD Severity

When ECG findings were analyzed, 91% of the mild group, 70% of the moderate group and 63.4% of the severe group were in normal sinus rhythm (p < 0.001) (Supplementary Table-3). At the time of visit to the ED or OC, 13.1% of the mild group, 33.2% of the moderate group and 36.6% of the severe group had sinus tachycardia (p < 0.001). Atrial fibrillation was detected in 11.6% of the severe group, 10.2% of the moderate group and 3% of the mild group, statistical significance was borderline (p: 0.051) (Supplementary Table-3).

When ECHO findings were analyzed, EF (p: 0.098) and sPAP levels (p:0.334) of the groups were similar. Tricuspid Insufficiency was present in 37.4% of the mild group, 48.4% of the moderate group and 40.7% of the severe group (p: 0.013) (Supplementary Table-3). In chest X-ray examination of the current visit, the frequency of pleural effusion (22.1%) and pneumonia (9.8%) was highest in the severe group (respectively p < 0.001, p < 0.001) (Supplementary Table-3).

In laboratory parameters, the severe group had the highest CRP (48.8 ± 34.3, p < 0.001), the lowest glomerular filtration rate (77.7 ± 25.9, p: 0.032), and the lowest eosinophil count (111.6 ± 148.3, p: 0.008) (Supplementary Table-4).

Considering all these characteristics, as the severity of the ROMA classification increases, age, the degree of functional impairment, the need for non-pharmacological treatment, the frequency of emergency department visits and hospitalisations, the frequency of abnormal vital signs, and the frequency of abnormal radiological findings also increase.

Primary Outcomes

As shown in Table 3, discharge from ED or OC at the current visit was highest in the mild group (73.7%), while 10.2% in the moderate group and 1.2% in the severe group (p < 0.001). At the current visit, 26.3% of the mild group, 89.8% of the moderate group and 98.3% of the severe group had been hospitalized (p < 0.001). Most of the moderate group (87.1%) were admitted to the ward and most of the severe group (62.6%) to the ICU (p < 0.001, p < 0.001, respectively). NIV and IMV administration was mostly in the severe group (p < 0.001, p < 0.001 respectively). According to these findings, it has been determined that as the severity of ROMA increases, the need for intensive care and ventilation support also increases.

Table 3 Distribution of Primary Outcomes by COPD Severity Group

Secondary Outcomes

Table 4 shows secondary endpoint results. Accordingly, as ROMA severity increased, in-hospital mortality (p < 0.001) and 30-day mortality after discharge (p = 0.015) increased, while 90-day mortality and 30- and 90-day readmission rates remained unchanged (p = 0.258, p = 0.712, and p = 0.681, respectively).

Table 4 Distribution of Secondary Outcomes by COPD Severity Group

Primary and secondary outcomes of AE-COPD groups according to Rome classification are shown in Figure 1. Survival analysis revealed no difference between the moderate and severe exacerbation groups (Figure 2), with a survival time of 36.5 days in the severe exacerbation group and 34.3 days in the moderate exacerbation group (p = 0.764).

Figure 1 Primary and secondary outcomes between groups of AE-COPD according to Rome classification. Abbreviations: ICU, intensive care unit; NIV, non-invasive ventilation; IMV, invasive mechanical ventilation. Notes: Discharged, patients discharged home from the emergency department or outpatient clinic at the current visit; Hospitalized, patients admitted to hospital (ward or intensive care unit) from the emergency department or outpatient clinic at the current visit; 30-days mortality, post-discharged 30 days mortality rate; 90-days mortality, post-discharged 30 days mortality rate; 30-days readmission, post-discharged 30 days readmission rate; 90-days readmission, post-discharged 90 days readmission rate.

Figure 2 Survival analysis in the moderate and severe groups.

Logistic regression analysis (Table 5) examining factors influencing exacerbation severity identified the following as significant: the number of emergency visits in the past year (p = 0.004, OR = 0.70), ICU admission in the past year (p = 0.001, OR = 1.33), low FEV1 (p < 0.001, OR = 1.00), LTOT (p = 0.019, OR = 1.13) and CPAP/BPAP at home (p = 0.004, OR = 1.39).

Table 5 Factors Affecting the Severity of Exacerbation (Logistic Regression Analysis)

Discussion

In our country, in data from 15 centers, we found that the Rome classification was sensitive to discharge from ED/OC, hospitalization, ICU risk, NIV/IMV risk, inhospital mortality and post-discharged 30-days mortality in AE-COPD.

When we examined the previous studies on Rome validation, we obtained similar results in terms of primary endpoints.^5–7 In the study of Zeng et al involving 7712 patients, Rome classification was found very successful in determining the risk of ICU admission, NIV and IMV.⁵ The sensitivity of the Rome classification was also proven in a study of 740 cases by Lee et al.⁶ In the Turkish population, we found that the Rome classification was not only successful in determining the risk of ICU admission, NIV/IMV, but also in the decision to discharge from the ED or OC.

Zeng et al found no difference between the moderate and severe AE-COPD groups on inhospital mortality, but inhospital mortality was higher in these groups than in the mild AE-COPD group.⁵ Reumkens et al also reported that inhospital mortality was higher in the moderate and severe groups than the mild group.⁷ In our study, similar to Lee’s study,⁶ there was no inhospital mortality in the mild AE-COPD group, while inhospital mortality was higher in the severe group than in the moderate group. In Zeng’s study,⁵ the post-discharged 60-day mortality risk was higher in the moderate and severe AE-COPD groups. Reumkens reported that 30- and 90-day mortality was highest in the severe AE-COPD group.⁷ In our study, 30-day mortality was higher in the severe group, similar to Reumkens.⁷ 90-day post-discharged mortality was also higher in the severe AE-COPD group but there was no statistical significance. In addition, time to mortality was similar between the groups.

In the COPD Audit study, factors associated with inhospital mortality in AE-COPD were CCI, need for NIV/IMV and respiratory acidosis.⁸ Of these, respiratory acidosis and ventilatory support correspond to the severe group in the Rome classification. Hence, it can be said that ROMA classification is a valuable predictor for inhospital mortality. However, in our opinion that one of the shortcomings of the Rome classification is that comorbidities are not included in this classification. Whereas, similar to the COPD Audit,⁸ other studies have found that comorbid conditions contribute to inhospital mortality in AE-COPD.^9,10

As opposed to all other studies, we also investigated readmission rates. Although 30- and 90-day readmission rates were higher in the severe AE-COPD group, there was no statistically significant difference. However, the severity of the disease is not the single parameter that influences readmission, there can be effect some social, economic or diseases related factors. Risk factors for 30-day readmission in the RACE study¹¹ included elevated Hospital Anxiety Depression Scale (HADs), hospital-acquired pneumonia, and frequent antibiotic use in the previous year. Risk factors for 90-day readmission in the same study were grade 3 to 4 tricuspid yetmezlik, 2 or more moderate COPD exacerbation in the last year, severe exacerbation history in the last year, immunosuppression, frequent antibiotic use in the last year and hospital admission from ED.¹¹ It is difficult to assess the impact of disease on readmission because readmission may vary according to the social structure, ethnic characteristics and income level of the country. However, prospective studies on readmission may reveal the impact of the severity of exacerbation based on Rome classification.

The fact that the parameters used in the ROME classification are easily accessible in our country facilitates the use of the classification in daily practice. In our study, as in other studies,^5–7 the lowest PaO2 was in the severe group and the highest level was in the mild group. However there was no statistical differences. In the study by Reumkens et al. PaO2 was found highest in the mild and severe group and lowest in the moderate group.⁷ Lee et al obtained a similar result, the mean PaO2 of the middle group was lower than the other groups.⁶ In Zeng’s study, the best oxygen levels were found in the middle group.⁵ The clustering of hypoxic patients in the middle group in two studies and the clustering of the best oxygenation in the middle group in another study creates a contradiction. This suggests that there is a need for further studies on the subject.

In our study, all other parameters except PaO2 in the Rome criteria were statistically significant between the groups. Before the Rome classification, respiratory failure parameters were used to determine the severity of AE-COPD.¹² In the ROME criteria, CRP is added.⁴ CRP, an easily accessible, cost-effective and potent acute phase protein, has long been used to assess the need for antibiotics in AE-COPD.^13,14 In Zeng’s study,⁵ CRP elevation was at the lowest level in the moderate group, whereas in Lee⁶ and Reumkens’⁷ study, CRP was the highest in the moderate group. In our study, the severe group had the highest CRP levels. There is inconsistency between the studies and this is one of the issues that needs to be investigated more.

In our study, besides CRP, other infection parameters such as WBC and neutrophils were highest in the severe group. This finding suggests infections as the primary etiology of severe exacerbations. Another remarkable and statistically significant laboratory parameter was eosinophils. In our study, the eosinophil count was lowest in the severe group and highest in the mild group. Lee’s study has a similar result to ours, although not statistically significant,⁶ but Zeng’s study is similar to ours. A recent meta-analysis showed that high blood eosinophils (above 300 cells/μL or 2%) may predict the risk of moderate-to-severe exacerbations of COPD in certain subgroups,¹⁵ however, the studies included in the meta-analysis were retrospective and the need for well-designed prospective studies was emphasized. The results of three studies on ROME validation, including ours,^5,6 are inconsistent with this meta-analysis. It is an issue that needs to be investigated.

The comorbidity results of our study were similar to Lee’s study.⁶ There was no significant difference between the groups in terms of comorbidity. This finding is the opposite of the results in Zeng’s study.⁴ There may be several reasons why comorbid conditions may have different effects in studies. Comorbidities are based on self-report of the patients or medical records. Additionally, lifestyle, healthcare and socioeconomic differences in the countries where studies were conducted may affect comorbidities. All comorbidities may not have been included in the studies to date. Most studies focus on cardiovascular and metabolic comorbidities. However, many COPD patients may have osteoporosis or other comorbid conditions that have not yet been identified. CCI is commonly used in COPD studies.¹⁶ It is an appropriate clinical tool for measuring comorbidities and assessing their impact on patient survival.¹⁶ The CCI is a validated method that can predict 1-year mortality risk and disease burden. It is reliable in different clinical groups and settings and can therefore be used in clinical research and practice to determine the impact of comorbidities on clinical outcomes.^16,17 In our study, comorbidities alone had no effect on the severity of AE-COPD, but the CCI was highest in the severe group and this was statistically significant. However, in Lee and Reumkens’ study, CCI was similar between groups.^6,7 This difference between the studies may be due to the reasons mentioned above. However, there are studies showing the predictive effect of high CCI in severe COPD exacerbation.^18,19 Comorbidities are not included in the ROME criteria, as they were previously included in GOLD’s AE-COPD severity system. However, the importance of comorbidities in COPD cannot be denied for both stable and exacerbation periods. It seems like important to work on improving the ROME criteria in this direction.

Another distinctive finding of our study was that most of the severe group used only short-acting bronchodilators, whereas most of the mild group used only long-acting inhalers. Thus, we once again proved the positive effect of long-acting inhalation on exacerbation severity in COPD. We did not find any other study investigating the effect of short-acting bronchodilator on the severity of AE-COPD in the ROME validation studies. Our study is important as it is the first in this field. One of the noteworthy issues for our country that needs to be addressed was that more than half of the patients in all groups had nebulizers. There is a need for research on the high number of nebulizer prescriptions, the reasons for this, and the expectations of patients. The data to be obtained will be important in terms of contributing to our health system.

Our study has some limitations. The first is that it was retrospective and the second is that it was short-term. The reasons for these limitations were the urgent need for Rome validation in our country and the possibility of obtaining the fastest results via this method. Another limitation was that the VAS was estimated from nurse observation or from the dyspnea severity description in the epicrisis. However, similar method was used in other Rome validation studies. There may be bias in the mild group because of the estimated assessment for VAS.

The first strength of our study is that it is the first study on Rome validation in our country and the fourth study globally. Another important feature is that it represents almost the whole country since data is collected from 15 different centers. It is also the first article to investigate the effect of ROME classification on 30- and 90-day readmission. It is also important in terms of showing for the first time the effect of short-acting inhaler therapy on AE-COPD severity according to the ROME classification.

Conclusion

This validation study demonstrated that the Rome criteria perform well in predicting patient discharge from the ED and OC, ICU/ward admission, need for NIV/IMV, inhospital mortality and post-discharged 30-day mortality. The existence of objective criteria will provide a clearer framework for physicians to approach AE-COPD. The ROME classification has an important mission in using a common language among physicians in the management of AE-COPD. New, multicentre studies integrating comorbidities are needed to develop the Rome classification and optimise it.