Validation of the Rome Severity Classification of Chronic Obstructive Pulmonary Disease Exacerbation: A Multicenter Cohort Study

Introduction

Acute exacerbations of chronic obstructive pulmonary disease (AECOPD) is a leading cause of disease-associated morbidity and mortality among patients with chronic obstructive pulmonary disease (COPD).¹ AECOPD also accounts for impaired lung function and poor quality of life and is the largest component of the socioeconomic burden of COPD.^2–4 Early assessment of the severity of AECOPD may facilitate risk-stratified clinical management, including outpatient treatment or early supported discharge for patients with mild AECOPD and timely escalation or appropriate palliation for patients with severe AECOPD.⁵ Several prognostic scores that stratify patients hospitalized for AECOPD according to their risk of short-term mortality have been published, the most notable being BAP-65 and DECAF.^6,7 Most of the scores performed well in the derivation cohort, but the results of subsequent validation studies were controversial.^8,9 Thus, current international guidelines do not recommend the use of a prognostic score for predicting the risk of adverse outcomes among patients with AECOPD admitted to the hospital. The Global Initiative on Obstructive Lung Disease (GOLD) has been recommending a classification of AECOPD severity based on post facto medication use and hospitalizations,⁴ which cannot provide practical information for making clinical decisions.

In light of this, a group of international COPD experts recently proposed a new severity classification of AECOPD, called the Rome proposal, through a Delphi process based on a thorough literature review and discussion.¹⁰ In this new classification, six objectively measured variables are used to mark the event severity: dyspnea (assessed by a visual analog scale (VAS), which is on a scale of 0–10, arterial oxygen saturation (SaO2), respiratory rate (RR), heart rate (HR), serum C-reactive protein (CRP) and, in selected cases, arterial blood gases (ABG). Based on these variables, AECOPD is subsequently classified as mild, moderate or severe. However, since the severity classification of AECOPD by the Rome proposal is based on the Delphi methodology, its predictive performance needs to be validated in real-world settings. To date, a few studies applied this new severity classification in small and single-center cohorts with AECOPD,^11–13 but to our knowledge, it has not been validated in large multicenter cohorts or in Chinese populations with AECOPD.

The aim of this study is to assess the validity of the Rome severity classification in distinguishing the severity of AECOPD based on short-term mortality and other adverse outcomes, including ICU admission, MV and IMV, et al, through a large, real-world and multicenter cohort of patients hospitalized for AECOPD in China.

Materials and Methods

Ethical Considerations

Our study complies with the Declaration of Helsinki. And it was approved by the Ethics Committee on Biomedical Research, West China Hospital of Sichuan University, and the Ethics Committee of the other nine academic medical centers that participated. Written informed consent was obtained from all participants.

Study Design and Participants

We performed a secondary analysis based on the data collected from the prospective, multicenter and noninterventional cohort study, MAGNET AECOPD (MAnaGement aNd advErse ouTcomes in inpatients with acute exacerbation of COPD) Registry study (ChiCTR2100044625) in China. In the MAGNET study, adult inpatients diagnosed with AECOPD were consecutively enrolled between September 2017 and July 2021 in ten hospitals and followed-up by telephone, outpatient visits, or rehospitalization when necessary. The admission, arrangement of auxiliary examinations and treatment of patients were at the discretion of the attending physicians, and no additional direct intervention was performed. The inclusion criteria, as well as the diagnosis criteria of AECOPD include: (1) a history of COPD defined according to 3 items: 1) exposure to risk factors (eg, tobacco smoking, specific environmental exposure); 2) long-term dyspnea (progressive, on exertion or persistent), chronic cough, or sputum production; 3) post-bronchodilator spirometry testing performed (forced expiratory volume in 1-second/forced vital capacity ratio (FEV1/FVC) <70%); and (2) an acute worsening of respiratory symptoms resulting in additional therapy. Patients were excluded from the analysis if they met any of the following criteria: (1) age less than 40 years; (2) no available information on parameters to assess the severity according to the Rome classification, including HR, RR, CRP, SaO2 or the evidence of VAS scores.

Data Collection and Severity Classification

A standardized case report form including baseline demographics, comorbidities, symptoms, vital signs, laboratory tests, radiological findings, treatments and adverse outcomes was completed for every patient enrolled in the MAGNET AECOPD Registry study. RR, HR and other vital signs were taken and recorded within 2 hours after admission. Almost all blood test results were obtained within 24 hours of admission.

Based on the information gathered at admission, all AECOPD patients were categorized as mild, moderate, or severe (Table S1). First, severe AECOPD events are defined by arterial blood gas values indicating hypercapnia (PaCO2>45 mm Hg) and acidosis (pH< 7.35). If one’s arterial blood gas values could be obtained and show hypoxemia (PaO2<60 mmHg) and/or hypercapnia (PaCO2 >45 mmHg) but no acidosis (pH >7.35), the patient would be identified as moderate. Patients were also classified as moderate when they met at least 3 of the 5 variables: dyspnea VAS≥5, RR≥24 breaths/min, HR≥95 bpm, CRP≥10 mg/L or resting SaO2<92% when breathing ambient air or usual oxygen prescription. The other patients were directly categorized as mild. As VAS scores were not routinely obtained in clinical practice, we retrospectively determined VAS scores based on the medical records and nurse’s description of the severity of dyspnea on admission.

Study Outcomes

The primary outcome was defined as 60-day all-cause mortality after admission. Secondary outcomes involved in-hospital all-cause mortality, ICU admission, MV, IMV and length of stay (LOS). The usage of glucocorticoids and antibiotics was also included in the analysis, as they can reflect disease severity.

Statistical Analysis

Quantitative variables with normal distribution were denoted as the mean values with standard deviation (SD) and compared using ANOVA tests. Quantitative variables with skewed distribution were depicted as medians with interquartile ranges (IQRs) and were compared using the Mann–Whitney U-test. The Kolmogorov–Smirnov test was used to assess the normality of distributions. Qualitative variables (categorical variables) were displayed as absolute frequencies with percentages, and Pearson’s chi-squared test (Fisher’s exact test for frequencies <5) was used for group comparisons. Dunn’s test with a Bonferroni correction for multiple comparisons was applied. Univariate regression analysis reporting odds ratios (ORs) with 95% confidence intervals (95% CIs) was conducted to determine whether there was a relationship between the Rome severity classification and adverse outcomes. Time-to-event analyses were performed with Kaplan‒Meier curves to evaluate the cumulative risks of 60-day mortality among the mild, moderate, and severe groups. All statistical analyses were conducted using SPSS version 22.0 (IBM, New York, United States). All P values were two-tailed, and a P value <0.05 indicated a statistically significant difference.

Results

Study Population

A total of 14,007 patients were consecutively enrolled in the MAGNET AECOPD Registry study. Among them, 7712 were included in this analysis. The main reasons for exclusion were as follows: (1) age less than 40 years (n=27); (2) lacking HR record on admission (n=11); (3) lacking RR record on admission (n=28); (4) lacking CRP record (n=3805); (5) lacking SaO2 record on admission (n=1297); and (6) lacking evidence of VAS on admission (n=1127). The mean age of the population was 72.68±10.61 years, and 77.6% were male. Approximately 18.9% were active smokers. According to the Rome severity classification, 3230 (41.88%) AECOPD inpatients were categorized as mild, 3110 (40.33%) were categorized as moderate and 1372 (17.79%) were categorized as severe. The 60-day mortality and in-hospital mortality rates were 3.1% and 2%, respectively. A total of 785 (10.2%) patients were admitted to the ICU during the hospital stay, 2000 (26.9%) patients received MV, and 378 (4.9%) patients received IMV during their hospital stay. The flow chart of the study is shown in Figure 1.

Figure 1 Flow chart of the study. Abbreviations: AECOPD, acute exacerbation of chronic obstructive pulmonary disease; HR, heart rate; bpm, beats per minute; RR, respiration rate; CRP, C-reactive protein; SaO2, Arterial Oxygen Saturation; VAS, visual analog scale.

Baseline and Clinical Characteristics of Included AECOPD Patients

The baseline characteristics are summarized according to the severity of AECOPD based on the Rome classification in Table 1. Gender and current smoker distribution were comparable among groups. It is surprising that the patients who experienced a mild AECOPD event were older, while those who experienced a severe AECOPD event were slightly younger. Patients were more likely to experience reduced mobility, a history of exacerbation in the last year, or accept long-term home oxygen therapy if their severity gradings were classified more severe. A negative association was observed between body mass index (BMI) and severity of AECOPD, as the proportion of BMI≤17 kg/m2 in the mild group were less than those in the moderate or severe group. Additionally, the FEV1 pred% of the mild group was significantly higher compared to the moderate or severe group. Higher DECAF scores and BAP-65 scores were related to more severe severity according to the Rome classification. The incidence of most comorbidities, including coronary heart disease, heart failure, arrhythmia, stroke, bronchiectasis, interstitial lung disease (ILD), chronic pulmonary heart disease, active cancer, diabetes, chronic renal failure, anxiety or depression and osteoporosis, was higher in the mild or moderate group than in the severe group. The prevalence of hypertension, pulmonary tuberculosis, obstructive sleep apnea-hypopnea syndrome (OSAHS) and gastroesophageal reflux disease (GRED) was generally comparable among the groups.

Table 1 Baseline Characteristics of Patients According to AECOPD Severity Based on the Rome Severity Classification

Table 2 shows symptoms, vital signs and laboratory parameters for patients stratified according to the Rome classification. The moderate and severe groups were more likely to have sputum and lower diastolic blood pressure than the mild group. The complete blood counts demonstrated higher red blood cell (RBC), white blood cell (WBC) and neutrophil percentages in the moderate and severe groups compared with the mild group; in contrast, the moderate and severe groups had lower eosinophil percentages than the mild group. The levels of N-terminal pro-B-type natriuretic peptide (NT-pro-BNP) and troponin T (cTNT) were higher in the moderate and severe groups than in the mild group. Radiologic abnormalities, including consolidation and pleural effusion, were more commonly seen in patients classified as moderate.

Table 2 Clinical Features of Patients According to AECOPD Severity Based on the Rome Severity Classification

Variables Included in the Rome Severity Classification

As shown in Tables 3 and S2, patients in the moderate group had significantly faster RR (21 vs 20 breaths/min), HR (93 vs 82 bpm), higher CRP (17.1 vs 6.78 mg/L), PaCO2 (49.0 vs 38.0 mmHg), and proportion of dyspnea with VAS score >5 (33.2% vs 13.3%) as well as lower levels of SaO2 (95.7 vs 97.0%) and PaO2 (78.0 vs 84.7 mmHg) than those in the mild group (all P < 0.001). The value of PaCO2 and proportion of VAS score >5 indicated a significantly worse breathing condition in patients classified as severe compared with those classified as moderate (68.05 vs 49.0 mmHg, 100% vs 33.2%, respectively). However, the value of CRP (11.09 vs 17.1 mg/L) was unexpectedly lower and the level of PaO2 (85.4 vs 78.0 mmHg) was unexpectedly higher in the severe group than in the moderate group.

Table 3 Criteria for Determining AECOPD Severity According to the Rome Classification

Clinical Outcomes According to the Rome Severity Classification

The comparison of outcomes is shown in Table 4, and the multiple comparison is shown in Figure 2. The incidence of ICU admission (6.4% vs 12.0% vs 14.9%, P <0.001), mechanical ventilation (11.7% vs 33.7% vs 45.3%, P <0.001) and invasive mechanical ventilation (1.4% vs 6.8% vs 8.9%, P <0.001) increased with the severity of AECOPD from the mild group to the severe group according to the Rome severity classification. Although the moderate and severe groups had higher 60-day mortality than the mild group (3.5% vs 1.9%, 4.3% vs 1.9%, P <0.05, respectively), mortality in the severe group was very close to that in the moderate group (3.5% vs 4.3%, P >0.05). The results for in-hospital mortality showed the same trend, difference of in-hospital mortality between the moderate and severe groups was not significant (2.5% vs 2.6%, P >0.05) despite the moderate group and severe group had higher mortality than the mild group respectively (2.6% vs 1.1%, 2.5% vs 1.1%, P <0.05). The administration of systemic glucocorticoids increased with the severity of AECOPD from the mild group to the severe group (P<0.001), while the moderate group received antibiotics more often than the mild or severe group.

Table 4 Outcomes According to AECOPD Severity Based on the Rome Classification

Figure 2 Multiple comparison of adverse outcomes in inpatients with varied AECOPD severity according to the Rome classification (*P value<0.05). Abbreviation: ICU, intensive care unit.

The results of univariate logistic analysis on adverse outcomes in inpatients with AECOPD are shown in Table 5 and Table 6. Similarly, the risk of death at 60 days after admission was significantly higher in the moderate and severe groups than in the mild group (ORs: 2.38 vs 1, 1.92 vs 1.00, respectively, P=0.001), while there was no significant difference in the risk of 60-day mortality between the severe group and the moderate group (ORs: 0.81 vs 1.00, P=0.206). The increase in the Rome severity classification was significantly associated with an increased risk of ICU admission (mild vs moderate vs severe ORs: 1.00 vs 1.97 vs 2.55, P <0.001) and IMV (mild vs moderate vs severe ORs: 1.00 vs 5.15 vs 6.91, P <0.001). The Kaplan–Meier curves also demonstrated similar results (Figure 3). That is, the severe and moderate groups had significantly worse 60-day survival than the mild group (both P <0.05), but the survival was not significantly different between the severe group and the moderate group (P >0.05).

Table 5 Univariate Logistic Analysis on 60-Days Mortality in Inpatients with AECOPD

Table 6 Univariate Logistic Analysis on ICU Admission and Invasive Ventilation Use in Inpatients with AECOPD

Figure 3 Kaplan–Meier estimates in-hospital survival in patients with varied AECOPD severity. Abbreviation: AECOPD, acute exacerbation of chronic obstructive pulmonary disease.

Discussion

Through a large multicenter cohort of AECOPD patients, we revealed that the Rome severity classification could excellently distinguish the risk of ICU admission, MV and IMV. However, more studies are needed to determine whether it can reliably discriminate the risk of short-term mortality.

Several studies have been published, aiming to validate the Rome severity classification in AECOPD patients. A retrospective study conducted in 200 Spanish inpatients with AECOPD revealed that the Rome classification lacked the capacity to classify the severity of AECOPD compared with the Spanish COPD Guidelines (GesEPOC) classification.¹¹ Carmen et al applied the Rome classification to a cohort of 364 hospitalized patients with AECOPD in the Netherlands and found that the Rome classification can differentiate between exacerbation events with different short-term mortality rates.¹² The study conducted by Lee et al in Korea found excellent performance of the Rome classification for predicting ICU admission and the need for noninvasive mechanical ventilation (NIV) or IMV and acceptable performance for predicting in-hospital mortality by comparing the Rome classification with the DECAF score and GesEPOC 2021 criteria.¹³ Our findings concerning the association between the Rome severity classification and the need for ICU admission and MV were highly consistent with Lee’s findings. However, the Rome classification worked poorly in our cohort compared to Lees and Carmen’s studies in identifying the difference in mortality rates between the moderate and severe groups. The inconsistent results might be attributed to differences in the characteristics of patient cohorts and the geographical setting of different studies. The global variability in the available resources to treat patients with AECOPD and local customs may affect the criteria for hospital visits and admissions and thus may also contribute to the differences in study results. Notably, the three studies mentioned above are all single-center, small-sample studies, which inevitably induces selection bias and weakens the power of these studies. In the present study, we consecutively included unselected inpatients with AECOPD from 10 tertiary general hospitals in China, which should represent real-world situations.

The physiologic parameters (VAS, HR, RR, SaO2) and inflammatory biomarkers (CRP) included in the Rome severity classification are all easy to obtain and have been tested as prognostic factors of AECOPD in some previous studies. The VAS offers the benefit of quantitatively representing ventilatory demand on a scale from 0 to 10,¹⁴ and the scale has been validated against respiratory loads in patients with COPD.¹⁵ The proportion of VAS scores >5 increased with disease severity in our cohort. Several studies showed that an elevated HR or RR was associated with exacerbation,^16,17 readmission^9,18 and correspondingly early mortality in COPD patients.^19,20 We also found faster HR and RR in the moderate and severe groups than in the mild group in the study. Furthermore, the two variables could be measured easily and noninvasively in the clinic.²¹ It is not difficult to foresee that the two variables could facilitate the application of the Rome classification in the management of COPD, as smartphones and wearable device technology are gradually used to facilitate early detection and treatment of AECOPD.^22,23 CRP, an acute-phase protein that can be measured accurately within minutes at the point of care, is a biomarker for assessing AECOPD since elevated CRP is associated with the need for antibiotics and higher mortality.^24,25 Although using CRP as a marker of airway inflammation may lack specificity, CRP is widely recognized as a useful and sensitive marker of infections and AECOPD.²⁶ SaO2 is a reflection of gas exchange in AECOPD patients. It is more practical and widely available to measure pulse oximetry in all clinical settings, although SaO2 is less reliable than arterial blood gas analysis. Some studies showed that a reduction in oxygen saturation was associated with AECOPD risk;^23,27 unfortunately, the change in SaO2 from baseline was not used to distinguish mild versus moderate events because of its unavailability in this post-hoc analysis. Therefore, future studies are needed to validate the reduction in SaO2 in assessing the severity of AECOPD. Acute respiratory failure with hypoxemia and/or hypercapnia and acidosis is the first and possibly most important clinical feature observed in patients with severe AECOPD,²⁸ and the mortality risk is higher at lower pH values.²⁹ Based on this fact, arterial blood gas analysis was the only criterion that was used to determine the severe group in the Rome classification. Theoretically, the incorporation of objective, easy and ready-to-measure variables in this proposed classification may assist in a better delineation of clinically different AECOPD.

However, the Rome classification should not be considered as a complete version. First, the criteria included in the severity classification are single-minded based on a review of the literature and discussion. ABG alone may not be sufficient to determine severe AECOPD events. It is unquestionable that COPD is a heterogeneous illness underpinned by diverse clinical characteristics and pathophysiological mechanisms.³⁰ Although there are several markers that potentially indicate AECOPD severity, none have received widespread acceptance. As stated by Ramakrishnan et al, the different endotypes of AECOPD severity cannot be assessed by solely focusing on inflammatory or pathophysiological parameters.³¹ Our research team previously discovered that blood urea nitrogen (BUN), a component of BAP-65, was linked to an increased risk of in-hospital death and other adverse outcomes in AECOPD inpatients.^6,32 Moreover, the baseline parameters, especially dyspnea in the stable stage, may have an impact on how this exacerbation may turn out.^33,34 Comorbidities like heart failure, arrhythmia, coronary heart disease, and diabetes, which are highly prevalent in COPD patients, also contribute to event severity in the real-life clinical setting. Therefore, baseline parameters and the existence of comorbidities may also need to be considered to incorporate into the severity assessment for AECOPD. Second, the threshold setting seems to be arbitrary. Taking CRP as an example, in our research, serum CRP levels were interestingly the highest in moderate events, with 62.2% of patients having a CRP above 10 mg/L. Similar results were also found by the two studies mentioned earlier and conducted in the Netherlands and Korea, respectively.^12,13 This revealed that a cutoff value of 10 mg/L may lack specificity to distinguish a moderate event from a severe one, and the mechanism linking CRP to the exacerbations of COPD may be complex and requires further researches. Probably due to the two flaws mentioned above, the Rome classification is valid in discriminating between mild and more severe AECOPD but fails to distinguish between moderate and severe AECOPD. Last, it is currently difficult to suggest optimal medical treatments according to AECOPD severity based on the Rome classification. In this cohort study, patients admitted with mild Rome AECOPD were slightly older and suffered from more comorbidities, suggesting that these factors, rather than the severity of acute respiratory events, contributed to the indication for hospitalization. Therefore, whether the Rome classification could be used as an indicator for hospitalization for AECOPD is debated. In addition, patients in the severe group needed more ICU admissions and mechanical ventilation than those in the moderate group, while the mortality did not increase accordingly. A possible explanation is that the strengthened treatments received by severe AECOPD patients may contribute to an improve in short-term survival; a more likely explanation is that this classification does lack a distinction between moderate and high mortality risks because of the flaws mentioned above. Consequently, the Rome classification needs to be optimized in the selection of variables and corresponding thresholds based on prospectively studies.

To our knowledge, this is the first large-scale multicenter cohort study to validate the Rome classification. Moreover, the consecutive inclusion of unselected inpatients with AECOPD and comprehensive collection of information in our study ensured high data quality and true associations between the Rome classification and risk of adverse outcomes in the real-world setting. Importantly, this study provided important initial insights into the distinctive value of the proposed classification in the Chinese population, and the Rome severity classification is the first step to differentiate clinically different AECOPD. Nevertheless, our study has several limitations. First, the VAS score was retrospectively evaluated based on the medical records and nurse’s description of the severity of dyspnea on admission, which inevitably leads to a bias. Fortunately, not all patients require a VAS score to be classified, and only those who already meet two out of the 5 moderate criteria need a VAS score to determine whether they are moderate. In addition, an exact VAS score was not necessary; alternatively, we only needed to evaluate whether the patient had significant dyspnea (VAS≥5 vs <5), which facilitates the accuracy of this retrospective evaluation based on detailed medical records on admission. Second, our study only allows the applicability of the Rome classification to be assessed in the hospital setting and not in the primary care setting, and how the Rome classification works in outpatient clinics or communities is still unclear. Finally, the exclusion of patients because of missing data may result in selection biases and affect the external validity of the results. But the original data is from an noninterventional cohort study, and the arrangement of auxiliary examinations and treatment of patients were at the discretion of the attending physicians, which should reflect the real-world application circumstance of Roma classification in China. Additionally, we found that the excluded patients tend to have milder conditions, with lower mortality rates (data not shown) and a greater likelihood of being classified to the mild group. So even if these patients were included, we believe the results would hardly change. Thus, optimization and prospective validation of the classification and a deeper exploration of its prognostic applicability in making therapeutic decisions remain pending.

Conclusion

It is essential for an AECOPD severity assessment tool to accurately identify patients at high risk for adverse outcomes. In this large cohort study, the Rome severity classification demonstrated excellent performance in predicting ICU admission and the need for MV or IMV but failed to distinguish the risk of short-time mortality between the moderate and severe groups. Studies are warranted to validate whether it could accurately evaluate the risk of mortality, and further optimization may still be needed before clinical application.