Convergent Validity and Minimal Clinically Important Difference of the Maugeri Foundation Respiratory Failure Questionnaire (MRF-28) and the Chronic Obstructive Pulmonary Disease-Specific Health-Related Quality of Life questionnaire (VQ11)

Introduction

Chronic obstructive pulmonary disease (COPD) is associated with progressive dyspnea and reduced exercise tolerance, both of which encourage sedentary lifestyles and cause considerable deterioration of health-related quality of life (HRQoL).¹ Symptom relief and improved HRQoL should be major goals of COPD treatment.^2–4 Among the non-drug treatments, pulmonary rehabilitation (PR), a comprehensive personalized intervention including physical activities, health education and psychological support, has been shown to be beneficial, regardless of age, sex, severity of airflow limitation or place where it is performed (hospital, outpatient clinic, or home).² Indeed, recent studies have confirmed that home-based PR is safe and effective and improves HRQoL in these patients.^2,4–7

Although generic questionnaires can be used to assess HRQoL in COPD patients, disease-specific tools may be more appropriate to obtain better responsiveness.^8,9 Among respiratory disease-specific HRQoL questionnaires, the St George’s respiratory questionnaire (SGRQ)¹⁰ and the chronic respiratory questionnaire (CRQ)¹¹ are frequently used for COPD patients.⁴ However, the length of the SGRQ (more than 50 items generally requiring 20 mins)⁸ limits its routine use, and the individualization of the dyspnea domain of the CRQ likely increases its complexity for both patients and interviewers.¹² Therefore, a COPD-specific questionnaire was developed for quick and easy (ie, approximately 3–4 mins)¹³ assessment of HRQoL in outpatient settings, real-life studies, and/or routine individual care: the visual simplified respiratory questionnaire (VSRQ).³ The validity (comparison to SGRQ), reliability, and responsiveness of the VSRQ in COPD patients have already been demonstrated.^3,14 Nevertheless, in the past few years, two other respiratory disease-specific questionnaires have been increasingly used in PR centres: the Maugeri Foundation respiratory failure questionnaire (MRF-28)^5,15–20 and the COPD-specific HRQoL questionnaire (VQ11).^5,13,19,21 The interest in both, compared to the VSRQ, is that they provide a global score of HRQoL but also and most importantly, they evaluate special dimensions of HRQoL.

The MRF-28 is composed of 28 items covering three theoretical components (daily activity, cognitive function, and invalidity).¹⁵ Its validity and reliability were confirmed in 81 patients treated with home mechanical ventilation for chronic respiratory failure but only 12 of the patients had COPD.¹⁵ The validity therefore needs to be confirmed in a larger sample of COPD patients. Moreover, to our knowledge, the minimal clinically important difference (MCID) of the MRF-28 is unknown. The VQ11 comprises 11 items distributed across three components (functional: 3 items, psychological: 4 items, social: 4 items) and a top level combining them.¹³ The VQ11 is valid and reliable,^13,21 but its responsiveness for home-based PR needs to be determined, as does its MCID.¹³

In this study, we therefore assessed the validity, responsiveness, and MCID of the MRF-28 and the VQ11 in a retrospective study designed to evaluate the effect of individual home-based PR on HRQoL in COPD patients. The VSRQ, which is valid, reliable, sensitive,³ and specifically developed for COPD patients, was taken as the reference. We hypothesized that home-based PR would be feasible and effective in improving HRQoL in COPD patients, and that the MRF-28 and VQ11 would be useful tools to evaluate these benefits.

Materials and Methods

Study Design

This 6-year monocentric observational study, which included consecutive COPD patients managed by outpatient PR, respects the “strengthening the reporting of observational studies in epidemiology” (STROBE) statement.²² All data were collected prospectively and were entered into our rehabilitation computerized medical records (according to the routine/standard management of patient). Retrospective analysis was executed from data collected between 2012 and 2017. All patients were following a home-based PR program.

Briefly and as described elsewhere, this individualized home-based PR program consisted of a 90 min session once a week for 8 weeks, based on an educational needs assessment.^5,19 It comprised endurance exercise, the resumption of the physical activities of daily living, therapeutic education, psychosocial support, and motivational communication to encourage health behavior changes and self-management. This weekly session was conducted under the direct supervision of a team member. Patients were expected to continue performing the endurance exercises on their own the other days of the week, according to a personalized action plan.

Initially, individual endurance exercise on a cycle ergometer was performed in sequences of 10 mins or less. They were asked to exercise at least 5 days per week, with the goal being to reach 30 to 45 mins per day, in one or several sessions.^5,19,23 The exercise intensity was prescribed on the basis of dyspnea symptoms (generally 3–4 on Borg’s category scale with ratio properties) or effort perception (generally 11–13 on Borg’s rating of perceived exertion scale). The patients were encouraged to increase the time spent in daily living activities. In addition, three muscle strengthening exercises for the upper and lower limbs were systematically proposed, lasting 10–15 mins per day and using weights and dumbbells and/or elastic bands. Each exercise comprised 10 repeated movements. A 1 min recovery period was observed between exercises. Warm-up and stretching exercises were also recommended together with balance exercises whenever necessary.

The PR agreement given to the patients before PR included an accident protocol. An accident was defined as death, hospitalization or emergency care required for heart or orthopedic incidents during the 8 weeks of PR. The patient and/or the physical therapist could declare the accident. Patients were asked to interrupt all physical activities in the case of any abnormal sensations, especially chest or joint pain, and to contact both the rehabilitation team and the attending physician.

Participants

In this observational study, 485 patients with COPD who were following home-based PR were consecutively included. COPD diagnosis was confirmed according to international guidelines.²⁴ COPD patients were excluded if they had dementia or poorly controlled psychiatric illness, neurological sequelae, or bone and joint diseases preventing physical activity.

Assessments

Patients were evaluated at home just before (ie, T₀) and after PR (ie, T₂). During a medical consultation at baseline, sociodemographic (ie, age, sex, body mass, height, body mass index and socioeconomic deprivation score) and spirometric data (ie, forced expiratory volume in one second: FEV₁, forced vital capacity: FVC, FEV₁/FVC and the severity of airflow limitation), as well as treatment (ie, long-term oxygen therapy: LTOT, non-invasive ventilation: NIV, continuous positive airways pressure: CPAP) were collected. As described elsewhere,^5,19 the evaluation of exercise tolerance included a 10 times sit-to-stand test (10STS), a timed up-and-go test (TUG), and a 6 min stepper test (6MST). The 10STS consists of standing up until reaching full knee extension and sitting back down 10 times as fast as possible while seated at the front of a 42-cm-high chair.²⁵ During the TUG, patients stand up from a standard armchair, walk 3 m, turn, walk back, and sit down again as quickly as possible.²⁶ Last, the number of steps taken in 6 mins was collected during a 6MST as previously described.²⁷ The stepper (Go Sport^®, Grenoble, France), with a step height fixed at 20 cm, was placed near a wall to support patients if they became unbalanced or exhausted. The patients could freely regulate their stepping rate to reach the highest number of strokes.

Psychological status was assessed by the hospital anxiety and depression (HAD) scale,²⁸ and HRQoL was assessed by the MRF-28,¹⁵ VQ11,²¹ and VSRQ.³ The HAD scale determined the patient’s psychological state in terms of anxiety and depression.²⁸ This questionnaire has 14 items (7 items for each psychological state), with responses scored on a scale of 0–3 to indicate symptom frequency (the higher the frequency, the higher the score is). The MRF-28 is composed of 28 items covering three theoretical components (daily activity, cognitive function, and invalidity) and gives a final score ranging from 0 to 100, with higher scores reflecting a higher degree of impairment.¹⁵ The VQ11 comprises 11 items distributed across three components (functional: 3 items, psychological: 4 items, social: 4 items) and a top level combining these three components.²¹ Higher scores indicate less favorable quality of life. Last, the VSRQ is a valid, reliable, and sensitive questionnaire to assess HRQoL, specially developed for patients with COPD.³ It comprises eight visual analog scales ranging from 0 to 10, with a total score ranging from 0 to 80. Higher scores indicate better HRQoL. The MCID of the VSRQ is known: +3.4 units.³ Consequently, the VSRQ was taken as a reference for comparison with the MRF-28 and VQ11.

Ethical Approval

All procedures performed in this study were in accordance with the Helsinki declaration and its amendments. Moreover, the research protocol was approved by the Observational Research Protocol Evaluation Committee of the French-language Society of Pulmonology (CEPRO 2017-007), and written informed consent was obtained from each participant.

Statistical Analysis

Descriptive statistics included the number of cases and percentages for categorical variables, and median, range or means and standard deviations (SD) for continuous variables. Normal Gaussian distribution of the data was verified by the Shapiro-Wilk test. At the baseline (T₀), the participant characteristics included in the statistical analysis were compared to those of nonparticipants (ie, patients excluded because of refusal to participate or lost to follow-up) from Student's t-tests or Mann–Whitney U-tests for continuous data. For categorical data, the frequencies were compared using Pearson’s Chi-squared test or Fisher’s exact test when appropriate. To test the effect of PR, paired Student's t-tests or Wilcoxon matched pairs tests were used.

Spearman correlation coefficients were calculated to examine the convergent validity between the VSRQ (ie, the anchor) and the other questionnaires (ie, MRF-28 and VQ11). These correlation coefficients were classed as “very high” (r ≥ 0.90), “high” (0.70 ≥ r ≤ 0.89), “moderate” (0.50 ≥ r ≤ 0.69), or “low” (0.26 ≥ r ≤ 0.49).²⁹

As triangulation is recommended to measure MCID,³⁰ both anchor-based (using the VSRQ as the anchor) and distribution-based (half a standard deviation and a small Cohen’s effect size) methods were used to calculate the MCID for MRF-28 and VQ11. According to Cohen,³¹ the magnitude of the difference can be considered to be trivial (ES < 0.2), small (0.2 ≥ ES < 0.5), moderate (0.5 ≥ ES < 0.8), or large (ES ≥ 0.8). A receiver operating characteristic (ROC) curve approach was used, with dichotomized VSRQ responses (less than 3.4 units vs 3.4 units or more, as proposed by Perez et al³) as dependent variables, and the change score for each questionnaire as independent.³² An optimal cut-off value for each questionnaire ‒ ie, the one that maximized Youden’s index (maximizes both sensitivity and specificity) ‒ was identified.

Results

Description of Sample

A total of 485 patients were solicited to participate in the study, and 7 of them refused (Figure 1). The sociodemographic and medical characteristics at inclusion are summarized in Table 1. Differences between participants (n = 400) and nonparticipants (n = 78) are presented in Table 2. Mainly, the nonparticipants were excluded because of physical incapacity (n = 21), hospitalization (n = 16), stopping the home-based PR without explanation (n = 13), or loss of motivation to continue the program (n = 10, Figure 1). The nonparticipants were significantly older (68.3 ± 10.9 vs 63.5 ± 11.1 years) and had lower FEV₁ (34.3 ± 13.5 vs 40.1 ± 17.8%) and FVC (56.0 ± 18.7 vs 65.2 ± 20.0%) compared to participants (p ≤ 0.01, Table 2).

Table 1 Characteristics of Patients Solicited to Participate in the Study

Table 2 Differences Between Participants and Nonparticipants

Figure 1 Flow diagram of patients with chronic obstructive pulmonary disease managed by home-based pulmonary rehabilitation program.

Effects of Home-Based PR

The effects of the home-based PR on exercise tolerance (ie, 10STS, TUG, and 6MST) psychological status (ie, anxiety and depression from the HAD questionnaire), and HRQoL (ie, from each dimension of the MRF-28, VQ11 and VSRQ) are summarized in Table 3. All measures were significantly improved after the program (p < 0.0001, Table 3), except for the dimension of cognitive function on the MRF-28 (for which only a trivial effect was noted); all other measures showed small (but significant) or moderate effects (Table 3).

Table 3 Effect of the Pulmonary Rehabilitation Program

Convergent Validity of HRQoL Questionnaires

The associations between the anchor (ie, VSRQ) and HRQoL (ie, MRF-28 and VQ11) at T₀ and T₂ and the delta (ie, T₂-T₀) were all significant (p < 0.0001, Table 4). The correlation coefficients ranged from r = −0.372 to −0.686 for MRF-28, and r = −0.423 to −0.753 for VQ11 (Table 4).

Table 4 Correlations Between VSRQ and the Other Questionnaires at T0 and T2 and the Deltas (T0-T2)

Determination of MCID

As the lower limit of the area under the ROC curve was sometimes < 0.5 (0.481 for MRF-28 and 0.436 for VQ11), the relationship between the change score and the anchor was considered insufficient to estimate the MCID from this statistical analysis (ie, ROC curve). Therefore, distribution-based (half a standard deviation and a small Cohen’s effect size) methods were used to determine it (Table 5). For each variable, the mean was calculated from these two methods. The MCID were −5.2 units for MRF-28, −2.0 units for VQ11, and +3.4 units for VSRQ.

Table 5 Determination of Minimal Clinically Important Difference Using Two Statistical Methods

Discussion

This study shows that 16.1% of the patients did not complete the home-based PR. The two main reasons, physical incapacity (26.9%) and hospitalization (20.5%, Figure 1) explained 47.4% of PR abandonment. At baseline, these nonparticipants were significantly older and had lower FEV₁ than the participants (Table 2). These results are unsurprising since body mass index (non-significantly decreased in the current study, p = 0.184, Table 2), age, and FEV₁ are predictors of hospitalization and death in COPD patients.³³ Moreover, FEV₁ is known to be significantly associated with COPD exacerbations.³⁴ To summarize, the nonparticipants did not complete the PR because they were older and had greater airflow limitation, which is known to lead to physical incapacity, exacerbations, and thus hospitalizations and death in COPD patients.

In line with the literature,^5,19,35 this study confirms that a personalized and comprehensive home-based PR (with one visit per week for 8 weeks) including physical training, health education, psychological support, and motivational communication is feasible and effective in COPD patients and can significantly improve exercise tolerance, anxiety and depression, and HRQoL (Table 3). As previously indicated,⁵ home-based PR is therefore an alternative to outpatient or inpatient rehabilitation and may make it easier for a greater number of patients worldwide to benefit from PR.

The VSRQ is used to measure HRQoL in patients with pulmonary disease,^14,36 especially when PR is executed at home.^{5,16,19,23,35} It is a valid, reliable, and sensitive questionnaire,^3,14 and thus it was our reference (ie, the anchor) to confirm the convergent validity of the MRF-28 and VQ11. The results showed a significant improvement in VSRQ, MRF-28, and VQ11 after PR (Table 3). Moreover, significant correlations (Table 5) were found between the VSRQ and the other HRQoL questionnaires at T₀ (r = −0.685 and −0.691 for MRF-28 and VQ11, respectively) and T₂ (r = −0.686 and −0.753 for MRF-28 and VQ11, respectively). Significant correlations were also noted for the delta (T₂-T₀) between VSRQ and the other HRQoL questionnaires (Table 5). According to the literature,^13,15,21 these results together confirm the convergent validity of the MRF-28 and VQ11. The validity of the MRF-28 has already been reported, with a significant correlation with the SGRQ (r = 0.78) in patients treated for chronic respiratory failure by home mechanical ventilation.¹⁵ The convergent validity (p < 0.05, r = 0.71) between the VQ11 and SGRQ was also reported in COPD patients.²¹ Therefore, the MRF-28 and VQ11 have good measurement properties and are useful tools to measure COPD-specific HRQoL in routine practice.²¹

The present study suggests that MRF-28 and VQ11 could be used alone or as an adjunct to VSRQ to assess the global HRQoL. However, as VSRQ includes no subscale (and provides only global score of HRQoL), MRF-28 and VQ11 could be preferred. Indeed, the MRF-28 covers 3 subcomponents of HRQoL (ie, daily activity, cognitive function and invalidity)¹⁵ and the VQ11 also provides scores from 3 subcomponents (ie, functional, psychological, and social).²¹ Previously, Mahler³⁷ indicated that HRQoL refers to 3 components of health: physical (eg, physical capacity), psychological (eg, self-esteem, and), and social (eg, social relationship). Thus, an instrument dedicated to measure HRQoL needs to provide a global score, as well as evaluation of different subcomponents. Indeed, as indicated by Janssens et al,¹⁵ these subcomponents may be contributive to a finer analysis of the impact of PR on HRQoL in respiratory disease.

As noted by Smid et al,³⁸ the MCID is the smallest difference in a measurable clinical parameter perceived by the patient or clinician that indicates a meaningful change in the condition for better or worse. It can thus be used to assess the effect of therapies like home-based PR. MCID of −4 units has frequently been found for the SGRQ³⁹ and even +3.4 for the VSRQ.³ Therefore, using anchor-based (with the VSRQ as anchor) and distribution-based methods, the MCID for MRF-28 and the VQ11 were calculated and identified as −5.2 units and −2.0 units, respectively. Consequently, as MCID is patient derived scores that reflect changes in a clinical intervention that are meaningful for the patient,⁴⁰ future studies in PR could also determine the number of responders (ie, patients with HRQoL score superior to MCID), rather than only raw data (ie, score of HRQoL).

Limitations

The cut-off points of MCID (ie, −5.2 and −2.0 units for MRF-28 and VQ11, respectively) were established from distribution-based methods rather than anchor-based methods because the area under the ROC curve was slightly less than 0.5 (0.481 for MRF-28 and 0.436 for VQ11), which raised the question of whether these two questionnaires are able to make sensitive distinctions between COPD patients who improved by at least −3.4 units on the VSRQ and those who did not.³⁸ The current study therefore gives only an indicative value of the MCID of the MRF-28 and VQ11, rather than exact cut-off points. Moreover, 16.1% of patients did not complete the home-based PR. This result is close to that of previous studies (dropout ranged from 0 to 36%) that proposed PR for COPD patients.⁴¹ These nonparticipants had worse baseline values (ie, older and greater airflow limitation) than those who completed PR. As Smid et al³⁸ noted, this may well have affected the current results. Furthermore, the participants were all COPD patients and this should be considered when generalizing the results to other populations. Last, although significant correlations (p < 0.0001) with moderate to high coefficients of correlation were found between the VSRQ and the other questionnaires (MRF-28 and VQ11) at T₀ (ie, r = −0.685 and −0.691, respectively) and T₂ (ie, r = −0.686 and −0.753, respectively), lower significant coefficients of correlation between deltas were noted (ie, r = −0.372 and −0.423, respectively, p < 0.0001, Table 5). Despite the findings of similar coefficients of correlation,³⁸ moderate correlation coefficients (r ≥ 0.5) are ideally needed to prevent a wide range in MCID.⁴² These observations therefore suggest that the MCID of our study are indicative values and specifically adapted for PR in COPD patients.

Conclusion

This study shows that home-based PR is an efficient non-pharmacological therapy that improves exercise tolerance, HAD, and HRQoL in COPD patients. Moreover, the results confirm (from VSRQ) the convergent validity of the MRF-28 and VQ11. Therefore, both questionnaires have good measurement properties and are useful tools to measure COPD-specific HRQoL in routine practice. Last, we suggest MCID of −5.2 units and −2.0 units for the MRF-28 and VQ11, respectively, to assess the effect of PR in COPD patients.