Development and Validation Features of the Patient Benefit Index for the Treatment of Allergic Rhinoconjunctivitis with Allergen Immunotherapy

Introduction

Allergic rhinoconjunctivitis (ARC) is one of the most common diseases worldwide.^1–3 Recent epidemiological studies show an increasing prevalence of ARC in some countries.^3–7 For children, a current study of worldwide time trends suggests that the prevalence of non-infective rhinoconjunctivitis may no longer be increasing.⁸ According to data from the German Robert Koch Institute, about 15% of the German population suffer from ARC.⁹ Furthermore, data from a survey on 90,880 workers in Germany show a prevalence of pollen sensitization in 21.4%.¹⁰ In a representative telephone survey in 1004 adults in Germany, 33% reported an allergic disease confirmed by a physician, including 23.2% with pollen sensitization. Among those, 53% suffered from impaired performance because of their allergic symptoms.¹¹ A recent international systematic review reported that the prevalence of rhinitis was 18.0% in people without and 40.5% in people with atopic dermatitis.¹²

Due to the high burden of ARC and the associated loss of productivity, ARC is of high socioeconomic relevance.¹³ In particular, patients with ARC often suffer from comorbidities such as atopic dermatitis and can develop more serious diseases such as allergic asthma if not treated properly.^14,15

Today, a broad spectrum of symptomatic treatment options for ARC are available. But for many patients the treatment course is not satisfactory and relief is not long-lasting.^11,16,17 To date, allergen immunotherapy (AIT) is the only causal treatment available. After a treatment period of three years, it can lead to a reduction or even a complete elimination of ARC symptoms.¹⁸ The benefit of AIT from the patient’s perspective has not been evaluated yet.

To assess the patient-relevant treatment benefit of AIT, the German Center for Health Services Research in Dermatology (CVderm) developed the Patient Benefit Index Questionnaire specifically for allergen immunotherapy (PBI-AIT). The assessment of the therapeutic benefit from the patient’s point of view has gained importance in the rating of drugs over the last years.¹⁹ The German Pharmaceuticals Market Reorganisation Act (AMNOG), which was implemented in 2011, obliges pharmaceutical companies to present dossiers proving their drug’s additional benefit over comparative medication.²⁰ The Federal Joint Committee decides if an additional benefit of the drug is detectable and based on this decision, the price of the drug is negotiated. The PBI can provide insight into the patient-relevant benefits of a drug, providing the necessary differential between similar treatment options.

In this article, the development and the validation of the PBI-AIT for the assessment of patient-relevant benefit in the treatment of ARC with AIT are presented.

Materials and Methods

Data Protection and Ethics

All participants provided written informed consent. The study has been approved by the ethics committee of the Medical Association of Hamburg (PV3901) in 2011 and was conducted in accordance with data protection regulations.

Development of the PBI-AIT

The instrument for the assessment of benefit in patients with ARC using AIT was adopted from the PBI for allergic rhinitis (PBI-AR).¹⁶ It was adapted, and new AIT-specific need and benefit items were added. The development of the PBI-AIT was based on international standards in developing psychometric and biometric testing.^21–25 In an open survey, n = 100 patients with clinically confirmed ARC were asked to answer questions about their burden caused by ARC and their treatment needs in their own words. A group of experts including two dermatologists, two psychologists, two ENT (ear, nose and throat) specialists and two patients with ARC discussed these items and compared them with those included in the PBI-AR. The final questionnaire contains 33 items.

The PBI-AIT consists of two questionnaires. The first one is called Patient Needs Questionnaire (PNQ) and gathers information on the patient’s individual therapeutic needs. For each item a five-point Likert scale (0 = not important at all to 4 = very important) records the patient’s individual relevance. In the second questionnaire, the Patient Benefit Questionnaire (PBQ), the same items are presented again, however relating to the extent to which the treatment needs were met by the treatment (scale from 0 = treatment did not help at all to 4 = treatment helped a lot). Alternatively, the patients can tick the option “does not apply to me” in the PNQ and “did not apply to me” in the PBQ. These options make sure that only current needs of the individual patient are included in the total score.

Calculation of the Patient Benefit Index

The Patient Benefit Index is a single value reflecting the patient-relevant treatment benefit. First, scores from each item of the PNQ are multiplied by the score of the corresponding PBQ item. Then, the product is divided by the sum of all PNQ items. Lastly, the final PBI score takes a sum of these calculated values.

The PBI can range from 0 (no benefit) to 4 (maximum benefit).

According to pilot analyses, a PBI ≥ 1 is considered a threshold of relevant treatment benefit.²⁶

Validation of PBI-AIT Under Routine Conditions

1. Study design: longitudinal, non-interventional multicenter study.
2. Patients and study centers: Patients aged ≥18 years with a clinical diagnosis of ARC who started with AIT (sublingual or subcutaneous) or received symptomatic medication were included. Data collection took place between April 2011 and August 2013 in n = 127 German allergist practices. Patients eligible for the study were informed about the study by their physicians who invited them to participate.
3. Data collection: Patients with ARC and their physicians had to answer standardized questionnaires at five time points (hereafter called t1 to t5) in the course of 9–12 months. The questionnaires were divided into patients with symptomatic therapy and patients starting AIT, as indicated on the front page of each questionnaire. Patients with AIT were allowed to maintain a symptomatic medication in addition. Patients who needed less than nine months or longer than 12 months for the completion of five visits were also included in this analysis. In addition to treatment needs (PNQ) and treatment benefits (PBQ), data on clinical characteristics, severity of ARC, disease specific quality of life (Rhinoconjunctivitis Quality of Life Questionnaire, RQLQ), health status (European quality of life visual analogue scale, EQ VAS) and satisfaction with previous ARC treatment were recorded. The clinical results of the study are not subject of the present publication.

Statistical Analysis

All data were described with standard statistical parameters (frequencies for categorical data; mean value, standard deviation and median for continuous outcome data). Frequencies in patient characteristics of the two treatment groups at t1 were compared by chi-squared tests for independence and means were compared by unpaired t-tests. A Pearson product-moment correlation coefficient was calculated for examining the statistical relationship between EQ VAS, RQLQ global score, current ARC severity assessed by the physician, current ARC severity assessed by the patient, satisfaction with previous treatment and the PBI-AIT at t5. It was supplemented by a significance test on differences between correlation coefficients.^27,28

The change in PBI-AIT depending on treatment was analyzed by a two-way analysis of variance (ANOVA) with repeated measures on one factor (PBI-AIT at t1 and t5). The other factor was treatment (AIT vs symptomatic). The main focus was on the interaction effect. Missing values were not replaced. The alpha level was set at 5%. All analyses were performed with IBM SPSS Statistics for Microsoft Windows, Version 23.0 (IBM Corporation, Armonk, NY).

Results

Patients’ Characteristics

In total, data from n = 493 patients with ARC were obtained. Of them, most participants (n = 427) started AIT treatment and n = 66 patients continued to take symptomatic medication only. It took a mean of 292 days between the beginning of the study (t1) and the end of the study (t5) (Figure 1). More than half of the patients were female, mean age of patients was 37 years at t1. The patients who started AIT did not differ in age, gender and BMI > 30 from the patients who received symptomatic medication only (Table 1). Almost all patients reported a pollen allergy. Patients who started AIT more often reported an allergy to grass and cereals. Both groups did not differ in disease severity and disease-related quality of life limitations but did differ in their treatment satisfaction and patient-defined treatment benefit.

Table 1 Characteristics of the Participating Patients with ARC at t1

Figure 1 Timeline of the study (t1 to t5).

Figure 2 Patient Benefit Index (PBI) score distribution (median and interquartile range) of patients with allergic rhinoconjunctivitis and either allergen immunotherapy (AIT) or symptomatic medication at t1 and t5 (AIT: n = 279 at t1, n = 333 at t5; symptomatic medication: n = 56 at t1, n = 43 at t5).

There were n = 279 participants who reported receiving AIT and who completed the PBI at t1, n = 333 at t5; n = 226 patients with AIT completed the PBI-AIT at both timepoints.

Feasibility

At the beginning of the study (t1), no item had a higher share of missing values than 3.6% in patients with AIT (“be able to have a normal sex life”) in the PNQ (Table 2). Mean proportion of missing values per item was 2.0% ± 0.6% (median = 1.9%); mean number of missing values per AIT patient was 0.7 in the PNQ at t1. At the end of the study (t5), no item had a higher share of missing values than 5.2% (“avoid the occurrence of asthma”) in the PBQ (Table 3). Mean proportion of missing values per item was 3.7% ± 0.6% (median = 3.7%); mean number of missing values per patient was 1.2 in the PBQ at t5.

Table 2 Patient Needs Questionnaire Records per Item (t1). Only Patients with Allergen Immunotherapy, n = 422

Table 3 Patient Benefit Questionnaire Records per Item (t5). Only Patients with Allergen Immunotherapy, n = 349

At t1, the item with the highest proportion of patients stating “does not apply to me” in the PNQ (58.1%) was “be able to have a normal sex life” (Table 2). Mean proportion of “does not apply to me” records per item was 21.0% ± 16.4% (median = 15.9%). At the end of the study (t5), the same item had the highest share of “did not apply to me” records (62.8%) in the PBQ (Table 3). Mean proportion of “did not apply to me” records per item was 27.5% ± 17.3% (median = 24.4%).

Among the treatment goals that patients with AIT rated as the most important were “be able to breathe through my nose more freely”, “be able to stay outdoors without symptoms”, “not have itching on the eyes, nose or palate anymore”, “be healed of all symptoms”, “be permanently free of physical complaints” (mean importance ratings: 3.7 on a scale from 0 = not at all important to 4 = very important). Among the least important goals were “be comfortable showing myself more in public” and “be able to have a normal sex life” (mean importance ratings: 2.5). Among the highest rated treatment benefits were “have confidence in the therapy” (mean benefit: 3.0 on a scale from 0 = no benefit from the therapy to 4 = maximum benefit from the therapy in this aspect) and “have an easily applicable treatment” (mean benefit: 2.9). Among the least achieved benefits was “be healed of all symptoms” (mean benefit: 2.1) (Tables 2 and 3).

Distributional Characteristics of the PBI

The patients with AIT achieved a mean PBI of 1.8 ± 1.1 and a median PBI of 1.7 (mean PBI of patients with symptomatic treatment = 2.4 ± 1.0, median PBI = 2.3) at t1. At t5, patients with AIT displayed a mean PBI of 2.5 ± 1.0 and a median PBI of 2.6 (mean PBI of patients with symptomatic treatment = 2.4 ± 1.0, median PBI = 2.6) (Figure 2).

Signifying clinical relevance, 97.5% of the patients with symptomatic treatment and 91.1% of the patients with AIT derived benefit from the treatment (PBI score ≥ 1). Of all patients, 92.1% attained benefits from their current treatment. Findings indicated that patient treatment needs were fulfilled “almost completely” to “completely” (PBI scores of 3 to 4) for 38.5% of patients with symptomatic treatment and 35.4% with AIT.

The weighted items of the PBI-AIT showed discriminatory power with values between 0.53 and 0.93 (AIT only). The discriminatory power of the PNQ items (t5) was between 0.67 and 0.86 and of the PBQ items (t5) between 0.77 and 0.96. The PBI-AIT of patients with AIT was slightly skewed to the right at t1 (0.27) and moderately skewed to the left at t5 (−0.42).

Reliability

The internal consistency of the items of the PNQ and PBQ was satisfactory. Cronbach’s alpha was 0.98 (n = 71) in PNQ, 0.99 (n = 56) in PBQ and 0.98 for the weighted items (n = 44). It was calculated based on data of patients with AIT who stated at t5 that all treatment needs and benefits of the PBI-AIT applied to them.

Correlations with Other Patient and Physician Reported Outcomes

The PBI-AIT correlated with all tested physician and patient reported outcomes at t5. In particular, the PBI-AIT correlated with the current disease severity assessed by the physician (r = −0.26, n = 325, p < 0.001) and with the EQ VAS (r = 0.27, n = 327, p < 0.001, Table 4).

Table 4 Correlation of the Patient Benefit Index Questionnaire for Allergen Immunotherapy with Other Outcome Variables. Only Patients with Allergen Immunotherapy at t5

Furthermore, the PBI-AIT correlated with the current severity degree assessed by the patient (r = −0.37, n = 329, p < 0.001) and RQLQ global score (r = −0.35, n = 331, p < 0.001). The highest correlation was found between PBI-AIT and satisfaction with previous treatment (r = −0.57, n = 314, p < 0.001). It was significantly higher than the correlation between PBI-AIT and disease severity (assessed by the physician).

Treatment-Related Patient Benefit Changes

In a two-way ANOVA with repeated measures, an interaction between treatment and time of PBI-AIT measurement was found (F (1/263) = 8.46; p < 0.01, partial η² = 0.031). The mean PBI of the patients with AIT was significantly lower at t1 and improved (1.8 ± 1.1 at t1 and 2.5 ± 1.0 at t5). The PBI of those with symptomatic treatment did not change in the course of the study (2.4 ± 1.0 at t1 and 2.4 ± 1.0 at t5).

Discussion

The benefit of AIT from the patient’s point of view has not been evaluated yet, as there has not been an implemented instrument for assessing this specific patient benefit. Therefore, a Patient Benefit Index for the assessment of patient defined needs and benefits in the treatment of ARC with AIT was developed for future use as a patient-reported outcome parameter in clinical studies and in health services research.

Correspondingly, the aim of the present study was the validation and feasibility testing of this instrument in clinical routine.

The concept of the PBI differs from patient satisfaction concepts and even more so from quality of life because it is based on patient preference assessments. The method of benefit assessment used in the PBI is grounded on the method of goal attainment scaling and the goal-oriented outcome measurement.^29–33 These methods have been expanded to the PBI concept based on the German requirements by the German Pharmaceuticals Market Reorganisation Act (AMNOG), which was introduced in 2011. This implies the patient relevant benefit to be a primary target of benefit assessment.

With regard to patient acceptability and comprehensibility, PBI-AIT proved practicable in clinical routine. The good practicability is reflected in the low share of missing values in the PNQ and the PBQ.

The relatively high share of “does not apply to me” records of several PNQ and PBQ items could be explained by the seasonality of ARC and the associated varying burden of ARC. A possible explanation why the PBI could be calculated for fewer patients at t1 than at t5 might be that some patients were unsure at t1 which therapy to evaluate, since their AIT had only just begun and could therefore not be evaluated. Actually, the specification was that they should rate their last therapy, but since their AIT was written on the questionnaire, this could have been misleading for some patients at t1. This assumption is supported by the fact that at t1 the PBQ questionnaire (about having reached treatment needs by the treatment) was filled in significantly less often than the PNQ questionnaire (about treatment-related needs) by patients with a beginning AIT.

The moderately left-skewed distribution of the PBI at t5 means that the bulk of values lies to the right of the mean. This indicates that the study participants tended to rate their treatment needs as being reached by therapy. The correlation between PBI-AIT and the constructs “satisfaction with previous treatment” and “burden of ARC” (in terms of quality-of-life impairments, health state and current severity) indicates convergent validity. The fact that the correlation between PBI-AIT and treatment satisfaction is stronger than the correlation between the PBI-AIT and disease severity could indicate discriminant validity since the construct of treatment satisfaction is supposed to be closer to the concept of the PBI. Moreover, we found indications for sensitivity to change of the PBI-AIT: As expected, a stronger change of the PBI-AIT in patients who started with AIT could be observed than in patients with symptomatic treatment who already received symptomatic medication before the start of the study, and therefore no significant change would be expected during the study period.

The presented study is non-interventional and therefore an open label study, which implies that the knowledge of active treatment may affect patient-reported data. However, the purpose of the study was to assess the data on validity, reliability and feasibility of the PBI-AIT in a real-life setting. Thus, the open-label study design implies that there may be patient-related biases that are typically present in non-interventional studies compared to randomized controlled clinical trials. Another limitation of the study is that the analysis on internal consistency was performed based on a rather small sample size. The small sample size could be explained by the fact that many treatment needs and benefits did not apply to the study participants at the date of completion of the PBI-AIT as the symptoms of ARC mostly occur only temporary.

To conclude, the analysis indicates that the PBI-AIT is a feasible, valid and reliable instrument for the assessment of patient relevant benefits in patients with ARC who receive AIT. Therefore, the usage of the PBI-AIT in research and clinical practice is recommended. The use of the PBI-AIT will lead to a more specific understanding of the individual patient needs and benefits of AIT.