Assessment of Real-World Patient-Reported Outcomes in Patients Initiating Biologic Agents for the Treatment of Autoimmune Diseases: An Observational Study in Four Patient-Powered Research Networks

Introduction

Over the last 20 years, biologic therapeutic agents (biologics) have revolutionized the care of many autoimmune diseases and have become part of standard of care. Numerous clinical trials and large observational studies have proven the efficacy, clinical effectiveness, and reasonable safety profiles of biologics approved by regulatory authorities. Nevertheless, real-world data about the clinical effectiveness of biologics as assessed by improvements in patient-reported outcomes (PROs) in health domains, such as fatigue, that are shared across many autoimmune diseases are limited. Many PROs measure important health domains that are common to a diverse range of conditions and are easily understandable and relatable for patients (eg, the interference that pain has on one’s daily activities). Although disease-specific clinical trial outcomes and disease activity measures may incorporate some patient-reported data, these established composite measures may not adequately capture concepts that directly assess the impact of disease on patients’ daily life.

The Patient-Centered Outcomes Research Network (PCORnet) was funded by the Patient-Centered Outcomes Research Institute (PCORI) to enable high-quality research for people with chronic health conditions. PCORnet initially consisted of multiple large health systems contributing electronic health record data, health plan research networks contributing primarily administrative claims data, and 20 Patient-Powered Research Network (PPRN) registries. PPRNs were developed to facilitate direct-to-patient clinical research over the internet and/or through the use of smartphone applications and other technology to answer research questions important to patients for their health decision-making.^1,2 PPRNs engage a broadly inclusive population of patients that is not restricted to specific physician practices, medical centers, or care settings and obtain PROs and other data directly from patients outside the context of a clinical encounter or specific research protocol. Results from PROs collected in this manner may provide additive information to data from trials or other data sources about the real-world effectiveness of biologics and other treatments in health domains that may be traditionally overlooked but substantially impact practical aspects of daily activities and may inform clinical decision-making.

The most reliable and meaningful approach for inclusion of PROs in the evaluation of real-world effectiveness is uncertain. For example, clinical trial inclusion criteria often limit study participants to those with sufficiently high disease activity to be reasonably expected to experience improvement. A similar approach to the evaluation of PROs may seem appropriate from an analytic perspective (to avoid floor effects) but may limit the generalizability of results. Specifically, because not all chronic illnesses affect all health domains, patients who are not experiencing an abnormality in one or more health domains (eg, anxiety, sleep disturbance) may not be reasonably expected to improve. Furthermore, individual patients without abnormalities in a specific health domain may not assign importance to improvement in that domain. As part of a PCORI-funded collaborative demonstration project that brought together four PPRNs representing patients with autoimmune and inflammatory conditions, the objective of the study was to assess and compare the proportions of patient participants with different autoimmune diseases who had abnormalities in PROs measuring important general health domains at the time of initiation of treatment with biologics, as well as the effects of baseline abnormalities on subsequent improvement.

Methods

Four PPRNs contributed data for this study (each described below). For all PPRNs, PROs were collected using PROMIS^® short-form surveys or computer adaptive testing (CAT) versions of PROMIS instruments, and scores were reported as T-scores normalized to the general population in the United States. A score of 50 represents the population norm (average score) and each 10-unit increase or decrease in the score represents 1 standard deviation of the population norm. For domains that are desirable (eg, physical function), higher scores are better. For domains measuring symptoms that are undesirable (eg, pain interference), lower scores are better. The following PROMIS measures that were considered important by patient stakeholder participants of the PPRNs and collected by two or more PPRNs were included in this report: pain interference, fatigue, sleep disturbance, physical function, anxiety, depression, and social function. Of note, there were some PROs that were not routinely collected by some PPRNs as they were not prioritized highly enough by that patient population for routine data capture. For example, the inflammatory bowel disease (IBD) patient registry did not include measures of physical function, as IBD does not typically have as large an impact on physical function as inflammatory arthritis conditions do.

ArthritisPower was launched in 2014 as a partnership between the nonprofit Global Healthy Living Foundation, its associated CreakyJoints arthritis patient community, and rheumatology researchers at the University of Alabama at Birmingham. Led by a Patient Governor Group, the PPRN has more than 33,000 consented patient participants with inflammatory arthritis (including those with rheumatoid arthritis (RA), psoriatic arthritis (PsA), ankylosing spondylitis (AS) who were part of this study), and other conditions, who use the ArthritisPower mobile and desktop application to track their symptoms and treatments while participating in research.^3,4

PARTNERS PPRN includes healthcare providers, a clinician scientist-initiated research network (Childhood Arthritis and Rheumatology Research Alliance (CARRA)), three patient-advocacy groups (Arthritis Foundation, Lupus Foundation of America, Cure JM Foundation), and a quality-improvement learning network (Pediatric Rheumatology Care and Outcomes Improvement Network (PR-COIN)). PARTNERS included patient participants with juvenile idiopathic arthritis (JIA) for this study, and data were retrospectively obtained from the CARRA Registry, a clinic-based prospective observational registry in the United States and Canada.⁵

IBD-Partners (formerly known as Crohn’s and Colitis Foundation of America Partners) is an online research network created by the Crohn’s and Colitis Foundation and the University of North Carolina School of Medicine. It is one of the largest inflammatory bowel disease (IBD) research networks in the world and includes over 15,000 patient participants with IBD. Participants contribute data by completing an online survey twice a year.

The Vasculitis PPRN (VPPRN) is led by the University of Pennsylvania and is a partnership between the Vasculitis Clinical Research Consortium, which is an international network of medical centers, patient organizations, and researchers, and the Vasculitis Foundation, a patient advocacy organization. The VPPRN is comprised of people affected by vasculitis, a family of almost 20 rare diseases. The VPPRN has enrolled over 3000 patient partners including people diagnosed with vasculitis, parents of children with vasculitis, and caregivers of adult patients. Members of the registry complete online surveys to contribute data.

For all PPRNs, the date of first instance of initiation of treatment with a biologic was identified. Baseline PRO scores were defined as those collected within 30 days prior to, or up to 7 days after, first biologic initiation. Follow-up PRO scores were defined as those collected within 3 to 8 months following biologic initiation; if participants had more than 1 PRO collection during the follow-up time window, then PROs from the date closest to 6 months following biologic initiation were used. Due to limited follow-up PRO collection meeting these requirements in some patient networks at the time of analysis, only follow-up results from patients with JIA or IBD are reported. We chose to assess all biologics together as a single exposure because valid inferences could not be made between results from treatment with different biologics due to sample size and limitations in being able to adjust for between-treatment differences.

All analyses were conducted individually by each PRRN using their own data. For baseline PROs, the mean, standard deviation, median, and interquartile range were determined. Because participants who were not experiencing an abnormality in a domain at baseline were expected to be unlikely to experience significant benefit following any intervention, the proportion of patients with PRO scores ≥5 units (0.5 standard deviations) worse than the population norm (eg, ≥55 for pain interference and ≤45 for physical function) was determined. Follow-up PRO scores were assessed irrespective of discontinuation of the biologic (ie, intention to treat approach). For participants with JIA or IBD, a comparison was made between the follow-up and baseline PRO scores by calculating the mean improvement and the proportion of participants whose PRO score improved by ≥5 units. A 5-unit change corresponds to an effect size of 0.5 standard deviations and is typically considered a minimally important difference.⁶ These comparisons to baseline were repeated, restricted to participants with a baseline abnormality (ie, ≥5 units worse than the population norm) in each specific PRO domains.

The study protocol was approved by the University of Alabama at Birmingham Institutional Review Board, protocol number 160712003, and the study complied with the Declaration of Helsinki. The PPRNs did not share identifiable data with the study team. They conducted their own analyses of their data and shared aggregate results. Each PPRN had their own local IRB approval and informed consent process to routinely enroll patients in their registry and collect and analyze data for research purposes.

Results

The characteristics of study participants from each of the PPRNs are shown in Table 1. The mean age of participants with RA/PsA/AS and IBD was similar, and participants with vasculitis were somewhat older. All diseases showed a female predominance that was largest for RA/PsA/AS. The mean disease duration was highly variable, ranging from 1.6 years for JIA to 13.8 years for IBD.

Table 1 Characteristics of Participants Providing One or More Patient-Reported Outcome Assessments at the Time of Newly Initiating Treatment with a Biologic

Table 2 shows the baseline PRO scores among all participants included in the study. There was wide variation across autoimmune diseases in baseline PRO scores for all domains. The proportion of participants with baseline pain interference scores ≥55 ranged from 52% to 55% in JIA and IBD to 93% in RA/PsA/AS. Similarly, the proportion of participants with fatigue scores ≥55 ranged from 51% in IBD to 90% in RA/PsA/AS. Sleep disturbance scores were less frequently abnormal overall, but 71% of RA/PsA/AS participants and 26% of IBD participants reported scores ≥55. The proportion of participants with physical function scores ≤45 was more similar, ranging from 67% in vasculitis to 90% in RA/PsA/AS. Anxiety, depression, and social function were only reported for participants with IBD and vasculitis, and the proportion of participants with abnormalities in these PROs was higher in vasculitis for all three domains.

Table 2 Baseline Scores of Patient-Reported Outcomes at the Time of Newly Initiating Treatment with a Biologic Among All Participants

The most frequently abnormal PRO domains also varied across diseases. For example, among participants with vasculitis, anxiety was the most frequently abnormal domain (96%), and among participants with IBD, pain interference, fatigue, and anxiety were the most frequently abnormal domains (52%, 51%, and 49%, respectively). Among participants with RA/PsA/AS, pain interference, fatigue, and physical function were the most frequently abnormal domains (93%, 90%, and 90%, respectively). Although JIA and RA/PsA/AS are all forms of inflammatory arthritis, substantially fewer participants with JIA had abnormalities in pain interference (55%) and physical function (69%) compared to RA/PsA/AS.

Table 3 shows the changes in follow-up PRO scores among participants with JIA or IBD. The mean elapsed time from the baseline PRO collection to the follow-up PRO collection was 5.8 months for JIA and 6.4 months for IBD. Among participants with JIA and IBD with abnormalities in baseline scores, the proportions of patients experiencing an improvement of ≥5 units were similar, ranging from 43% to 56%. When assessing follow-up in all participants with JIA irrespective of baseline PRO scores compared to those with baseline abnormalities, the mean improvements in scores were smaller but still clinically relevant (range of mean change: 4.6 to 6.3 units of improvement), and the proportions of participants experiencing an improvement of ≥5 units were only somewhat lower (range: 41% to 48%). Among all participants with IBD irrespective of baseline PRO scores compared to those with baseline abnormalities, the mean improvements in scores were substantially smaller and some were likely not clinically relevant (range: 1.1 to 1.9 units of improvement), and the proportions of participants experiencing an improvement of ≥5 units were consistently lower (range: 27% to 31%).

Table 3 Change in PRO Scores from Baseline to Follow-Up Assessment Approximately 6 Months Later, Comparing Strata of All Participants and Those Participants with Baseline Scores ≥5 Units Worse Than the Population Norm

Discussion

We evaluated the PROs capturing health domains that are common to many autoimmune diseases at the time of initiation of treatment with biologics among participants in four PPRN patient registries that were created as part of PCORnet. Across autoimmune diseases, there was wide variation in baseline PRO scores and in the domains that were most frequently abnormal. Some of the observed differences across diseases may be attributable to differences in disease duration, patient age, or selection bias (ie, which patients chose to participate), but this is unlikely to explain all of the observed variation. Rather, the differences are likely to represent true differences in the health impacts and lived experiences of the different diseases; for example, pain interference was the most frequently abnormal domain in RA/PsA/AS, while anxiety was the most frequently abnormal domain in vasculitis. Among the diseases where sample sizes were sufficient (JIA and IBD), initiation of biologics was associated with improvements in PRO health domains, and the improvements were more pronounced when restricted to participants experiencing abnormalities in each specific domain at baseline. Of importance, the proportion of patients with abnormalities at baseline ranged from 26% to 93% across PRO domains and diseases. Abnormalities in some health domains (eg, sleep disturbance) were common but did not uniformly affect all patients with the chronic conditions that we studied.

We chose a priori to assess for 5-unit changes (equivalent to 0.5 standard deviations) in PROMIS scores. Historically, a change of 0.5 standard deviations has been found to closely approximate the minimal important difference in various health-related quality of life instruments.⁶ More recently, studies of PROMIS measures among patients with rheumatoid arthritis and systemic lupus erythematosus have reported minimal important differences of approximately 2 units,^7,8 while a study of PROMIS measures in patients with advanced-stage cancer found larger minimal important differences ranging from 2.5 to 6 units.⁹ More recent work in RA suggests that a difference of at least 5 units (as we used) represents a meaningful change for PROMIS Pain Interference and Fatigue. Given the seriousness of autoimmune diseases and the potential risks and economic cost of treatment with biologics, we assumed that a minimal important difference may be viewed as insufficient improvement by some patients treated with biologics and opted to assess a potentially greater change in PROMIS measures in this study. This threshold is consistent with the range recommended for a Minimally Important Change (MIC) based on a systematic review of 31 studies reporting a MIC for PROMIS measures.

This study had limitations, primarily as a result of the novel methods of data collection. We relied upon participants to supply the date of first use of biologics, which may have been inaccurate. Utilization of resources such as electronic health records, insurance claims, or pharmacy dispensing data may be useful to validate treatment initiation dates in future studies. Participants were encouraged to complete PROs at the time of initiation of biologics, but many did not, and therefore their data could not be used in this study. Similarly, many participants did not complete follow-up PROs in the needed time window after initiating biologics. It is possible that participants who completed follow-up PROs had treatment outcomes that were significantly different than participants who did not, but we cannot evaluate this with our data. We assessed improvement by determining the proportion of patients with ≥5 units of improvement, but this does not account for participants who may have experienced lesser improvement in some PRO health domains following initiation of certain treatments. We restricted our assessment to patients initiating treatment with biologics, and the results for initiation of other treatments may have been different.

Three of the four PPRNs collected PROs outside the context of clinical care and demonstrated the feasibility of this approach to assess the real-world effectiveness of the initiation of biologics. With the collection of PROs from larger numbers of participants and at more structured intervals, these types of data may be used to conduct observational studies of comparative effectiveness. As mentioned, one of the primary challenges in this study was obtaining baseline PRO measurements near the time of initiation of biologics. Patients often do not anticipate medication changes before they occur and may be too distracted to complete PRO measurements prior to starting the new therapy. A more targeted and dedicated effort, including enhanced communication with participants through the use of reminders or notifications and more effective communication about the importance and impact of PRO collection, may be needed. Inclusion of participants’ healthcare providers or healthcare records to identify instances of therapy initiation may enhance PRO data collection as well. As an important and new facilitator of such efforts, Remote Therapeutic Monitoring (RTM) programs whereby patients use software as a medical device (ie, a smartphone app) to record their health and symptoms became reimbursable by Medicare and other insurance programs in 2022. Programs like RTM and its sister program, Remote Physiologic Monitoring (insurance reimbursable beginning in 2019) likely will result in digital data capture of patients’ symptoms by software and/or biosensors to become a more mainstream part of routine care and facilitate efforts like ours to systematically capture PROs. Lessons learned from this demonstration project have yielded important insights about the need to protocolize data capture around medication initiation and scheduled follow-up intervals, with participant contact if needed to avoid missing data.¹⁰

Conclusions

As expected, many patients experienced improvement in PROs following initiation of treatment with biologics for autoimmune diseases. Nevertheless, a substantial proportion of participants did not exhibit abnormalities in all measured PRO domains prior to initiation of treatment with biologics, and patients without current abnormalities in a given domain appear to be less likely to experience improvement in that domain. In the conduct of real-world studies of PROs, consideration should be given to restricting analyses to participants with baseline abnormalities, much the same as clinical trials are restricted to participants with sufficiently active disease to demonstrate improvement. This approach may limit the generalizability of the results, although all eligible participants could be included in the study with the restrictions applied only at the time of data analysis of individual PRO domains (ie, patient participants need not be excluded entirely from the study, as is often done in clinical trials). At minimum, studies should likely include a priori analysis plans to stratify patients based upon baseline PRO scores to help better understand which patients respond best to which interventions. Similarly, it should not be assumed that all patients experiencing escalation of therapy will have meaningful abnormalities in all selected health domains at the onset of therapy. Thus, selection of PRO domains that most effectively capture the lived experience of patients with a specific condition is a critical first step in identifying PRO endpoints for pragmatic clinical research. Given the observed wide variation in baseline PRO scores and in the domains that were most frequently abnormal across autoimmune diseases, it is likely that the domains selected for study may need to be disease-specific, even among seemingly similar conditions. Alternatively, a different solution could be the use of a composite outcome that encompasses multiple PRO domains. For PROs to be reliably and meaningfully included in the evaluation of real-world medication effectiveness, more knowledge and careful consideration are needed to select the most appropriate patient populations and subgroups for inclusion and evaluation in studies measuring change in PROs.