An Easy-to-Implement Clinical-Trial Frailty Index Based on Accumulation of Deficits: Validation in Zoster Vaccine Clinical Trials

Introduction

Older adults are not a homogeneous population: while many people maintain a high degree of independence and good health as they age, many other older individuals become “frail”, a state of decreased physiologic reserve and increased vulnerability for morbidity and mortality.^1,2 Frail older adults often have different clinical presentations (including with infectious illnesses) and/or respond differently to interventions and treatments than their less frail counterparts;^3–5 Classifying frailty has therefore become an important topic within healthcare and clinical trial fields in recent years. This has become especially poignant in the current COVID-19 era, with the devastating impact of COVID-19 on frail older adults, and the need to understand differential responses to treatments and vaccines.⁶ Frail participants and those with multiple morbidities are often under-represented in randomized controlled trials (RCTs), and frail people are likely to have different outcomes and responses to the interventions being studied, raising questions about vaccine or drug effectiveness in real-world settings.⁷ This is being recognized by regulatory authorities, who increasingly request that efficacy and safety of a vaccine or drug be demonstrated in a frail cohort within a clinical trial setting. For instance, the European Medical Agency guidelines emphasize that treatments for vulnerable groups, such as frail individuals, should be appropriately studied for unique treatment effects.⁷

Even so, inclusion of frailty measures in clinical trials is rare, with barriers including a perception that measuring frailty is difficult or will add to participant and data collection burden. The fact that no consensus exists on frailty measurement may also contribute to such nihilism. It is true that multiple tools have been proposed in the literature to measure frailty.^8,9 Some approaches, such as the Frailty Phenotype¹⁰ and the Clinical Frailty Scale^11–13 require specific data collection at baseline (ie, the initial study visit). In contrast, the Frailty Index (FI) approach is based on accumulation of deficits and calculated using a wide-ranging set of deficits. Many (if not most) published deficit accumulation FI are generated from existing datasets, and thus do not require specific items to have been collected prospectively. This offers clear benefits for use of FI in clinical trials, including of vaccines, where detailed baseline health data tend to be collected along with measures of quality of life and symptoms in order to assess impact on possible illness outcomes, safety and reactogenicity.

With the aim of studying frailty in relation to performance of the recombinant Zoster vaccine, we constructed a FI based on data from clinical trials of using a large and relatively diverse sample of clinical study participants (N = 26,976, ≥50 years of age). This Clinical Trial-FI (CT-FI) was shown to be an accurate measure of frailty status compared with a prospectively defined FI (NCT02979639) and has also been applied in the ZOE-Frailty study (NCT03563183).^14,15

In the present manuscript, we aimed to describe and present the CT-FI as an achievable frailty measure for use in vaccine clinical trials, and to validate the CT-FI in relation to characteristics that are known to be preserved across different frailty indices. We also aimed to demonstrate that the CT-FI is a robust measure that is not sensitive to the choice of individual included/excluded deficits.

Methods

Study Design and Participants

The CT-FI was applied in the ZOE-Frailty study (NCT03563183) in order to examine vaccine efficacy, immunogenicity, safety and reactogenicity, with analyses stratified by level of frailty as a retrospective analysis of the ZOE-50 (NCT01165177) and ZOE-70 (NCT01165229) vaccine studies, which were RCTs of adjuvanted recombinant zoster vaccine compared with placebo in the prevention of herpes zoster illness and complications.^16,17 A full description of the ZOE-Frailty study is found at the study’s clinicaltrials.gov registration: https://clinicaltrials.gov/ct2/show/NCT03563183?term=064andcond=Zoster&draw=2andrank=1. The previously published ZOE-Frailty study was a retrospective analysis of 26,976 ZOE-50 and ZOE-70 participants (mean age 68.8 years, range: 48–96). Participants were found to represent a range of frailty from non-frail to frail. Vaccine efficacy against herpes zoster was robust (>90%) across frailty subgroups.¹⁵ For the present analyses further exploring and validating the CT-FI, all ZOE-Frailty participants with known baseline frailty were included, regardless of vaccine group assignment or vaccination status in the RCTs. Of the total 26,976 participants, 137 (0.5%) were missing the full FI, but of these, 6 had enough data to be defined as frail (FI >0.25) so only 131 had to be excluded because frailty status was unknown.

Generating the CT-FI

According to published guidelines, in order to be a candidate for inclusion in a deficit accumulation FI, any health deficit must fulfill the five basic criteria:²³

1. It must associate with health status.
2. Prevalence must generally increase with chronological age.
3. The deficit must not saturate too early in the general population.
4. Deficits must cover a range of systems.
5. Only when using a FI to serially track a given population over time, the deficits used to make up the FI need to remain constant from one iteration to the next; comparisons between samples, or drawn from different contexts or datasets, do not need to include the same deficits.

Accordingly, we identified 41 health deficits: 12 from the participant’s medical history, 25 from responses to the Short Form Survey-36 (SF-36)¹⁸ and 4 from responses to the EuroQol-5 Dimension (EQ-5D)¹⁹ questionnaires completed at baseline. The included comorbidities (defined based on the participant’s medical history) were selected as per prior published FI work.²⁰ These were not intended to be an exhaustive list of all possible medical conditions, but rather to represent a broad range of illnesses.

In order to avoid duplication of concepts included in both QoL measures, we selected 25 items from the SF-36 covering the domains: physical functioning, general health, vitality and mental health,¹⁸ and from the EQ-5D we included items for mobility, self-care, usual activities and anxiety/depression.¹⁹

According to established methods for FI generation, items are scored from 0 (deficit absent) to 1 (deficit present) and each participant’s deficits are summed and divided by the total number of potential deficits considered (here 41), with an adjustment for missing SF-36 or EQ-5D items, to generate a continuous index in the range of 0–1.^21,22 After each study participant’s continuous FI value was calculated, for the purposes of stratified analyses participants were also assigned to one of three frailty subgroups based on previously published cutoffs: FI ≤0.08 was classified as non-frail; FI >0.08 to ≤0.25 pre-frail; FI >0.25 frail.^15,20 Full details of the included deficits and scoring of CT-FI components are provided in Supplementary Methods and Tables S1 and S2.

Validation Methods

We employed a series of well-known methods to validate the CT-FI.^21–23

1. Distribution and associations with age and sexThe FI should increase monotonically with chronological age and based on literature generally follows a gamma distribution with a sub-maximal limit of approximately 0.70 that is independent of age.^24,25 We analyzed the relationship between the CT-FI and chronological age by means of a general linear model and obtained an estimate of the rate of deficit accumulation per year of age. Our approach was to fit a piecewise linear model in which we chose 70 years of age as the cutoff point. We also examined differences in frailty by chronological age. It is well established in the literature that frailty tends to be higher in women than in men (while mortality is higher in men at any given level of frailty),²⁶ so we examined sex differences in the CT-FI.^27,28

Statistical analyses were conducted using SAS Desktop 9.4 software.

Role of the Funding Source

This work was sponsored by GlaxoSmithKline Biologicals SA. GlaxoSmithKline Biologicals SA was involved in all stages of the study conduct and analysis, and took responsibility for all costs associated with the development and publishing of the present manuscript. GlaxoSmithKline Biologicals SA does not veto ongoing publications or control the decision about what journal to submit to, with ultimate decision on the target journal made by the coauthors. The lead and corresponding author MKA received no payment related to any aspect of this study.

Ethics

The parent ZOE studies were approved by Research Ethics Boards at participating sites; participants provided written informed consent in accordance with the Declaration of Helsinki. The present analyses, done as part of the ZOE Frailty Study (NCT03563183), were approved by Research Ethics Boards at ZOE frailty participant sites. The full list of sites participating in the ZOE frail study is found in the clinicaltrials.gov registration at: https://clinicaltrials.gov/ct2/show/NCT03563183?term=064andcond=Zoster&draw=2andrank=1.

Results

CT-FI Distribution and Associations with Age and Sex

The CT-FI followed a gamma distribution tailing to the right with maximum 0.70 and median 0.10 (Interquartile Range 0.05–0.17) (Figure 1A). The CT-FI was distributed more symmetrically as age increased (Figure 1B); mean score increased with age (Figure 2). Age was a significant factor in the piecewise linear model (p < 0.0001) with a rate of mean deficit accumulation per year of 0.0021 for participants <70 years and 0.0052 for those aged ≥70. There was also a significant “sex by chronological age” interaction effect in both age categories with the average rate of deficit accumulation per year for males and females being 0.0016 and 0.0025, respectively, for participants under 70 years and 0.0047 and 0.0058, respectively, for participants ≥70 years (Table S3).

Figure 1 Continued.

Figure 1 Clinical Trial Frailty Index (CT-FI) distribution. (A) Presents the CT-FI distribution in the entire sample. (B) Presents CT-FI distributions stratified by age category. Abbreviations: N, number of subjects in each age category; YOA, years of age.

Figure 2 Box and whisker plot of Clinical Trial Frailty Index by years of age. Abbreviation: N, number of subjects at each age. Notes: The boxes represent the 25% and 75% percentiles. The whiskers extend from the 10th to the 90th percentile. The line joins the mean value at each age. The dots represent the maximum values at each age group.

The percentage of individuals classified as non-frail, pre-frail and frail was 42.7%, 45.6% and 11.3%, respectively. The most common medical history deficits were hypertension and arthritis reported by 53.4% and 32.4% of all participants, respectively, while the least common medical history deficits were congestive heart failure and migraine reported by 3.6% and 2.6% (Table S4).

Mortality in Relation to the CT-FI

Table 1 presents the number of deaths by frailty status and sex. The percentage of participants who died during study follow-up (and who were classified per protocol as experiencing a fatal serious adverse event [SAE]) was 2.0%, 5.2%, and 11.7% in the non-frail, pre-frail, and frail subgroups, respectively. Although the rate of deficit accumulation was higher for females than males, the proportion of deaths among frail males was higher compared to frail females (ie, 16.0% versus 9.4%). Figure S1 presents the Kaplan–Meier estimates of the survivor function by frailty and sex.

Table 1 Number of Deaths by Frailty Status and Sex

In the univariate Cox regression analysis, sex, chronological age, and CT-FI were each significant predictors of survival. In the multivariable model, the FI hazard ratio remained significantly (p < 0.001) related to mortality even in the presence of chronological age and sex (Table 2).

Table 2 Survival Analyses by Cox Regression

Resampling Methods

In 1000 iterations of the Bootstrap analysis of the Cox proportional hazards univariate model, the CT-FI remained significantly related to mortality (HR 95% CI: 1.040, 1.064), demonstrating that the CT-FI was not sensitive to inclusion or exclusion of specific groups of items; ie, groups of items were not driving the observed association with outcomes (Table 2). Fifty Bootstrap replications (based on random re-selection of 30/41 CT-FI deficits) of the Kaplan–Meier estimates of the survivor function by frailty status are presented in Figure 3. The separation between frailty status categories remained clear for both males and females. The Jackknife analysis also showed consistent results in relation to survival. Across all 41 iterations of the univariate Cox model the FI hazard ratio ranged from 1.055 to 1.059, demonstrating that no single item was driving the observed association with outcomes (Table S5).

Figure 3 Bootstrap analysis of survival by frailty status. Survival curves present 50 replications of 75% resampling.

Internal Consistency

The three component categories of the CT-FI were moderately correlated with each other. The Spearman correlation between the medical history and the SF-36 and EQ-5D components were 0.32 and 0.22, respectively, while that between the SF-36 and EQ-5D components was 0.57. The overall Cronbach Coefficient Alpha was 0.65.

Known Groups Effects/Comparisons

The mean SF-36 and EQ-5D domain scores recorded at baseline are presented by frailty status in Table 3. All scores decreased consistently with increasing frailty. Of the domain scores that did not contribute to the construction of the CT-FI, the mean SF-36 bodily pain scores were 86.7 in non-frail participants, 72.3 in pre-frail participants and 49.9 in frail participants while the mean SF-36 role emotional scores were 94.5 in non-frail participants, 85.9 in pre-frail participants and 66.4 in frail participants.

Table 3 Mean SF-36 and EQ-5D Domain Scores at Baseline by Frailty Status

Discussion

Here and in our previous manuscript¹⁵ we describe, for the first time to our knowledge, the retrospective creation of a CT-FI from health status and patient reported outcome (PRO) data collected in a large vaccine clinical trial, and its subsequent validation. We have demonstrated that the CT-FI is readily implementable, valid and robust. Replicating known FI properties, the CT-FI increases with age, is higher in women than in men, and conforms to a gamma distribution with a consistent, sub-maximal upper limit to the proportion of deficits that any person can accumulate. The CT-FI is associated with mortality and is internally consistent. Furthermore, using resampling methods within the index, we have demonstrated that the properties and outcome associations of the CT-FI are not dependent on inclusion or exclusion of specific individual variables, in that these characteristics are robust to resampling or deletion of items within the index.

In the measurement of frailty, multidimensional interdisciplinary Comprehensive Geriatric Assessment (CGA) has been used extensively.^29,30 However, this is a complex measure that may be impractical in some contexts due to the required time and expertise of the person performing the assessment (ideally a geriatrician). No consensus has been reached on which tools are most suitable for studies or settings where CGA is not feasible or would place a high burden on both health-care staff and participants. It is therefore important to select or develop validated frailty measures that can be implemented efficiently across different settings.

Several distinct methods of assessing frailty in clinical studies have been developed over recent years. Widely used examples include the Frailty Phenotype,¹⁰ judgement-based descriptive scales such as the Clinical Frailty Scale,^11–13 and the FI based on accumulation of deficits.^22,31,32 The Frailty Phenotype is based on a predefined set of five criteria exploring the presence/absence of general signs or symptoms (unintentional weight loss, self-reported exhaustion, slow walking speed, weak grip strength, and low physical activity). From these responses a subject is assigned to one of three frailty categories: frail, pre-frail, or non-frail.¹⁰ The Clinical Frailty Scale is based on a global judgment-based assessment of frailty, in which an individual is scored in one of nine categories from very fit to very severely frail or terminally ill.¹³ Both of these approaches require that specific data collection be undertaken up-front in a clinical trial setting, which may not always have been done (for retrospective analyses) or may not be seen as feasible (for prospective data collection).

Here, we have used the accumulation of deficits approach which has numerous advantages for clinical trial settings. It is a well validated approach, having been used in many international population and clinical datasets.^26,33 It includes a wide range of medical conditions such as comorbidities, physical, and mental functioning. Its major distinctive trait is that instead of simply assigning a subject to one of a few categorical frailty states it provides a continuous distribution of frailty that allows a more accurate examination of the associations between frailty, ageing, other characteristics, and outcomes. Also, the deficit accumulation FI approach brings inherent flexibility, in that it can be generated using existing data as long as a sufficient number of deficits have been collected across different systems, which is often the case in clinical trials.^21,34 Previous studies have demonstrated that a FI can be constructed using data from existing population and clinical databases, and in particular data that would be collected in routine health assessments of older adults.²¹ FI have also been generated and used in some cardiovascular and chronic disease clinical trials.^35–38 We have extended this to generate and validate a CT-FI using information that is readily available in patient’s health history and PRO instruments that are typically recorded at screening or baseline visits of vaccine clinical trials, which differ from clinical trial of therapeutics in that the focus is on prevention in a cohort that generally does not share specific underlying comorbidity. As such, the FI approach to measuring frailty is eminently achievable to implement in clinical trials without increasing participant and operational burden.

Our CT-FI included 41 variables. In general, the more variables included in a FI, the more precise estimates become. The generally accepted minimum number of deficits is approximately 20, though stability and robust prediction of outcomes is generally best with ≥30 deficits.^21,32 By including a large number of health deficits, the FI approach allows for integration of a large amount of data while reducing dimensionality.³⁹ A common question is why the index is not weighted. There are multiple reasons for this. Not weighting enhances ease of calculation and generalizability across data sets. Validations in multiple settings and datasets have demonstrated that weighting is not required.^21,40 This can also be understood through the approach of including many deficits across multiple systems, which allows for self-weighting (eg, a serious health condition will likely be associated with functional and symptom impact, hence garnering additional deficits, while a less serious health condition would not). Notably, although this CT-FI was derived from a shingles vaccine study, we did not include shingles-specific risk factors or family history items as these are not generally included in FI based on published criteria.²¹ It is important to emphasize that the aim of a CT-FI for use in vaccine trials is to develop a general measure of frailty, not a disease-specific measure of susceptibility.

The properties of the CT-FI are consistent with characteristics of FI that have been widely replicated and preserved across datasets. These include the gamma distribution, shift towards more symmetric distribution with increasing age, and the limit to frailty of approximately 0.7.²⁵ Consistent with prior literature on sex differences, frailty was also higher in women than in men.^27,28 Survival analyses demonstrated that likelihood of death was higher for frail males compared with frail females. This is consistent with other studies showing that while men accumulate fewer deficits than women, any given level of deficit accumulation is more lethal for men.²¹ Female sex, younger age and lower CT-FI were all significant predictive factors for survival. Frailty was significantly predictive of survival independent of age (p < 0.001), ie, reflecting a state of increased vulnerability to adverse health outcomes for individuals independent of chronological age.²²

The CT-FI is not sensitive to the inclusion or exclusion of specific individual variables. This was demonstrated here by the use of the Bootstrap and Jackknife resampling methods. We observed that associations with mortality remained consistent despite these tests of variable composition, which is consistent with prior experience with these resampling techniques.^22,23 In other words, FI need not include a standard checklist of items across studies. This is a powerful and convenient property of the deficit accumulation approach to frailty. As long as a sufficient number of deficits are considered across multiple systems, standard methodology and criteria allow creation of a FI in any dataset or clinical context.²¹ In the context of clinical trials, this allows a CT-FI to be constructed retrospectively, using data that are already being collected; this has benefits in that A) it allows retrospective (or prospective) analysis of an existing clinical trial population to determine the distribution of frailty represented among participants, B) participant burden is minimized by not requiring additional frailty-specific data collection, and C) clinician judgment is not required to assign participants to frailty categories.

Our study is not without limitations. As frailty was not a pre-specified measure in the initial ZOE studies, the CT-FI was constructed with the available data elements; though indeed the retrospective assessment of frailty is the point of the present exercise. Although we previously demonstrated that the ZOE trials recruited a range of participants from non-frail to frail, as with most clinical trials the frailest individuals would not have met inclusion criteria.¹⁵ Strengths include the large study population (>26,000 participants across a wide age range and from diverse international settings), robust data collection for health history, PROs and outcomes in the context of a vaccine clinical trial.

Conclusion

We demonstrate how the CT-FI approach could be useful in other vaccine clinical trials even if they did not plan to evaluate frailty in advance. To our knowledge, this is the first retrospective application of the FI to a large vaccine clinical trial. The potential widespread applicability of this approach is particularly relevant as vaccines targeting older adults are gaining increasing public health prominence and importance; this is the case, for example, with development and testing of COVID-19 vaccines where assessment of the impact of frailty on vaccine efficacy is critical.^41,42

Frailty is increasingly recognized by the public, clinicians, decision-makers and regulators to have important impacts on the efficacy and safety of interventions such as vaccines and drugs. Therefore, it is important to be able to characterize frailty in study populations, either descriptively, or to allow a targeted recruitment of study participants across different levels of frailty. Frail individuals are often under-represented in clinical studies, and even when they have been included their degree of frailty remains unknown. Including frailty measures in clinical trials thus represents an important step forward in understanding how and whether interventions will benefit frail older adults. We have demonstrated that a robust and valid CT-FI can be generated retrospectively or prospectively using data already being collected in many vaccine clinical trials, so it does not add to participant burden or trial complexity.