Assessing effectiveness of aripiprazole lauroxil vs placebo for the treatment of schizophrenia using number needed to treat and number needed to harm

Introduction

Aripiprazole lauroxil (AL) is a long-acting injectable formulation of aripiprazole that is FDA-approved for the treatment of schizophrenia in adults based on the results of a 12-week, randomized, double-blind, placebo-controlled study in more than 600 patients with acute exacerbation of their illness.^1–4 In the pivotal study,¹ AL 441 and 882 mg administered every 4 weeks provided statistically significant and clinically meaningful improvements in schizophrenia symptoms at study endpoint compared with placebo, as measured by Positive and Negative Syndrome Scale (PANSS) total score reductions. In that study, initiation of an AL dosing regimen required oral aripiprazole supplementation for the first 21 days after the initial injection.¹

Although the primary efficacy outcome measure in clinical trials for schizophrenia is commonly reported as the change observed in PANSS total score, a point change on a rating scale can be difficult to clinically interpret and subsequently apply to treatment decision-making. Categorical/binary outcomes such as response vs nonresponse are more intuitive. To more fully understand the clinical relevance of statistically significant results of clinical trials, several metrics are available that quantify absolute effect sizes of a given treatment to better inform health care providers of the benefits and risks of that treatment. Number needed to treat (NNT) and number needed to harm (NNH) are measures of effect size and indicate how many patients would need to be treated with one agent instead of a comparator to encounter one additional outcome of interest.⁵ Lower NNTs are evidenced when there are large differences between the interventions under evaluation. For example, a NNT of 2 would be a very large effect size, as a difference is expected to be encountered after treating just two patients with one intervention vs the other. A NNT of 50 would indicate little difference between the two interventions, as it would require treating 50 patients to expect to observe a difference in the outcome. A negative NNT denotes an advantage for the comparator regarding the potential benefit.

NNH is used when referring to undesirable events associated with treatment. In general, a useful medication is one with a low NNT and a high NNH compared with another intervention. A low NNT and a high NNH would mean an individual patient is more likely to encounter a benefit than a harm. Although these values can vary by treatment indication, a rule of thumb is that single-digit NNTs (i.e., <10) for efficacy measures suggest that the intervention has potentially useful advantages and that double-digit or higher NNHs (i.e., ≥10) for adverse outcomes indicate that the intervention is potentially tolerable.⁵ A negative NNH denotes an advantage for the study medication regarding the potential harm.

The likelihood to be helped or harmed (LHH) is the ratio of NNH to NNT and, in general, a LHH >1 would mean the likelihood to be helped is greater than the likelihood to be harmed. For a LHH <1, the reverse is true. Choosing which NNH and NNT to use in calculating LHH requires careful consideration so that the outcomes being assessed are well matched and consistent with a patient’s values and preferences.^5–7

In contrast to NNT, Cohen’s d⁸ is a metric that defines effect size for continuous (not categorical) variables, such as the PANSS total score. Measured in standard deviation units, Cohen’s d is not clinically intuitive for most practitioners but is commonly described in research reports.

The aim of this post hoc analysis is to use these metrics (i.e., NNT, NNH, LHH, and Cohen’s d) to assess the evidence base supporting the use of AL as a treatment for schizophrenia in order to place this intervention into clinical perspective.

Methods

The study was conducted in accordance with the Declaration of Helsinki, 1964, and Good Clinical Practice principles (International Conference on Harmonisation, 1997). The study protocol, all amendments, and informed consent documents were approved by a qualified institutional review board at each study site (all institutional review boards are listed in Table S1), and all participants completed written informed consent before participating in any study procedures.

Data sources

Data for this post hoc analysis were collected during the randomized, double-blind, placebo-controlled, Phase 3, pivotal study of AL in patients experiencing an acute exacerbation of schizophrenia (Aristada, Alkermes, Inc., Waltham, MA, USA; www.clinicaltrials.gov registration number: NCT01469039).^1–3

Data extraction

Treatment response was evaluated using categorical efficacy outcomes extracted from the dataset using several PANSS thresholds. First, the proportion of patients in each treatment arm meeting the criterion of a ≥20%, ≥30%, ≥40%, or ≥50% reduction from baseline to endpoint on PANSS total score was determined. In addition, the time course of treatment response was evaluated by calculating the proportion of patients in each treatment arm meeting the criterion of a ≥30% reduction on PANSS total score at study days 8, 15, 22, 29, 57, and 85.

For safety and tolerability, adverse events (AEs) occurring at any time during the treatment period were recorded. Other safety and tolerability outcomes included the proportion of patients who discontinued due to an AE and due to an AE other than “schizophrenia” or “psychotic disorder.”

Statistical analysis

NNT and NNH, with the respective 95% CIs, for AL (each dose strength and pooled) vs placebo were computed for each outcome.⁹ When the NNT or NNH estimate was not statistically significant at the P<0.05 threshold (as noted when the 95% CI would contain infinity), the notation “ns” was provided. LHH values were calculated to illustrate potential tradeoffs for efficacy and tolerability outcomes. Formulae used for these calculations are presented below:

• ARI = (incidence on medication) – (incidence on placebo) = ƒ₁–ƒ₂
• The 95% CI was calculated by

• NNT (or NNH) = 1/ARI, and rounded up to the next highest whole number
• The CI for the NNT (or NNH) was calculated by taking the reciprocal of the lower and upper bounds of the CI for the ARI
• LHH = NNH/NNT

Medication effects over time were also assessed using Cohen’s d effect size for the PANSS total score difference of AL vs placebo at study days 8, 15, 22, 29, 57, and 85. Missing PANSS total scores for patients who terminated the study early (before day 85) were imputed using the last observation carried forward.

Results

Study population

Of 623 patients experiencing an acute exacerbation of schizophrenia randomized to placebo, AL 441 mg, or AL 882 mg every 4 weeks, 622 were analyzed for safety (all patients who received at least one dose of IM study drug) and 596 were analyzed for efficacy (all patients who received at least one dose of IM study drug and had at least one primary efficacy assessment after IM study drug). Overall patient characteristics have been described previously.¹ All patients were markedly to severely ill, with mean PANSS total scores 92.6, 92.0, and 93.9 for the AL 441 mg, AL 882 mg, and placebo groups, respectively.¹

Efficacy outcomes

In this post hoc analysis, individual AL doses (441 and 882 mg) evidenced NNT values <10 vs placebo for response, defined as a ≥30% decrease in PANSS total score from baseline (Figure 1; Table S2). When AL results were pooled for both doses, responder rates were 35.3% for AL vs 18.4% for placebo, yielding a NNT of 6 (95% CI: 5–11) at day 85 (Table S2). At the lowest response threshold (i.e., ≥20% reduction in PANSS total score from baseline to endpoint), NNT was more robust (NNT = 4); at higher thresholds of response (≥40% and 50%), smaller effect size estimates for NNT of response (10 and 26, respectively) were calculated vs placebo (Figure 1; Table S2). For the pooled AL dose group, the effect size for responders (i.e., PANSS total score reduction ≥30%) became significant as early as day 22 (Figure 2); NNT remained significant and was more robust at later time points (Figure 2).

Figure 1 PANSS responders (defined by PANSS total score reduction thresholds of 20–50%) with AL 441 mg, 882 mg, or AL doses pooled: NNT and 95% CI vs placebo, at endpoint. Note: *Upper bound 95% CI is 216. Abbreviations: AL, aripiprazole lauroxil; CI, confidence interval; NNT, number needed to treat; ns, not significant at the P<0.05 threshold; PANSS, Positive and Negative Syndrome Scale.

Figure 2 PANSS responders (≥30% reduction from baseline PANSS total score) by study day with AL doses pooled: NNT and 95% CI vs placebo by days on therapy. Abbreviations: AL, aripiprazole lauroxil; CI, confidence interval; NNT, number needed to treat; ns, not significant at the P<0.05 threshold; PANSS, Positive and Negative Syndrome Scale.

Cohen’s d

Effect sizes calculated using Cohen’s d, examining the difference between PANSS total score for AL vs placebo at days 8, 15, 22, 29, 57, and 85, paralleled the magnitude of the effect sizes seen with NNT for PANSS response (Table 1). Pooling the two doses of AL, the Cohen’s d for the PANSS total score change vs placebo at day 85 was 0.61 (95% CI: 0.44–0.79), representing a moderate effect size and being comparable to the NNT of 4 observed for responders defined by a ≥20% reduction in PANSS total score from baseline to endpoint.

Table 1 PANSS total score: change from baseline* and Cohen’s d for AL vs placebo (LOCF)

Tolerability outcomes

Study completion rates were higher for patients taking either dose of AL than for placebo. After pooling both doses of AL, the NNT estimate vs placebo for study completion was 6 (95% CI: 4–11) (Table 2). Examining discontinuation due to AEs, rates were higher among patients randomized to placebo, yielding a NNH estimate of −8 (95% CI: −6 to −15) for the pooled doses of AL vs placebo, indicating an advantage for AL in avoiding this outcome. When the AEs of “schizophrenia” or “psychotic disorder” were excluded from the NNH analysis as reasons for discontinuation, AE-related discontinuation rates remained higher for placebo, but the NNH estimates were no longer statistically significant.

Table 2 Rates and NNT vs placebo for study completion and NNH vs placebo for AE-related discontinuation (safety population)

Akathisia was the only AE with an incidence ≥5% in each AL group and at least twice the rate of placebo, producing a NNH of 14 (95% CI: 9–33) for pooled AL doses vs placebo (Table 3). Among the other AEs with an incidence of ≥2% that occurred more frequently in both AL groups than in the placebo group and resulted in statistically significant estimates of NNH values for pooled doses of AL vs placebo were toothache, blood creatine phosphokinase increased, and weight increased. The AEs of schizophrenia, agitation, and psychotic disorder produced NNH values that were negative and statistically significant (Table 3), suggesting an advantage for the pooled AL dose groups vs placebo in avoiding this outcome.

Table 3 AE rates (Incidence ≥2% in any treatment arm) and NNH vs placebo (safety population)

LHH

A limited number of LHH calculations could be done because when the NNT and/or NNH are not statistically significant, LHH cannot be reliably calculated. In addition, when the NNH is a negative number, LHH is rendered meaningless. Nonetheless, using the NNT for response (≥30% reduction from baseline in PANSS total score) for the pooled doses of AL vs placebo and the NNH for akathisia, the LHH was 14/6 or 2.3. Thus, treatment with AL was 2.3 times more likely to result in a therapeutic response than a complaint of akathisia. A caveat, as noted in the discussion, is that in this study, akathisia was not commonly encountered after the second or third injection. Because the rate of AE-related discontinuations was higher for placebo than for AL, it is not possible to contrast the NNT for response with the NNH for discontinuation due to an AE.

Discussion

Although inferential statistics can help determine whether a given result represents a probable outcome (as opposed to one occurring by chance), effect sizes are essential in helping determine if an outcome is clinically important. An outcome may be statistically significant but clinically irrelevant. NNT (and NNH) is a useful way of assessing clinical relevance because it reflects the magnitude of a therapeutic benefit in “patient units” instead of point changes on a rating scale or a Cohen’s d, which measures effect size in standard deviation units. Kraemer and Kupfer¹⁰ propose that NNTs of 3 (rounded up from 2.3), 4 (rounded up from 3.6), and 9 (rounded up from 8.9) correspond to a Cohen’s d of 0.8, 0.5, and 0.2, respectively, representing effect sizes that are “large,” “medium,” and “small.” In general, the therapeutic effects of AL vs placebo from this post hoc analysis resulted in single-digit NNT values. Effect size differences from placebo generally became weaker as the threshold for response was increased from ≥20% to ≥50%. A meta-analysis of studies of other atypical antipsychotics reported an overall mean response rate for long-acting injectables of 47% vs 24% for placebo (NNT = 4) based on a ≥20% improvement in PANSS total score;¹¹ thus, AL 441 and 882 mg were comparable with respect to this outcome, with an observed NNT of 4 (95% CI: 3–6). Similar analyses have been published regarding other antipsychotics, including paliperidone palmitate,^12,13 iloperidone,¹⁴ cariprazine,¹⁵ lurasidone,¹⁶ and pimavanserin.¹⁷

Individual AE outcome differences for pooled AL dose groups vs placebo generated NNH values in the double digits and produced LHH values that were consistently favorable. Clinically, additional considerations include the time to onset of the AE vs time to onset of a therapeutic response, as well as the severity and duration of the AE. The AE in question may be easily manageable if it is short-lived and not serious. For akathisia, median durations were 13, 15, and 22 days for the placebo, AL 441 mg, and AL 882 mg groups, respectively. Two patients discontinued because of an AE of akathisia, and the majority (>75%) of akathisia events had an onset before the second injection.^1,18 No cases of akathisia occurred in the AL 882 mg group beyond 1 month after the first injection.¹

In contrast, the comparison of pooled AL dose groups vs placebo produced negative NNH estimates for several individual AEs, suggesting that AL was advantageous over placebo for those outcomes. Three were statistically significant, including AEs coded as schizophrenia (NNH = −16), agitation (NNH = −26), and psychotic disorder (NNH = −30). This makes intuitive sense, as AL would be expected to treat the symptoms of schizophrenia. Moreover, in a previous analysis, AL demonstrated decreases in ratings of agitation and hostility, with the antihostility effect being independent of the general antipsychotic effect.¹⁹

A limitation of this analysis is that given the rigid patient selection criteria of a clinical trial, results from this pooled analysis of a 12-week trial may not be generalizable to all patients seen in clinical practice. In addition, the trial was conducted using a fixed-dose design, in contrast to clinical practice, where dose adjustments are generally made based on individual symptom relief and/or the development of tolerability issues. Furthermore, a major limitation of NNT and NNH analyses is that these metrics are limited to dichotomous outcomes. Other effect size measures are necessary when describing continuous outcomes, such as mean changes in PANSS scores or mean changes in fasting plasma glucose levels.²⁰ Cohen’s d addresses this limitation and provides a way to assess effect size of drug-placebo differences for continuous variables.

Conclusion

These descriptive analyses of supportive endpoints using NNT and NNH provide more meaningful clinical information that is simpler to interpret than absolute point changes on clinical rating scales, and they are more intuitive than a comparison of clinically relevant outcomes for AL vs placebo measured via continuous variables (e.g., PANSS total score). The magnitude of the NNT effect sizes for PANSS response paralleled those observed using Cohen’s d when examining change in PANSS total score. Using the metrics of NNT and NNH, AL 441 and 882 mg administered every 4 weeks were efficacious and well-tolerated interventions for the treatment of patients experiencing an acute exacerbation of schizophrenia.