The Clinically Meaningful Score Difference and Meaningful Score Regions in Positive and Negative Syndrome Scale Total Score from Post Hoc Analyses of a Phase 3 Pivotal Clinical Trial of Transdermal Asenapine in Patients with Schizophrenia

Introduction

Schizophrenia is a debilitating chronic condition with considerable morbidity and is a major cause of disability worldwide.¹ The condition is characterized by positive symptoms (eg, hallucinations, delusions) that are generally amenable to treatment with antipsychotic medications, along with negative symptoms (eg, affective flattening, alogia, asociality, avolition) and cognitive impairment, both of which are responsible for long-term morbidity and functional impairment.^2–4 Because of the potentially debilitating symptoms associated with schizophrenia, patients experience psychosocial sequelae and low rates of functional recovery.¹ Additionally, these consequences give rise to high rates of unemployment, homelessness, incarceration, divorce, and social isolation and a decreased overall quality of life.¹

HP-3070 is the first asenapine transdermal system (patch) approved by the US Food and Drug Administration (FDA) in 2009 for once-daily use in adults with schizophrenia.^5,6 In a phase 3 placebo-controlled trial, once-daily applications of HP-3070 7.6 mg/24h or 3.8 mg/24h demonstrated significant improvement compared with placebo in Positive and Negative Syndrome Scale (PANSS) total scores.⁵ Differences from placebo in the least-squares mean (95% CI; adjusted P) change from baseline to Week 6 in PANSS total scores were −4.8 (−8.06 to −1.64; adjusted P=0.003) and −6.6 (−9.81 to −3.40; adjusted P<0.001) for 7.6 mg/24h and 3.8 mg/24h, respectively.⁵ HP-3070 was well tolerated, with a systemic safety profile similar to sublingual asenapine.⁵ In addition, a post hoc analysis demonstrated that HP-3070 was superior to placebo in reducing schizophrenia-associated hostility,⁷ and another post hoc analysis demonstrated efficacy across other domains such as negative symptoms and disorganized thought.⁸

The PANSS is the gold standard for measuring positive, negative, and general psychopathology to assess symptom severity in schizophrenia and is routinely used to determine treatment effect by identifying a mean change in score.^9–11 Changes in PANSS total scores are correlated with improvement in categories on the Clinical Global Impression (CGI) rating scales (CGI-Improvement [CGI-I] and CGI-Severity [CGI-S]), designed to describe a patient’s overall clinical state.^10,12 CGI rating scales have been used previously as anchors to assess the clinical relevance of changes in negative symptoms as measured by the PANSS.¹⁰

Although statistical significance is useful in evaluating treatment efficacy in clinical trials, it is not descriptive of the size of a treatment effect or its potential clinical relevance.^10,13 To aid in determining a magnitude of score change that would equate to a clinically meaningful improvement for patients, the FDA has established recommendations for analyzing clinical outcome data in their 2023 FDA Patient-Focused Drug Development (PFDD) Guidance.^14,15 This FDA guidance defines meaningful score difference (MSD) as the difference between two clinical outcome assessment scores that would be viewed as clinically meaningful from a patient’s perspective.¹⁵ Meaningful score regions (MSRs), which are separated into regions by severity of disease at baseline, determine the regions of meaningful treatment effect that correspond to the patients’ experience.¹⁵ Determining the MSD and MSR may help clinicians better assess treatment results as measured by the PANSS.^10,15 Although the PANSS is not a patient-reported outcome, identifying the magnitude of PANSS total score change that corresponds to a clinically meaningful outcome would be relevant to patients and their care.

This post hoc analysis of the pivotal phase 3 HP-3070 clinical trial aimed to determine the MSD in PANSS total score and the MSRs in PANSS total score by the severity of the disease at baseline as assessed by CGI-S.

Materials and Methods

Study Design

Data were analyzed from a pivotal, randomized, phase 3, multicenter, placebo-controlled, double-blind study conducted from August 2016 to November 2017 in patients with schizophrenia, the principal results of which have been published (NCT02876900, registered on August 3, 2016; study details: https://clinicaltrials.gov/study/NCT02876900).⁵ Briefly, the study consisted of a 3- to 14-day screening/run-in period; a 6-week double-blind treatment period where patients were randomized (1:1:1) to HP-3070 7.6 mg/24h, HP-3070 3.8 mg/24h, or placebo; and a 30-day follow-up.⁵

The study assessed the efficacy and safety of HP-3070 transdermal patches (Hisamitsu Pharmaceutical Co., Inc., Tokyo, Japan) in adults with schizophrenia experiencing acute exacerbations at the time of screening.^5,6 The primary efficacy endpoint was change from baseline to Week 6 in PANSS total score.⁵ The key secondary endpoint was change from baseline in Week 6 CGI-S score.⁵ Other secondary endpoints included weekly change in PANSS total score and CGI-S score, weekly CGI-I score, and weekly proportion of PANSS and CGI-I responders.⁵ PANSS responders were defined as patients with ≥30% reduction in PANSS total score.⁵ The percentage change in the PANSS total score was calculated based on an unadjusted approach (without subtracting 30 from all scores) used in the phase 3 study. CGI-I responders had a score of 1 (very much improved) or 2 (much improved). The intention-to-treat (ITT) analysis set included all consented and randomized patients.

The primary efficacy variable was analyzed using a mixed-model repeated-measures analysis utilizing all data available during the double-blind treatment period (complete and partial). The PANSS total score was calculated via the sum of all 30 items; if one or more items were missing at a given assessment, the total score was noted as missing. Missing PANSS total scores were not imputed for the primary analysis. PANSS scores obtained 24 hours after removal of the double-blind study medication on Day 42 or PANSS scores obtained >24 hours after discontinuation of the double-blind study medication where the patient switched to an approved antipsychotic treatment were excluded. For the CGI-I score calculation, numerical values 1 through 7 corresponding to each CGI-I response category were used, and the “not assessed” category was treated as a missing numerical value.

Patients

Eligible study participants were aged ≥18 years, were experiencing an acute exacerbation of schizophrenia, had a baseline CGI-S score of ≥4 (moderately, markedly, or severely ill), and had a PANSS total score of ≥80, with scores of ≥4 in ≥2 of the PANSS items for conceptual disorganization, delusions, hallucinatory behavior, and unusual thought content at screening and baseline.⁵ Participants were excluded if they had been diagnosed with schizophrenia <6 months prior to the screening visit, had a current DSM-5 diagnosis other than schizophrenia, or were known to be resistant/refractory to antipsychotic treatment, including asenapine.⁵ The study was approved by an independent ethics committee or institutional review board at each study center and was performed in compliance with the International Council for Harmonisation Good Clinical Practice Guidelines and adheres to the ethical principles of the Declaration of Helsinki (Supplementary Table 1). All patients provided written informed consent.

MSD and MSR Analysis

The data available at baseline and Week 6 (without imputation) were included in MSD/MSR analysis: PANSS total score and CGI-S at baseline, CGI-I and CGI-S collected at Week 6, and change and percentage change in PANSS total score at Week 6. The PANSS total score comprises 30 items scored from 1 to 7, with 1 indicating “absent” (ie, no symptom present for the item), making the lowest possible PANSS total score 30, which can distort calculations of percentage change.¹⁶ Accordingly, some statistical literature suggests adjusting PANSS total scores by subtracting 30 points from all scores before obtaining the percentage change.¹⁶

The CGI-I was used as an anchor to quantify improvement in the PANSS total score. Responders were defined as those with CGI-I=1, 2 (“very much improved” and “much improved”, respectively). Data for those with CGI-I=1, 2, 3 (“very much improved”, “much improved”, and “minimally improved”, respectively) are reported in the supplementary section (Supplementary Table 2). The correlation of the PANSS total score with CGI scores was evaluated via Pearson and Spearman correlation coefficients and P values. Unadjusted and adjusted approaches were used.^16,17 Robust correlations were used to confirm the confidence in translating differences in the anchors to differences in PANSS total scores. Relationships between the PANSS total scores and CGI-S at baseline and Week 6 were evaluated and presented via a scatterplot matrix.

MSD and MSRs in PANSS total score were determined at Week 6 via the unadjusted approach, which was applied in the parent study.⁵ Anchor-based methods were supplemented with empirical cumulative distribution function curves to establish a threshold—or a range of thresholds—that would constitute a meaningful within-patient change in PANSS total score from a CGI-I perspective. The following were identified:

• MSD: the change and percentage change (50^th percentile) in PANSS total score from baseline to Week 6 for at least 50% of responders (CGI-I=1, 2).
• MSRs: the change and percentage change (25th and 75th percentiles) in PANSS score from baseline to Week 6 by baseline severity CGI-S (CGI-S=4, defined as “moderately ill”; CGI-S=5, defined as “markedly ill”; and CGI-S=6, defined as “severely ill”) for responders (CGI-I=1, 2).

The percentage change in PANSS total score was also calculated via the adjusted approach.

Results

Demographics and baseline characteristics are shown in Table 1. A total of 616 patients from the ITT analysis set (HP-3070 7.6 mg/24h, HP-3070 3.8 mg/24h, placebo) were evaluated in this post hoc analysis. Mean (standard deviation [SD]) age was 42.0 (11.7) years, 61% of patients were male, and 76% were White. At baseline, mean (SD) PANSS total score was 96.6 (9.5). After 6 weeks of treatment, mean (SD) change from baseline in PANSS total score versus placebo was −21.3 (14.4), translating to a mean (SD) percentage change (unadjusted) of −22.1% (15%) and (adjusted) of −32.4% (22%) (Table 2). Mean (SD) CGI-S score at baseline was 4.9 (0.5) and mean (SD) CGI-S score at Week 6 was 3.8 (1.0). Mean (SD) CGI-I score at Week 6 was 2.6 (1.0) (Table 2).

Table 1 Demographic and Baseline Characteristics (ITT Analysis Set)a

Table 2 CGI-S, CGI-I, and PANSS Total Score Week 6 Characteristicsa

Positive correlations of PANSS total score (unadjusted) with CGI-S at baseline and both CGI-I and CGI-S at Week 6 were observed (Supplementary Figure 1). According to the Pearson correlation coefficients, PANSS total score positively correlated with CGI-S score at baseline (r = 0.53, P < 0.0001), and both change from baseline (r = 0.81, P < 0.0001) and percentage change from baseline (r = 0.82, P < 0.0001) in PANSS total score (unadjusted) at Week 6 positively correlated with CGI-I score (Supplementary Table 3). Similar results were observed based on Spearman correlation coefficients; CGI-S score at baseline (r = 0.50, P < 0.0001) and change from baseline (r = 0.80, P < 0.0001) and percentage change from baseline (r = 0.82, P < 0.0001) in PANSS total score (unadjusted) at Week 6 positively correlated with CGI-I score. As demonstrated using Pearson correlation coefficients, PANSS total score positively correlated with CGI-S score at baseline (r = 0.53, P < 0.0001), and both change from baseline (r = 0.81, P<0.0001) and percentage change from baseline (r = 0.82, P < 0.0001) in PANSS total score (adjusted) at Week 6 positively correlated with CGI-I score. Similar results were observed based on Spearman correlation coefficients: PANSS total score was positively correlated with CGI-S score at baseline (r = 0.50, P < 0.0001), and change from baseline (r = 0.80, P < 0.0001) and percentage change from baseline (r = 0.82, P < 0.0001) in PANSS total score (adjusted) at Week 6 positively correlated with CGI-I score. MSD and MSRs in change from baseline to Week 6 in PANSS total scores overall and by baseline severity (defined by CGI-S) for responders are shown in Table 3 and Figure 1. The mean MSD in change from baseline to Week 6 in PANSS total score for at least 50% of responders (CGI-I = 1, 2) was −30. The mean MSRs of change from baseline to Week 6 in PANSS total score for those who were identified as moderately ill, markedly ill, or severely ill at baseline were [−36, −23], [−37, −24], and [−44, −27], respectively. MSD and MSRs in change from baseline to Week 6 in PANSS total scores overall and by baseline severity (defined by CGI-S) for responders with CGI-I scores of 1, 2, or 3 are shown in Supplementary Table 4 and Supplementary Figure 2.

Table 3 MSD and MSRs in Change From Baseline to Week 6 in PANSS Total Score by CGI-S at Baseline for Responders

Figure 1 Change in PANSS total score from baseline to Week 6 for respondersa (A) overall and (B) by baseline CGI-S. aResponders are defined as CGI-I scores of 1 or 2. Abbreviations: CGI-I, Clinician-Rated Clinical Global Impression-Improvement; CGI-S, Clinician-Rated Clinical Global Impression-Severity; PANSS, Positive and Negative Syndrome Scale.

Using the unadjusted approach, the MSD and MSRs of percentage change from baseline to Week 6 in PANSS total scores overall and by baseline severity (defined by CGI-S) for responders were obtained and are shown in Table 4 and Figure 2. The mean MSD in percentage change from baseline to Week 6 in PANSS total score for at least 50% of responders (CGI-I=1, 2) was −32%. The mean MSRs of percentage change from baseline to Week 6 in PANSS total score for those who were identified as moderately ill, markedly ill, or severely ill at baseline were [−39%, −25%]; [−39%, −25%]; and [−41%, −25%], respectively. Meaningful regions of change and percentage change from baseline to Week 6 in PANSS total score corresponding to severity at baseline for responders (CGI-I = 1, 2) are shown in Figure 3. MSD and MSRs in change from baseline to Week 6 in PANSS total scores overall and by baseline severity (defined by CGI-S) for responders with CGI-I scores of 1, 2, or 3 are shown in Supplementary Table 5 and Supplementary Figure 3. Meaningful regions of change and percentage change from baseline to Week 6 in PANSS total score corresponding to severity at baseline for responders (CGI-I = 1, 2, 3) are shown in Supplementary Figure 4.

Figure 2 Percentage change in PANSS total score (unadjusted) from baseline to Week 6 for respondersa (A) overall and (B) by baseline CGI-S. aResponders are defined as CGI-I scores of 1 or 2. Abbreviations: CGI-I, Clinician-Rated Clinical Global Impression-Improvement; CGI-S, Clinician-Rated Clinical Global Impression-Severity; PANSS, Positive and Negative Syndrome Scale.

Figure 3 Meaningful regions of change and percentage change (unadjusted) in PANSS total score corresponding to severity at baseline for responders (CGI-I =1, 2). Box and whisker plots display the 25th (left edge of box), 50th (line inside box), and 75th (right edge of box) percentiles of the PANSS score distributions corresponding to each CGI-S level. Whiskers indicate scores ± 1.5×interquartile range. Means are represented by dots. Abbreviations: CGI-I, Clinician-Rated Clinical Global Impression-Improvement; CGI-S, Clinician-Rated Clinical Global Impression-Severity; PANSS, Positive and Negative Syndrome Scale.

Using the adjusted approach, MSD and MSRs of percentage change from baseline to Week 6 in PANSS total score overall and by baseline severity (defined by CGI-S) for responders (CGI-I = 1, 2) were obtained and are shown in Supplementary Table 6. MSD and MSRs of percentage change (adjusted) from baseline to Week 6 in PANSS total score overall and by baseline severity (defined by CGI-S) for responders (CGI-I = 1, 2, 3) are shown in Supplementary Table 7. The mean MSD in percentage change (adjusted) from baseline to Week 6 in PANSS total score for at least 50% of responders (CGI-I=1, 2) was −46%. The mean MSRs of percentage change (adjusted) from baseline to Week 6 in PANSS total score for those who were identified as moderately ill, markedly ill, or severely ill at baseline were [−58%, −39%]; [−56%, −37%]; and [−55%, −34%], respectively. Meaningful regions of percentage change (adjusted) from baseline to Week 6 in PANSS total score corresponding to severity at baseline for responders (CGI-I = 1, 2) are shown in Supplementary Figure 5.

Table 4 MSD and MSRs of Percentage Change From Baseline to Week 6 in PANSS Total Score (Unadjusted) by CGI-S at Baseline for Responders

Discussion

The current analysis estimated the MSD and MSRs based on the 2023 FDA PFDD guidance¹⁵ in order to determine what PANSS score changes were clinically meaningful in people with schizophrenia. The estimate from the MSD analysis indicated a clinically MSD of −32% in PANSS total score from baseline to Week 6. The estimate from the MSR analysis indicated clinically MSRs of [−39%, −25%]; [−39%, −25%]; and [−41%, −25%] in percentage change in PANSS total score from baseline to Week 6 for those with a CGI-I=4, 5, 6 at baseline, respectively. In clinical practice, a change of −25% to −41.5% in PANSS total score constitutes a clinically meaningful improvement in score, which could represent more of an ideal change than a minimal difference/range.

Utilizing 5 different approaches, a previous study identified the minimum clinically important percentage change in PANSS score as ranging from 7.3% to 18.5% in those with predominantly negative schizophrenia symptoms.¹⁰ A previous publication established that minimal improvement (CGI-I=3) was associated with a 27% decrease in the PANSS score for those with predominantly negative symptoms.^10,18 While the MSRs identified in this study are somewhat higher than previously reported, these estimates are based on PANSS total score among patients with an acute exacerbation, which shows a substantial resolution after the acute phase subsides. The prior published estimates focused on a target population of patients with primarily stable negative symptoms that are unlikely to display a large change in symptom severity over time.

At present, there is variability in the literature regarding the use of an adjusted or unadjusted approach for calculating the percentage change in PANSS total score in schizophrenia trials.^{9,12,16,17,19–21} Because the PANSS total score comprises 30 items for which the lowest possible score is 1 (“absent”), the lowest possible PANSS total score is 30, which can distort calculations of percentage change.¹⁶ Consequently, using the unadjusted PANSS total scores to calculate percentage change may underestimate the true treatment effects and provide a lower estimate of percentage change, especially in studies with low baseline severity.^16,17 Thus, use of the adjusted approach has been suggested.^16,17

Linking scores from a psychometric scale such as PANSS total score with CGI can assist in the interpretation of clinical trial results, including justification for the definition of response as a decrease of 20% or 30% from baseline in PANSS total score, which otherwise appears arbitrary. The methodology of calculating MSD and MSRs provides clinicians with an estimate of the expected treatment effect in people with schizophrenia and can aid in their evaluation of the effectiveness of HP-3070 in their patients. When examining groups of patients participating in randomized controlled clinical trials, these estimates can be used as a threshold to identify individual patients with clinically meaningful improvement. For example, an MSD estimate of 30% is often used as a threshold to define responders in studies of patients with acute exacerbations of schizophrenia.^10,22 This definition of response serves as a basis of the calculation of clinician-friendly effect sizes such as number needed to treat.²³ Studies that report outcomes based on lower thresholds of improvement may not provide as meaningful information regarding clinical relevance.¹⁰

Limitations of this post hoc analysis are that the patients in the pivotal phase 3 HP-3070 clinical trial were required to have an acute exacerbation of schizophrenia and that patients with significant comorbidities, including substance abuse, were excluded. Another limitation is that the current MSD and MSR estimates obtained were based on CGI scores evaluated from the clinicians’ judgement and not from patients’ perspectives. In addition, the calculation of MSD and MSR produces dichotomous outcomes (either meeting the threshold or not), which differs from calculating a continuous effect size such as standardized mean difference (ie, Cohen’s d) that can offer more precision. Patient-related outcomes could improve the meaningfulness of these data by more directly addressing the score changes that would translate to meaningful results for patients.²⁴ Additionally, the MSR definition included in this analysis, which follows the April 2023 FDA guidance,¹⁵ establishes the MSR by merging the estimates from the 4 different vantage points (raw score vs percentage change combined with the different criteria for improvement). Although this approach offers a more comprehensive perspective on the data, it may introduce conceptual challenges, broaden the range of MSR estimates, and ultimately increase the heterogeneity of the results. However, although imperfect, this approach is a step forward in the interpretation of the meaningfulness of clinical trial results. The MSD and MSR estimates reported here can help clinicians interpret individual PANSS changes in routine practice and may support the specification of responder thresholds in future trials.

Conclusion

A change of −32% (unadjusted) and −46% (adjusted) was identified as the clinically MSD of percentage change in PANSS score from baseline to Week 6. Previous studies have established the efficacy and safety of HP-3070 treatment in patients with schizophrenia.^5,7,8 This post hoc analysis adds to the current body of evidence on HP-3070, contributing to the ongoing discussion of what constitutes a clinically meaningful response in patients with an acute episode of schizophrenia.