Hummingbird Study: Results from an Exploratory Trial Assessing the Performance and Acceptance of a Digital Medicine System in Adults with Schizophrenia, Schizoaffective Disorder, or First-Episode Psychosis

Introduction

Schizophrenia is a psychotic disorder that presents global health challenges and is among the leading causes of disability.^1,2 The global prevalence of schizophrenia has increased from approximately 13 million people in 1990 to over 20 million in 2016.³ In Europe, the economic burden of psychotic disorders was estimated to be €93.9 billion in 2010.⁴ Schizophrenia, schizoaffective disorder, and first-episode psychosis are similarly presenting mental illnesses.⁵ Schizophrenia consists of psychosis, which can present as multiple symptoms, such as delusions, hallucinations, disorganized behavior, and a lack of awareness of disease.⁵ Schizoaffective disorder involves schizophrenia symptoms in the presence of major depressive episodes, with or without bipolar mania.⁵ First-episode psychosis refers to a period after an initial psychotic episode and recovery.⁵ The therapeutic goals for schizophrenia and other psychotic disorders include symptom control and relapse prevention.^1,6 Antipsychotics, such as aripiprazole, olanzapine, quetiapine, and risperidone, are considered the first-line pharmacotherapy for these psychotic disorders.^1,6

Adherence to therapy is a major challenge in patients with psychotic disorders.⁷ An estimated 41–50% of patients with schizophrenia, and approximately 55% of patients with first-episode psychosis, are not fully adherent with their medication.^7,8 Psychotic symptoms, such as unawareness of illness, poor memory, depression, paranoid delusions, and hallucinations, could underlie nonadherence in these patients.^1,7 Poor adherence to antipsychotics has been associated with higher rates of violence, hospital admission, substance abuse, and an increased risk of mortality.^9,10 Many tools are used to measure adherence, such as self-reporting, direct visualization, biomarkers and metabolites, pharmacy prescription data, and medication event-monitoring systems.¹⁰ Electronic adherence monitoring (EAM), which measures when medication is accessed, has been considered the gold standard of adherence measurement, and a recent meta-analysis found that EAM-measured adherence to oral antipsychotics was approximately 70%.¹¹ These adherence tools, however, may be associated with disadvantages.^12,13 For example, self-reporting and observational methods may be affected by patients reporting incorrect information or tampering with an adherence device (either purposefully or by accident), and measuring drug metabolism through assays can be expensive and intrusive.^12,13

The digital medicine system (DMS) measures ingestion of a medication using an ingestible sensor that is either embedded within a tablet^14,15 or embedded within an inert tablet and co-encapsulated with an oral medication tablet.¹⁶ After ingestion of the tablet, the sensor is activated by stomach fluid and sends a signal to a wearable sensor patch attached to the patient’s torso.¹⁶ A smartphone application collects these data from the patch via Bluetooth and transfers them to a secure digital health data server.¹⁶ The data can be accessed by patients through the smartphone application or by healthcare providers and caregivers through separate web portals.¹⁶ The DMS is intended to encourage greater self-management of mental illness by helping the patient and their healthcare provider track their medication-taking behaviors to support informed treatment discussions.¹⁶

Incorporating smartphones in adherence management offers new opportunities and benefits—most patients with psychosis who own mobile phones use them daily, and smartphone ownership, while lower among those with psychosis than in the general population, is also growing.^17–19 Although studies have shown that patients with schizophrenia may not have negative experiences with mobile devices,²⁰ data collection could present a barrier for patients with paranoia or who are uncomfortable with this information being shared.¹⁷ Data collection also introduces numerous complex ethical issues regarding the privacy and security of patient information, which must be addressed to ensure patient and public trust in digital medicine.¹⁷ To protect patient privacy, the DMS is patient-centric, and allows specific authorization as to which healthcare providers, caregivers, and family members can access the information collected. The DMS uses industry-standard encryption protocols for data transmission from the patch to the mobile device, and from devices to the server; DMS data can be accessed only through the sponsor’s servers, which are also protected with industry-standard security features.

In previous Phase 2 clinical studies, the DMS was evaluated in the United States (US) in subjects with schizophrenia, bipolar I disorder, and major depressive disorder who were considered clinically stable and were receiving oral aripiprazole.^21,22 The first DMS used aripiprazole paired with a sensor and was approved in the US as monotherapy for schizophrenia or bipolar 1 disorder, and as an adjunctive treatment for major depressive disorder.²³ The DMS in this study used ingestible and wearable sensors certified by the British Standards Institute for use in Europe as class IIa medical devices (Conformité Européenne #559,373).²⁴

In this phase 4 pragmatic study (the Hummingbird Study), we evaluated the clinical utility of the DMS in patients diagnosed with schizophrenia, schizoaffective disorder, or first-episode psychosis potentially presenting with acute mental illness, who were receiving oral aripiprazole, olanzapine, quetiapine, or risperidone, and who also had a smartphone to use in the study. This study was performed using the National Health Service (NHS) Mental Health Trusts in the United Kingdom (UK), which presented an opportunity to evaluate the DMS in a different treatment system outside of the US.

Methods

Study Design and Participants

This was an 8-week open-label, single-arm, multicenter, phase 4 pragmatic study performed at 5 NHS Foundation Trusts in the UK. The study was conducted in accordance with local laws, the International Conference on Harmonization Good Clinical Practice Consolidated Guideline, and the Declaration of Helsinki. The study protocol was approved by the London – City and East Research Ethics Committee and was registered with ClinicalTrials.gov (NCT 03568500) and with EudraCT (#2017-004602-17). All subjects provided written informed consent prior to participating in the study.

Subjects for this study were identified using database searches conducted at each study site per provider discretion, from a range of clinical populations, including those being treated by acute-care teams or by community services. Key eligibility criteria included subjects being aged between 18 to 65 years; having a diagnosis of schizophrenia, schizoaffective disorder, or first-episode psychosis; and having a prescription for oral aripiprazole, olanzapine, quetiapine, or risperidone. Eligible subjects were able to complete onboarding and use the DMS by downloading and using the application on their personal smartphone with internet connectivity. Subjects were excluded from the study if they had an intellectual developmental delay or disorder, major neurocognitive disorder, or another condition that might impact the subject’s ability to participate in the trial or interact with the DMS, or if they were advised to not participate in the trial per their healthcare provider’s judgement. Full eligibility criteria and the study design have been previously published.¹⁶

Procedures

Subjects were screened for up to 1 week, followed by an 8-week assessment period while using the DMS (Appendix s-Figure-1). A safety follow-up telephone call was performed 2 weeks after the last visit. The DMS included a drug–device combination of the patient’s prescribed oral antipsychotic medication tablet co-encapsulated with an ingestible event marker (IEM) embedded in an inert tablet (Proteus Digital Health Inc., Redwood City, CA), a sensor patch (Disposable Wearable Sensor version 5 [DW-5], Proteus Digital Health Inc., Redwood City, CA) that was applied to the torso to detect ingestion of the antipsychotic, and a smartphone application for subjects with a web portal for healthcare providers (Otsuka Medical software version 2.1; Otsuka Pharmaceutical Development and Commercialization, Inc., Princeton, NJ). This study was conducted using both Android and iOS versions of the smartphone application. An integrated call center was available to answer any technical questions on the use of the DMS from the study subjects and participating investigators or site staff.

Outcomes

The primary endpoint was the mean proportion of days with good patch coverage. Good patch coverage was defined as either having the patch worn with data collected for 80.0% or greater of the time or having an IEM detected within a given day of the assessment period. The primary endpoint was calculated per subject as days of good patch coverage divided by all assessment days (from first drug intake date until the last drug intake date for each subject). Each subject’s coverage value was then averaged to calculate a mean for each of the 3 diagnostic categories, as well as for the overall study population. The secondary endpoint of adherence metric was calculated as the mean proportion of each subject’s number of detected IEMs divided by the expected number of IEMs on assessment days of good patch coverage.

Exploratory endpoints related to device performance were the mean proportion of days that subjects wore the patch during the assessment period, and the mean number of IEMs registered on the digital health data server divided by the expected number of IEMs during the assessment period. Exploratory endpoints related to acceptance were responses to the Subject Usability and Satisfaction Scale on week 8. Other exploratory endpoints assessed included changes from baseline to week 8 in Patient Activation Measure-Mental Health (PAM-MH)²⁵ and Clinical Global Impression-Severity of Illness Scale (CGI-S)²⁶ scores as measured in subjects with baseline and postbaseline scores. The number of times per day that subjects used the application and the number of times per week that healthcare providers used the web portal were also captured.

Safety was assessed as the incidence of adverse events (AEs), treatment-emergent adverse events (TEAEs), serious AEs, and reports of suicide or suicidal ideation. All AEs were classified using preferred terms of the standardized Medical Dictionary for Regulatory Activities version 22.0. TEAEs of medical device (patch) site irritation were graded by the healthcare provider using the Skin Irritation Scoring System (grades 0–7).²⁷

Statistical Analyses

As this was a feasibility study, no statistical comparisons or power calculations were performed. A sample size of 60 subjects was planned for this study with an anticipated discontinuation rate of 25.0%. Performance and acceptance analyses were assessed in the intent-to-treat population, which was defined as all subjects who entered the trial and used the DMS. Safety analyses were performed in the safety population, which was defined similarly to the intent-to-treat population. Descriptive statistics were used for all endpoints. Continuous variables were summarized by means, medians, or ranges, and relevant quartiles, standard deviations (SD), or standard errors of the mean. Categorical variables were summarized using frequency distributions. No imputation was performed for other missing data, unless specified otherwise.

Role of the Funding Source

All aspects of the trial (design; data collection, analysis, and interpretation; writing the report; and decision to submit the paper for publication) were managed by the sponsor, including oversight of the contract research organization. The corresponding author (JCF) had full access to all data in the study and had final responsibility for the decision to submit for publication.

Results

Subjects

From May 25, 2018 to March 22, 2019, 55 subjects were screened and 44 were enrolled in the trial, of whom 54.5% (24/44) completed the trial (Figure 1). One subject discontinued from the trial without ingesting any study medication. Most subjects had a diagnosis of schizophrenia (40.9%; 18/44), followed by first-episode psychosis (36.4%; 16/44), and schizoaffective disorder (22.7%; 10/44; Table 1). All but 1 of the subjects enrolled received antipsychotic medication (Table 1). Overall, subjects used the DMS for 1760 days (aripiprazole group: 711 days; olanzapine group: 865 days; quetiapine group: 184 days [no subjects enrolled used risperidone]). Of the 20 subjects who discontinued the trial early, 65.0% (13/20) used the Android version of the application, 15.0% (3/20) used the iOS version, and 20.0% (4/20) did not report the version. Further, of those who discontinued the trial early, 85.0% (17/20) contacted the call center (a total of 100 calls) for assistance with patch-related issues. Six subjects withdrew consent; of these, one felt uncomfortable wearing the patch and taking medication, and another found it difficult to cope with the technology, reporting that it made them feel “anxious.”

Figure 1 Subject disposition. aOf the subjects enrolled, 43 received a treatment and were included in the intent-to-treat and safety sample analyses.

Table 1 Subject Demographics and Baseline Characteristics

Performance

For the 8-week assessment period, the mean proportion of days with good patch coverage was 63.4% (SD: 26.6) for the overall population (Table 2). No noticeable difference was observed between psychiatric conditions with respect to the proportion of time with good patch coverage over the assessment period (Table 2). A decline in the proportion of time with good patch coverage from weeks 1–4, then followed by an increase from weeks 4–5, was observed in all groups over the assessment period, except for subjects with schizoaffective disorder. Subjects with schizoaffective disorder exhibited a decrease in time with good patch coverage during weeks 1–2, followed by a rebound during weeks 2–5 to a greater value than for the first week (Figure 2). A decline in time with good patch coverage was observed over weeks 5–8 in all groups regardless of psychiatric condition (Figure 2). For the overall population, the mean proportion of days over the 8-week assessment period that subjects wore the patch was 55.1% (SD: 27.8) (Table 2). No noticeable differences in the proportion of time that subjects wore the patch were observed between psychiatric conditions (Table 2).

Table 2 Summary of Digital Medicine System Performance Endpoints Related to Patch Wear (Intent-to-Treat Population)

Figure 2 Proportion of time with good patch coverage over the assessment period (intent-to-treat population). Data are means ± standard error.Note: Arrow represents scheduled provider visit during week 4.

The proportion of IEMs ingested and registered on the digital health data server over the 8-week assessment period was 56.4% (SD: 25.6) for the overall population (Table 3). For subjects with schizophrenia or with first-episode psychosis, the proportions of IEMs registered on the digital health data server were comparable (Table 3). Subjects with schizoaffective disorder had the lowest proportion of IEMs registered (Table 3). The mean adherence metric for the 8-week assessment period was 86.6% (SD: 14.5) for the overall population (Table 3). Subjects with schizophrenia and first-episode psychosis had relatively consistent adherence metrics over the assessment period (Figure 3). Subjects with schizoaffective disorder, however, had a lower mean and higher variability in adherence metrics compared with other psychiatric conditions over the assessment period (Table 3; Figure 3).

Table 3 Summary of Digital Medicine System Endpoints Based on IEM Detection (Intent-to-Treat Population)

Figure 3 Adherence metric over the assessment period (intent-to-treat population). aOne subject in the schizoaffective disorder group was excluded because they had 0 days of good patch coverage. Data are means ± standard error. Arrow represents scheduled provider visit during week 4.

Acceptance

When surveyed on their experience of the DMS, 53.5% (23/43) of subjects were somewhat satisfied, satisfied, or extremely satisfied with the DMS, and 51.2% (22/43) of subjects found the DMS somewhat helpful, helpful, or extremely helpful in managing their condition (Appendix s-Table 1). Proportions of subjects who reported it somewhat easy, easy, or extremely easy to use the DMS: 69.8% (30/43); to apply the patch, 67.4% (29/43); and to use the mobile application, 60.5% (26/43; Appendix s-Table 1).

When asked about the patch, 60.5% (26/43) of subjects somewhat agreed, agreed, or strongly agreed with not minding the patch, and 58.1% (25/43) of subjects needed assistance in changing and pairing patches (Appendix s-Table 1). An insignificant decrease in mean PAM-MH scores from baseline (64.5) to week 8 or early termination (63.6) was observed. No change in the mean CGI-S score from baseline (2.6) to week 8 or early termination (2.6) was observed. Overall, subjects used the smartphone application 0.5 times per day (SD: 0.3), and healthcare providers accessed the web portal 1.8 times per week (SD: 0.8).

Data from the call center indicated that 11 subjects telephoned 34 times for patch-related issues, and 9 times for issues related to smartphone application usage. Further, research or clinical staff telephoned the call center 23 times for patch-related issues during onsite visits for subjects in need of assistance with the system.

Safety

Over the 8-week assessment period, AEs were reported in 9 subjects (Table 4). No deaths, serious AEs, or AEs related to suicide or suicidal ideation were reported during the study (Table 4). TEAEs were reported in 9 subjects, all of which were associated with medical device site irritation due to the adhesive patch (Table 4). Most of these TEAEs were considered mild in severity. One TEAE was considered of moderate intensity and resolved after discontinuation of the DMS. Four subjects discontinued the study due to a TEAE (Table 4). Most (88.9% [8/9]) patch-related skin irritation scores were from 0–2, which were not considered medically significant (Table 5).

Table 4 Summary of Safety (Safety Population)

Table 5 Skin Irritation Scoring System for Patch-Related Skin TEAEs (Safety Population)

Discussion

In this phase 4 pragmatic study in subjects with schizophrenia, schizoaffective disorder, or first-episode psychosis who were receiving aripiprazole, olanzapine, or quetiapine, good patch coverage with the DMS was reported 63.4% of the time over the 8-week assessment period. Overall, the DMS generated an adherence metric of 80.0% or greater, reflecting the percentage of IEMs detected that would be expected when good patch coverage was reported. The DMS was generally well-tolerated in this study population and no new safety findings were reported. The incidence of TEAEs (20.9%), which were all associated with patch-related skin reactions, was consistent with reports from previous trials with the DMS (32.8%²¹ and 34.7%²²). Together, these results support the use of the DMS in a broader population of patients and with different types of antipsychotics than seen in previous clinical trials.^21,22 Specifically, this study included patients with schizoaffective disorder or first-episode psychosis and those who may have presented with acute illness. Moreover, these findings suggest that the DMS could be successfully used with aripiprazole and other oral antipsychotics, such as olanzapine and quetiapine.

The proportion of time that subjects had good patch coverage in this study (63.4%) was lower compared with a previous 8-week phase 2 trial with the DMS (80.1%).²² Moreover, a lower proportion of time spent wearing the patch was reported in this study (55.1%) compared with previous phase 2 trials with the DMS (70.7%²¹ and 77.9%²²). Although the proportion of time with good patch coverage appeared to decline from baseline to week 8, an increase between weeks 4–5 was noticed in all subjects, regardless of psychiatric condition. This increase may have been due to the scheduled interaction with a healthcare provider at week 4, which suggests that healthcare providers could increase patient engagement during visits, possibly by reviewing ingestion data of their patients using the web portal. A possible explanation for why less patch wear was reported in this study compared with previous reports may be related to issues with pairing the patch and smartphone application, as indicated by the number of calls from subjects and healthcare providers to the call center.

To our knowledge, this was the first study to evaluate the DMS where subjects could use their own smartphone, which included the iOS and Android application platforms. Compatibility of the DMS with various smartphones across diverse subject demographics may have resulted in user error and affected subjects’ ability to successfully engage with the smartphone application and pair patches. Additionally, this study had fewer scheduled interactions or touchpoints with healthcare providers (3 interactions) compared with previous clinical trials with the DMS (4 to 5 interactions).^21,22 This study also included subjects with acute mental illness, whereas previous trials recruited only subjects who were already on stable antipsychotic doses and did not have acute psychotic symptoms.^21,22 These features of the study design provided a more real-world setting to evaluate performance and acceptance of the DMS. Issues with pairing patches with the smartphone application, the fewer number of touchpoints, and the inclusion of subjects with acute mental illness may have negatively influenced patch coverage metrics in this study.

In this study, the DMS showed good acceptance in the overall subject population, as evidenced by responses to the user-experience survey. Overall, subjects were satisfied with using the DMS and generally found its components easy to use. However, less than half of subjects found it easy to pair the patch with the smartphone application, and over half of subjects needed help with changing or pairing patches. This may indicate a lack of familiarity with pairing Bluetooth devices, or that compatibility issues between the patch and smartphone application were encountered. Acknowledging the limitations of cross-study comparisons, the proportion of subjects in this study who were satisfied with the DMS (53.5%) and found it helpful in managing their condition (51.2%) appeared to be lower compared with a previous phase 2 study in subjects with schizophrenia who were stable on medication (78.3% and 70.0%, respectively).²¹ These findings may be explained by similar factors that affected patch coverage. Specifically, having fewer scheduled interactions, the inclusion of subjects with acute mental illness, and issues related to pairing the patch to the smartphone application may explain why fewer subjects were satisfied with the DMS in this study compared with prior reports.

The proportion of IEMs detected (56.4%) and the adherence metric (86.6%) in this study were similar to findings in previous phase 2 trials with the DMS (59.4% and 73.9%–88.6%, respectively).^21,22 These results, along with considerations that this study included different populations, acutely ill subjects, and the use of different types of smartphones suggest that the DMS could be a clinically useful tool to measure intake of medication in diverse populations with varying disease severity. This is beneficial, as accurate measurements of adherence can be particularly challenging in patients with severe mental illness.⁷

A potential limitation of this study was the small sample size, especially in the group with schizoaffective disorder (n=10), which could have contributed to the wide variability in the proportion of time with good patch coverage and adherence metric observed in this group. Also, allowing subjects to use their own smartphone that was compatible with the application could have led to lower-than-planned enrollment. A limitation pertaining to patient comfort with data collection is that paranoid symptoms were not assessed, thus conclusions cannot be drawn on whether patients with paranoia could use the DMS successfully. Further, patients were excluded if they had a condition judged to impair their ability to engage with the DMS, which could have included patients who refused to comply with data collection policies. Of note, 2 patients withdrew consent due to discomfort with the DMS, although it is unclear if this was due to paranoia, discomfort with data collection, or another reason. Another limitation may be that information on the smartphone operating systems and reasons for which subjects and healthcare providers contacted the call center were not fully collected. Such data could have been useful to better understand the source of compatibility issues between the patch and smartphone application, which may have affected the performance and acceptance of the DMS. Moreover, to gain better insights to the real-world applicability of the DMS, future studies could benefit from enrolling more subjects per psychiatric condition and collecting more information on smartphones used in the study and related technical issues. These studies could also be enhanced by examining the impact of sharing the DMS’s data with patients, both on adherence metrics and the quality of their relationships with healthcare providers. Lastly, a more user-friendly design may be necessary to reach a broader population of patients who have serious mental illnesses and concerns with the process of data collection.

Conclusions

Overall, the DMS observed good patch coverage and high treatment adherence metrics over the 8-week assessment period. Most subjects were satisfied with using the DMS and generally found its components easy to use. Moreover, the DMS was associated with a safety profile that was comparable to that of previous reports.^21,22 Together, these findings suggest that the DMS could be a useful tool to measure intake of various antipsychotics, including olanzapine and quetiapine, in addition to aripiprazole. However, compared with previous studies,^21,22 the lower values for patch-related endpoints and subject satisfaction^21,22 suggest that regular clinical contact, more support and engagement from the care team, and better training may be required to reinforce performance and acceptance of the DMS in patients with psychotic disorders. This study reflects use of the DMS in a more real-world setting where patients would likely use their own smartphone. A goal for future studies is to better understand smartphone ownership in patient populations to help optimize testing processes of the DMS’s smartphone application. Further evaluation of the DMS in larger clinical trials including different types of antipsychotics and patients with various psychotic disorders may be necessary to establish its clinical utility and to determine its impact on improving treatment adherence in patients with serious mental illness.