User Acceptability and Technical Robustness Evaluation of a Novel Smart Pill Bottle Prototype Designed to Support Medication Adherence

Introduction

Up to half of the people taking medication for chronic conditions are non-adherent.^1,2 Medication non-adherence hinders disease control and places a significant negative burden on patients’ quality of life. Notably, the World Health Organization (WHO) concludes that improving patients’ adherence may have a greater effect on patients’ health than any other improvement in therapy.¹ Besides clinical impact, non-adherence also has economic impact as it may lead to increased use of healthcare resources.³

Over the last decade, a wide range of promising smart devices has been developed that can monitor medication adherence.⁴ Data from these smart devices provide information that is granular – eg more time points – than a pill count, and more objective than patient self-report. These data can also be complimentary to – and even help explain – clinical and biochemical laboratory findings such as response biomarkers and drug concentrations.^5–8 Finally, these smart devices can directly support interventions, patient self-monitoring, and help appropriate implementation of a prescribed medication regimen.⁹

The technology for medication monitoring of solid doses by smart pill bottles (SPBs) is rapidly developing with new options for electronics, sensors, real-time data collection, and direct communication between patients and healthcare professionals or trialists.¹⁰ However, the currently used SPBs still suffer from the limitation that they only measure the opening of the bottle and not whether a dose has actually been taken out and – if so – how many.^11–13

Recently, a novel smart pill bottle prototype (SPBP), the Elucid Pill Connect system, has been developed with the aim of providing insights into non-adherence and supporting patients to better manage their medicines.¹⁴ The system consists of a regular pill bottle equipped with an automated dispense and reminder system that connects to a mobile app. Within a predefined timeframe, only the prescribed number of pills is released and at the moment of dispense the time and temperature are logged. The SPBP has potential for both daily practice and clinical trials. However, the usability, reliability and acceptability need to be established first before wide-scale implementation.

Therefore, this proof-of-concept study was designed to test this possible new medication adherence intervention by the means of human factor testing and product evaluation in healthy volunteers. It aims to elucidate user acceptance and technical robustness of the first prototype of an SPB, to ultimately inform the development of a final version ready for scale-up.

Methods

Study Design

This proof-of-concept trial involved 10 healthy volunteers using the SPBP with placebo capsules for 14 days between June and July 2019. Afterwards, the SPBP was evaluated by all subjects to understand the impact and barriers to its use. This study was registered at the Dutch trial registry (Trial NL7835).

Participants

The study was performed with volunteers recruited through the University Medical Center Groningen in the Netherlands. Subjects were eligible for the trial if all of the following criteria were met: age >18 years, own an Android or iPhone mobile phone with operating system versions Android ≥6.0 or IOS ≥11.4, able to operate the mobile phone themselves and provide informed consent. There were no exclusion criteria assessed prior to enrollment. Exclusion criteria assessed after enrollment were the withdrawal of informed consent and unavailability of the subject to participate in the trial.

Twenty-one individuals were asked for their interest in participation; 19 (90%) showed interest and were recruited and assessed for their availability during the study period. From this pool, 10 volunteers were randomly selected and balanced by gender, age group, education level and smartphone operating platform in order to achieve a heterogeneous group. All participants provided written consent to participate in the study.

Procedures

First, all participants were asked to attend a presentation where they received information on the trial and a short training on how to use the SPBP. Subsequently, all subjects had two site visits. At Visit 1, they received their SPBP connected to a temperature logger and containing a sufficient number of placebo capsules. They also received written information and a short explanation on SPBP use. At Visit 2, they returned the bottles and temperature loggers and completed the questionnaire and a short interview on problems that occurred and suggestions to optimize the system. During the first week of the study, all participants were instructed to dispense one morning dose. In the second week, all participants were instructed to dispense a morning dose and an evening dose. For each dose, the participants had a time frame of 6 hr to perform the dispensing. When the participants did not dispense within the first 3 hr, an SMS reminder was sent.

Technical Specifications of the Smart Pill Bottle

The SPBP (Elucid Pill Connect system, eLucid mHealth Ltd, Manchester, United Kingdom) is a patented system that allows for electronic dispenses of medication units (tablets or capsules), registering the time, bottle temperature and number of units that are dispensed. The smart bottle consists of a standard white plastic pill bottle with an automated dispensing module inserted. This module is powered by a small battery. The system is operated by a mobile phone application, and can only be unlocked when a dose is due. This prevents double dosing. In case, for example, serious side-effects occur during a clinical trial, bottles can be centrally locked from a distance. Data are sent real-time to a dedicated platform, which can be viewed by healthcare professionals or trialists, see Figure 1. Via the app, users have insight in their dosing schedule and get SMS reminders when their dose is due or when they are about to forget their medication. For the purpose of the trial, bottles were made transparent instead of white so that the dispensing mechanism could be easily observed.

Figure 1 Data transfer of the SPBP system. Notes: Dispenses are performed through an app on the mobile phone. Time and temperature data are registered when the dose is successfully dispensed, and then loaded to an online platform.

Questionnaire

All participants completed the questionnaire that included the System Usability Scale (SUS),^15,16 additional satisfaction questions and the Medication Adherence Report Scale (MARS).¹⁷ The questionnaire incorporated the five-point Likert scale where one is “always” and five is “never”. SUS is a measure for user acceptance of a new system, where a user rates 10 questions on topics such as the need for support, training, and complexity of the system. The outcomes are scored and the overall value ranges from 0 to 100, where high scores indicate good usability. The additional satisfaction questions were self-designed. More details can be found in the Supplementary Materials. MARS is a scale consisting of five topics, where a subject rates self-assessed engagement in non-adherent behavior and where higher scores indicate higher levels of adherence. The validated Dutch translation was used.

Temperature Validation

To validate the temperature sensor inside the SPBP (LMT85-Q1, Texas Instruments, Dallas, Texas, United States) a calibrated temperature data logger (Testo 174T, Testo SE & Co. KGaA, Lenzkirch, Germany) was attached to each SPBP. Temperature data of the internal sensor were recorded at every dispense, whereas the external logger recorded every 5 mins. As it was assumed that environmental temperature is quite stable, two external temperature measurements were averaged around the time of the sensor measurement.

The mean temperature difference between the bottle sensors and the external loggers was calculated per bottle. To evaluate the systematic bias of the temperature sensor, a Bland–Altman plot was used for all measured data points.¹⁸ In this plot, the difference between the two paired temperature measurements (y-axis) is set out against the average of the two measurements (x-axis). The limit of agreement (LOA) represents 95% of the differences between the two measurement methods (± 2 SD).

Outcomes

The primary outcome was the percentage of capsules successfully dispensed according to the intended dosing schedule. Secondary outcomes were user acceptance, measured with the SUS, additional satisfaction questions, self-reported adherence (MARS) and technical validation, ie correct dispense and logging of dispense data, sending of reminders, central locking at distance and temperature validity.

Outcomes were reported with descriptive statistics where appropriate.

Ethics

On the basis of the study protocol, the Medical Ethics Review Board of the University Medical Center Groningen (METc UMCG, number: 2019/332) exempted this study from the Medical Research Involving Human Subjects Act. This research followed the declaration of Helsinki and the General Data Protection Regulation. No clinical data were collected or used during this study and there were no changes to subjects’ regular care routines.

Results

Participants

The characteristics of the 10 study participants are shown in Table 1. Participants had equal gender balance and varied in age group, education level, smartphone platform and experience with following a medication schedule (regarded as chronic medication use). All participants completed both study visits.

Table 1 Characteristics of the Study Participants (Total n=10)

Dispensing and Adherence

During the study, a total of 168 capsules out of 190 were successfully dispensed, which implicates an overall dispense adherence of 88% to the intended dosing schedule. Four participants had an adherence rate of 100% during the study. A total of 22 doses were missed by six participants. Of these missed doses, 12 were due to human error (eg forgot the bottle at home) and 10 were missed because of a product design problem (with nine of those occurring in the same bottle, further discussed in the technical evaluation). The participants had a high self-reported adherence with an average MARS total score of 23.6 out of 25; the participants’ responses are reported in Figure 2.

Figure 2 Medication adherence report scale scores.

User Acceptance

The majority of the participants judged the system usability acceptable, with a mean SUS score of 79.3 (range: 57.5 to 97.5). The responses of the participants to the SUS questionnaire are shown in Figure 3. In the questionnaire, most participants (80%) thought the system was easy to use; two participants thought that most people would need training of which one participant needed to learn a lot before they could get going. No participants needed the support of a technical person. In total, 70% liked to use this system frequently.

Figure 3 System usability score for the smart pill bottle prototype.

In the additional questions, as shown in Figure 4, all but one user found both the dispenser as the application simple to use. However, they found it uneasy to carry the dispenser around (30%), and handling the system was more difficult than standard drug packaging (40%). Half of the group found benefits from this system. Most participants (80%) were positive towards the idea that a healthcare professional received information about their medication intake, while two participants marked this question as “not applicable”, as there was no actual information shared with healthcare professionals in this study. Being reminded by the system and seeing upcoming doses was appreciated by 60% of the participants. Only three of the participants reported problems with taking their medicine at the correct time, where four liked help for taking their medicine on time.

Figure 4 Additional questions on the smart pill bottle prototype.

Technical Evaluation

All bottles’ batteries remained functional during the length of the trial. Nine bottles were in proper condition after the end of the study. In one bottle, the automated dispensing mechanism got jammed due to a technical problem. Dispensing data were correctly logged and sent to the platform for all dispenses. All capsules that were dispensed were confined to the permitted time window.

In the short interview, seven participants reported that they encountered an error during the study. Two participants reported that not all capsules were dispensed successfully due to an error in the dispensing mechanism. Two participants reported a situation where a double dose was dispensed. Other errors were easily resolved by the participants in their next attempt by shaking the bottle and trying to dispense again.

Furthermore, two participants reported that they got reminders after they already performed a dispense. In these situations, data were not rapidly synchronized but loaded in the next dispense at most times, while the participants were connected to Wi-Fi and to the mobile network at the moment of dispense. Moreover, participants reported that they did not always get their reminders when needed.

Temperature

There were 188 data points where the recorded temperature of the SPBP sensor could be compared with the external temperature logger data, with temperatures that ranged from 13.1°C to 26.6°C, as shown in Figure 5. Temperature data of Android phones were more detailed as there was one decimal available, whereas iOS phones only recorded whole numbers. The overall mean temperature difference was −0.82°C, where the mean temperature difference per bottle ranged from −1.37°C to −0.21°C. These data implicate that the temperatures that were measured within the SPBP are lower than the measurements of the external logger. When all data points were combined, the mean temperature difference between the methods was −0.78°C (SD: 0.70, LOA: -2.15, 0.58), see Figure 6.

Figure 5 Comparison of temperatures that were measured simultaneously by the sensor inside the smart pill bottle prototype (x-axis) and by the calibrated external temperature logger (y-axis).

Figure 6 Bland–Altman to evaluate the bias between the measurements of the internal temperature sensor and external temperature logger. Note: In this plot, the difference between the two paired measurements is set out against the average of the two paired measurements.

Lock-Out

On the last day, all bottles were successfully locked. There were no dispenses recorded during this lock-out period, and in the short interview all participants confirmed that no dispense was possible.

Outcomes per Adherence Type

The main method and score outcomes were stratified per adherence type as determined by the number of dispensing attempts that were made by the participant, as shown in Figure 7. Participants with a 100% intended adherence missed no capsules and regarded the system with a lower SUS score. However, one participant still regarded themself as non-adherent. There was one capsule count discrepancy caused by a double dispense. Of the three participants that had 90–99% dispense adherence, one told during the interview that she did not miss a dispense and regarded herself as adherent. All three rated the SUS score high. All participants with a lower SPBP-measured adherence also reported being non-adherent during both the interview and the questionnaires.

Figure 7 Association of smart pill bottle-defined dispense adherence rates (<90%, 90–99% and 100%) versus various alternative adherence indicators. Note: A low MARS score is defined as a score of 24 or less.Abbreviations: MARS, Medication Adherence Report Scale; SUS, System Usability Scale.

Suggestions for Optimization

Several participants recommended to personalize the timing of the reminders. Two participants asked for a strict time point to receive the messages, of which one asked for an earlier timing to fit the message into their morning schedule. One participant wanted to set the time of the messages himself. Another suggestion regarding reminders was to use pop-ups in the app instead of SMS reminders. One participant suggested to do more with the phone during the training and to explain how to install sound options when reminders come in. Another participant had her phone constantly silenced, she reported that it works better for her just to set an alarm on her phone (as it overrules the silence).

There were a few suggestions to improve the system. Two participants suggested an option in the application where you can look back which doses were due, or that all medication taken is put into a graph. Moreover, one participant wanted an emergency option, so access to the medication would be granted when the mobile phone would crash.

Discussion

In this study, we evaluated dispense adherence rates, user acceptance and technical robustness of a novel smart pill bottle prototype. While both adherence and acceptability were high, the dispense and reminder systems require further optimization.

All participants completed the trial and were adherent to their placebo dosing schedule, which was reflected in the self-assessed adherence. The user acceptance was high, as the majority of the participants saw benefits in the system and thought it was easy to use. To increase user acceptance, some suggestions were given to optimize the system.

For the technical robustness, all dispensing moments were within the predefined time frames and the measured temperature accuracy was within the accuracy range as given by the sensor manufacturer,¹⁹ but did not reach the expected average of ±0.4°C. In total, 95% of all measurements ranged between a difference of −0.5 to 2.1°C. This accuracy may be regarded as acceptable for less temperature-sensitive drugs, as temperature differences may still exist between the in-and outside of the bottle. For highly temperature-sensitive drugs, this discrepancy needs further investigation. Furthermore, all participants were locked out successfully at the end of the trial.

However, two issues were identified in the technical aspects of the bottle. First, a manufacturing error for the prototype was signaled and this has been reported to the manufacturer. Accordingly, the manufacturer has introduced a new process and quality assurance to prevent this from happening again. Second, the double dispense events could be attributed to the study-specific use of see-through bottles instead of the common white bottles, as the sensor in the dispensing mechanism was possibly saturated with scattered infrared light.²⁰ Indeed, both participants that encountered this problem were outside when they performed the dispense. The manufacturer will therefore increase the power of the sensor and provide a secondary block of dark plastic to the bottles.

The observed high adherence rates were significantly higher than real-world adherence rates. Indeed, trial participation generally improves adherence behavior. While the general impact of being electronically monitored may be low,^13,21 this system was especially designed to improve adherence by involving patients in their dose schedule and sending reminders. Previously, reminders were shown to improve adherence.^22–27 According to our findings, these reminders could be more beneficial when better timed and personalized by end-users.

Assessment of SPBP Assets

The SPBP tested in this study records the exact time, temperature and number of capsules that are dispensed. Furthermore, all data could be accessed in real-time. This could potentially improve intake data for trials as the trialist can immediately take action and have an open discussion with the participant based on the provided adherence overview. However, this could also increase the stigma of being monitored and raise privacy concerns. Another downside is that the moving parts in the dispensing mechanism have a risk of blockage and may have a higher impact on battery durability.

An additional opportunity of this system is the temperature monitoring. In clinical trials, drug storage conditions are tightly regulated before giving out to the patient to ensure product quality and drug stability.²⁸ However, when at home, patients may store their medication at less suitable places, eg in a bathroom or a windowsill in the sun, where the medication can reach high temperatures.²⁹ As the container is air-sealed with a cap and only the required amount is dispensed, the content will be minimally exposed to humidity. Adding a sound alarm may increase participants’ awareness when the temperature is outside the recommended storage range. For this purpose, the temperature should be measured continuously.

Another asset of the system is its ability to send reminders. Text reminders have previously shown to be a simple and effective way to increase adherence.^23–27 On the other hand, a high frequency of reminders may introduce response fatigue. In this system, reminders are sent only when this information is relevant (ie when doses are not taken), which may even trigger the subjects to take the medication before the reminder time.³⁰ While in this study these reminders were sent at a certain time point, the system is suitable to be adjusted to user preferences.

Strengths and Limitations

Usability studies are an important step in the development of user-friendly technologies, as they provide valuable user feedback on needs and expectations. Strengths of this study were that participants were randomly selected to obtain a heterogeneous group, and the use of multiple methods to assess adherence and satisfaction.

However, there were also some limitations, including generalizability and sample size. As this study did not use prescribed medication, no formal medication adherence could be tested. Furthermore, the participants in this study were on average young, healthy, and half of them was not accustomed to a medication schedule. While the age groups and education levels that were covered in this study could be possible target groups for this device, their experiences may differ from the experiences of actual medication users, who may feel more unwell and may use the device for a longer period. Younger participants are also in general more comfortable with new e-technology. Whether higher age groups could benefit from this device is not covered in this study. For participants that do not possess or know how to operate a smartphone, other non-smartphone-connected devices (MEMs, blister packs, week boxes) may be more suitable to track and support adherence.

While the small sample size could be considered a limitation, similar numbers have previously been reported for other pilot studies on system usability.^31–33

Implications for Future Research

This study gives insight in the development process of a mobile phone-connected pill bottle, provides design and implementation considerations and may stimulate further research in features to improve and monitor adherence. Suggested improvements from this study related to optimization of the manufacturing process and personalization of the app-based reminder system. When these improvements to the SPBP are made, a follow-up trial with the next SPB version is recommended, until 100% robustness and high user satisfaction is shown.

Ultimately, application of these SPBs could be diverse. For example in clinical trials, these pill bottles would be suitable to assess non-adherence, but also in pharmacokinetic research, where knowledge of the precise time of intake, dose and storage environment of the drug are pivotal for estimations of exposure and explanation of individual variability in drug response.

Conclusion

The SPBP was well accepted and this study informs further technical optimization and follow-up studies. Taking into account patient needs and usability is important in the development of new technologies. The issues that were uncovered may stimulate further research and advancements in the smart adherence devices area.