Formative usability evaluation of a fixed-dose pen-injector platform device

Introduction

Pen injectors enable patients to treat themselves safely and conveniently with biopharmaceuticals through self-injection. Such devices, originally developed for use with insulin, have found widespread use across a variety of indications.^1,2 The increasing use of biological drugs that cannot be administered orally and the trend (for both cost and convenience reasons) toward patient self-administration mean that the use of such devices is expected to grow strongly in the future.² To meet the needs of different drug-dosing regimens, pen injectors with different operating principles have been developed. Variable-dose pens, where the user selects the required dose for each injection, is the largest category, with a multitude of different devices and solutions available on the market. Fixed-dose pens, where the same preset dose is delivered in every injection, is a smaller category, with only a limited number of options currently available.²

The development of safe and reliable medical devices, such as pen injectors, requires the application of knowledge of human capabilities and limitations to the design of artifacts, known as usability or human-factor engineering. The documented application of usability methods, including usability testing, throughout the design cycle is also required by regulatory authorities.^3–5 Typically, the usability-testing process during new-device development is divided into three parts. First, early formative studies are conducted during early development with the aim of providing user feedback iteratively to refine the device design and instructions for use (IFU). Late-stage formative tests are performed toward the end of development to confirm that the device is suitable for its intended use and likely to pass the usability part of design validation. Finally, summative testing, also known as usability validation, is carried out to provide objective evidence that the intended use has been met and that the device can be reliably and safely used by the intended user population.⁶

Several usability studies on pen injectors have been reported, ranging from formative work aimed at identifying and quantifying handling errors, studies focused on assessing task completion, late-stage formative studies to confirm overall safety and ease of use, to validation studies.^7–17 Although these studies provide important and valuable information on possible handling difficulties and usability issues, it is worth noting that all reported studies have dealt with variable-dose pen injectors. There is thus little information specifically related to fixed-dose pen injectors available in the literature.

This article presents a late-stage formative study of a fixed-dose pen-injector platform device. In order to provide a benchmark in terms of usability performance, a different fixed-dose pen injector available on the market was included in the study. Specifically, the study was designed and carried out with the aim of understanding whether the platform device and its IFU are suitable for users with different characteristics and whether the pen injector would be likely to pass future summative testing in specific indications.

Materials and methods

Objectives

In this late-stage formative study, the primary objective was to evaluate whether participants could use the FixPen™ fixed-dose platform prototype devices safely and effectively relying on the IFU as only support and to benchmark the device against a fixed-dose pen injector available in the market. Further objectives were to capture usability issues that may affect user performance, preference, and satisfaction and to obtain participants’ subjective ratings on properties and performance of the device.

Compliance with ethics guidelines

The study was conducted in accordance with Ypsomed AG’s standard procedures, including internal review of the study protocol and supervision of the study by medical services. The study complies with the principles laid out in the User Experience Professionals Association Code of Professional Conduct and the European Pharmaceutical Market Research Association Code of Conduct. As a simulated-use study, it did not require approval by an ethics committee.

Study devices and materials

Two devices were included in the study. The FixPen (Ypsomed AG, Burgdorf, Switzerland) is a fixed-dose disposable pen injector intended for the subcutaneous self-injection of drugs in the context of various treatments requiring relatively frequent (daily or every 2–3 days) injections of a fixed dose.¹⁸ The device contains a 3 mL cartridge and works according to the push–pull principle (pull to load a dose, push to inject). It is manually operated and designed with a gearing mechanism to optimize injection force and dose-button extension. It also provides audible feedback (“clicks” after loading a dose and completion of injection) and visible feedback (viewing window) to the user during the injection. It is presented in Figure 1A. The Forteo™ pen (Lilly France S.A.S, Fegersheim, France) injector is a device used for the delivery of teriparatide to patients suffering from osteoporosis.¹⁹ It was introduced to the market in 2008, and has similar characteristics in terms of cartridge type, injection volume, and user interface to the FixPen, with the difference that it neither has any window indicating the status of the injection nor features an audible “click” signal after completed loading of a dose. Neither of the two pens in the study requires priming before injection of the first dose. The Forteo pen is presented in Figure 1B. The Forteo pen was selected over the only other fixed-dose pen currently on the market (the Lyxumia pen, introduced in 2013) because of the Lyxumia’s shorter use history and very limited market penetration.

Figure 1 The FixPen (A) and Forteo (B) disposable pen injectors used in the study. Notes: For the FixPen, a neutral label was used; for the Forteo, the product label was removed.

For the simulated injections, standard 31 G 6 mm injection-pen needles (Mylife ClickFine; Ypsomed AG, Burgdorf, Switzerland), water-filled 3 mL insulin-type cartridges (Sanofi, Paris, France), and injection pads (Ypsomed AG) were used. IFU were prepared in a similar format for the two devices to control for differences in learning effect. The tasks performed by the user to perform an injection are shown in Figure 2, and involved the actions of removing a protective cap, attaching a needle, loading a dose, injecting, holding for 5 seconds, removing the needle, and replacing the cap.

Figure 2 Tasks performed by the user during an injection.

Participants and groups

The FixPen has been developed as a platform device, implying that it may be used for different therapies. However, one important therapeutic area for this type of pen injector is known to be osteoporosis, which typically concerns postmenopausal women.^20,21 To reflect this situation, two user groups were defined in the study. The first user group comprised females aged ≥45 years, representing potential osteoporosis patients. As osteoporosis does not lead to any specific impairments directly relevant to using an injection pen, postmenopausal women in general (as opposed to patients suffering from osteoporosis) were selected as users. The target was to reach eight to ten participants in this group, with five as a minimum requirement. The second group consisted of male and female participants of various ages in two categories: either naïve with respect to device use or with previous experience of using pen injectors, representing potential users from other indications. In this group, the target number of participants and minimum requirement was five in each category. A minimum of five participants in each category to be evaluated is in line with current recommendations for this type of study.³

Procedure

The handling study followed standard protocols and procedures based on single-site visits. Trained staff (ie, one moderator) guided each study subject’s visit according to the procedure presented in Figure 3. Each subject first responded to a prestudy questionnaire, and then studied the IFU before performing a simulated injection into the pad situated on a desk in front of them and answering questions on the injection and the device. Having completed the three steps with the first pen, the subject then repeated them with the second pen. Subjects were randomized to test the two pens in a crossover setup to control for order, as well as learning effects. Each study subject thus performed two simulated injections in total (one with each pen). Each simulated injection was video-recorded and followed by the moderator, who was present at all times. During the injections, the moderator observed and recorded any usability issues, consisting of use errors or near misses, and intervened as soon as the study subject experienced difficulties. The set of questions at the end of each injection involved asking the subjects to rate the performance and ease of use of the study device on a 4-point Likert scale.

Figure 3 Sequence of events during the handling tests. Abbreviations: IFU, instructions for use.

Results

Participants

The study-participant population is characterized in Table 1. A total of 20 participants were recruited and participated. The target number of participants (ten for group 1 and five each for groups 2A and 2B) was reached in all groups. Overall, 80% of the participants were female, 25% had previous experience using pen injectors, and 5% reported suffering from some kind of impairment. The mean age of the participants was 50 years, with a range of 21–74 years.

Table 1 Participant characteristics

Injection success and observed user errors

The initial injection-success rate, defined as tasks 1–5 being successfully completed without any intervention by the moderator (holding after injection was not considered critical to success), was 90% for both pens across all users. All users were ultimately successful in using both devices. The initial success rates per user group are presented in Figure 4. The highest initial success rate was achieved in group 1 and the lowest in group 2B. There were no near misses. An overview of all observed use errors for all tasks is provided in Table 2. A total of 27 errors were seen for the 40 injections, corresponding to an overall error rate of 0.68 errors per injection. The needle-removal task was by far the largest source of user error, responsible for close to 60% of all observed errors. The tasks of loading the dose and holding after injection were the next major contributors, together responsible for about 30% of all errors. The two study devices exhibited comparable error rates (0.70 for the FixPen and 0.65 for the Forteo pen), and given the similarity in rates and handling sequence, the results for the two pens were pooled to look at learning effects. The data for the first and second injections in Table 2 show that except for the errors in loading the dose, there was no discernable reduction in error rate from the first to the second injection. The highest error rates were seen in group 1 and the lowest in group 2B, with experienced users in the latter group making less than half the errors of the females >45 years of age in the former group.

Figure 4 Initial injection-success rate. Notes: Success rate defined as the fraction of injections where tasks 1–5 (cap removal, needle attachment, dose loading, needle insertion, injection) were completed by the user without any involvement by the moderator. All injections were ultimately successful. Group 1, females >45 years of age; group 2A, naïve users; group 2B, experienced users.

Table 2 Observed user errors

User feedback

Figure 5 presents self-reported data on convenience in using the device. Both devices were well appreciated by the subjects. The FixPen was given the highest score by ≥85% of subjects on the different convenience aspects, whereas the Forteo pen received the highest score from ≥70% or more. The FixPen received higher ratings on overall attractiveness and injection feedback (certainty that the dose had been loaded and injected), whereas the Forteo pen was given a higher rating for the grip of the dose button. The acceptability of the force to load the dose was given somewhat higher ratings for the FixPen over the Forteo Pen, whereas the acceptability of the injection force was given the highest ratings by all subjects for both pens. The perceived feedback on completion of injection was examined separately, as presented in Figure 6. For the Forteo pen, subjects reported relying mainly on the position of the dose button, whereas for the FixPen the audible click and the display window were used as additional sources of feedback from the device.

Figure 5 User-feedback scores on usability questions asked at the end of the handling test. Notes: Users self-reported convenience on a scale of 1–4, where 4 represented the highest degree of convenience. Each bar provides a breakdown of the percentage of users assigning a certain agreement score to a question, with the absolute number of users per score given as a numeral.

Figure 6 User feedback on which pen feature indicated that the injection had been completed. Notes: Results combined for all user groups (n=20).

Discussion

There were four user errors made during tasks 1–5 leading to an intervention by the moderator and thus to the injection being considered as initially unsuccessful. One subject experienced difficulties attaching the needle, two subjects initially forgot to load the dose, and one subject only partially loaded the dose before attempting to inject. As the moderator intervened as soon as the subjects started having difficulties, rather than waiting until he or she proved unable to continue, it is not known which user errors would have led to an injection failure in the absence of any moderator intervention. However, the fact that the moderator interventions for the errors related to needle attachment, and forgetting to load the dose consisted of merely referring the subject to the IFU, is taken as an indication that these participants eventually would have been able to complete the tasks unassisted and thus achieve a successful injection. The subject who only partially loaded the dose was directly assisted with this task by the moderator. It is believed to have been less likely that this subject would have been capable of completing the task alone.

It is worth noting that the lowest rates of initial injection success were seen with the experienced users in group 2B, whereas the injection-naïve females in group 1 had the highest success rates. The relatively low rates achieved by the experienced users in group 2B was somewhat counterintuitive, and is believed to be due to the fact that they were seen to spend less time studying the IFU before the injections and more likely to adhere to ingrained behavior compared to the other user groups.⁷

A total of 27 user errors were observed, most of which concerned needle handling (17 errors). This is in line with previous studies finding needle handling to be the main source of user error.⁷ As the needle handling and interface between the pen and the needle are standardized, such errors are by definition unrelated to the design of the individual device. Holding after injection was the second-largest source of user error (four errors). This task is known to be a major error source, and it is widely acknowledged that users often have difficulties respecting a predefined holding time.²⁰ Loading the dose was associated with three user errors: two with the FixPen and one with the Forteo pen. The FixPen errors were due to the subjects forgetting to load the dose, whereas for the Forteo pen the error was an incomplete loading of the dose. The former errors may well have been related to the IFU, whereas the latter, which was the one requiring direct assistance by the moderator, could possibly have been related to the mechanism of the pen. The Forteo pen continuously emits audible clicks during dose loading, and thus does not directly indicate a complete loading of the dose, whereas the FixPen instead clicks once the dose has been completely loaded.

With regard to user error rates, it is useful to compare the results from the present study with rates reported in the literature. The present study saw rates of 0.70 errors per injection for the first injection and 0.65 for the second injection. This can be compared with work on a variable-dose insulin-type pen, where rates of 2.39 for the first and 1.94 for second injection were found, and variable-dose pen injectors for infertility treatment and diabetes, reporting error rates of 1.75 and 1.85, respectively.^7,8,10 The handling of variable-dose pens is more complex than for the devices in the current study in that such pens typically require a priming step and selection of the dose before injection. At the simpler end of the spectrum of handling complexity, a study on an autoinjector requiring neither needle attachment nor loading of the dose reported error rates of 0.53 for the first injection and 0.14 for the second injection.²² It thus appears that the error rates seen in the current study fall neatly in between those observed for devices requiring more complex handling and those reported from devices with fewer and simpler handling steps.

The user feedback on performance was overall very positive with high ratings on all usability questions asked for both devices. The FixPen received somewhat higher ratings than the Forteo pen on general attractiveness, how well the device fit in the user’s hand, and feedback on dose loading and injection completion. The Forteo pen was rated somewhat more highly than the FixPen on grip of the dose button. These differences in rating can most likely be attributed to the differences in design between the two devices. The FixPen is smaller, and the Forteo pen has a large mushroom-shaped dose button. Furthermore, the FixPen has a window indicating the progress on dose loading and injection not present in the Forteo pen (see Figure 1). It also provides audible feedback of injection completion not provided by the Forteo pen. As further indicated by the scores on the two devices reported in Figure 6, it thus appears that these distinct design features of the FixPen are well appreciated by users.

Conclusion

Both study devices were able to be successfully used by all study subjects. The observed rates of user error were similar for both devices and in line with the results reported previously for other disposable dial-and-dose systems. Most user errors concerned needle handling, with holding after injection being the second-largest source of errors. Both findings are consistent with other results reported in the literature.

The FixPen was well appreciated by all users, with maximum usability feedback scores reported by ≥90% on handling forces and device feedback, and by ≥85% on fit and grip of the device. The presence of clear audible and visible feedback on completion of injection in the FixPen was seen to be a significant improvement over the Forteo pen injector. The observation that the FixPen can be safely and efficiently used by all user groups provides confidence that the device and IFU in their current form will pass future summative testing in specific applications.

Acknowledgment

This study was conducted with funding from Ypsomed AG.

Disclosure

Both authors work for Ypsomed AG. The authors report no other conflicts of interest in this work.

References

1.		French DL, Collins JJ. Advances in parenteral injection devices and aids. In: Neema S, Ludwig JD, editors. Pharmaceutical Dosage Forms: Parenteral Medications – Volume 3: Regulations, Validation and the Future. London: Informa Healthcare; 2010:71–75.
2.		Thompson I, Lange J. Pen and autoinjector drug delivery devices. In: Kolhe P, Shah M, Rathore N, editors. Sterile Product Development: Formulation, Process, Quality and Regulatory Considerations. Heidelberg: Springer; 2013:331–356.
3.		Association for the Advancement of Medical Instrumentation. Human Factors Engineering: Design of Medical Devices. Arlington (VA): AAMI; 2010.
4.		International Organization for Standardization. Medical devices – part 1: application of usability engineering to medical devices. 2015. Available from: https://www.iso.org/standard/63179.html. Accessed March 7, 2018.
5.		US Food and Drug Administration. Applying human factors and usability engineering to medical devices: guidance for industry and Food and Drug Administration staff. 2016. Available from: https://www.fda.gov/downloads/MedicalDevices/./UCM259760.pdf. Accessed December 12, 2017.
6.		Wiklund M, Kendler J, Strochlic AY. Usability Testing of Medical Devices. Boca Raton (FL): CRC Press; 2010.
7.		Lange J, Richard P, Bradley N. Usability of devices for self-injection: results of a formative study on a new disposable pen injector. Med Devices (Auckl). 2014:7:195–203.
8.		Schertz JC, Saunders H, Lang B, Arriagada P. The redesigned follitropin alfa pen injector: results of the patient and nurse human factors usability testing. Expert Opin Drug Deliv. 2011;8(9):1111–1120.
9.		Rohrer TR, Winter F, Qvist M, Kappelgaard AM. Comparison of intuitiveness, ease of use and preference among three prefilled, disposable growth hormone injection pens. Expert Opin Drug Deliv. 2013;10(12):1603–1612.
10.		Fujioka K, Sparre T, Sun LY, Krogsgaard S, Kushner RF. Usability of the novel liraglutide 3.0 mg pen injector among overweight or obese adult patients with or without prior injection experience. J Diabetes Sci Technol. 2016;10(1):161–174.
11.		Rapaport R, Saenger P, Schmidt H, et al. Validation and ease of use of a new pen device for self-administration of recombinant human growth hormone: results from a two-center usability study. Med Devices (Auckl). 2013;6:141–146.
12.		Carter J, Beilin J, Morton A, de Luise M. Usability, participant acceptance, and safety of a prefilled insulin injection device in a 3-month observational survey in everyday clinical practice in Australia. J Diabetes Sci Technol. 2009;3(6):1425–1438.
13.		Pfützner A, Schipper C, Niemayer M, et al. Comparison of patient preference for two insulin injection pen devices in relation to patient dexterity skills. J Diabetes Sci Technol. 2012;6(4):910–916.
14.		Fuchs GS, Mikkelsen S, Knudsen TK, Kamp T, Kappelgaard AM. Ease of use and acceptability of a new pen device for the administration of growth hormone therapy in pediatric patients: an open-label, uncontrolled usability test. Clin Ther. 2009;31(12):2906–2914.
15.		La Rue S, Malloy J. Evaluation of the dual-chamber pen design for the injection of exenatide once weekly for the treatment of type 2 diabetes. J Diabetes Sci Technol. 2015;9(4):815–821.
16.		Pachon JA, Kivitz AJ, Heuer KU, Pichlmeier U. Assessing usability, label comprehension, pen robustness and pharmacokinetics of a self-administered prefilled autoinjector pen of methotrexate in patients with rheumatoid arthritis. SAGE Open Med. 2014;2:2050312114564241.
17.		Jeannerot F, Stüdeli T, Gunther-Lavergne L, Hirning D, Schertz J. Usability engineering study in the European Union of a redesigned follitropin alfa pen injector for infertility treatment. Expert Opin Drug Deliv. 2016;13(9):1221–1229.
18.		Ypsomed AG. FixPen – the easy-to-use fixed dose pen. 2017. Available from: https://www.ypsomed.com/yds/products/pen-injectors/fixpen/overview.html. Accessed December 12, 2017.
19.		Eli Lilly. Forteo [user’s manual]. 2017. Available from: http://pi.lilly.com/us/forteo-user_manual.pdf. Accessed December 12, 2017.
20.		Melton J. The prevalence of osteoporosis: gender and racial comparison. Calcif Tissue Int. 2001;69(4):179–181.
21.		Hodsman AB, Bauer DC, Dempster DW, et al. Parathyroid hormone and teriparatide for the treatment of osteoporosis: a review of the evidence and suggested guidelines for its use. Endocr Rev. 2005;26(5):688–703.
22.		Lange J, Richard P, Bradley N. Usability of a new disposable autoinjector platform device: results of a formative study conducted with a broad user population. Med Devices (Auckl). 2015;8:255–264.