Usability of devices for self-injection: results of a formative study on a new disposable pen injector
Jakob Lange,1 Philipp Richard,1 Nick Bradley2
1Ypsomed AG, Burgdorf, Switzerland; 2Bergo, Glasgow, United Kingdom
Abstract: This article presents a late-stage formative usability study of a pen-injector platform device. Such devices are used for the subcutaneous delivery of biopharmaceuticals, primarily for self-administration by the patient. The study was conducted with a broad user population, defined to represent user characteristics across a range of indications. The goals of the study were to confirm that the pen could be used without recurring patterns of use errors leading to hazardous situations, to evaluate the comprehension of the instructions for use (IFU), and to determine if training is necessary. In the study, a total of 36 participants in six groups (health care providers, caregivers, adolescents, diabetics with retinopathy, diabetics with neuropathy, and patients with arthritis) each read the IFU, prepared the device, and performed two simulated injections into an injection pad. Any use errors, near misses, or deviations from the IFU procedure were recorded. The overall success rate (injection completed by the participant without need for assistance) was 94% for the first and 100% for the second injection. Ninety-two percent of the participants reported that they felt confident using the device, 100% found the IFU helpful, and 75% found the device positively comfortable to use. Overall, a total average of 3.35 deviations and errors per user and injection were recorded (there were no near misses). Subtracting the errors without any potential for negative consequences for the injection or the user (trivial deviations), as well as those related to attaching and removing the pen needle (independent of the design of the pen itself), led to an average of 1.31 potentially relevant deviations per user and injection. It was concluded that the pen injector together with the IFU could be safely and efficiently used by all user groups without any training, and thus that the device and IFU in their current form are well suited for use in a range of specific applications.
Keywords: human-factor engineering, injection pen, handling study, user error, instructions for use, UnoPen™
Pen injectors are used for the subcutaneous delivery of biopharmaceuticals, primarily for self-administration by the patient. First developed to replace vials and syringes for the treatment of diabetes with insulin, such injection devices have been an important part of the drug-delivery device family for several decades.1 The combination of the worldwide increase in diabetes, the trend toward biological drugs that cannot be administered orally, and the growing pressure on health care costs that pushes toward patient self-injection means that their importance is expected to grow further in the future.2–4
Usability, or human-factor engineering, refers to the application of knowledge of human capabilities and limitations to the development and design of artifacts and systems. Successful development of safe and reliable medical devices requires the application of usability methods throughout the design cycle, and documented usability testing is an important part of the information required by regulatory authorities in order to grant marketing authorization.5–7 Usability evaluation during device development is typically divided into three parts. First comes early formative testing, conducted in the early development stages with the aim of providing feedback from users at various stages of the design process, in order to iteratively refine the design, the packaging and its instructions for use (IFU). This is followed by late-stage formative testing carried out to gain certainty that the device is suitable and likely to pass design validation. Last comes summative testing, performed at the end of development, in order to provide objective evidence that the intended use has been met and that the device can be safely and reliably used by the intended patient population.8
Recently, there has been a move from indication-specific device development (mainly for diabetes) to device platforms intended to be used across a wide range of therapies and indications.9–11 This requires a different approach to formative usability work involving user groups with a broader range of user properties and raises the question of how to set up a usability-engineering program in line with regulating agencies’ expectations without a known specific user population. One possibility here, and the one selected for the platform examined in the current study, is to adopt a two-tiered approach whereby the platform device first undergoes formative testing with a broad user population recruited to reflect general user properties rather than those of a specific indication. In a second step, the device is then customized for a given application and subjected to further formative testing, followed by design validation with the corresponding specific user population.
Most of the usability work on pen injectors found in the literature has been studies comparing different devices for a specific indication and focusing on patient preference and ease of use.12–19 Some formal summative testing of a given device across the intended user groups in the selected indication has also been reported.20,21 With regard to formative testing with the intent of identifying potential hazards and providing design input, only limited work on pen injectors can be found in the literature.14,15 To our knowledge, no studies involving users selected to be applicable across different indications have been published.
This article presents a late-stage formative study of a platform device with a broad user population, defined to represent user properties across a range of indications. Specifically, the study was designed and carried out with the goal of understanding whether the platform device and its IFU are suitable for users in all intended applications, and whether the device would be likely to pass summative usability tests in specific indications.
Materials and methods
In this late-stage formative study, the aim was to test the UnoPen™’s (Ypsomed AG, Burgdorf, Switzerland) platform-prototype devices as well as the IFU with a sample of users having a variety of properties, in order to understand whether the device and the IFU are appropriate for a wide range of potential user populations.
The primary goal of the study was to confirm that the UnoPen™ can be used by a user population exhibiting different properties without recurring patterns of use errors leading to hazardous situations. The secondary goal was to understand the comprehension of the IFU and gather insight about how the IFU can be improved. Additionally, it is expected that the results of the study may provide information on whether training needs to be considered.
The UnoPen™ is a disposable multiple variable-dose pen injector holding a 3 mL cartridge. The device is fully manual and similar in design to current insulin pen injectors on the market. It is presented in Figure 1. The UnoPen™ operates according to the principles of dial to dose and push to inject.1 It is designed for ease of use, with a geared dosing mechanism providing reduced injection force and a dose scale with large, easy-to-read numbering.22,23
Figure 1 The UnoPen™ (Ypsomed AG, Burgdorf, Switzerland) disposable pen injector used in the study.
Participants and groups
The UnoPen™ is intended as a platform device to be employed across different medical indications and patient groups. Therefore, no specific indication was used to define the user groups. The focus was instead on selecting relevant user properties that can reasonably be expected to be found in a wide range of applications. Ideally, any user population for subsequent products would be a subset of the user groups thus defined.
Table 1 presents the six defined user groups, together with the screening criteria for each group. The different user groups reflect possible differences in the abilities of potential end users. Therefore health care providers (HCPs) in group 1 have clinical knowledge and training, which will positively affect their performance with the device in comparison to lay users (groups 2–6). Caregivers (CGs) in group 2 will have full command of their mental and physical abilities; therefore, they are likely to perform better than participants with impairments in groups 4–6. Further, as CGs inject the drug into someone else, the role type is different in comparison to other lay users who self-inject.
Table 1 Definition of the user groups and screening criteria for each group
Due to developmental and educational factors, which may affect their abilities in understanding device use, adolescents were assigned to a separate group (group 3). Diabetics with retinopathy (DR; group 4), diabetics with neuropathy (DN; group 5) and arthritic patients (AR; group 6) were also separated into distinct groups based on the type of impairment. Here, participants in group 4 will have visual impairments, participants in group 5 will have tactile impairments, and participants in group 6 will have motor impairments. Each of these groups therefore may present different challenges for user–interface design.
It is currently recommended to have five to eight participants per user group for formative studies.5 The target number of participants to be recruited was therefore set to eight per user group, with the aim of reaching a minimum of five active participants per user group.
Facilities and equipment
Usability tests were conducted partially in a specialized center at Napier University in Edinburgh, United Kingdom and partially at a market-research facility (Progressive Partnerships) in Glasgow, United Kingdom. Mannequins, present at the Edinburgh facility, were used for the injections performed by user groups 1 and 2, whereas groups 3, 4, and 5 injected into a pad (Ypsomed AG) used at both facilities. The equipment used for the injections was the UnoPen™ disposable pen injector with water-filled cartridges as well as pen needles (mylife Clickfine®; Ypsomed AG).
The market-research agency recruited participants across the six groups according to the screening criteria defined in Table 1. The participants were scheduled to attend an individual 60-minute session. The activities carried out by each participant were 1) read and sign a consent form, 2) listen to background information and study brief, and 3) conduct the handling tests. The sequence of events during the handling tests is depicted in Figure 2. All activities were accompanied by a researcher who noted results and observations. The handling tests were recorded on video.
Figure 2 Sequence of events during the handling tests.
As shown in Figure 2, after participants read the IFU, they were asked to perform the first of two injections (performing all steps in the IFU), which involved either injecting 60 units (to simulate a large injection) or 20 units (to simulate a small injection). In a random fashion, half of participants across all groups were asked to do a 60-unit injection first, and the other half were asked to do a 20-unit injection first. Participants were allowed to refer to the IFU at any time, but did not receive any training on device use. The researcher observed and recorded all use errors, near misses, or deviations from the IFU procedure. Once the participant completed the last step (needle disposal), they were asked questions on any use errors, near misses, or deviations identified by the researcher or any difficulties they experienced in understanding or using the device and/or IFU.
Before performing the second injection, participants were asked to watch a 1-minute news summary. The purpose of this distractor task was to minimize short-term memory recall from the previous injection. The second injection was then performed, and the same questions were asked as before. Participants were again allowed to refer to the IFU at any time. Once the questions on the second injection had been answered, the participants were asked a specific set of questions and asked to rate their confidence and comfort in using the device as well as the helpfulness of the IFU.
The characteristics of the participants in the study are presented in Table 2. A total of 43 participants were recruited, of whom 36 participated in the study. The minimum number of five participants per group was reached in five of the six groups. Overall, 78% of the participants were female, 81% had previous experience using pen injectors, and 28% suffered from some kind of impairment. Thirty-three sessions were conducted in the center in Edinburgh, and three sessions conducted in Glasgow.
Table 2 Characteristics of participants
Injection success rate, confidence, comfort, and IFU usefulness
All participants except two persons in the AR group were successful (ie, required no assistance) in performing the first injection, which corresponded to a 94% success rate. Everybody succeeded in performing the second injection, meaning a success rate of 100% for this injection. The detailed results are given in Table 3.
Table 4 presents the data on confidence in using the device. Overall, 69% of the participants reported that they would strongly agree with the statement that they felt confident in using the UnoPen™, whereas 22% reported that they would agree with this statement. The combined rating for the two categories was 92% of all participants. Looking at the rating per user group, it appears that 100% of participants in all groups except the CG and AR groups would either agree or strongly agree that they felt confident in using the device.
Overall, 28% of participants rated the UnoPen™ as very comfortable to use, whereas 47% rated it as comfortable to use, giving a combined result of 75% of participants rating the device as at least comfortable to use. Only 8% of the participants rated the UnoPen™ as uncomfortable to use, with the remaining 17% rating the degree of comfort as neutral. The results per user group are shown in Table 5.
Table 6 presents the reported degree of usefulness of the IFU. Overall, 75% of the participants rated the IFU as very helpful and 25% as somewhat helpful bringing the total to 100% of participants finding the IFU to varying degrees helpful.
Errors and deviations from IFU procedure
Table 7 presents an overview of all observed user errors and deviations from the procedure described in the IFU. No near misses were recorded. Overall, a total average of 3.35 errors and deviations per injection were observed. However, a large number of the observations concerned errors and deviations without any potential negative impact on the user and of no possible consequence for the outcome of the injection. These observations were lumped together and categorized as trivial, whereas the remainder were labeled potentially relevant. It should be noted that the potentially relevant errors and deviations were not directly linked with a negative consequence for the user, but rather actions that under certain circumstances could lead to a problem for the user (a kind of risky behavior). Only two of a total of 154 observed potentially relevant errors led to an actual problem for the participant.
As can be seen in Table 7, the fraction of trivial errors and deviations varied significantly across the user steps, being as high as 89% of the total for the first user step (attaching needle), but less than 5% for steps three and six (selecting the dose and disposing of the needle). Overall, 35% of the observed errors and deviations were defined as trivial.
The split of potentially relevant errors and deviations across the user steps is shown in Figure 3. The largest proportion (35%) of all potentially relevant errors occurred when disposing of the needle (step 6), and significant proportions were observed when priming (step 2, 24%) and holding after injection (step 5, 21%). Attaching the needle (step 1, 4%), selecting the dose (step 3, 4%), and injecting (step 4, 12%) were associated with fewer observed potentially relevant errors or deviations.
Figure 3 Potentially relevant user errors and deviations from the IFU, split between user steps over all user groups and all injections (n=71).
A breakdown of the observed potentially relevant errors and deviations per injection and user group is provided in Table 8. The error rate varied significantly between user groups, ranging from 3.75 errors per participant for the first injection in the DN group to 1.00 for the second injection in the CG group. The overall rate was 2.39 for the first and 1.94 for the second injection, equivalent to an improvement of 19% between the injections. This improvement or learning effect also varied across the user groups, with the CG group showing the largest (39%) and the HCP group showing the lowest (11%) learning effect.
The overall success rate in performing injections was well above 90% for the first injection and 100% for the second injection. It is worth noting that the two participants who failed in performing the first injection made errors that led to problems not with the pen itself but with the needle: in one case not removing the inner needle cover when priming, and in the other case inserting the needle at an angle for injection and thereby bending the needle in the process. It is worth noting that this success rate was reached with the IFU as the only support, thus indicating that no training is required to reach satisfactory success rates.
The reported degree of confidence in using the device was above 90% across all participants and user groups. Degrees below 100% were reported by participants in the AR and the CG groups. The results for the AR group may be related to the fact that this group had the highest fraction of users without any previous pen experience (five of seven participants). It is not known why the CG group reported low values.
With regard to the degree of comfort experienced in using the UnoPen™, it is interesting to note that the HCP and CG groups consistently reported lower degrees of comfort than the impaired participants (DR, DN, and AR groups). It can only be speculated whether this was because the device is particularly well suited for impaired users, if HCPs and CGs are more critical than other users, or if the device was perceived to be less comfortable when injecting others as opposed to when self-injecting.
The IFU was well appreciated by all participants with very little variation across user groups. Overall, a large number of user errors and deviations from the procedure described in the IFU were observed (average of 3.35 errors per injection). Here, it is important to keep the overall success rate in performing injections in mind, which means that only a very small minority of the observed user errors actually led to failure. In fact, more than a third of the observations concerned errors and deviations without any potential negative consequences for the injection or the user (trivial deviations).
Furthermore, of the remaining two-thirds (potentially relevant deviations), about half were estimated to be due to users with previous pen experience sticking to their habits rather than not understanding or misinterpreting the instructions. For instance, observed errors such as not priming, not tapping the cartridge before priming, priming with the needle pointing downward, not holding for the required time after injection, or using the inner needle cover for needle disposal are all examples of what users acknowledged doing with their regular devices. Many participants thus either accidentally reverted to their usual behavior or intentionally decided to do it their way instead of following the IFU.
It is also worth noting that the user steps involving attachment and disposal of the needle together accounted for almost 40% of the potentially relevant deviations, or an average of 0.87 errors per injection. This can be compared with the observed average of 0.39 errors per injection during the same user steps reported in another study on a comparable disposable injection pen.21 It is not known why the observed rate in the present study was so much higher. However, about half of the deviations concerned using the inner needle cover for disposal, ie, an example of ingrained behavior rather than misunderstanding. Furthermore, these steps are essentially independent of the design of the pen injector, as the interface between pen and needle is standardized and thus by definition the same for all pens.24 Looking at the number of potentially relevant errors and deviations for the remaining user steps (priming, dose setting, injecting, and holding), the average was 1.31 errors per injection, which can be compared with an average of 1.36 errors per injection during the same user steps reported in the other study.21
The fact that potentially relevant deviations were observed during the actual injection step in 24% of the cases may be considered remarkable. However, of these 17 deviations, 14 were related to participants not keeping the display window in view while injecting. This error deprived the user of visual feedback on the progress of the injection; however, tactile and audible feedback was still provided and allowed all participants committing this error to complete the injection successfully.
There were large differences in the numbers of potentially relevant errors and deviations for all user steps observed in the different user groups. Thus, participants in the HCP and CG groups had averages well below two errors per injection, the adolescent group had around two, and the DR, DN, and AR groups had averages well above two and even above three in some cases. These observed differences correlate well with differences both in experience and in dexterity and visual acuity between the user groups. It is believed that it is the difference in experience rather than level of impairment that explains the observed differences in error rate.
The number of potentially relevant errors and deviations was observed to decrease between the first and second injections in all user groups, although this decrease (learning effect) varied strongly between groups. That the HCP group showed the lowest learning effect is believed to be due to the majority of their errors being deliberate deviations rather than actual misinterpretations. The above-average learning effect in the AR group is presumably due to it having a large number of participants without any previous pen experience. It is not understood why the CG group exhibited such a high learning effect, but it is believed that it may have been an artifact, due to the low numbers of both errors and users for the second injection, which makes the rate calculation very sensitive to small absolute variations in numbers.
With regard to how the results of this formative study impact further usability work on the UnoPen™, the general observation that the device can be safely and efficiently used by all tested user groups provides confidence that the device and IFU in their current form will pass future summative testing in specific applications. The significant differences in use patterns and number of errors that have been observed between different groups suggest that further formative work, possibly resulting in changes to the IFU and/or device design, may be useful in certain applications, eg, where user populations include naïve and impaired patients. Depending on the options available for a certain application, it may be considered to add user training for cases where this could be expected to be beneficial, eg, user populations similar to the ones who in the current study showed above-average numbers of errors in combination with strong learning effects.
The UnoPen™ together with the IFU could be safely and efficiently used by all user groups without any training, with an overall success rate in performing injections above 90% for the first injection and 100% for the second injection, and a reported degree of confidence in using the device above 90% across all participants and user groups. The IFU was well appreciated by 100% of the participants.
Although a large number of user errors and deviations from the procedure described in the IFU were observed, more than a third of the observations concerned errors and deviations without any risk of negative user consequences, and another quarter were associated with use of the needle rather than the pen injector itself. A noticeable fraction of errors and deviations were also due to experienced users sticking to their (erroneous) habits rather than not understanding or misinterpreting the instructions.
Little variation was observed between the different user groups in injection success rate, in-use confidence, and comfort, as well as in perceived IFU usefulness, whereas large differences between the groups were seen in number of errors and deviations, as well as the improvement from the first to the second injection. The observation that the device can be safely and efficiently used by all tested user groups provides confidence that the device and IFU in their current form will pass future summative testing in specific applications.
The study was conducted by Bergo, United Kingdom with funding from Ypsomed AG. Jakob Lange and Philipp Richard work for Ypsomed AG. Nick Bradley works for Bergo, United Kingdom. The authors report no other conflicts of interest in this work.
Fry A. Insulin delivery device technology 2012: where are we after 90 years? J Diabetes Sci Technol. 2012;6(4):947–953.
Selam JL. Evolution of diabetes insulin delivery devices. J Diabetes Sci Technol. 2010;4(3):505–513.
French DL, Collins JJ. Advances in parenteral injection devices and aids. In: Nema S, Ludwig JD, editors. Pharmaceutical Dosage Forms: Parenteral Medications – Regulations, Validation and the Future. 3rd ed. Colchester, UK: Informa Healthcare; 2010:71–75.
Thompson I, Lange J. Pen and autoinjector drug delivery devices. In: Kolhe P, Shah M, Rathore N, editors. Sterile Product Development. AAPS Advances in the Pharmaceutical Sciences Series 6. New York: Springer; 2013:331–356.
Association for the Advancement of Medical Instrumentation. Human Factors Engineering – Design of Medical Devices. Arlington (TX): AAMI; 2009.
Association for the Advancement of Medical Instrumentation. ISO 62366:2007: Medical Devices – Application of Usability Engineering to Medical Devices. Arlington (TX): AAMI; 2007.
US Food and Drug Administration. Draft guidance for industry and Food and Drug Administration staff – applying human factors and usability engineering to optimize medical device design. 2011. Available from: http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm259748.htm. Accessed March 10, 2014.
Wiklund M, Kendler J, Strochlic AY. Usability Testing of Medical Devices. Boca Raton (FL): CRC; 2011.
Simpson I. Two key questions in platform injection device development. 2012. Available from: http://www.pharmaceuticalonline.com/doc/two-key-questions-in-platform-0001. Accessed March 10, 2014.
Thompson I. Matching a range of injectors with both patients’ and pharma partners’ needs. ONdrugDelivery. 2012;(33):36–38.
Kaufman SR. Understanding which types of design should be driving innovation at drug delivery device suppliers. ONdrugDelivery. 2013; (45):39–43.
Carter J, Beilin J, Morton A, De Luise M. Usability, participant acceptance, and safety of a prefilled insulin injection device in a 3-month observational survey in everyday clinical practice in Australia. J Diabetes Sci Technol. 2009;3(6):1425–1438.
Olsen BS, Lilleøre SK, Korsholm CN, Kracht T. Novopen Echo for the delivery of insulin: a comparison of usability, functionality and preference among pediatric subjects, their parents, and health care professionals. J Diabetes Sci Technol. 2010;4(6):1468–1475.
Pfützner A, Schipper C, Niemayer M, et al. Comparison of patient preference for two insulin injection pen devices in relation to patient dexterity skills. J Diabetes Sci Technol. 2012;6(4):910–916.
Fuchs GS, Mikkelsen S, Knudsen TK, Kamp T, Kappelsgaard AM. Ease of use and acceptability of a new pen device for the administration of growth hormone therapy in pediatric patients: an open-label, uncontrolled usability test. Clin Ther. 2009;31(12):2906–2914.
Asakura T, Jensen KH. Comparison of intuitiveness, ease of use, and preference in two insulin pens. J Diabetes Sci Technol. 2009;3(2):312–319.
Asakura T, Seino H, Jensen KH. Patient acceptance and issues of education of two durable insulin pen devices. Diabetes Technol Ther. 2008;10(4):299–304.
Wong M, Abdulnabi R, Carey MA, Fu H. A randomized, cross-over comparison of preference between two reusable insulin pen devices in pen-naïve adults with diabetes. Curr Med Res Opin. 2013;29(5):465–473.
Hey-Hadavi J, Pfeil A, Deeb LC, et al. Ease of use and preference for a new disposable self-injection pen compared with a reusable pen for administering recombinant human growth hormone: a multi-center, 2-month, single-arm, open-label clinical trial in parent-caregiver dyads. Clin Ther. 2010:32(12):2036–2047.
Rapaport R, Saenger P, Schmidt H, et al. Validation and ease of use of a new pen device for self-administration of recombinant human growth hormone: results from a two-center usability study. Med Devices (Auckl). 2013;6:141–146.
Schertz JC, Saunders H, Hecker C, Lang B, Arriagada P. The redesigned follitropin alfa pen injector: results of the patient and nurse human factors usability testing. Expert Opin Drug Deliv. 2011;8(9):1111–1120.
DuPont. Disposable injector pen uses new low-friction grade of DuPont™ Delrin® for healthcare applications [press release]. Geneva: DuPont; 2011 [Oct 18]. Available from: http://www2.dupont.com/Medical_Device_Material/en_US/news_events/article20111018.html. Accessed March 10, 2014.
Ypsomed AG. UnoPen™ – The intuitive dial and dose disposable pen [product description]. Burgdorf: Ypsomed AG; 2011. Available from: http://www.ypsomed.com/tl_files/ypsomed/data/documents/yds_downloads/products/91003_FAS_UP.pdf. Accessed March 10, 2014.
International Organization for Standardization. ISO 11608-2:2012: Needle-Based Injection Systems for Medical Use – Requirements and Test Methods. Part 2: Needles. 2nd ed. Geneva: ISO; 2012.
© 2014 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.