Development of a Measure to Assess Attitudes Towards Nasal versus Autoinjector Glucagon Delivery Devices for Treatment of Severe Hypoglycemia
Received 7 June 2022
Accepted for publication 17 September 2022
Published 21 November 2022 Volume 2022:15 Pages 3601—3615
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Konstantinos Tziomalos
Sanjay K Bajpai,1 M Janelle Cambron-Mellott,2 Oliver Will,2 Jiat-Ling Poon,1 Qianqian Wang,1 Beth D Mitchell,1 Eugenia Y Peck,2 Jane Babrowicz,2 Nedina K Raibulet,1 Christopher J Child,1 Kathleen Beusterien2
1Eli Lilly and Company, Indianapolis, IN, USA; 2Cerner Enviza, New York, NY, USA
Correspondence: Sanjay K Bajpai, Eli Lilly & Company, 893 S Delaware St, Indianapolis, IN, 46225, USA, Tel +1 317 931 9828, Email [email protected]
Background: For individuals managing diabetes, the administration of glucagon for severe hypoglycemia can be lifesaving, yet, until recently, there were no easy-to-use devices for these stressful emergencies. New products have emerged to meet this need, including nasal glucagon (NG) and auto-injector glucagon (AI). This study evaluated the psychometric properties of a new measure, the Glucagon Device Attitudes Questionnaire (GDAQ), in assessing attitudes toward NG and AI from the perspectives of persons with diabetes on insulin (PWDs), caregivers, and acquaintances.
Methods: Developed based on qualitative research, the GDAQ consists of 38 rating items for each device and 16 direct-elicitation of attitudes of device relative to each other. It was administered to participants via a cross-sectional online survey. Twenty-six rating items were included in principal component analysis and confirmatory factor analysis. Items comprising each factor were averaged to form scales. Additionally, 12 direct elicitation items were averaged to form an overall “Attitudes” scale. Reliability and validity analyses were conducted. Descriptive statistics were provided for the rating items not included in the factor analysis.
Results: A total of 405 PWDs, 313 caregivers, and 305 acquaintances participated. Three factors were identified: “Prepared and Protected” (7 items), “Hesitation” (12 items), and “Device Perceptions by Others” (7 items); factor loadings ranged from 0.13 to 0.92, 0.50 to 0.89, and 0.16 to 0.92, respectively. Cronbach’s alpha for the four scales ranged from 0.76 to 0.96. Correlations of the scales with their global item ranged from 0.30 to 0.90. The items outside of the factor analysis showed good distribution in responses and differentiation between the two devices.
Discussion: This study supports the validity and reliability of the GDAQ, which successfully conceptualizes attitudes towards devices for administering glucagon among different respondent groups. Use of the GDAQ can help guide the development and testing of new glucagon drug/device combinations.
Keywords: diabetes, severe hypoglycemic events, glucagon delivery device, patient attitudes
Devices that deliver rescue medications play an important life-saving role in a wide range of conditions, including asthma, hemophilia, chronic obstructive pulmonary disease, and allergy. For those individuals managing diabetes, this includes the administration of glucagon in the context of severe hypoglycemic events. Severe hypoglycemia, defined as a severe event characterized by altered mental and/or physical status requiring external assistance for recovery,1–3 is a serious and potentially life-threatening event and has been reported as affecting 30–40% of adults with Type 1 diabetes and 7–25% of adults with Type 2 diabetes yearly.4–6 Severe hypoglycemia is responsible for 4–10% of all deaths among those with Type 1 disease,2 and is associated with significant treatment costs, loss of productivity and burden.5,7–10
Individuals experiencing a severe hypoglycemia episode may be unconscious, seizing, or in an altered mental state and hence require the assistance of another person to actively administer carbohydrates, glucagon, or take other corrective actions.10 The unstable nature of glucagon resulted in conventional emergency glucagon kits being complex to use, and in the stress of a severe hypoglycemia emergency, can constitute a significant obstacle for those trying to help a person with diabetes.11–17
New products to deliver glucagon were approved by the US Food and Drug Administration (FDA) over the last two years to address a need for more practical and easy-to-use methods of glucagon delivery during severe hypoglycemia episodes. These products include: nasal glucagon (NG; Lilly), which is a dry powder ready-to-use synthetic glucagon,18 an autoinjector glucagon (AI; Xeris) that delivers a strict room-temperature (storing between 20° and 25°C [68° to 77°F]) stable liquid glucagon19 and a premixed dasiglucagon injection (Zegalogue; Zealand) that provides a peptide analog of human glucagon. Such delivery methods would likely improve the confidence, efficacy, accuracy, and likelihood of such treatments being administered in these stressful emergency situations. Specific device attributes, such as being easy to use and carry around, having minimal steps for delivery, and being compact and discreet, have been shown to translate into greater treatment satisfaction and across multiple disease contexts.20,21 Indeed, optimizing existing therapy delivery, or switching to a suitable alternative, can help avoid unnecessary escalation of treatment and health-care resources.22
Although measures have been developed to assess attitudes for medication delivery devices in diabetes,23–25 none focus on devices for delivering a rescue medication like glucagon. Moreover, none evaluate such devices from the perspective of caregivers and acquaintances, as well as persons with diabetes. Given the recent regulatory approval of three novel glucagon delivery devices designed to be easy-to-use, along with previous study findings that persons with diabetes, caregivers, and acquaintances value alternatives, it is important to be able to quantify attitudes towards NG versus AI delivery among persons with diabetes, caregivers, and acquaintances.
Previous work by the authorship group utilized qualitative interviews among 45 persons with diabetes on insulin, caregivers, and acquaintances to gain insight into the hypothetical features of a new glucagon device that are important to these groups.26 Features that participants indicated they would like to see in a new device included easy to use, uncomplicated, pre-mixed/ready-to-use, small/easy to carry, easy instructions, and not having a needle or long needles. If they had such a device, participants reported that they would feel more prepared and protected, in terms of being ready and having something on hand that is quick and easy to use, feeling more secure, as well as not being nervous or stressed. Individuals would also be more confident that others could assist in case of an emergency. Regarding their attitudes towards the device(s), participants indicated there could potentially be less hesitation in using a NG device over AI because others would feel more confident with a device without a needle, as well as not needing to locate and determine an injection site.
These qualitative results were used to guide the development of a new measure, the Glucagon Device Attitudes Questionnaire (GDAQ). The current study sought to evaluate the measurement properties of this new measure in assessing attitudes toward NG and AI from the perspective of persons with diabetes on insulin, caregivers, and acquaintances.
Materials and Methods
In a cross-sectional online survey, the GDAQ was administered to persons with diabetes on insulin, caregivers, and acquaintances. All participants were 18 years of age or older. Persons with diabetes had self-reported type 1 or type 2 diabetes and were currently on insulin (type 1 – basal-bolus, mix/premix, or continuous insulin and must have started within 1 year of being diagnosed; type 2 – basal, bolus, basal-bolus, mix/premix, or continuous insulin). Caregivers were individuals who live with or help a person who has type 1 or type 2 diabetes who was currently on insulin (if type 1, must have started within 1 year of diagnoses) and provide care on a regular basis for diabetes. Acquaintances were individuals who knew a person with diabetes (type 1, type 2, or unaware of the type), but do not provide care on a regular basis for their diabetes. Participants who selected having 10 or more conditions and professionally licensed health-care providers also were excluded.
Persons with diabetes, caregivers, and acquaintances were recruited in the US through ailment and general population panels maintained by Cerner Enviza Profiles and partners. Panel members received an e-mail with a general introduction to the availability of a new study and those interested completed an online screener, which was custom developed for the purposes of this study. The screener included a question asking potential participants to select which condition(s) they have from a list; those who did not select diabetes were asked if they are a caregiver of/or know someone who has any of the conditions from the list. If an interested individual qualified for the study, the screener provided a brief text-based introduction that explained the purpose and scope of the study. All participants completed an Informed Consent Form.
Exemption status from review by an ethics committee was received from the Sterling IRB on March 1, 2019. This research met the terms of the US Department of Health and Human Service’s Policy for Protection of Human Research Subjects at 45 C.F.R. §46.101 (b) category 2 which refers to research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior, unless information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and any disclosure of the human subjects’ responses outside the research could reasonably place the subjects at risk of criminal or civil liability, or be damaging to the subjects’ financial standing, employability, or reputation.
Glucagon Device Attitudes Questionnaire (GDAQ)
The GDAQ was developed based on findings from concept elicitation interviews with 45 persons with diabetes, caregivers, and acquaintances. The detailed results from this qualitative research are reported elsewhere.26 The measure includes rating items for each device, as well as direct elicitation items. Prior to rating items, respondents were provided a description of severe hypoglycemia, which stated that individuals with severe hypoglycemia may be in an altered mental state, having seizures or unconscious, cannot take care of themselves during a severe hypoglycemia event and require the help of another person. It also stated that glucagon must be administered by another person. Respondents were also provided descriptions of NG and AI devices, in a randomized order. The NG description provided the instructions of use including the following 1. Prepare (eg, “Remove the shrink wrap by pulling on red stripe. Open the lid and remove the device from the tube”), 2. Spray (eg, “Hold the device between fingers and thumb. Do not test before use. Insert the tip gently in one of the nostrils until finger(s) touch the outside of the nose. Push the plunger all the way in. The dose is complete when the green line is no longer showing”), and 3. Assist (“Turn patient on side. Call for medical help.”). The AI description included the following instructions of use: 1. Prepare (eg, “Tear open pouch and remove device. Tear at notch. Pull of red cap”), 2. Inject (eg, “Choose injection site and expose skin. Push down on skin to start. Hold down for 5 seconds. Wait for window to turn red”), and 3. Assist (eg, “Turn patient on side. Call doctor.”).
Rating Items for Each Device
The GDAQ was developed specifically to compare attitudes toward NG versus AI. It is comprised of 38 rating items asked of each device (7-point rating scales, ranging from “very unlikely” to “very likely” or “not at all” to extremely’). For each of these items, participants rated each of the two devices, which were randomized with respect to which was shown first. Of the 38 rating items, 33 were answered by all respondent groups, of which seven represented global assessments of the device (eg, “overall ease of use”, “overall like or dislike”), while two items were answered by caregivers and acquaintances only, two by persons with diabetes and caregivers only, and one by persons with diabetes only.
Direct Elicitation Items
In addition, the GDAQ includes 16 direct elicitation questions asking which device they would prefer, with a 7-point scale ranging from “Very strongly prefer NG” to “Very strongly prefer AI”. Again, the two devices were randomized in being the left anchor versus right anchor for these items. Four of these compared NG versus AI globally on: “overall preference for device”, “feeling prepared and protected with device”, “being less likely to hesitate in using device”, and “overall [perceived] ease of using device”. The remaining 12 items assessed the following attributes: “size of the device”, “being less complicated”, “feeling comfortable”, “method of delivering glucagon”, “causing less worry”, “overall convenience of the device”, “ability of children to use the device”, “administering glucagon to child”, “ease of preparing the device and medication for use”, “overall confidence in using the device correctly”, “carrying around the device” and “being easy to learn to use”.
Cognitive Debriefing Pre-Test Interviews
In an initial pre-testing phase, two rounds of cognitive debriefing interviews were conducted among persons with diabetes on insulin, caregivers, and acquaintances of persons with diabetes (n = 15 in each group) to obtain feedback on the GDAQ. Three trained interviewers followed a think aloud approach to obtain participant feedback on the GDAQ to ensure that the instructions and items were relevant and understandable and that they were being interpreted consistently by participants. Interviewer prompts included “What does this item mean to you?” and “How did you arrive at your answer?”. The interviews were conducted via telephone with a secure web sharing application so that respondents could view the questionnaire during the interview.
To confirm the content validity of the questionnaire, a content analysis was performed of the cognitive interview data, in which all responses were organized and evaluated by item. The moderators met intermittently to discuss the findings and potential revisions that should be made to the questionnaire. The revisions were then tested in the second round of cognitive debriefing interviews to confirm whether further changes were needed. Based on the findings, revisions were made to 11 of the questions initially included in the measure, and the GDAQ was then finalized; Table 1 details the revisions.
Table 1 Revisions to GDAQ
Rating Items for Each Device
This analysis focused on the 33 rating items that were answered by all respondent groups. First, attribute ratings were reverse coded as needed such that higher ratings indicate more favorable responses. A principal component analysis (PCA) was first performed to identify potential scales within the GDAQ that were answered by all groups. The PCA included rating items answered by all respondent groups, with the exception of the seven global assessment items. Separate PCAs were run for all respondents in aggregate and by respondent group, as well as by device, for a total of eight PCAs. All PCAs were run using the SAS PRINCOMP procedure. The number of factors were determined by examining the scree plot and the percent variance explained for all respondent groups. Attribute groupings were determined, for the most part, by the factor loadings.
Once the number of factors and attribute groupings were identified, the attributes were assigned to their corresponding factors and confirmatory factor analyses (CFA) were conducted using a structural equation model with the SAS CALIS procedure. Comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR) were assessed. In order to be able to compare factors by device and across respondent groups, the same attributes were assigned to the same factors for each device and each respondent group. Items comprising each factor were then averaged to create scales. Separate scales were computed for NG and AI. Separate CFAs were run for all respondents in aggregate and by each respondent group, as well as by device, for a total of eight CFAs.
Cronbach’s alpha was computed to assess the internal consistency of each scale. A Cronbach’s alpha of 0.70 is recognized as an acceptable threshold for assessing the internal consistency reliability of a scale of items.27,28 In addition, Cronbach’s alpha was computed after removing each rating item from its scale. Inter-item correlations were also computed to assess how well the items comprising a scale hung together. Optimal mean inter-item correlation values range from 0.2 to 0.4.29 Items were further evaluated and considered for movement to another scale if they had the Cronbach’s alpha increased when the item was removed from the scale for both NG and AI groups.
To assess predictive validity of the scale, each scale was correlated with its global rating item (with that item removed from the scale, if applicable). In addition, means, standard deviations, minimums, maximums, and the proportion of respondents choosing the lowest and highest response categories were computed for the 12 rating items not included in the factor analyses; this included the 7 global assessment rating items rated by all respondent groups as well as the 5 rating items that were not answered by all respondent groups.
Direct Elicitation Items
Direct elicitation items were recoded so that scores above four were in favor of NG, then the 12 non-global items were averaged to create a “Attitude” scale. Cronbach’s alpha was computed to assess the internal consistency of the “Attitude” scale, and inter-item correlations were also computed to assess how well the items comprising the “Attitude” scale hung together. Finally, the “Attitude” scale was correlated with its global rating item, “overall preference for device”.
A total of 1030 respondents met the inclusion criteria and completed the study; 405 respondents qualified as persons with diabetes on insulin, 320 qualified as caregivers, and 305 qualified as acquaintances. Quality checks led to the removal of 7 caregivers who indicated a lack of attention; n = 3 completed the survey in less than half the median time, and n = 4 used the same rating for each item in three or more of the GDAQ item rating sections that had both positively worded and negatively worded attributes (note that scores of 4, indicating neutrality, were allowed). Therefore, the final analysis sample included 405 persons with diabetes on insulin, 313 caregivers, and 305 acquaintances.
Device Rating Items
Principal Component Analysis & Confirmatory Factor Analysis results
The results for the PCA analyses were similar for the total sample and three respondent groups; persons with diabetes on insulin, caregivers, and acquaintances. They were similar by device as well. The 3-factor solution was chosen based on the scree plot and the percent variance explained for all respondent groups (60.0% for NG, Figure 1; 60.7% for AI, Figure 2). Within the 3-factor solution, we looked at the loadings of the attribute on the factor, especially which loading was the largest. In general, the majority of items gathered on one of two factors for every respondent group; in cases where they did not, they still loaded reasonably well within the same factor. A third group of items had low loadings and changed the factor they loaded on between respondent groups, so the decision was made to cordon these attributes into their own factor. Therefore, we assigned the attributes to three factors, 1) “Prepared and Protected” (7 items), 2) “Hesitation” (12 items), and 3) “Device Perceptions by Others” (7 items) (Table 2) and ran CFA goodness-of-fit metrics.
Table 2 Final Factor Loadings for Scales by Respondent Group
Figure 1 Principal component analysis results for total sample: Scree plot and variance explained for NG rating items.
Abbreviation: NG, nasal glucagon.
Figure 2 Principal component analysis results for total sample: Scree plot and variance explained for AI rating items.
Abbreviation: AI, autoinjector glucagon.
The final assignment of attributes to factors as well as the factor loadings for the final scales are shown in Table 2, and fit metrics for the CFA are shown in Table 3. Table 4 provides descriptive statistics for device rating items not included in factor analyses by respondent group. Factor loadings ranged from 0.13 to 0.92 for 0.13 to 0.92 for “Prepared and Protected”, from 0.50 to 0.89 for “Hesitation”, and from 0.16 to 0.92 for “Device Perceptions by Others”. SRMR is below 0.1 for the AI model among caregivers and for both devices among acquaintances, indicating adequate fit.
Table 3 Fit Metrics for Confirmatory Factor Analysis
Table 4 Descriptive Statistics for Device Rating Items Not Included in Factor Analyses by Respondent Group
Cronbach’s alpha for each scale by respondent group is shown in Table 5.
Table 5 Cronbach’s Alpha for Final Scales by Respondent Group
Prepared and Protected
Cronbach’s alpha for “Prepared and Protected” ranged from 0.76 to 0.82 for NG and from 0.82 to 0.86 for AI, indicating that this scale has good internal consistency. Examination of Cronbach’s alpha if item is deleted showed that removal of “Likely to use device correctly” would only marginally improve alpha if deleted across respondent groups (largest improvement in alpha = 0.02 from 0.76 to 0.78 for acquaintances for NG only). In addition, among acquaintances, removal of “Confident get/gave full dose” would minimally improve alpha (both devices; largest improvement = 0.017, from 0.82 to 0.83). These improvements were small enough not to warrant removing these items from the scale.
Cronbach’s alpha for “Hesitation” ranged from 0.91 to 0.96 for NG and from 0.94 to 0.95 for AI, indicating that this scale has excellent internal consistency. Examination of Cronbach’s alpha if item is deleted showed that removal of “Embarrassed” would only very minimally improve alpha if deleted for respondent groups (both devices) and acquaintances (AI only) (largest improvement in alpha = 0.002; alpha remains 0.95). As this finding was not consistent across respondent groups and devices, Cronbach’s alpha is already high, and there was minimal improvement, this item was kept in the scale.
Device Perceptions by Others
Cronbach’s alpha for “Device Perceptions by Others” was 0.81 for NG across all respondent groups and ranged from 0.81 to 0.87 for AI, indicating this scale has good internal consistency. Examination of Cronbach’s alpha if item is deleted showed no items that would improve alpha if deleted across all respondent groups. Deleting “Likely to use device correctly (other people)” would improve alpha marginally by 0.01–0.02 for persons with diabetes and caregivers for both devices and by 0.01 for acquaintances for NG only. In addition, deleting “Comfortable to use (other people)” would improve alpha by 0.00–0.03 for persons with diabetes and acquaintances, but would not improve alpha for caregivers. Finally, deletion of “Difficulty in finding location (other people)” would improve alpha by 0.02 for caregivers and would improve alpha 0.00–0.02 for persons with diabetes and acquaintances for NG only. Thus, we retained all items in this scale.
Prepared and Protected
Inter-item reliabilities for the “Prepared and Protected” scale for NG ranged from 0.20 to 0.86 for persons with diabetes, 0.15 to 0.83 for caregivers, and 0.05 to 0.79 for acquaintances. Most items had small to large positive correlations with each other and were significant at P<0.05. Inter-item reliabilities for the “Prepared and Protected” scale for AI ranged from 0.17 to 0.81 for persons with diabetes, 0.30 to 0.87 for caregivers, and 0.14 to 0.83 for acquaintances. All items had small to large positive correlations with each other and were significant at P<0.05 for all respondent groups.
Inter-item reliabilities for the “Hesitation” scale for NG ranged from 0.50 to 0.82 for persons with diabetes, 0.44 to 0.77 for caregivers, and 0.28 to 0.74 for acquaintances. All inter-item correlations were moderate to large and were significant at P<0.001. Inter-item reliabilities for the “Hesitation” scale for AI ranged from 0.39 to 0.82 for persons with diabetes, 0.39 to 0.81 for caregivers, and 0.38 to 0.88 for acquaintances. All inter-item correlations were moderate to large and significant at P<0.001.
Device Perceptions by Others
Inter-item reliabilities for the “Device Perceptions by Others” scale for NG ranged from −0.12 to 0.83 for persons with diabetes, −0.04 to 0.74 for caregivers, and 0.06 to 0.33 for acquaintances. Most inter-item correlations were significant at P<0.05 and were small to large positive correlations. Inter-item reliabilities for the “Device Perceptions by Others” scale for AI ranged from 0.00 to 0.81 for persons with diabetes, 0.16 to 0.79 for caregivers, and 0.27 to 0.76 for acquaintances. Most inter-item correlations were significant at P<0.05 and small to moderate positive correlations.
Correlations Between the Scales and Global Items
Correlations between the scales and their global items are shown in Table 6.
Table 6 Correlations of Scales with Global Item by Respondent Group
Prepared and Protected
For NG, the correlation between the “Prepared and Protected” scale with its global item, “Feel prepared and protected”, was 0.60 for persons with diabetes and caregivers and 0.55 for acquaintances. For AI, the correlation between the “Prepared and Protected” scale with its global item, “Feel prepared and protected” was 0.63 for persons with diabetes, 0.74 for caregivers, and 0.69 for acquaintances.
For NG, the correlation between the “Hesitation” scale with its global item, “Hesitate in using device”, was 0.79 for persons with diabetes and 0.72 for caregivers and acquaintances. For AI, the correlation between the “Hesitation” scale with its global item, “Hesitate in using device” was 0.78 for persons with diabetes, 0.76 for caregivers, and 0.80 for acquaintances.
Device Perceptions by Others
For NG, the correlation between the “Device Perceptions by Others” scale with its global item, “Likelihood to prefer (other people)” was 0.37 for persons with diabetes, 0.40 for caregivers, and 0.30 for acquaintances. For AI, the correlation between the “Device Perceptions by Others” scale with its global item, “Likelihood to prefer (other people)” was 0.45 for persons with diabetes, 0.53 for caregivers, and 0.58 for acquaintances.
Rating Items for Each Device Not Included in Factor Analyses
Descriptive statistics for the 12 ratings items not included in the factor analyses (the 7 global rating items asked by all respondent groups and the 5 rating items asked of only some respondent groups) are shown in Table 4 for each device by respondent group. Except for “Overall perceived ease of use of NG” among caregivers and acquaintances, where the minimum rating was 2, the full range of responses were used for these items for each device by all respondent groups. There is clear differentiation between the two devices on all global rating items. For NG, the proportion of respondents with scores at floor (minimum score) ranged from 0.0% to 59.7% and ceiling (maximum score) ranged from 3.8% to 62.9% across these rating items and respondent groups. For AI, the proportion of respondents with scores at floor (minimum score) ranged from 1.2% to 60.3% and ceiling (maximum score) ranged from 2.6% to 47.9% across these rating items and respondent groups. “Comfortable asking others to administer device in an emergency” was above the threshold of 15% for floor effects for both NG and AI among persons with diabetes and caregivers, and “Likelihood to call 911 before use” and “Likelihood to call 911 after use” were above the threshold of 15% for floor effects for NG and AI among caregivers and acquaintances. In addition, “Overall ease of use for children” was above 15% for AI for all respondent groups. Few items were under the threshold of 15% for ceiling effects for both devices across respondent groups. However, all global items were able to differentiate perceptions between NG and AI.
Direct Elicitation Items
Cronbach’s alpha for the “Attitudes” scale was 0.96 for all respondent groups. Examination of Cronbach’s alpha if item is deleted showed no items that would improve alpha if deleted across all respondent groups. The only item showing improvement in alpha if deleted was “Administering glucagon to child” among persons with diabetes only. This item would only improve alpha by <0.001, and as alpha is already very high, all items were retained in this scale.
Inter-item reliabilities for the “Attitudes” scale ranged from 0.46 to 0.84 for persons with diabetes, 0.49 to 0.76 for caregivers, and 0.52 to 0.78 for acquaintances. All inter-item correlations were large and were significant at P<0.001.
Correlations Between the Scales and Global Items
Correlation between the “Attitudes” scale with its global item, “Overall preference for device” was 0.90 for persons with diabetes, 0.86 for caregivers, and 0.87 for acquaintances (Table 6).
The current study describes the development and validation of the GDAQ, a comprehensive measure of attitudes towards glucagon delivery device(s) that addresses an important unmet need in the literature. Drug and device developers have increasingly recognized the importance of integrating patient attitudes and patient-centered design strategies into the development of their products. Clinical treatment recommendations have also shifted towards a more patient-centered approach, with the American Diabetes Association noting the critical importance of integrating individuals’ values and attitudes in the development of treatment plans.3
This approach to device development, however, relies upon the ability to accurately and effectively assess patient attitudes and attitudes using validated measures. Despite this, there currently exist no validated measures that focus on attitudes toward new glucagon delivery devices. Further, those more general satisfaction measures that do exist tend to possess ceiling effects,30–32 lack specificity for distinguishing attitudes among different devices, and are not appropriate for assessing the attitudes of caregivers and acquaintances. The GDAQ addresses each of these limitations and this broader gap in the literature.
Importantly, the GDAQ was developed in line with the guidelines put forward by the FDA concerning patient reported outcomes33 and a review of previous scale development research, including the Medication Delivery Device Assessment Battery (MDDAB)23,25 and the Diabetes Injection Device-Preference Questionnaire (DID-PQ).24,34 The GDAQ was designed to align with these previous measures, as well as provide insight into attitudes towards specific glucagon devices. In addition to this solid empirical foundation, the qualitative concept elicitation phase also helped inform the development of question phrasing and scale domains.26
The study findings confirmed the measurement properties of the GDAQ and supported the results of the initial concept elicitation research.26 The GDAQ was able to effectively differentiate between attitudes concerning NG when compared to AI and represents an innovative approach in utilizing a reference when providing ratings to achieve better differentiation and address the ceiling effects inherent to similar patient-reported measures.
Ultimately, the ability to accurately assess attitudes towards devices for the administration of medications among persons with diabetes, caregivers and acquaintances can help guide drug and device development. In the context of severe hypoglycemia emergencies, an easier-to-use delivery method would likely improve the confidence, efficacy, accuracy, and likelihood of use among caregivers and acquaintances. This in turn would serve to reduce the burden and stress on persons with diabetes and those assisting in a severe hypoglycemia emergency, as well as the possibility to reduce health-care costs at large through limiting severe hypoglycemia-related emergency responses and emergency department visits.
The GDAQ yielded three distinct scales. These included the 1) “Prepared and Protected” scale, which addressed issues concerning how safe and prepared people feel carrying the device, 2) “Hesitation” scale, which addressed levels of fear and panic associated with the use of the device, and finally, and 3) “Device Perceptions by Others” scale, which considers the perspective of “others”, who may be less familiar with severe hypoglycemia-related emergencies and glucagon devices. The reliability measurements from this robust sample of participants further supports the use of these three scales. The feedback from the cognitive interviews informed revisions to the GDAQ to ensure that items are understood and interpreted in a uniform way among respondents. Notable revisions included rewording that clarified the person using the device, the age of the person in question, and terminology that was more easily understood (eg, reassured versus secure).
The current findings should be considered in light of the study limitations. The online administration of the survey can limit generalizability to populations less familiar with online technology. Further, those surveyed had not used NG and AI devices and thus provided feedback and responses based on hypothetical descriptions. It is also possible that AI devices received more favorable reviews based on social desirability and some participants’ unwillingness to disclose a fear of needles or a desire for easier-to-use devices given their acknowledged role as a person with diabetes, caregiver or acquaintance. Finally, we did not examine test-retest reliability of the GDAQ, or the sensitivity and specificity of the device, which should be a focus of future research.
The current study confirmed the content validity and reliability, including the internal consistency, of the GDAQ. The measure, with three distinct scales, successfully conceptualizes device attitudes among persons with diabetes, caregivers and acquaintances. The GDAQ represents an important tool in device assessment and one that can help guide the development and testing of new drugs and devices in the glucagon delivery space.
AI, autoinjector; AQ, acquaintance; CFA, confirmatory factor analysis; CFI, comparative fit index; CG, caregiver; FDA, Food and Drug Administration; GDAQ, Glucagon Device Attitudes Questionnaire; NG, nasal glucagon; PCA, principal component analysis; PWD, person with diabetes on insulin; RMSEA, root mean square error of approximation (RMSEA); SRMR, standardized root mean square residual.
Data Sharing Statement
The datasets used and/or analyzed during the current study are available from Eli Lilly for reasonable request subsequent to Eli Lilly approvals of appropriate documentations, such as, appropriate confidentiality and data retention agreements as well as detailed protocols, analytic plans and publication/disclosure plans for the proposed usage of the data.
Ethics Approval and Informed Consent
All participants completed an Informed Consent Form. Exemption status from review by an ethics committee was received from the Sterling IRB on March 1, 2019 (Study ID 6494). This research met the terms of the US Department of Health and Human Service’s Policy for Protection of Human Research Subjects at 45 C.F.R. §46.101 (b) category 2 which refers to research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior, unless information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and any disclosure of the human subjects’ responses outside the research could reasonably place the subjects at risk of criminal or civil liability, or be damaging to the subjects’ financial standing, employability, or reputation.
Consent for Publication
All participants completed an Informed Consent Form (ICF) electronically. In the ICF, participants were informed that the results of the study may be published. Exemption status from review by an ethics committee was received from the Sterling IRB on March 1, 2019. This research met the terms of the US Department of Health and Human Service’s Policy for Protection of Human Research Subjects at 45 C.F.R. §46.101 (b) category 2 which refers to research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior, unless information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and any disclosure of the human subjects’ responses outside the research could reasonably place the subjects at risk of criminal or civil liability, or be damaging to the subjects’ financial standing, employability, or reputation.
Errol J. Philip PhD for writing support.
All authors are responsible for the work described in this paper. All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.
This study was funded by Eli Lilly and Company. Cerner Enviza received funding from Eli Lilly and Company to conduct this study and to write and submit the paper for publication.
SKB, JLP, QW, BDM, and NKR are full-time employees of Eli Lilly and Company (Indianapolis, IN, USA). CJC is a full-time employee of Eli Lilly and Company (Erl Wood Manor, Windlesham, UK). JLP, BDM, and CJC own stock in Eli Lilly and Company. MJCM, OW, EYP, JB, and KB are employees of Cerner Enviza, who provided consulting services on this study. The authors report no other conflicts of interest in this work.
1. Workgroup on Hypoglycemia. Defining and reporting hypoglycemia in diabetes: a report from the American Diabetes Association workgroup on hypoglycemia. Diabetes Care. 2005;28:1245–1249. doi:10.2337/diacare.28.5.1245
2. Seaquist ER, Anderson J, Childs B, et al. Hypoglycemia and diabetes: a report of a workgroup of the American Diabetes Association and the Endocrine Society. Diabetes Care. 2013;36(5):1384–1395. doi:10.2337/dc12-2480
3. American Diabetes Association. Diabetes advocacy: standards of medical care in diabetes 2020. Diabetes Care. 2020;43(Suppl. 1):S203–S204. doi:10.2337/dc20-S016
4. UK Hypoglycaemia Study Groups. Risk of hypoglycaemia in types 1 and 2 diabetes: effects of treatment modalities and their duration. Diabetologia. 2007;50(6):1140–1147. doi:10.1007/s00125-007-0599-y
5. Frier BM. Hypoglycaemia in diabetes mellitus: epidemiology and clinical implications. Nat Rev Endocrinol. 2014;10:711–712. doi:10.1038/nrendo.2014.170
6. Edridge CL, Dunkley AJ, Bodicoat DH, et al. Prevalence and incidence of hypoglycaemia in 532,542 people with type 2 diabetes on oral therapies and insulin: a systematic review and meta-analysis of population based studies. PLoS One. 2015;10(6):e0126427. doi:10.1371/journal.pone.0126427
7. Goldstein D, Chodick G, Shalev V, Thorsted BL, Elliott L, Karasik A. Use of healthcare services following severe hypoglycemia in patients with diabetes: analysis of real-world data. Diabetes Ther. 2016;7(2):295–308. doi:10.1007/s13300-016-0169-8
8. McEwan P, Larsen Thorsted B, Wolden M, Jacobsen J, Evans M. Healthcare resource implications of hypoglycemia-related hospital admissions and inpatient hypoglycemia: retrospective record-linked cohort studies in England. BMJ Open Diabetes Res Care. 2015;3(1):e000057. doi:10.1136/bmjdrc-2014-000057
9. Bronstone A, Graham C. The potential cost implications of averting severe hypoglycemic events requiring hospitalization in high-risk adults with type 1 diabetes using real-time continuous glucose monitoring. J Diabetes Sci Technol. 2016;10(4):905–913. doi:10.1177/1932296816633233
10. American Diabetes Association. 10 Microvascular complications and foot care: standards of medical care in diabetes-2018. Diabetes Care. 2018;41(Suppl 1):S105–s118. doi:10.2337/dc18-S010
11. Pontiroli AE, Ceriani V. Intranasal glucagon for hypoglycaemia in diabetic patients. An old dream is becoming reality? Diabetes Obes Metab. 2018;20(8):1812–1816. doi:10.1111/dom.13317
12. Yale JF, Dulude H, Egeth M, et al. Faster use and fewer failures with needle-free nasal glucagon versus injectable glucagon in severe hypoglycemia rescue: a simulation study. Diabetes Technol Ther. 2017;19(7):423–432. doi:10.1089/dia.2016.0460
13. Harris G, Diment A, Sulway M, Wilkinson M. Glucagon administration - undervalued and undertaught. Pract Diab Int. 2001;18(1):22–25. doi:10.1002/pdi.138
14. Deacon B, Abramowitz J. Fear of needles and vasovagal reactions among phlebotomy patients. J Anxiety Disord. 2006;20(7):946–960. doi:10.1016/j.janxdis.2006.01.004
15. Nir Y, Paz A, Sabo E, Potasman I. Fear of injections in young adults: prevalence and associations. Am J Trop Med Hyg. 2003;68(3):341–344. doi:10.4269/ajtmh.2003.68.341
16. Pfutzner A, Schipper C, Niemeyer M, et al. Comparison of patient preference for two insulin injection pen devices in relation to patient dexterity skills. J Diabetes Sci Technol. 2012;6(4):910–916. doi:10.1177/193229681200600423
17. Taddio A, Ipp M, Thivakaran S, et al. Survey of the prevalence of immunization non-compliance due to needle fears in children and adults. Vaccine. 2012;30(32):4807–4812. doi:10.1016/j.vaccine.2012.05.011
18. Rickels MR, Ruedy KJ, Foster NC, et al. Intranasal glucagon for treatment of insulin-induced hypoglycemia in adults with type 1 diabetes: a randomized crossover noninferiority study. Diabetes Care. 2016;39(2):264–270. doi:10.2337/dc15-1498
19. Pharmaceuticals X. Xeris pharmaceuticals achieves phase 3 milestones of its ready-to-use glucagon rescue pen for the treatment of severe hypoglycemia. Xeris Pharmaceuticals, Inc.; 2018. Available from: https://globenewswire.com/news-release/2018/02/15/1349229/0/en/Xeris-Pharmaceuticals-Achieves-Phase-3-Milestones-of-its-Ready-to-Use-Glucagon-Rescue-Pen-for-The-Treatment-of-Severe-Hypoglycemia.htm.
20. Barbosa CD, Balp MM, Kulich K, Germain N, Rofail D. A literature review to explore the link between treatment satisfaction and adherence, compliance, and persistence. Patient Prefer Adherence. 2012;6:39–48. doi:10.2147/PPA.S24752
21. Stewart KD, Johnston JA, Matza LS, et al. Preference for pharmaceutical formulation and treatment process attributes. Patient Prefer Adherence. 2016;10:1385–1399. doi:10.2147/PPA.S101821
22. Prevention CfDCa. Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER). National Center for Health Statisitcs Web site; 2014. Available from: https://www.cdc.gov/qdrl/b2questevalu/QECognInter210.htm.
23. Matfin G, Van Brunt K, Zimmermann AG, Threlkeld R, Ignaut DA. Safe and effective use of the once weekly dulaglutide single-dose pen in injection-naive patients with type 2 diabetes. J Diabetes Sci Technol. 2015;9(5):1071–1079. doi:10.1177/1932296815583059
24. Matza LS, Boye KS, Stewart KD, Paczkowski R, Jordan J, Murray LT. Development of the Diabetes Injection Device Experience Questionnaire (DID-EQ) and Diabetes Injection Device Preference Questionnaire (DID-PQ). J Patient-Rep Outcomes. 2018;2:43. doi:10.1186/s41687-018-0068-z
25. Yu M, Brunt KV, Milicevic Z, Varnado O, Boye KS. Patient-reported outcomes in patients with type 2 diabetes treated with dulaglutide added to titrated insulin glargine (AWARD-9). Clin Ther. 2017;39(11):2284–2295. doi:10.1016/j.clinthera.2017.10.002
26. Bajpai SK, Cambron-Mellott MJ, Peck E, et al. Perceptions about glucagon delivery devices for severe hypoglycemia: qualitative research with patients, caregivers, and acquaintances. Clin Ther. 2019;41(10):2073–2089.e2076. doi:10.1016/j.clinthera.2019.08.006
27. Altman DG, Bland JM. Statistics notes. Units of analysis. BMJ. 1997;314(7098):1874. doi:10.1136/bmj.314.7098.1874
28. Nunnally JC. Psychometric Theory.
29. Briggs SR, Cheek JM. The role of factor analysis in the development and evaluation of personality scales. J Pers. 1986;54(1):106–148. doi:10.1111/j.1467-6494.1986.tb00391.x
30. Atkinson MJ, Sinha A, Hass SL, et al. Validation of a general measure of treatment satisfaction, the Treatment Satisfaction Questionnaire for Medication (TSQM), using a national panel study of chronic disease. Health Qual Life Outcomes. 2004;2:12. doi:10.1186/1477-7525-2-12
31. Peterson RA, Wilson WR. Measuring customer satisfaction: fact and artifact. Acad Mark Sci Rev. 1992;20(1):61. doi:10.1007/BF02723476
32. Andrew S, Salamonson Y, Everett B, Halcomb EJ, Davidson PM. Beyond the ceiling effect: using a mixed methods approach to measure patient satisfaction. Int J Mult Res Approaches. 2011;5(1):52–63. doi:10.5172/mra.2011.5.1.52
33. Food and Drug Administration. Patient-reported outcome measures: use in medical product development to support labeling claims; 2009.
34. Matza LS, Stewart KD, Paczkowski R, Coyne KS, Currie B, Boye KS. Psychometric evaluation of the Diabetes Injection Device Experience Questionnaire (DID-EQ) and Diabetes Injection Device Preference Questionnaire (DID-PQ). J Patient-Rep Outcomes. 2018;2:44. doi:10.1186/s41687-018-0064-3
© 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.