Acceptability of and willingness to pay for using a smartphone-based vaccination application in a Vietnamese cohort

Introduction

Advances in vaccine research have led to an unprecedented reduction in morbidity and mortality associated with various diseases.¹ However, incidents of previously eliminated vaccine-preventable diseases (VPDs) are still being reported around the world.^2–4 Previous studies have suggested that insufficient vaccination coverage and poor compliance to vaccine schedules, either delayed or earlier-than-recommended vaccination, rather than the effectiveness of vaccines themselves, remain significant contributors to these outbreaks.^2–5 Commonly cited explanations for inadequate coverage and lack of timely vaccination include missing vaccination appointments, being unaware of vaccination schedules, and intentional delaying of getting vaccinated partly due to the dissemination of inadequate information regarding side effects of vaccination.^6–8

One promising solution to improve vaccination coverage and compliance is the use of vaccination schedule reminder.^9,10 A Cochrane review of 75 studies from 10 countries reported growing evidence supporting the use of postcards, text messages, and autodialer to improve the receipt of immunization.⁹ For example, text messaging has been advocated as a low-cost solution to remind parents of the vaccination schedules for their children, thereby improving vaccination rates, especially in the rural areas of low-income countries.¹¹ Other communication technologies, such as automated phone calls and interactive computer videos and e-mails, have also been shown to be useful, for instance, in promoting the initiation of human papillomavirus vaccination and enhancing vaccination compliance.¹² In particular, with the advances in smartphone technologies, Katib et al¹³ described how smartphone-based applications could be utilized to record and track whether children in rural communities had received the recommended vaccinations. Moreover, a randomized trial, conducted by Chen et al,¹⁴ demonstrated the effective use of a smartphone-based application in improving vaccination coverage and enabling health care professionals to identify individuals who might have missed their vaccination schedule in rural China.

In Vietnam, the Expanded Program on Immunization (EPI) has contributed to the prevention of approximately 2.3–5.7 million VPD incidents and 10,000–26,000 fatal cases during the period of 1980–2010 as well as the eradication of polio and tetanus in 2000 and 2005.¹⁵ Despite these achievements, the country continues to experience outbreaks of VPDs. For example, 15,000 cases of measles – a disease that was covered under the EPI program – were reported in 2014.¹⁶ Of those infected, 86% were either having unconfirmed vaccination status or not being vaccinated at all.¹⁶ Moreover, untimely vaccination has been shown to more likely happen among children from poorer and less educated households in rural and remote areas where access to vaccination information, such as vaccination schedule, benefits, and safety of vaccines, is limited.¹⁶ This inadequate access to proper information about vaccination might lead to problems of forgetting or not knowing when and where to get vaccinated along with the hesitation toward vaccination due to safety concerns. Furthermore, intentional delay and rejection of vaccination due to the fear of adverse events following immunization (AEFIs) have been shown to greatly impact the effectiveness of vaccination programs. For example, it has been reported that the dissemination of inadequate information regarding AEFIs by the media contributed to a decrease in over 30% in the coverage of hepatitis B vaccination in Vietnam in 2013.¹⁷ Nonetheless, the country has started taking initiatives to tackle such problem, including the use of vaccination management smartphone applications, which have been piloted to go beyond being a simple schedule reminder tool.¹⁸ Specifically, such an application could serve as a more personalized system to record, for example, other health information each time a user is being vaccinated. This feature might help patients and health care providers better monitor one’s health conditions to avoid complications and reduce the potential adverse effect. Moreover, patients might use the applications to register in a national health information system – an important measure in accurately monitoring vaccination uptake.

To fully evaluate the effectiveness, adoption, and feasibility of implementing mobile phone applications into vaccination management programs in Vietnam, a greater knowledge about the perception and financial feasibility of such strategy among the key populations for vaccination in the context of Vietnam are warranted. In this study, we sought to determine the willingness to use and willingness to pay (WTP) for smartphone-based vaccination management applications as well as the potential-associated factors among Vietnamese adults at a vaccination clinic. With smartphone users accounting for the majority of mobile phone user populations in both the cities and rural areas of Vietnam,¹⁹ smartphone-based applications might serve as an effective platform to educate and deliver important information about vaccination and immunization to a diverse population, especially those in areas that would be otherwise difficult to reach.

Subjects and methods

Study setting and sampling method

A cross-sectional study was conducted between March and April 2016 in an urban vaccination clinic that provided both paid and free vaccinations under the national EPI at the Hanoi Medical University in Vietnam. The average number of vaccinees, including both adults and their children at this clinic, ranges from 20 to 30 persons each day. Convenience sampling was adopted to recruit a total of 429 participants, who either came to get vaccinated or brought their children for vaccination, according to the following inclusion criteria: 1) participants who received vaccination at the clinic; 2) participants who were at least 18 years old; and 3) participants who provided their written consents to participate in the study. There was no exclusion criterion. To ensure their confidentiality and comfort while participating in the study, the questionnaire was administered to each participant in a private consultation room at the clinic. There, we explained to each participant the risks and benefits of using smartphone-based vaccination management applications as well as additional application functions that could be provided with co-payments.

Measurements and instruments

Sociodemographic questionnaire

Data about gender, ethnicity, current educational attainment, religion, current marital status, employment status, and monthly household income of the participants were collected.

Acceptability of using smartphone-based vaccination management applications

Using a structured questionnaire, we asked the participants about their current usage and knowledge of general and vaccination management smartphone applications. In addition, we assessed their perception about the necessity of and willingness to use such applications given the existing availability of information about vaccination from community health workers and on the Internet.

WTP for smartphone-based vaccination management applications

We evaluated the WTP among the participants based on their gender, age, ethnicity, current educational attainment, religion, marital status, and employment status. To determine the amount of WTP, we used a bidding game technique, which included double-bounded dichotomous and open-ended questions. Briefly, the participants were asked a series of yes/no questions with regard to their WTP (Figure 1). First, we asked them about their attitude regarding the necessity of smartphone-based vaccination management applications using a 5-point Likert scale (from very necessary to very unnecessary). Among the participants who answered “neutral”, “necessary”, or “very necessary”, we asked them about their willingness to use such applications. Second, for the participants who answered “yes” to the willingness to use inquiry, we asked them whether they were willing to pay 200,000 Vietnamese dong (VND; US $9) for a smartphone-based vaccination management application. Depending on their responses, we would then present two bidding options: higher bid for clients who have answered “yes” and lower bid for clients who have answered “no”. The bidding process continued until the bid was equal to four times or one-eighth of the initial proposed pricing. Finally, participants were asked an open-ended question, “What is the maximum price you would be willing to pay for a smartphone-based vaccination management application?”

Figure 1 The bidding process (unit: US$). Notes: N indicates unwillingness to pay; Y indicates WTP. Abbreviation: WTP, willingness to pay.

Statistical analyses

Data were analyzed by STATA software version 12.0 (StataCorp LP, College Station, TX, USA). A P-value of <0.05 was considered statistically significant. A stepwise logistic model with a threshold of P-value <0.2 was used to identify factors associated with the acceptability of and WTP for using a smartphone-based vaccination management application.

Ethical approval

The protocol used in this study was approved by the ethics committee of the Hanoi Medical University. All participants signed written informed consents prior to their participation, acknowledging their understanding of the study’s purpose, their rights to withdraw from the study at any time, and the protection of their collected data and confidentiality.

Results

Sociodemographic characteristics of the participants

Overall, the majority of the participants were female (73.5%), belonged to the Kinh ethnicity (97.7%), and were in their late 20s (26.8±6.2 years old; Table 1). Most had obtained an education that was above the high school level (63.2%) and did not associate with a religion (97.4%). Among the participants who brought their children for vaccination, 90.6% were living with a spouse or partner. In contrast, 75.3% of the participants who self-presented for vaccination were single, divorced, or widowed. In this cohort, a number of participants were either white-collar workers (43.4%) or students (37.8%). Of whom, 66.7% of those who brought their children for vaccination were white-collar workers, and 56.9% of those who self-presented for vaccination were students. Regarding the household monthly income, the participants made an average of 15.9±32.2 million VND in which participants who brought their children for vaccination made more than those who self-presented for vaccination (20.8±52.8 vs 13.4±11.4 million VND, P<0.01). Significant differences were also found between those who brought their children for vaccination and those who self-presented for vaccination with regard to gender, ethnicity, education, marital status, employment, and age.

Table 1 Sociodemographic characteristics of the participants Notes: aParticipants who brought their children to receive vaccination at the selected clinic. bParticipants who went to the clinic to get vaccinated. cP-values referred to the comparisons of the listed characteristics between the participants who brought their children to receive vaccination and those who got vaccinated at the selected clinic. Abbreviation: VND, Vietnamese dong.

Acceptability of and WTP for using smartphone-based vaccination management applications

In this cohort, we observed a high percentage of smartphone usage (90.1%; Table 2). Of whom, 68.5% reported a frequent usage. However, of those who reported using smartphone applications, only 5% claimed to be aware of existing smartphone-based vaccination management applications. Interestingly, among those who were aware of the applications, 57.9% reported that they had used them in the past. Moreover, we observed a greater usage among the participants who brought their children for vaccination, compared to those who self-presented for vaccination (81.8% vs 25.0%, P=0.01).

Table 2 Acceptability of using smartphone-based vaccination management applications Notes: aParticipants who brought their children to receive vaccination at the selected clinic (n=157). bParticipants who went to the clinic to get vaccinated (n=267). cP-values referred to the comparisons of the listed characteristics between the participants who brought their children to receive vaccination and those who got vaccinated at the selected clinic.

Regarding the willingness to use the smartphone-based vaccination applications, 69.6% of all participants felt that such application was necessary. Of the 367 participants who felt at least neutral about the necessity of these applications, 93.7% were willing to use the apps. Importantly, among those who were unwilling to use smartphone-based vaccination management applications or thought that they were unnecessary, 44.4% felt that there was sufficient vaccination-related information available online, 32.2% praised the adequacy of vaccination schedule reminders from community workers, and only 4.7% were accounted for not having access to a smartphone.

WTP for smartphone-based vaccination management applications

Overall, the majority of participants (79.1%) were willing to pay on average 92.7 thousand VND for a smartphone-based vaccination management application (Table 3). This high level of WTP appeared to be consistent across all evaluated sociodemographic characteristics among the participants who felt at least neutral about the necessity of these applications and willing to use such applications. Within this population, the least amount of WTP was observed for those who had completed secondary school education or below (44 thousand VND), and those who were freelancers (59.9 thousand VND). In contrast, the largest amount of WTP was observed among those who were unemployed (290.6 thousand VND), <20 years old (146.5 thousand VND), and students (107.3 thousand VND). No significant difference was found between those who were willing to pay and those who were not willing to pay for smartphone-based vaccination management applications in terms of gender, age, ethnicity, education level, religion, marital status, employment, and types of vaccinees. There was also no significant difference observed between these two groups with regard to the WTP amount.

Table 3 WTP for smartphone-based vaccination management applications Notes: aWTP data were reported of the participants who were at least neutral about the necessity of and willing to use smartphone-based vaccination management applications (n=344; Table 2). bP-values referred to the comparisons of the listed characteristics between the participants who were willing to pay and those who were unwilling to pay for smartphone-based vaccination management applications. cParticipants who brought their children to receive vaccination at the selected clinic. dParticipants who went to the clinic to get vaccinated. Abbreviations: VND, Vietnamese dong; WTP, willingness to pay.

Associated factors of acceptability of and WTP for using smartphone-based vaccination management applications

Participants, who were at least 30 years old, were less likely to be willing to use smartphone-based vaccination management applications, compared to participants who were younger than 20 years old (OR =0.26, 95% CI =[0.07, 0.94]; Table 4). Moreover, those in the highest household income quintile were less likely to use such applications, compared to those belonged to the lowest income quintile (OR =0.24, 95% CI =[0.07, 0.90]). On the other hand, compared to those without an employment, participants, who were freelancers, were more likely to use the applications (OR =5.54, 95% CI =[1.27, 24.2]). Frequency of using smartphone applications was not significantly associated with the willingness to use vaccination applications in this cohort (OR =0.38, 95% CI =[0.11, 1.31]).

Table 4 Associated factors with the acceptability of and WTP for using smartphone-based vaccination management applications Notes: aOnly the factors that were significantly correlated with the acceptability, WTP, or the WTP amount are reported in this table. ***P<0.01; **P<0.05. Abbreviations: VND, Vietnamese dong; WTP, willingness to pay.

Regarding the WTP, male participants who self-presented to get vaccinated and those who reported having knowledge of the vaccination applications were less likely to be willing to pay for the applications (OR =0.41, 95% CI =[0.20, 0.85], and OR =0.28, 95% CI =[0.08, 0.99], respectively). Moreover, although simply having access to information about vaccination was not found to be significantly associated with the WTP, participants who believed that hospital staff’s advice regarding vaccination was the most reliable information source were less likely to pay for the vaccination applications, compared to those who had gained vaccination information from community health workers (OR =0.31, 95% CI =[0.13, 0.76]). In contrary, being more financially secured was found to increase the likelihood of being willing to pay for vaccination applications as evident in the analysis between participants in the highest and lowest household income quintiles (OR =3.46, 95% CI =[1.08, 11.11]).

In this cohort, we also found that participants who had attained high school education were willing to pay a higher amount for vaccination management applications, compared to those whose education remained at or below secondary school (coefficient =70.41 thousand VND, 95% CI =[29.25, 111.58]). In addition, participants who got remainders of vaccination schedules from parents, instead of grandparents, were likely to pay a higher amount for vaccination applications (coefficient =124.68 thousand VND, 95% CI =[20.53, 228.83]). On the other hand, male participants who self-presented for vaccination and those who were between 20 and 29 years old were found to be more likely to pay less for vaccination applications, compared to participants who brought their children to get vaccinated (coefficient =−48.76 thousand VND, 95% CI =[−91.02, −6.49]) and participants who were below 20 years old (coefficient =−44.25 thousand VND, 95% CI =[−85.80, −2.69]), respectively. In contrast to the WTP, we did not find a significant association between household income and WTP amount (coefficient =35.27, 95% CI =[−11.59, 82.14]).

Discussion

In the present study, we reported high levels of acceptability of, willingness to use, and WTP for smartphone-based vaccination management applications among Vietnamese parents whose children were vaccinated at an urban vaccination clinic in Vietnam. Despite a high smartphone use, we identified a gap in the awareness of such applications among Vietnamese adults. On the other hand, we discovered several sociodemographic factors among this population, who might serve as the initial outreach target of the applications in future implementation effort, including higher education, younger age, and being employed as freelancers. Lastly, in addition to increased education about the benefits of the application that differentiates it from available vaccination information available on the Internet, a stronger collaboration among application developers, funders, health care professionals, and community health workers might also help maximize the application’s visibility, adoption, and effectiveness.

The high level of acceptability of and WTP for using a smartphone-based vaccination management application in this study was comparable to that of vaccination reminders via smartphone reported in other studies. For example, Nigerian parents with children at vaccination age occupied 69%, 62.6%, and 77% of the surveyed population who accepted smartphone-based vaccination reminders in Ilorin, Benin and Lagos, and Nigeria, respectively.²⁰ Within the context of Vietnam, a study of antiretroviral treatment recipients found that 63.5% of the participants were willing to use smartphone applications to support their treatment adherence.²¹ On the other hand, a study examining Vietnamese active Internet users found that the acceptability of using smartphone-based applications to assist smoking cessation was only 26.8%.²² Despite a potential variation among studies of different target interventions, the high level of willingness to use a smartphone-based vaccination management application in this study suggests that a well-developed application would be well perceived and utilized.

Regarding the WTP for the applications, the average amount of WTP in this study was higher than that reported for antiretroviral treatment adherence supporting application, which was 51,000 VND (US $2.5) per month.²¹ However, in this cohort, we also found that although people who had a lower monthly household income were more likely to use the applications, they were less willing to pay for the applications. Therefore, even though building and introducing smartphone-based vaccination management applications might be financially beneficial for application developers and vendors, a co-payment system and insurance coverage of the applications could help reach other populations who might be at a greater need, but otherwise might not be able to finance the associated expenses.

Furthermore, in contrast to the participants who were unemployed, we observed that younger adults with higher educational attainment were more likely to use and pay a higher amount for the applications. On the other hand, people who claimed to have knowledge of the vaccination applications were less likely to pay for such applications. Coupled with the finding that the biggest reason for being unwilling to use the applications reported in this study was the adequacy of vaccination information on the Internet, it might be important to ensure the quality and creditability of such information regarding vaccination and vaccination applications. In fact, it has been shown that inaccurate information about vaccines and immunization remains widely disseminated and improperly presented on the Internet, targeting a range of audiences, especially those who might not be as prepared to differentiate such misinformation in the popular media. As demonstrated previously, this issue, however, supports the use of smartphone applications that go beyond the role of a schedule reminder to serve as a more curated tool for vaccination management, education, and communication with health care teams.¹⁸

Interestingly, people perceiving that vaccination information from hospital staff was more reliable compared to information from community health workers were less likely to pay for the applications. This finding suggests that increased visibility of the applications and promotion of their benefits among health care professionals in hospitals might help improve the adoption of such applications. Even though they were not measured in this study, a better understanding of preferences for specific features of a smartphone-based vaccination management application might also provide useful insights into maximizing the utility and acceptability of such application. Lastly, strategies to address the potential unmet needs associated with using the applications, such as financial burden, potential issues in data management, and confidentiality protection as well as user integration into the existing health care infrastructures, might enable application developers to better differentiate the application’s features from the available online resources and information provided from hospital staff.

In this cohort, besides the parents whose children are at the age for vaccination, we also identified several other characteristics of people who might serve as a promising initial outreach target population for the application. However, it is important to note that such associations were drawn from the analysis of vaccinees at a single urban vaccination clinic. Moreover, in addition to potential bias associated with self-reporting, convenience sampling in a cross-sectional study might prevent us from generalizing the findings to the general Vietnamese population.

Conclusion

Smartphone application presents a promising innovation in health care service delivery. In this study, we provided evidence supporting the favorable perception and feasibility of implementing smartphone applications for vaccination management among the Vietnamese vaccinees. Our findings also suggest a need for building a stronger partnership among the application developers, family members, community health care workers, and hospital staff to ensure proper access to and awareness of quality education and information regarding vaccines, vaccination, and vaccination management applications. This, in turn, might help further enhance the acceptability and financial feasibility of using such applications as well as their potentials for improving vaccination coverage and compliance.

Disclosure

Dr Carl Latkin reports grants from the NIH. The other authors report no conflicts of interest in this work.

---

References

1.		World Health Organization [homepage on the Internet]. Immunization; 2017. Available from: http://www.who.int/topics/immunization/en/. Accessed April 27, 2017.
2.		Akmatov MK, Kretzschmar M, Krämer A, Mikolajczyk RT. Timeliness of vaccination and its effects on fraction of vaccinated population. Vaccine. 2008;26(31):3805–3811.
3.		Vitek CR, Wharton M. Diphtheria in the former Soviet Union: reemergence of a pandemic disease. Emerg Infect Dis. 1998;4(4):539–550.
4.		Park YJ, Eom HS, Kim ES, Choe YJ, Bae GR, Lee DH. Reemergence of measles in South Korea: implications for immunization and surveillance programs. Jpn J Infect Dis. 2013;66(1):6–10.
5.		Smith PJ, Humiston SG, Parnell T, Vannice KS, Salmon DA. The association between intentional delay of vaccine administration and timely childhood vaccination coverage. Public Health Rep. 2010;125(4):534–541.
6.		Capanna A, Gervasi G, Ciabattini M, et al. Effect of mass media on influenza vaccine coverage in the season 2014/2015: a regional survey in Lazio, Italy. J Prev Med Hyg. 2015;56(2):E72–E76.
7.		Gibson DG, Ochieng B, Kagucia EW, et al. Mobile phone-delivered reminders and incentives to improve childhood immunisation coverage and timeliness in Kenya (M-SIMU): a cluster randomised controlled trial. Lancet Glob Health. 2017;5(4):e428–e438.
8.		Morin A, Lemaître T, Farrands A, Carrier N, Gagneur A. Maternal knowledge, attitudes and beliefs regarding gastroenteritis and rotavirus vaccine before implementing vaccination program: which key messages in light of a new immunization program? Vaccine. 2012;30(41):5921–5927.
9.		Jacobson Vann JC, Jacobson RM, Coyne-Beasley T, Asafu-Adjei JK, Szilagyi PG. Patient reminder and recall interventions to improve immunization rates. Cochrane Database Syst Rev. 2018;1:CD003941.
10.		Manakongtreecheep K. SMS-reminder for vaccination in Africa: research from published, unpublished and grey literature. Pan Afr Med J. 2017;27(Suppl 3):23.
11.		Domek GJ, Contreras-Roldan IL, O’Leary ST, et al. SMS text message reminders to improve infant vaccination coverage in Guatemala: A pilot randomized controlled trial. Vaccine. 2016;34(21):2437–2443.
12.		Francis DB, Cates JR, Wagner KPG, Zola T, Fitter JE, Coyne-Beasley T. Communication technologies to improve HPV vaccination initiation and completion: A systematic review. Patient Educ Couns. 2017;100(7):1280–1286.
13.		Katib A, Rao D, Rao P, Williams K, Grant J. A prototype of a novel cell phone application for tracking the vaccination coverage of children in rural communities. Comput Methods Programs Biomed. 2015;122(2):215–228.
14.		Chen L, Du X, Zhang L, et al. Effectiveness of a smartphone app on improving immunization of children in rural Sichuan Province, China: a cluster randomized controlled trial. BMC Public Health. 2016;16:909.
15.		Jit M, Dang TT, Friberg I, et al. Thirty years of vaccination in Vietnam: Impact and cost-effectiveness of the national expanded programme on immunization. Vaccine. 2015;33(Suppl 1):A233–A239.
16.		Minh An DT, Lee JK, Van Minh H, et al. Timely immunization completion among children in Vietnam from 2000 to 2011: a multilevel analysis of individual and contextual factors. Glob Health Action. 2016;9:29189.
17.		Li X, Wiesen E, Diorditsa S, et al. Impact of adverse events following immunization in Viet Nam in 2013 on chronic hepatitis B infection. Vaccine. 2016;34(6):869–873.
18.		Nguyen NT, Vu HM, Dao SD, Tran HT, Nguyen TXC. Digital immunization registry: evidence for the impact of mHealth on enhancing the immunization system and improving immunization coverage for children under one year old in Vietnam. Mhealth. 2017;3:26.
19.		Nielsen. Nielsen Vietnam Smartphone Insight Report. Hanoi, Vietnam, 2017.
20.		Ibraheem RM, Akintola MA. Acceptability of reminders for immunization appointments via mobile devices by mothers in Ilorin, Nigeria: A cross-sectional study. Oman Med J. 2017;32(6):471–476.
21.		Tran BX, Houston S. Mobile phone-based antiretroviral adherence support in Vietnam: feasibility, patient’s preference, and willingness-to-pay. AIDS Behav. 2012;16(7):1988–1992.
22.		Tran BX, Le XTT, Nguyen PN, et al. Feasibility of e-Health interventions on smoking cessation among vietnamese active internet users. Int J Environ Res Public Health. 2018;15(1):165.