Back to Journals » Journal of Multidisciplinary Healthcare » Volume 15

Factors Affecting Medical Students’ Continuance Intention to Use Mobile Health Applications

Authors Xu Q, Hou X , Xiao T, Zhao W 

Received 16 July 2021

Accepted for publication 21 February 2022

Published 8 March 2022 Volume 2022:15 Pages 471—484

DOI https://doi.org/10.2147/JMDH.S327347

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 4

Editor who approved publication: Dr Scott Fraser



Qian Xu,1– 3 Xiaorong Hou,1,2 Tingchao Xiao,3 Wenlong Zhao1,2

1College of Medical Informatics, Chongqing Medical University, Chongqing, People’s Republic of China; 2Medical Data Science Academy, Chongqing Medical University, Chongqing, People’s Republic of China; 3Department of Medical Informatics Library, Chongqing Medical University, Chongqing, People’s Republic of China

Correspondence: Wenlong Zhao, College of Medical Informatics Chongqing Medical University, Chongqing, People’s Republic of China, Tel +86 23 6571 4682, Email [email protected]

Background: With the dramatic growth in smartphones, mobile health applications (apps) in the field of healthcare or medicine, which are characterized by strong operability, flexibility and interactivity, provide a supplementary approach to medical learning. The aims of this study were to awaken medical students to pay more attention to the learning function of mobile health app and gain deeper insight into our understanding of the factors influencing medical students’ mobile health apps continuance intention for enhancing practical utilization.
Methods: A cross-sectional survey was conducted with a self-designed questionnaire administered to medical students at Chongqing Medical University. Data were collected from 450 participants from March to April 2019. SPSS V.25 was used for the descriptive statistical analysis of the results. Smart PLS 3.2.7 was used to construct a structural equation modelling framework to estimate the influencing factors of medical students’ continuance intention to use mobile health apps.
Results: The external characteristics of mobile health apps had a positive influence on perceived ease of use (β = 0.378, P < 0.001). Perceived ease of use had a positive impact on perceived usefulness (β = 0.573, P < 0.001). Perceived ease of use (β = 0.195, P = 0.011), perceived usefulness (β = 0.450, P < 0.001) and subjective norms (β = 0.255, P < 0.001) had a positive impact on attitude towards using mobile health apps. Additionally, perceived usefulness (β = 0.202, P < 0.001) and attitude (β = 0.730, P < 0.001) had a remarkable influence on continuance intention. Furthermore, the mediating effect of attitude between perceived usefulness and continuance intention was significant (β = 0.329, P < 0.001).
Conclusion: Both perceived usefulness and attitude have a significant effect on medical students’ mobile health apps continuance intention. So, we suggest apps providers and developers evoking and maintaining users’ interest in mobile health apps and perfecting apps’ features from various aspect. Also, more attention should be paid on social relationship to introduce medical students to familiarize mobile health apps and strengthen adhesiveness of medical students.

Keywords: mobile apps, medical students, structural equation modelling, continuance intention

Introduction

Currently, medical students are faced with many challenges in their studies and the complex health care service process in their careers.1 Medical education has historically been lecture based,2 covering various subjects, such as physiology and pathology. More importantly, healthcare staffs work in hospitals need to continuously acquire knowledge and skills to advance their education and respond to clinical events and even life-long learning to track the latest research and provide competent practice. Thus, it is necessary to have quick and timely access to information and knowledge, such as clinical guidelines, drug resources, medical calculators, and the latest scientific evidence without restrictions.3

Smartphones, as hand-held devices, are widely used among people in daily life and have transformed the way in which individuals to acquire information and deliver communication without the limitations of time and space. For example, the advantage of smartphones has become more apparent with the COVID-19 pandemic between doctors and patients.4 Clearly, smartphones are also a prevalent and effective learning tool for medical study and clinical activities of medical students,5 healthcare professionals6 and younger clinicians today.7 Studies have shown that 95.8% of medical students own a smartphone for academic,8 and 94.4% of interns owned a smartphone for the purposes of work.9 Similar research conducted by Loredo E Silva MP has found that almost all medical students have a smartphone and that a total of 96.8% use it during lectures, classes, and meetings.10 A critical mass of smartphones leads to an increasing number of mobile applications (apps), which are a type of software developed to run on a mobile device for a targeted purpose,11 such as e-mail and social media. Gradually, mobile apps related to medicine or health are released and called mobile health apps, they are designed for medical students, clinical practitioners or the public for special purposes,12,13 such as Visible Body for learning anatomy, ONE DROP for diabetes patient’s management. It has been reported that there are 10,275 apps distributed in the medical and healthcare and fitness categories,14 and the downloads were expected to reach almost 6 billion in 2020.15 Nowadays, mobile health apps have been pervasively used among medical students and clinical caregivers. About 32.4% of fourth-year medical students at the Leipzig Medical School used medical apps.16 Besides, medical students and junior doctors have owned 1~5 mobile health apps and used many times per day.17 Approximately,84% of medical students believe that mobile health apps are very beneficial in course learning, information search and academic performance.18–20

Mobile health apps in the field of healthcare or medicine, which are characterized by strong operability, flexibility and interactivity, have gradually received increasing attention from researches.21 Some researchers have attempted to evaluate the valuable mobile health app of learning in basic medical sciences and clinical skills acquisition from features with visual richness, scientific comprehensiveness, auditory richness, usability, user-friendliness, self-assessment and professional.22,23 Then, mobile health apps, usually serve as interventions, have showed better efficacy in medical education or clinical practice by randomized controlled trial (RCT). A RCT involving physiotherapy students has suggested the effectiveness of a mobile health apps as a complement to traditional education for developing skills in ultrasound and palpation of the shoulder region.24 Another study has examined using a special mobile health app to educate undergraduate nursing students in providing care with regard to infant airway obstruction.25 Lozano-Lozano M has designed an experiment of a RCT, and physiotherapy students with Ecofisio mobile app tended to obtain adequate professional competencies.26

Generally, we summarize two aspects of research on mobile health apps. Early research focused on investigations of the usage condition of mobile health apps with cross-sectional surveys in different groups, such as healthcare workers,27 pharmacy students,28 medical students,29 nursing students30 and general practitioners.31 These studies have found both healthcare providers and medical students embrace mobile health apps as a powerful tool for supporting a range of healthcare education settings and clinical practice,32 and improving medical students’ learning efficiency and clinical operation skills, even assisting medical students transition from having theoretical knowledge to performing clinical operations.33 However, the study also showed medical students still were unaware of the importance of medical apps as resources in the learning process,34 leading to minimal practical use of these apps and short duration,35 although majority of students had installed medical apps on the smart devices.16 Currently, there is still inadequate attention on medical students’ continuance motivation of using mobile health apps. Hence, it is vital to identify the to improving practical usage.

Later, several theoretical models have been proposed in earlier studies to seek the factors that influence users’ mobile apps usage behavior. Initially, the technology acceptance model (TAM), originally developed by Davis in 1989, was used measure the success of new technology.36 The TAM is a highly cited model in regard to information technology and has been proved be a reliable model for understanding medical students,37 the public38 and diabetes patients’ adoption of mobile apps39. This work clarified perceived ease of use and perceived usefulness are positively associated with the attitude or intention. Furthermore, studies incorporated the TAM with an investment model and UTAUT2 with ECM model have revealed factors influencing users’ continence intention to use mobile apps,40,41 such as effort expectancy, facilitating conditions. However, the factors influencing medical students’ continuance intention to use mobile health apps are still unclear, particularly in social motivation. The theory of planned behavior (TPB) is a social-cognitive theory that provides a useful framework for predicting and understanding health-related behaviors by attitude, subjective norms and perceived behavioral control.42 One study illustrated that attitude, perceived behavioral control and subjective norms better explained the students’ adoption intention of mobile learning.43 Particularly, subjective norms is positively correlated with intention to improve their oral health behaviors,44 indicating that social relationship such as classmate and teachers, may produce effect on individuals’ continuance intention to use one technology or service. Thus, we combine TAM with TPB, which may facilitate acquisition of the factors influencing medical students’ continuance intention to use mobile health apps.

Objective

Previous studies have revealed the huge potential in clinical knowledge acquire and attempted to explore the predictors influencing medical students’ or healthcare workers’ behavioral intention to use mobile apps related to medicine or health. But few attention has been paid on medical students’ continuance intention to use mobile health apps in spite of high installed rate. Therefore, in this study, we aim to gain deeper insight into the factors influencing medical students’ continuance intention to use mobile health apps for enhancing practical utilization and awaken medical students to pay more attention to the learning function of mobile health apps.

Conceptual Model and Hypotheses

The conceptual model consists of 6 constructs that were originally designed based on the TAM and TPB: perceived ease of use, perceived usefulness, attitude, subjective norms and continuance intention. In addition, the external characteristics of mobile apps, such as the memory size required by educational apps, which is problematic,45 are introduced to the model. The research model is presented in Figure 1. By taking into account numerous studies and interviews, we proposed seven research hypotheses, which are shown below.

Figure 1 Research model: a hybrid model of TAM and TPB model.

External Characteristics

One study about assessment criteria of mobile health apps have demonstrated that size of app is one of the most important desired features for users. A smaller size of app may raise to apps’ installation rate, because smartphones need 150 to 200 MB free space in their internal memory for smooth functionality of their apps.46 Similarly, apps with a size of 150MB and more are found to have markedly reduced install rate, implying that user may have a tendency to use apps with small size due to the smartphones’ limited storage space. In addition, a study in basic medical sciences has illustrated that a mobile app of anatomy with visual richness, scientific comprehensiveness, auditory richness, affordability, user-friendliness, self-assessment, interactive content and user support is perfect.22 Likewise, a mobile app with an easy-to-use graphic interface is beneficial to users.47 Therefore, the external characteristics of mobile health apps, such as size, interface or availability of them may have impact on users intention to use them. One these bases, we propose hypotheses:

H1: Mobile health apps’ external characteristics will positively affect medical students’ perceived ease of use of a mobile health app.

Perceived Ease of Use, Perceived Usefulness, Attitude and Continuance Intention

According to the original model of TAM, perceived ease of use and perceived usefulness are key determinants of attitude and behavioral intention. Initially, a survey has suggested that perceived usefulness and ease of use affect university students continue intention to use mobile health applications, it does not involve empirical study.35 Further studies have clarified that both perceived ease of use and perceived usefulness play crucial role in user’s continuance intention of one technology or service indirectly and directly.48,49 A large number of findings have also clarified that perceived usefulness and perceived ease of use associate with users’ attitude toward usage of mobile clinic registration system,50 eHealth Systems,51 distance learning52 and electronic patient portals.53 Besides, it is reported that perceived ease of use produce impact on perceived usefulness in one technology.54,55 Furthermore, other researches have revealed the positive link between perceived usefulness and continuance intention to use e-learning56–67 and apps.58 The attitude represents the positive or negative beliefs and judgments of individual about the consequences and behavioral characteristics, attitude contributes to changes of behavioral intention. Gu59 and Guo60 have revealed the correlations between attitude and patients’ intentions for continuous usage of mHealth services. Sittig S also has pointed out that psychological attributes play a role in the willingness to use a mobile health app among College nursing students.61 For mobile health apps, perceived ease of use and perceived usefulness are related to the accessibility and user-friendliness features of the apps, which may affect users’ attitude and possibility of continuing intention. One these bases, we established the following hypotheses:

H2: Medical students’ perceived ease of use positively affects their perceived usefulness of using a mobile health app.

H3: Medical students’ perceived usefulness positively affects their attitude towards using a mobile health app.

H4: Medical students’ perceived ease of use positively affects their attitude towards using a mobile health app.

H5: Medical students’ perceived usefulness positively affects their continuance intention towards using a mobile health app.

H6: Medical students’ attitude towards using a mobile health app positively affects their continuance intention towards using it.

Subjective Norms

Subjective norms refer to social pressure, which implying that an individual’s behavior will be shaped by pressure in society mostly derive from social network, such as colleagues, leadership and groups. A study by Hyun-sun Seo also has shown that most students consumed fast food under the influence of their friends.62 It is also considered that subjective norms can have a vital influence on attitude through the development of social influence processes on attitudes.63 A survey has shown that peer recommendation influenced near 70% of students to choose the mobile health apps usage.35 Research has pointed that O2O user’s attitude in the mobile commerce environment is motivated by subjective norms.64 Based on the deduction, the following hypothesis was formulated:

H7: Medical students’ subjective norms will affect their attitude towards using a mobile health app.

Methods

Sample Size and Sampling Method

First, we conducted a pilot test among 30 medical students between November and December 2018, and some items were modified to indicate the purposes of the survey. The questionnaires showed good validity and reliability. The next stage was estimating the sample size with the following formula:

n= Z α/22×(1-P)/Pδ2

In this study, we assumed a 50% favorable perception by extracting the data from the literature on mobile health apps usage rates ranging from 0.4 to 0.78,9,19,65 retrospectively and then setting a margin of error of 10% and a 95% confidence level. The estimated sample size that would statistically be convenient in this study was 385 participants. To compensate for an estimated 15% dropout rate due to incomplete surveys and withdrawals, 450 questionnaires were distributed based on a convenience sampling method,66 which is a non-probability sampling approach, at Chongqing Medical University from March to April 2019. We declared the data in the investigation only for research at the beginning of the questionnaire and all the participants attended the survey voluntarily and anonymously. The survey followed Declaration of Helsinki and was approved by the ethics committee at Chongqing Medical University (#2018011), and informed consent was taken before participates’ participation in the study.

Data Collection and Exclusion

The survey was divided into three sections. 1) The first section collected demographic data, such as the respondent’s gender, grade, and major. 2) The second section included items regarding device for the Internet, the amount of time spent using smartphones, the number and class of installed mobile health apps. 3) The third section was based on a literature review and summary and referred to the TAM and TPB. The scale included six parts and 21 measurement items: attitude consisted of three measurement items, subjective norms contained three measurement items, perceived usefulness occupied four measurement items, perceived ease of use had five measurement items, external characteristics consisted of three measurement items and continuance intention had three measurement items. To measure the weight of the responses, we used a five-point Likert-type scale ranging from “strongly disagree” to “strongly agree”. Before conducting the survey, the investigators received training, which included the purpose of this research, research methods, and explanations of questionnaire items. Then, the trained investigators distributed anonymous self-administered Chinese questionnaires to medical students in the library and classrooms. Additionally, we removed the questionnaire with missing values and same choices.

Statistical Analysis

Data were analyzed using SPSS 25 and Smart PLS 3.2.7. SPSS V.25 was used for the descriptive statistical analysis of the questionnaire. First, Cronbach’s alpha was used to check the reliability of the latent variables. The acceptable score for this index is equal to or greater than 0.7.67 To examine the research model and hypotheses, structural equation modeling (SEM) was carried out based on partial least squares (PLS). It is a powerful method of analyzing complex models using smaller samples59,68. Additionally, the composite reliability, the average variance extracted (AVE), and standardized factor loadings were used to determine convergent validity. Acceptable scores for the factor loadings and composite reliability (CR) were set at equal to or greater than 0.7. For the AVE, the acceptable score was equal to or greater than 0.5.69 In addition, to investigate discriminant validity, the square root of the AVE of each structure was compared with the correlation coefficient of all structures. If the square root of the AVE of each structure was larger than the correlation coefficient or latent variables, then the questionnaire had good discriminant validity.

Results

Descriptive Statistics

We removed invalid responses, collecting 421 questionnaires, for a valid response rate of 93.6%. The details of the respondents’ demographic information were shown in Table 1. There was a greater proportion of female participants than male participants (66.8% female, 33.2% male). In total, we selected 232 participants based on the criterion of having at least one year of experience using mobile health apps to study.40 This number met the requirement that the sample size be 5~10 times the number of items.70 Table 1 shows the following. Smartphones became the most popular device for the Internet. About a half of participants spent 2~5 hours in using smartphones. The majority of participants installed 1~5 mobile health apps. The mobile health apps, which were distributed in the categories of drug or medical information databases and educational tools, were the most popular among medical students.

Table 1 Demographic Characteristics of the Participants

Measurement Model Testing

We performed a confirmatory factor analysis to test the measurement equation, including reliability and validity tests. Reliability is a test to check the extent to which the measurement of a test remains consistent after being repeated on the subject and in the same conditions, referring to Cronbach’s alpha and the CR of the scales. The Cronbach’s alpha of the six latent variables was between 0.656 and 0.818, which were all close to or greater than 0.65,71 and the CR values were 0.8072 or above, meeting the criterion (0.70) and suggesting that the scale was reliable and internally consistent. The validity is the accuracy of interpretations made from the results of measurements in a study, and it was supported by convergent validity and discriminant validity. For convergent validity, the AVE reflects the total number of variance indicators capable of representing each latent variable and these values of all items achieved greater than 0.5, which was desirable. The values of the two factor loadings were 0.689 and 0.685, lower than the recommended criterion of 0.7. However, they were still retained because they satisfied the values for the AVE and CR.40,73 Therefore, the convergent validity of this model was good. Discriminant validity was established, as the square roots of the AVE shown in Table 2 were greater than any of the correlation estimates.74 See Table 3 and Table 4.

Table 2 Testing Results of the Hypothetical Model

Table 3 Factor Loadings, Cronbach’s Alpha, CR and AVE

Table 4 Discriminant Validity

The Structural Equation Model Path Effect

Regarding the proposed research model, all hypotheses were supported, showing as Figure 2. H1 was verified, the external characteristics of mobile health apps had a positive influence on perceived ease of use (β=0.378, P<0.001), and the explained variance was 14.3%. Perceived ease of use was positively associated with perceived usefulness (β=0.573, P<0.001), with an explained variance of 33.0%, H2 was also verified. Perceived usefulness had an impact on attitude (β=0.450, P<0.001), with an explained variance of 54.2%. Besides, the relationship from perceived ease of use (β=0.195, P=0.011) and subjective norms (β=0.255, P<0.001) to the attitude towards using mobile health apps were significant. Therefore, H4 and H7 were also verified. Additionally, H5 and H6 were verified, and the perceived usefulness (β=0.202, P<0.001) and attitude (β=0.730, P<0.001) played a vital role on medical students’ mobile health apps continuance intention, moreover the explained variance of the two factors to continuance intention of mobile health apps peaked at 74.9%, see Table 2. Furthermore, the mediating effect of attitude between perceived usefulness and continuance intention was significant (β=0.329, P<0.001).

Figure 2 Model results.

Discussion

In this study, the proportion of female participants was higher than that of male participants, and similar research results were demonstrated by Briz-Ponce L.20 The reason could be that women have a growing tendency of enrolling in medical schools. In particular, the number of nursing students accounted for a large majority. Furthermore, it is reported that among the 2075 undergraduate freshmen admitted in 2020 at China Medical University, the total male-to-female ratio was 35.78:63.22. Similarly, this phenomenon is also observed at Chongqing Medical University with male-to-female ratio of 3:7. In this regard, the most of the medical students use smartphones as regular devices for the Internet, which means high rate of smartphones ownership among medical students. Related researches have also administrated smartphone ownership rates of medical students in Brazil and India are as high as 98.7%10 and 96%14 respectively. Approximately, 50% of the medical students in this study spend more than 2~5 hours on the Internet every day, this phenomenon is common among young people, and more attention should be paid on preventing the risk of digital addiction. Additionally, the most frequently used apps among medical students are correlated with medical learning, containing drug/medical information databases (156, 67.2%), such as drug references, followed by medical information references (154, 66.4%), which is totally different from other investigations among medical students.74 Another study has found medical students tend to use mobile health apps related to fitness and dietary to track physical activity, weight loss, calorie counting, not medical or clinical learning. Therefore, various requirements should be considered when we recommend mobile health apps to individuals.

According to the results, first, this study finds that the external characteristics of mobile health apps, including the operation interface, memory size and operational stability, are determinants of medical students’ perceived ease of use. This result is consistent with a previous study showing that medical students expressed that the memory size of apps is an issue in selecting educational apps.45 And, medical students will take the memory size of mobile health apps into consideration when they plan to install one, which might be less of a concern in the future due to the increasing memory capacity of smartphones. Hence it also reminds apps developers to produce mobile health apps with less memory size and friendly graphic interface. Moreover, it is important to update version of mobile health apps in time to maintain steady operation.

This study shows that perceived ease of use is a determinant of perceived usefulness. This result is in accord with previous studies demonstrating that ease of use, such as simple navigation, enhances the experience of users.75 Similar result appeared in researches about users’ sustained usage of health and fitness apps,40 e-Banking adoption,52 acceptance of electronic patient portals.54 So, mobile health apps’ developers should improve the function of apps in details and enhance user’s experience to meet their requirement. Furthermore, the finding also shows that perceived ease of use and perceived usefulness have a direct effect on the attitude towards using mobile health apps. Previous studies have confirmed that both perceived ease of use and perceived usefulness are strong predictors in predicting peoples’ attitude towards a system.76,77 Likewise, perceived ease of use and perceived usefulness significantly affected users’ attitude toward using e-learning, e-banking and eHealth systems. This implies that medical students will show a more positive attitude towards mobile health apps and prefer to use them if mobile health apps are easy to install and operate. Simultaneously, app developers should understand the need of medical students and perfect mobile health apps’ function constantly. Indeed, educators should capture the latest medical health apps and recommend to students to improve the rate of usage.

Usually, subjective norm is a predictor variable to intention, such as intention to improve oral hygiene behavior and intention to get COVID-19 vaccinated,78 few attentions are paid on the relationship between subjective norm and attitude. However, the finding of this study reveals subjective norms’ direct impact on individuals’ attitude. Similar result illustrates that subjective norms plays a role mainly by affecting attitudes towards pesticide. It implies medical students will improve motivation to use mobile health apps when they receive suggestions from social relationship. Especially, students hear of someone’s successful experience with mobile health app, they are prone to use them. Therefore, we should consider the differences based on medical students’ professional background.

There is a significant relationship between perceived usefulness and continuance intention. Normally, when medical students use mobile health apps and expect their attributes to produce advantageous results, their intention to continue to use the apps will be strengthened. Past studies proved that perceived usefulness has a positive impact on the usage behavior of users.79,80 It is reported that perceived usefulness is a crucial factor in continuance intention to use clinical information system,81 smart device82 and health apps.83 Hence, it is crucial to focus on continuous improvement in mobile health apps’ quality and reputation, and also social media, brochure or poster should be capitalized on to advocate the benefits of mobile health apps for attract users attention. Conversely, a study exhibits that perceived usefulness is not significant influences continuance intention to use mobile payment,84 which may result from diverse characteristic of technology or service. Moreover, attitude has a direct effect on medical students’ mobile health apps continuance intention. Analogously, attitude shows powerful explanation to continuance intention with regard to social media-based health management systems,60 continued use intentions of a gamified85 and continued intention toward e-learning.86 This indicates that medical students own good attitude toward mobile health apps, they are more likely to sustained usage of them. So, efforts should be expended to change users’ awareness of mobile health apps.

Also, perceived usefulness has an indirect effect on continuance intention via attitude. Some studies have confirmed the mediating effect of attitude, implying that attitude plays a vital role in persons’ behavior. Zainal N has proved that attitude mediates the relationship between trust in electronic word-of-mouth source and intention to follow.87 Susilowati has demonstrated that attitude toward money played a mediating role between financial literacy and perceived financial confidence on students’ financial behaviour.88 Hence, more encouragement should be put into practice and motivation also should be stimulated, which making medical students develop habit of using mobile health apps.

Theoretical and Practical Implications

Firstly, based on the TAM and TPB models, this study presents a joint model of medical students’ continuance intention to use mobile health apps and validates empirically at one university. Secondly, the proposed research model contains TAM, TPB and introduce novel construct (external characteristics of mobile health apps). Finally, the findings provide support for medical students’ continuance intention to use mobile health apps. Therefore, this foundational contribution is established new research model and identify medical students’ continuance intention to use mobile health apps. Besides, this study is also to shed the light and adds the literature on medical students’ continuance intention to use mobile health apps.

This study enriches understanding of the factors influencing medical students’ continuance intention to use mobile health apps, which will be beneficial for apps developers and users. For apps developers, it is necessary to produce mobile health apps with less memory size, improve the function of apps in details and enhance user’s experience. For users, in particular, course teachers are authoritative and should recommend some excellent mobile health apps to students, which assist medical students in developing habit of using mobile health apps.

Strengths and Limitations

Nowadays, many researches have conducted in the field of mobile apps with SEM. They proposed different models for exploring the influencing factors of users’ behavior. Majority of the relevant studies, however, do not absorb the mobile health app’ external characteristics as a factor to the model. Also, scholars never show concern on the medical students’ continuance intention of mobile health apps. This study attempts to catch influencing elements that can predict medical students’ continuance intention of mobile health apps. As a result, we find perceived usefulness and attitude are the key determinants of medical students’ continuance intention to use. Meanwhile, this study provides new ideas and directions in the research area of usage of mobile health apps among medical students and proposes suggestions on promoting medical students to keeping on using mobile health apps. These advances can be used as a reference for mobile apps providers and medical educators to formulate intervention strategies. This study has still existed several limitations. As the first limitation, the data collection method involved convenience sampling at one medical university, covering only a limited number of participants, which may produce bias in this study. In future studies, samples should be selected based on stratified random sampling or multistage. Then, demographic characteristic can be discussed as moderator variable in the research model. Lastly, payment willingness and perceived risk should be explored and tested. Because the price and security for privacy affect users’ choice.80–90

Conclusion

Grounded in the TAM and TPB, this study introduces the external characteristics of mobile health apps to explore the factors influencing mobile health apps continuance intention. The results show that all hypotheses were supported. The external characteristics of mobile health apps had a significant effect on perceived ease of use. Furthermore, perceived ease of use has a direct impact on perceived usefulness. In addition, attitude was affected by perceived ease of use, perceived usefulness and subjective norms, which has been clarified in many studies. Finally, both perceived usefulness and attitude had a significant effect on medical students’ mobile health app continuance intention. Overall, this model has good interpretability in regard to medical students’ mobile health apps continuance intention. At the same time, this study provides a novel view for future research as well as for educational providers or marketers who wish to intervene to increase medical students’ continuance intention to use mobile health apps.

Abbreviations

App, Application; SEM, Structural equation modeling; PLS, Partial least squares; CR, Composite reliability; AVE, Average variance extracted; TAM, Technology acceptance model; TPB, Theory of planned behavior; UTAUT, Unified theory of acceptance and use of technology.

Acknowledgments

We would like to thank all the participants who took part in the questionnaire survey. Furthermore, we are also very thankful to all the reviewers for their careful examination of this paper.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Funding

This study was supported by Ministry of Education in China’s Project of Humanities and Social Sciences (20YJCZH043), Intelligent Medicine Research Project of Chongqing Medical University (ZHYX2019006), Chongqing Federation of Social Sciences (PY2018043) and Intelligent Medicine Research Project of Chongqing Medical University (ZHYX202029).

Disclosure

The authors report no conflicts of interest for this work.

References

1. Jebraeily M, Fazlollahi ZZ, Rahimi B. The most common smartphone applications used by medical students and barriers of using them. Acta Inform Med. 2017;25(4):232–235. doi:10.5455/aim.2017.25.232-235

2. Snashall E, Hindocha S. The use of smartphone applications in medical education. Open Med J. 2016;3(Suppl–3, M6):322–327. doi:10.2174/1874220301603010322

3. Boruff JT, Storie D. Mobile devices in medicine: a survey of how medical students, residents, and faculty use smartphones and other mobile devices to find information. J Med Libr Assoc. 2014;102(1):22–30. doi:10.3163/1536-5050.102.1.006

4. Iyengar K, Upadhyaya GK, Vaishya R, Jain V. COVID-19 and applications of smartphone technology in the current pandemic. Diabetes Metab Syndr. 2020;14(5):733–737. doi:10.1016/j.dsx.2020.05.033

5. Bartwal J, Nath B. Evaluation of nomophobia among medical students using smartphone in North India. Med J Armed Forces India. 2020;76(4):451–455. doi:10.1016/j.mjafi.2019.03.001.

6. Xu X, Griva K, Koh M, et al. Creating a smartphone app for caregivers of children with atopic dermatitis with caregivers, health care professionals, and digital health experts: participatory co-design. JMIR Mhealth Uhealth. 2020;8(10):e16898. doi:10.2196/16898

7. Pujari A, Saluja G, Agarwal D, Selvan H, Sharma N. Clinically useful smartphone ophthalmic imaging techniques. Graefes Arch Clin Exp Ophthalmol. 2021;259(2):279–287. doi:10.1007/s00417-020-04917-z

8. Shah J, Haq U, Bashir A, Shah SA. Awareness of academic use of smartphones and medical apps among medical students in a private medical college? J Pak Med Assoc. 2016;66(2):184–186.

9. O’Connor P, Byrne D, Butt M, et al. Interns and their smartphones: use for clinical practice. Postgrad Med J. 2014;90(1060):75–79. doi:10.1136/postgradmedj-2013-131930

10. Loredo Silva EMP, de Souza Matos BD, da Silva Ezequiel O, Lucchetti ALG, Lucchetti G. The use of smartphones in different phases of medical school and its relationship to internet addiction and learning approaches. J Med Syst. 2018;42(6):106. doi:10.1007/s10916-018-0958-x

11. Liu C, Zhu Q, Holroyd KA, Seng EK. Status and trends of mobile-health apps for iOS devices: a developer’s perspective. J Syst Softw. 2011;84(11):2022–2033. doi:10.1016/j.jss.2011.06.049

12. Nabi J, Cone EB, Vasavada A, et al. Mobile health app for prostate cancer patients on androgen deprivation therapy: qualitative usability study. JMIR Mhealth Uhealth. 2020;8(11):e20224. doi:10.2196/20224

13. Duffy S, Norton D, Kelly M, et al. Using community health workers and a smartphone application to improve diabetes control in Rural Guatemala. Glob Health Sci Pract. 2020;8(4):699–720. doi:10.9745/GHSP-D-20-00076

14. Gavali MY, Khismatrao DS, Gavali YV, Patil KB. Smartphone, the new learning aid amongst medical students. J Clin Diagn Res. 2017;11(5):JC05–JC08. doi:10.7860/JCDR/2017/20948.9826

15. Research2Guidance. MHealth app market sizing 2015–2020 data report to size opportunities in the mHealth app market; 2015.

16. Sandholzer M, Rurik I, Deutsch T, Frese T. Medical students’ expectations towards an implementation of a family medicine textbook as a comprehensive app in Germany. J Med Syst. 2014;38(10):125. doi:10.1007/s10916-014-0125-y

17. Payne KB, Wharrad H, Watts K. Smartphone and medical related App use among medical students and junior doctors in the United Kingdom (UK): a regional survey. BMC Med Inform Decis Mak. 2012;12(1):121. doi:10.1186/1472-6947-12-121

18. Sayedalamin Z, Alshuaibi A, Almutairi O, et al. Utilization of smart phones related medical applications among medical students at King Abdulaziz University, Jeddah: a cross-sectional study. J Infect Public Health. 2016;9(6):691–697. doi:10.1016/j.jiph.2016.08.006

19. Lepp A, Barkley JE, Karpinski AC. The relationship between cell phone use and academic performance in a sample of U.S. college students. SAGE Open. 2015;15(1):1–9. doi:10.1177/2158244015573169

20. Briz-Ponce L, Juanes-Méndez JA, García-Peñalvo FJ, Pereira A. Effects of mobile learning in medical education: a counterfactual evaluation. J Med Syst. 2016;40(6):136. doi:10.1007/s10916-016-0487-4

21. Iskandarsyah A, Shabrina A, Siswadi AGP. Usability and acceptability of JAGA SEHAT: mobile application to improve knowledge about healthy lifestyle. J Multidiscip Healthc. 2022;15:115–124. doi:10.2147/JMDH.S342913

22. Mansouri M, Bigdeli S, Dehnad A, et al. Exploring the features of mobile phone application of anatomy in basic medical sciences: a qualitative study. BMC Med Educ. 2020;20(1):231. doi:10.1186/s12909-020-02145-x

23. Gladman T, Tylee G, Gallagher S, et al. A tool for rating the value of health education mobile apps to enhance student learning (MARuL): development and usability study. JMIR Mhealth Uhealth. 2020;8(7):e18015. doi:10.2196/18015

24. Fernández-Lao C, Cantarero-Villanueva I, Galiano-Castillo N, et al. The effectiveness of a mobile application for the development of palpation and ultrasound imaging skills to supplement the traditional learning of physiotherapy students. BMC Med Educ. 2016;16(1):274. doi:10.1186/s12909-016-0775-1

25. Kim SJ, Shin H, Lee J, Kang S, Bartlett R. A smartphone application to educate undergraduate nursing students about providing care for infant airway obstruction. Nurse Educ Today. 2017;48:145–152. doi:10.1016/j.nedt.2016.10.006

26. Lozano-Lozano M, Galiano-Castillo N, Fernández-Lao C, et al. The ecofisio mobile app for assessment and diagnosis using ultrasound imaging for undergraduate health science students: multicenter randomized controlled trial. J Med Internet Res. 2020;22(3):e16258. doi:10.2196/16258

27. Abolfotouh MA, BaniMustafa A, Salam M, et al. Use of smartphone and perception towards the usefulness and practicality of its medical applications among healthcare workers in Saudi Arabia. BMC Health Serv Res. 2019;19(1):826. doi:10.1186/s12913-019-4523-1

28. Owensby JK, Kavookjian J. Pharmacy students’ perceptions of the usefulness of motivational interviewing and the use of mobile health applications on patient counseling in the future. Curr Pharm Teach Learn. 2017;9(4):568–575. doi:10.1016/j.cptl.2017.03.019

29. Hisam A, Shafique MU, Khurshid MN, et al. Usage and types of mobile medical applications amongst medical students of Pakistan and its association with their academic performance. Pak J Med Sci. 2019;35(2):432–436. doi:10.12669/pjms.35.2.672

30. Sheikhtaheri A, Kermani F. Use of mobile apps among medical and nursing students in Iran. Stud Health Technol Inform. 2018;248:33–39.

31. Liu Y, Ren W, Qiu Y, et al. The use of mobile phone and medical apps among general practitioners in Hangzhou City, Eastern China. JMIR Mhealth Uhealth. 2016;4(2):e64. doi:10.2196/mhealth.4508

32. Wallace S, Clark M, White J. ‘It’s on my iPhone’: attitudes to the use of mobile computing devices in medical education, a mixed-methods study. BMJ Open. 2012;2(4):e001099. doi:10.1136/bmjopen-2012-001099

33. Nord A, Svensson L, Hult H, Kreitz-Sandberg S, Nilsson L. Effect of mobile application-based versus DVD-based CPR training on students’ practical CPR skills and willingness to act: a cluster randomised study. BMJ Open. 2016;6(4):e010717. doi:10.1136/bmjopen-2015-010717

34. Greene LR, Spuur KM. Undergraduate use of medical radiation science mobile applications. Radiography. 2018;24(4):352–359. doi:10.1016/j.radi.2018.04.012

35. Jabour AM, Rehman W, Idrees S, Thanganadar H, Hira K, Alarifi MA. The adoption of mobile health applications among university students in health colleges. J Multidiscip Healthc. 2021;14:1267–1273. doi:10.2147/JMDH.S310539

36. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. Mis Q. 1989;13(3):319–340. doi:10.2307/249008

37. Briz-Ponce L, García-Peñalvo FJ. An empirical assessment of a technology acceptance model for apps in medical education. J Med Syst. 2015;39(11):176. doi:10.1007/s10916-015-0352-x

38. Palos-Sanchez PR, Saura JR, MÁ RM, Aguayo-Camacho M. Toward a better understanding of the intention to use mhealth apps: exploratory study. JMIR Mhealth Uhealth. 2021;9(9):e27021. doi:10.2196/27021

39. Zhang Y, Liu C, Luo S, et al. Factors influencing patients’ intentions to use diabetes management apps based on an extended unified theory of acceptance and use of technology model: web-based survey. J Med Internet Res. 2019;21(8):e15023. doi:10.2196/15023

40. Cho H, Chi C, Chiu W. Understanding sustained usage of health and fitness apps: incorporating the technology acceptance model with the investment model. Technol Soc. 2020;63:101429. doi:10.1016/j.techsoc.2020.101429

41. Carlos T, Diogo S, Tiago O. Exploring the influential factors of continuance intention to use mobile apps: extending the expectation confirmation model. Inf Syst Front. 2020;1–15. doi:10.1007/s10796-018-9864-5

42. Khani Jeihooni A, Jormand H, Harsini PA. The effect of educational program based on beliefs, subjective norms and perceived behavior control on doing pap-smear test in sample of Iranian women. BMC Womens Health. 2021;21(1):290. doi:10.1186/s12905-021-01419-w

43. Azizi SM, Khatony A. Investigating factors affecting on medical sciences students’ intention to adopt mobile learning. BMC Med Educ. 2019;19(1):381. doi:10.1186/s12909-019-1831-4

44. Shi H, Wang J, Huang R, et al. Application of the extended theory of planned behavior to understand Chinese students’ intention to improve their oral health behaviors: a cross-sectional study. BMC Public Health. 2021;21(1):2303. doi:10.1186/s12889-021-12329-9

45. Lau C, Kolli V. App use in psychiatric education: a medical student survey. Acad Psychiatry. 2017;41(1):68–70. doi:10.1007/s40596-016-0630-z

46. Anthony Berauk VL, Murugiah MK, Soh YC, Chuan Sheng Y, Wong TW, Ming LC. Mobile health applications for caring of older people: review and comparison. Ther Innov Regul Sci. 2018;52(3):374–382. doi:10.1177/2168479017725556

47. Salomé GM, Ferreira LM. Developing a mobile app for prevention and treatment of pressure injuries. Adv Skin Wound Care. 2018;31(2):1–6. doi:10.1097/01.ASW.0000529693.60680.5e

48. Abdul-Halim NA, Vafaei-Zadeh A, Hanifah H, Teoh AP, Nawaser K. Understanding the determinants of e-wallet continuance usage intention in Malaysia. Qual Quant. 2021;1–27. doi:10.1007/s11135-021-01276-7

49. Xie C, Jia S, He C. An empirical study on the factors affecting elderly users’ continuance intention of shared nurses. Risk Manag Healthc Policy. 2020;13:1849–1860. doi:10.2147/RMHP.S261827

50. Lai YH, Huang FF, Yang HH. A study on the attitude of use the mobile clinic registration system in Taiwan. Technol Health Care. 2015;24(Suppl 1):S205–11. doi:10.3233/THC-151076

51. Kalayou MH, Endehabtu BF, Tilahun B. The applicability of the modified Technology Acceptance Model (TAM) on the sustainable adoption of eHealth systems in resource-limited settings. J Multidiscip Healthc. 2020;13:1827–1837. doi:10.2147/JMDH.S284973

52. Rizun M, Strzelecki A. Students’ acceptance of the COVID-19 impact on shifting higher education to distance learning in Poland. Int J Environ Res Public Health. 2020;17(18):6468. doi:10.3390/ijerph17186468

53. Honein-AbouHaidar GN, Antoun J, Badr K, Hlais S, Nazaretian H. Users’ acceptance of electronic patient portals in Lebanon. BMC Med Inform Decis Mak. 2020;20(1):31. doi:10.1186/s12911-020-1047-x

54. Kim J, Park HA. Development of a health information technology acceptance model using consumers’ health behavior intention. J Med Internet Res. 2012;14(5):e133. doi:10.2196/jmir.2143

55. Hsiao SJ, Tseng HT. The impact of the moderating effect of psychological health status on nurse healthcare management information system usage intention. Healthcare. 2020;8(1):28. doi:10.3390/healthcare8010028

56. Lee MC. Explaining and predicting users’ continuance intention toward e-learning: an extension of the expectation-confirmation model. Comput Educ. 2010;54(2):506–516. doi:10.1016/j.compedu.2009.09.002

57. Wang L, Lew SL, Lau SH, et al. Usability factors predicting continuance of intention to use cloud e-learning application. Heliyon. 2019;5(6):e01788. doi:10.1016/j.heliyon.2019.e01788

58. Chen Y, Yang L, Zhang M, Yang J. Central or peripheral? Cognition elaboration cues’ effect on users’ continuance intention of mobile health applications in the developing markets. Int J Med Inform. 2018;116:33–45. doi:10.1016/j.ijmedinf.2018.04.008

59. Gu D, Guo J, Liang C, et al. Social media-based health management systems and sustained health engagement: TPB perspective. Int J Environ Res Public Health. 2019;16(9):1495. doi:10.3390/ijerph16091495

60. Guo X, Chen S, Zhang X, Ju X, Wang X. Exploring patients’ intentions for continuous usage of mHealth services: elaboration-likelihood perspective study. JMIR Mhealth Uhealth. 2020;8(4):e17258. doi:10.2196/17258

61. Sittig S, Hauff C, Graves RJ, et al. Characteristics of and factors influencing college nursing students’ willingness to utilize mHealth for health promotion. Comput Inform Nurs. 2020;38(5):246–255. doi:10.1097/CIN.0000000000000600

62. Seo H, Lee S-K, Nam S. Factors influencing fast food consumption behaviors of middle-school students in Seoul: an application of theory of planned behaviors. Nutr Res Pract. 2011;5(2):169–178. doi:10.4162/nrp.2011.5.2.169

63. Fulk J. Social construction of communication technology. Acad Manag J. 1993;36(5):921–950. doi:10.2307/256641

64. Xinyin L, Kaiying X, Wei C. Research influential factors of O2O user information behavior in the mobile commerce environment. Libr Inf Service. 2015;59(07):23–30. doi:10.13266/j.issn.0252-3116.2015.07.003

65. Robinson T, Cronin T, Ibrahim H, et al. Smartphone use and acceptability among clinical medical students: a questionnaire-based study. J Med Syst. 2013;37(3):9936. doi:10.1007/s10916-013-9936-5

66. García-Viola A, Garrido-Molina JM, Márquez-Hernández VV, Granados-Gámez G, Aguilera-Manrique G, Gutiérrez-Puertas L. The influence of gamification on decision making in nursing students. J Nurs Educ. 2019;58(12):718–722. doi:10.3928/01484834-20191120-07

67. Zeller RA. Measurement error, issues and solutions. In: Kempf-Leonard K, editor. Encyclopedia of Social Measurement. Elsevier; 2005:665–676. doi:10.1016/B0-12-369398-5/00109-2.

68. Rigdon EE. Choosing PLS path modeling as analytical method in European management research: a realist perspective. Eur Manag J. 2016;34(6):598–605. doi:10.1016/j.emj.2016.05.006

69. Hair JF. Multivariate Data Analysis. Pearson Education International; 2010.

70. Santosa AD, Taufik N, Prabowo FHE, et al. Continuance intention of baby boomer and X generation as new users of digital payment during COVID-19 pandemic using UTAUT2. J Financ Serv Mark. 2021;26(4):259–273. doi:10.1057/s41264-021-00104-1

71. Juárez-García DM, de Jesús García-solís M, Téllez A. Adaptation and validation of the health belief model scale for breast self-examination in Mexican women. Value Health Reg Issues. 2020;23:30–36. doi:10.1016/j.vhri.2019.11.006

72. Fornell C, Larcker DF. Evaluating structural equation models with unobservable variables and measurement error. J Market Res. 1981;4(2):337–346. doi:10.1177/002224378101800104

73. Ali F, Kim WG, Li J, et al. A comparative study of covariance and partial least squares based structural equation modelling in hospitality and tourism research. Int J Contemp Hosp Manag. 2018;30(1):416–435. doi:10.1108/IJCHM-08-2016-0409

74. Morse SS, Murugiah MK, Soh YC, Wong TW, Ming LC. Mobile health applications for pediatric care: review and comparison. Ther Innov Regul Sci. 2018;52(3):383–391. doi:10.1177/2168479017725557

75. Dr E, Rauniar R, Rawski G, et al. Technology acceptance model (TAM) and social media usage: an empirical study on Facebook. J Enterp Inf Manag. 2014;27(1):6–30. doi:10.1108/JEIM-04-2012-0011

76. Weng C, Tsai CC, Weng A. Social support as a neglected e-learning motivator affecting trainee’s decisions of continuous intentions of usage. Australas J EducTechnol. 2015;31(2):177–192. doi:10.14742/ajet.1311

77. Ramírez-Correa PE, Arenas-Gaitán J, Rondán-Cataluña FJ. Gender and acceptance of E-learning: a multi-group analysis based on a structural equation model among college students in Chile and Spain. PLoS One. 2015;10(10):e0140460. doi:10.1371/journal.pone.0140460

78. Yahaghi R, Ahmadizade S, Fotuhi R, et al. Fear of COVID-19 and perceived COVID-19 infectability supplement theory of planned behavior to explain iranians’ intention to get COVID-19 vaccinated. Vaccines. 2021;9(7):684. doi:10.3390/vaccines9070684

79. Grenier Ouimet A, Wagner G, Raymond L, Pare G. Investigating patients’ intention to continue using teleconsultation to anticipate postcrisis momentum: survey study. J Med Internet Res. 2020;22(11):e22081. doi:10.2196/2208

80. Lacka E, Chong A. Usability perspective on social media sites’ adoption in the B2B context. Ind Mark Manag. 2016;54:80–91. doi:10.1016/j.indmarman.2016.01.001

81. Hadji B, Dupuis I, Leneveut L, Heudes D, Wagner JF, Degoulet P. Determinants of continuance intention in a post-adoption satisfaction evaluation of a clinical information system. Stud Health Technol Inform. 2014;205:990–994.

82. Cho J, Lee HE. Post-adoption beliefs and continuance intention of smart device use among people with physical disabilities. Disabil Health J. 2020;13(2):100878. doi:10.1016/j.dhjo.2019.100878

83. Cho J. The impact of post-adoption beliefs on the continued use of health apps. Int J Med Inform. 2016;87:75–83. doi:10.1016/j.ijmedinf.2015.12.016

84. Franque FB, Oliveira T, Tam C. Understanding the factors of mobile payment continuance intention: empirical test in an African context. Heliyon. 2021;7(8):e07807. PMID: 34458632; PMCID: PMC8379467. doi:10.1016/j.heliyon.2021.e07807

85. Panattil SJ, George A, Joy MM. The role of informational feedback as a game mechanic on user perceptions, attitudes and the intention to continue using a gamified health behaviour change support system. Health Mark Q. 2021;26:1–21. doi:10.1080/07359683.2021.1995639

86. Rajeh MT, Abduljabbar FH, Alqahtani SM, et al. Students’ satisfaction and continued intention toward e-learning: a theory-based study. Med Educ Online. 2021;26(1):1961348. doi:10.1080/10872981.2021.1961348

87. Zainal N, Harun A, Lily J. Examining the mediating effect of attitude towards electronic words-of mouth (eWOM) on the relation between the trust in eWOM source and intention to follow eWOM among Malaysian travellers. Asia Pac Manag Rev. 2017;22(1):35–44. doi:10.1016/j.apmrv.2016.10.004

88. Susilowati N, Latifah L. College student financial behavior: an empirical study on the mediating effect of attitude toward money. Adv Sci Lett. 2017;23(8):7468–7472. doi:10.1166/asl.2017.9500

89. Hati SRH, Zulianti I, Achyar A, Safira A. Perceptions of nutritional value, sensory appeal, and price influencing customer intention to purchase frozen beef: evidence from Indonesia. Meat Sci. 2021;172:108306. doi:10.1016/j.meatsci.2020.108306

90. Wang Y, Zheng N, Xu M, et al. Hierarchical identifier: application to user privacy eavesdropping on mobile payment app. Sensors. 2019;19(14):3052. doi:10.3390/s19143052

Creative Commons License © 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.