Back to Archived Journals » Smart Homecare Technology and TeleHealth » Volume 8

Medical Professionals and Medical Apps: Why Oversight Must Come from Within

Authors Krieger WH , Cotton BP

Received 9 April 2021

Accepted for publication 9 August 2021

Published 24 August 2021 Volume 2021:8 Pages 1—7

DOI https://doi.org/10.2147/SHTT.S315073

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Professor Aryya Gangopadhyay



William H Krieger,1 Brandi P Cotton2

1Department of Philosophy, The University of Rhode Island, Kingston, RI, USA; 2Renfrew Center, Boston, MA, USA

Correspondence: William H Krieger Email [email protected]

Abstract: Phone or tablet-based healthcare applications, or “medical apps,” play an important role in an evolving healthcare system. The effect of medical apps on consumers has been welldocumented; however, little attention has been paid to the impacts that apps have had on medical professionals, people whose best interests like in ensuring that medical apps positively impact patient outcomes. After a brief introduction, introducing the spectrum of problems surrounding medical apps, this paper will move its focus to issues of concern for medical practitioners who prescribe or use medical apps as a part of their approach to medical care. Given the current lack of regulatory oversight of medical apps and noting the potential for improper use of these mHealth technologies, the authors will argue that as qualified, well suited, and interested parties, medical professionals should help to shape this new regulatory and ethical landscape. Additionally, before concluding, the authors will provide concrete examples of ways that medical professionals have put these ideas into practice.

Keywords: mHealth apps, regulation, technology, digital care, virtual care

Introduction

Mobile devices are redefining how health care is administered, monitored, and delivered through a range of specialized technologies called mobile medical applications (also known as mHealth apps or medical apps). Apps are pieces of software that can be installed and run on a variety of devices, primarily smartphones and tablets, and medical apps are a fast-growing part of the app market, with over 318,000 mHealth apps currently being sold,1 targeting a wide range of users. Medical apps serve a number of purposes; ranging from medical devices on the one hand, to lifestyle, education, and entertainment apps on the other.2 Information sources, such as Medscape, provide the user with up-to-date, on-demand access to medical and pharmaceutical information. Others, ranging from Fitbits and Wi-Fi scales to glucose monitors, respond to user-inputted health data, either manually entered or via external hardware, providing health advice and support based on proprietary algorithms. Still others act as an interface for doctors,3 medical residents and faculty,4 pharmacists,5 nurses,6 and dieticians7 as a means to collect data upon examination, or as an interface for telemedicine in resource-poor areas.8

Medical apps have been touted as an unqualified boon to healthcare. However, a critical analysis reveals that medical apps have both positive and negative effects on consumers. In general, evolving ideas about the nature of medical authority and patient autonomy have changed the traditional, authoritarian model of medicine. While medical professionals used to be the only source of healthcare (or sources, if they encouraged the patient to get a second opinion), technology has changed the doctor patient relationship dramatically. In this changing landscape, the “patient” has been rebranded as the “healthcare consumer.” While a patient might wait for direction, a consumer relies on his or her own judgment, has the ability to synthesize data and epidemiological studies, and browses the healthcare “marketplace”. Whether by reading reviews or purchasing apps and other products, the health consumer is an active partner in medical decision making. In fact, as the consumer is making the final choice, s/he is, in a way, the senior partner in this relationship. “More and more this decentralization and democratization places the patient in the driver seat and relegates the physician to the role of a (trusted?) mentor”.9 Health consumers are integrating these data into their daily lives, determining what a “good day” is and changing their lives and lifestyles in response to the feedback (or perhaps better branded “reflexive self-monitoring”)10 provided by their relationships with their digital health devices. Yet the medical marketplace model is potentially fraught with complications for the consumer, requiring the skills to synthesize complex medical studies, and discern the authenticity and relevancy of medical information. Despite these cautions, however, this model is generally thought to be superior to the days when the doctor spoke and the patient complied.11,12 In addition to increasing personal autonomy on the consumer side, increasing shared decision making unburdens the medical professionals from having to serve as the sole gate keeper to medical knowledge. On the negative side, medical apps have the potential for providing information that is unregulated, untested, and biased, leading consumers to make decisions that might be against their own self-interests or wishes.13,14

Although the aforementioned “consumer-side” issues are interesting, they lie outside the scope of this paper. The authors’ current focus is on the fact that, to date, the impact of medical apps on medical professionals remains comparatively underdeveloped.15 As unfiltered, un-vetted, potentially faulty or intentionally misleading information from medical apps can impact medical consumers abilities to make important health decisions, and given the fact that government agencies have limited their scope of authority over mHealth apps, another qualified group is needed to help to ensure that apps are appropriate and effective for use as health management tools. As there is no such regulatory or consulting body at present to fill this need, medical providers need to fill the current vacuum that exists in medical app oversight and curation.

The Promise and Peril of Medical Apps

Medical apps, along with the marketplace they inhabit, have given medical consumers a feeling of autonomy and empowerment. This shift away from the old authoritarian model of medicine to a more consumer-based view is relatively new, having taken root in the 20th century as a response to changing social ideas of authority, paired with changing epistemological views of medical and other knowledge.16 Media sources herald mobile apps as the new revolution, bringing truth to the masses.17 Arguments can be made that more information and new information sources are to be lauded as empowering, However, without context, disclosure, or prior knowledge, there is no certainty that consumers are receiving data-driven medical advice. Lacking oversight, there are inherent risks and high stakes in expecting the public to manage this newly available technology; recent estimations suggest that 50% of people searching on Google could not distinguish between search results and advertisements.18 This was the case even when the word “Ad” was clearly visible. Further, Kolowich shows that this problem extends to academic settings; university students were unable to conduct basic research, to filter their results, or to seek readily available assistance.19

Although this problem is pressing, it predates medical apps, as doctors have long addressed patients’ concerns that were largely influenced by television or other media. Health information (or misinformation), however, has never been more voluminous or easier to access. An early hope was that the Food and Drug Administration regulations would oversee medical app monitoring and safety. Unfortunately, the FDA regulates only those apps the creator has designated (in advance) as medical devices,20 a position which has resulted in confusion for consumers (and healthcare professionals). In spite of the fact that people may assume an app being prescribed by a physician is reliable and empirically validated, many apps recommended by physicians and prescribers do not meet these criteria, and further, apps categorized by their makers as educational, lifestyle or entertainment apps are not subject to FDA regulation. For example, one of the authors’ primary care physicians had the author link a Fitbit to the medical provider’s app. This action allowed the author’s doctor to monitor a variety of health measures (eg, steps, exercise, and sleep patterns) more accurately than if the doctor had to rely on the author to truthfully report these data. While Fitbits are innocuous (although the author has seen that the steps “counted” by the Fitbit do not necessarily match up with those recorded by the author’s smartwatch), physicians do use step trackers, heart rate monitors, and the like, as a part of their diagnostic toolbox, all without having any insight into the proprietary algorithms that produce those data. While there is a real difference between a fitness band and an insulin pump, there is no easy way to set a red line where, regardless of the app makers wishes (as noted above, given the FDA’s stance on only regulating apps that self-identify as medical devices), an app would be regulated by the FDA. As a result, many apps (with varying degrees of accuracy) are used by medical professionals to gain insight into health or as diagnostic tools (discussed below), all without any scrutiny by any regulatory group (other than the app store).

An additional worry for health consumers, medical apps may provide unfiltered data, data that can be used to “diagnose” health issues that may not actually be present or that may not warrant intervention. This is not a new phenomenon. Medical professionals have always had to deal with a measure of under-determinism when they make diagnoses. In fact, some physicians believe that exposing patients to medical uncertainty can be a positive experience. Referring to breast cancer screenings and other diagnostic tools that are used to separate medical signal from noise, Hatch makes the case that “uncertainty is the great unspoken secret of medicine and that by ignoring this fundamental uncertainty we are doing real harm to ourselves”.21 To expose this issue and better explain uncertainty’s role in medicine, Hatch speaks of a spectrum of certainty, taking into account the evidence that physicians have, the confidence they should have, and the potential for harm if they are wrong. Although health consumers have met these uncertainties in the past - often when encountering complex medical decisions - medical app data can highlight this tension without pairing it with the necessary guidance, education, and context that medical professionals provide.

Even with their training, doctors feel the same pressures to diagnose and treat their patients. As such, medical providers, like health consumers, have to sometimes fight their own inclinations and authoritarian impulses to help find the best outcomes.22 While apps can help provide information to the practitioner, the current regulatory environment makes it difficult for medical professionals to find the requisite data to evaluate apps, to trust their proprietary algorithms, to use them properly, or to know when they malfunction (significant on its own, but also leading to questions about whether the prescribing practitioner, the app maker, or the app store would assume liability when this happens). In a study involving Ob/Gyn practitioners, researchers discovered that although clinicians found smartphones to positively impact clinical knowledge and stress levels, they were inadvertently breaching HIPAA laws by transmitting sensitive health data from their phones without proper protection and deletion.23 Absent guidance, well-meaning healthcare specialists could violate privacy or safety laws via usage of medical apps risk breaching patient confidentiality, and expose their practice to litigation.

Research and Development

Absent regulation, manufacturers of medical apps will likely continue to work to make sure that their products will reach consumers without outside interference. Obviously, this unregulated approach would not result in the best (most efficacious and safe) apps making it to market. Ideally, medical groups should strive to establish guidelines to enable manufacturers to minimize the aforementioned risks associated with medical apps. Rather than wait for established and universally-accepted guidelines, select universities and medical schools are attempting to fill this power vacuum. For instance, the University of Utah created a program – now replicated in both the US and Canada – known as “Appy Hour,” a regular event devoted to the use and evaluation of medical apps.24 Research findings suggest that students’ ability to select and evaluate apps was greatly improved after such training.25 These successes show the promise of medical app curation when led by groups that understand the ethical responsibility they have for prescribing and promoting these pieces of medical technology. Hopefully, this trend will filter its way upstream to the point that regional or national bodies would step in to create guidelines for their medical providers.

While national medical groups may have once been resistant to assume responsibility for regulation, prior experience with Electronic Health Records (EHRs) illustrated the importance of prescribers’ involvement in evolving technologies that greatly impact practice. To this point AMA CEO James Madara: warned the physician community not to make the same mistake it did with EHRs, noting that physicians ‘should have been more forceful in their participation’ during the initial stages of EHR development.26 Given that physicians could be held responsible for acting on bad data provided by an app they prescribed, Madara advocates for physicians to take on a leadership role in medical app development to make sure that these medical adjuncts provide safe, reliable, private, data to patients and medical professionals, separating these from the many ineffective or potentially harmful apps that are currently in circulation. Opinion 8.8 in the American Medical Association’s Code of Medical Ethics shows that Madara’s views are in line with already established best practices.27

Opinion 8.8 of the AMA code of ethics argues that physicians are summoned to take an advisory role in the face of newly approved medical technologies. Physicians’ professional commitment to advance scientific knowledge and make relevant information available to patients, colleagues, and the public carries with it the responsibility to report suspected adverse events resulting from the use of a drug or medical device. Further, this section addresses the fact that, even in cases where there are in place pre- and post-marketing studies of these technologies, medical professionals are in a unique position to capture reports that slip through those studies. As the professionals who prescribe and monitor the use of drugs and medical devices, physicians are best positioned to observe and communicate about adverse events. Recognizing that medical professionals have a responsibility to their patients when it comes to using medical apps), the AMA has taken two steps to work the problem. First, the organization recently completed a Digital Health Study on physician use and interest in digital health28 in which the AMA identified a number of areas of interest and concern for physicians who are encountering a wide variety of digital health technologies. As these technologies range from Fitbit style monitors to EMRs, the AMA continues to create space via ongoing discussion in their AMA Wire service for physicians and medical students.29 Secondly, the AMA and other organizations formed a partnership called Xcertia, an e-healthcare collaborative, which includes a growing number of high-profile organizations. Together, these organizations making up Xcertia created a set of privacy and security guidelines for mHealth adoption. “The guidelines are intended to address concerns about quality and safety that may be discouraging physicians form integrating mHealth apps into their practice”.30

In addition to the AMA, Mt Sinai’s Icahn School of Medicine has developed Prescription Universe (RxUniverse), a service that provides lists of medical apps that have been tested and approved for use. RxUniverse is a multi-platform application that delivers digital health to patients at the point of care. Healthcare professionals can discover the right app from a curated, evidence-based selection and immediately ‘prescribe’ it directly to a patient’s mobile device. With RxUniverse, providers can confidently and immediately deliver the best digital medicine to empower individual patients or population.31 Regulation and expertise are critical in this arena, as research findings suggest that medical apps can provide both clinical advantages and disadvantages in managing a broad range of chronic conditions. Select apps have demonstrated positive results, such as improving skin cancer prevention,32 while others have demonstrated potentially dangerous effects, such as those which inaccurately track glucose levels for the management of diabetes mellitus.33 Or provide poor quality assessment tools for people suffering from asthma.34 These findings suggest a critical need for rigorous empirical studies and for leadership by people who have the expertise to understand them. As such, the public needs to be educated about providers’ expertise in these matters. Additionally, even in cases when providers use medical apps, some healthcare professionals are unwilling to recommend apps to their patients, citing a “lack of knowledge of effective apps and the lack of trustworthy sources to access them”.35 Their solution, a curated selection of effective healthcare apps, begs the question of who is best able to provide this service to healthcare professionals. Given the lack of federal or industry oversight (explained above, and mirrored by other studies on this issue,36 the authors of this paper suggest that healthcare professionals will need to step in to address this need.

Success Stories

Although the authors of this paper argue for medical oversight of medical apps, this oversight can occur in a number of different ways (and at different stages of medical practice). In this section, we will show how these medical devices can be of real benefit to physicians, in different ways.

In a training environment, properly curated and vetted medical apps can be as crucial to learning as are other advances in reforming medical training, in keeping with research showing the efficacy of flipped classrooms and other pedagogical and curricular changes designed to engage contemporary student learning styles, and the like.37 In one study, apps used in medical simulations have been shown to be a critical component of student learning. The mARble project uses an app, developed at the Hanover Medical School to update traditional moulage, simulating triage scenarios for training purposes. Markers represent commonly found wound patterns and an iPhone camera detects such markers on a “victim”s skin as a simulated wound, allowing the student to explore and treat the injury as if it were an actual examination. Such apps transform traditionally unrealistic props meant to aid in training, adding authenticity which is critical to medical simulation practices. In providing this experience, mARble has been found to increase exam scores among users.38 Designed for use within an academic environment, an app approved by a medical school improved medical training among students illustrating effective use and positive outcomes.

In regions where healthcare needs are underserved, creating tailor-made apps to assist in providing care to patients could potentially transform care in areas which lack access to medical facilities and supplies. A group at The University of Rwanda’s School of Medicine at the College of Medicine and Health Sciences, created a needs assessment baseline study to inform app developers of actual needs in remote communities in Rwanda. Using a User Centered Design Framework (UCD) – a method of medical app software development which focuses on healthcare providers’ expectations and needs - has been both efficient and effective.39 According to Rusatira, with proper oversight and local knowledge, an offline-accessible medical app would improve patient outcomes, increase information storage and flow, and would educate traveling medical professionals who may need to work outside their standards of training due to limitations in local resources. Here, the oversight predates actual app development, and highlights how medical feedback and expertise can lead to the development of apps of high quality used in areas of high need.

On the provider side, Gordon et al suggest that there are a variety of possible existing systems (either in isolation or in combination) that could help to cure the curation problem noted above. They suggest that app education could be made a part of the profession by tasking healthcare education or governing organizations to create continuing education courses devoted to medical apps and similar technologies. Added to this, medical professionals could also turn to healthcare organizations to create “digital formularies,” safe lists that providers could use when prescribing mHealth apps. Granting that formularies are not without their issues (moving selection to organizations who may be as interested in price or other non-clinical factors as they are in efficacy and safety), using this sort of model could create workflows that would easily connect with current medical records and health insurance systems.40

Even in situations when a medical apps is both safe and effective, Omada Health found that human contact can still make a good app create better healthcare. Omada Health created an app designed to help educate and change health habits for prediabetic Medicare recipients. This is not just a medical app. The Omada program is a combination of digital tools, including as app and a digital scale, online educational materials, and access to live coaches, all approved by the CDC Diabetes Prevention Recognition Program. A study conducted on the efficacy of this system found that: Of the 501 people who enrolled in the online program, 95% completed at least four of the weekly educational lessons, and 92% completed nine or more lessons. Less than 2% failed to complete at least one lesson in the first 16 weeks of the program.41 This study showed more than a high rate of program completion (the app automatically tracked user activity, sending text reminders to them if they were falling behind on logging their caloric intake, if they missed sessions, or if they did not weigh themselves regularly.) In fact, despite the fact that the population was older (the average age was 68.8 years), which generally is not correlated with a high level of comfort with new technology, the Omada study found that “Participants lost 8.0% of their initial body weight at 6 months and 7.5% at 12 months.” This program can serve as an example of well-regulated medical apps being integrated into patient care, and it also highlights the potential for savings in public health dollars, as diabetes management costs an average of $8000 per year per patient.

Conclusion

Medical apps are increasingly central to both health consumers and healthcare professionals. When selected and evaluated with care, medical apps potentially improve patient care outcomes.42 However, as the authors have argued, so long as medical apps lack FDA or other governmental oversight, the burden of evaluation must involve other groups focusing on positive medical health outcomes. As outside groups (non-profit and industry) have, to date, not been able to take up this burden, the authors, agreeing with other studies showing the impact of this lack of oversight,43 argue that this oversight will have to fall to those who are using mHealth apps in their practices, medical researchers and practitioners, both singly and organizationally. Further, medical and nursing schools should consciously evaluate the apps they bundle for their students, ensuring safety and clinical efficacy, as well as teaching their students the importance of consistent evaluation of apps as they emerge on the market. Similarly, in medical practices, the same care used to choose a company to handle Electronic Medical Records (EMRs) should be applied to selecting medical apps for both office-based use and patient use, based on rigorous empirical studies. Most importantly, health professional schools and other leaders in healthcare should work with health consumer groups to establish clear standards for the evaluation and implementation of medical apps as provider and patient tools.

Summary

Medical apps have been touted as an unqualified boon to healthcare. However, a critical analysis reveals that medical apps have both positive and negative effects on both health consumers and practitioners. Setting aside “consumer-side” issues, the authors’ current focus is on the fact that, to date, the impact of medical apps on medical professionals remains comparatively underdeveloped. As unfiltered, un-vetted, potentially faulty or intentionally misleading information from medical apps can impact medical consumers abilities to make health decisions, and given the fact that government agencies have limited their scope of authority over mHealth apps, the authors argue that medical providers need to fill the current vacuum that exists in medical app oversight and curation.

Acknowledgments

The authors would like to acknowledge and thank Dr. Steven Hatch for reading and commenting on an early version of this paper and we would like to thank the editor and reviewers for making a number of recommendations that had a substantial (positive) impact on this work.

Disclosure

The authors report no conflicts of interest in this work.

References

1. Franklin R. 11 surprising mobile health statistics. Mobius MD; 2019. Available from: https://mobius.md/2019/03/20/11-mobile-health-statistics/. Accessed April 7, 2021.

2. Krieger WH. Medical apps: public and academic perspectives. Perspect Biol Med. 2013;56(2):259–273. doi:10.1353/pbm.2013.0013

3. Horowitz B. 75 percent of physicians prefer apple iPad, iPhone: survey; 2011. Available from: http://www.eweek.com/news/75-percent-of-physicians-prefer-apple-ipad-iphone-survey. Accessed April 1, 2021.

4. Katz‐Sidlow RJ, Ludwig A, Miller S, Sidlow R. Smartphone use during inpatient attending rounds: prevalence, patterns and potential for distraction. J Hosp Med. 2012;7(8):595–599. doi:10.1002/jhm.1950

5. Felkey B, Fox B. Emerging technology at the point of care. J Am Pharma Assoc. 2002;5(Suppl 1):S50–S51.

6. Wentzel J, van Gemert-pijnen J, editors. Antibiotic information app for nurses. eTELEMED 2014. The Sixth International Conference on eHealth, Telemedicine, and Social Medicine; Barcelona, Spain; 2014.

7. Lieffers JR, Vance VA, Hanning RM. Use of mobile device applications in Canadian dietetic practice. Can J Diet Pract Res. 2014;75(1):41–47. doi:10.3148/75.1.2014.41

8. Keating L. Need to see a doctor? Here are the best apps for on-demand health care. TechTimes; 2015. Available from: http://www.techtimes.com/articles/29060/20150128/need-to-see-a-doctor-here-are-The-best-apps-for-on-demand-healthcare.htm. Accessed August 1, 2020.

9. Alexander J, Joshi G. Smartphone application-based medical devices: twenty-first century data democratization or anarchy? Anesth Analg. 2016;123(4):1046–1050. doi:10.1213/ANE.0000000000001502

10. Lupton D. Personal data practices in the age of lively data. In: Daniels J, Gregory K, McMillan Cottom T, editors. Digital Sociologies. Policy Press; 2017:335–350.

11. Chen Y, Yang L, Hu H, Chen J, Shen B. How to become a smart patient in the era of precision medicine? In: Shen B, editor. Healthcare and Big Data Management. Advances in Experimental Medicine and Biology. Vol. 1028. Springer. 2017;1–16. ISBN 978-981-10-6040-3.

12. Guadagnoli E, Ward P. Patient participation in decision making. Soc Sci Med. 1998;47(3):329–339. doi:10.1016/S0277-9536(98)00059-8

13. Paget L, Frosch DL. What will it take to reduce the app gap? J Gen Intern Med. 2016;31(12):1408–1409. doi:10.1007/s11606-016-3831-y

14. Wicks P, Chiauzzi E. Trust but verify – five approaches to ensure safe medical apps. BMC Med. 2015;13:205. doi:10.1186/s12916-015-0451-z

15. Lewis T, Wyatt J. mHealth and mobile medical apps: a framework to assess risk and promote safer use. J Med Internet Res. 2021;16(9):e210. doi:10.2196/jmir.3133

16. Tomes N. Remaking the American Patient: How Madison Avenue and Modern Medicine Turned Patients into Consumers. University of North Carolina Press; 2016.

17. Socheski J. Will apps replace doctors? 14 top applications of 2013. ReferralMD; 2013. Available from: https://getreferralmd.com/2013/12/will-apps-replace-doctors. Accessed April 7, 2021.

18. Charlton G. Do 50% of adults really not recognise ads in search results? Search Engine Watch; 2016. Available from: https://searchenginewatch.com/2016/04/27/do-50-of-adults-really-not-recognise-ads-in-search-results/. Accessed April 7, 2021.

19. Kolowich S. What students don’t know. Inside Higher Ed; 2011. Available from: https://www.insidehighered.com/news/2011/08/22/erial_study_of_student_research_habits_at_illinois_university_libraries_reveals_alarmingly_poor_information_literacy_and_skills. Accessed April 7, 2021.

20. US Food and Drug Administration. Device Software Functions Including Mobile Medical Applications; 2015. Available from: https://www.fda.gov/MedicalDevices/DigitalHealth/MobileMedicalApplications/default.htm. Accessed April 7, 2021.

21. Hatch S. Snowball in a Blizzard. New York: Basic Books; 2016:5.

22. Srivastava R. Speaking up — when doctors navigate medical hierarchy. N Engl J Med. 2013;368:302–305. doi:10.1056/NEJMp1212410

23. Santos F, Sousa H, Muglu J, Navaratnarajah R. Smarter with smartphones? A survey on the use of smartphones and medical apps in obstetrics and gynecology. Eur J Obstet Gynecol Reprod Biol. 2016;206:e186. doi:10.1016/j.ejogrb.2016.07.455

24. Casucci T. Mobile technologies: appy hour. University of Utah. 2012. Available from: http://campusguides.lib.utah.edu/EcclesMobileDevices/appyhour. Accessed April 7, 2021.

25. Rodis J, Aungst T, Brown T, Cui Y, Tam L. Enhancing pharmacy student learning and perceptions of medical apps. JMIR Mhealth Uhealth. 2016;4(2):e55. doi:10.2196/mhealth.4843

26. Sweeney E. AMA CEO: docs must get more involved with mHealth apps. FierceHealthcare; 2016. Available from: https://www.fiercehealthcare.com/it/ama-ceo-docs-need-to-be-more-involved-mhealth-apps. Accessed June 26, 2020.

27. American Medical Association. Code of medical ethics opinion 8.8; 2016. Available from: https://www.ama-assn.org/delivering-care/required-reporting-adverse-events. Accessed June 20, 2020.

28. American Medical Association. Digital health study: physicians motivations and requirements for adopting digital clinical tools. American Medical Association; 2016. Available from: https://www.ama-assn.org/sites/default/files/media-browser/specialty%20group/washington/ama-digital-health-report923.pdf. Accessed June 20, 2020.

29. American Medical Association. AMA wire health IT. Available from: https://wire.ama-assn.org/practice-management/health-it. Accessed June 20, 2020.

30. Robeznieks A. Xercia releases updated mobile health-app guidelines for comment. AMA Digital; 2019. Available from: https://www.ama-assn.org/practice-management/digital/xcertia-releases-updated-mobile-health-app-guidelines-comment. Accessed February 26, 2011.

31. Icahn School of Medicine at Mount Sinai. Rx Universe Digital Medicine Toolkit. Rxuniverse.com; 2018. Available from: http://rxuniverse.com/. Accessed November 26, 2020.

32. Robinson J, Friedewald J, Desai A, Gordon E. A randomized controlled trial of a mobile medical app for kidney transplant recipients: effect on use of sun protection. Transplant Direct. 2016;2(1):e51. doi:10.1097/TXD.0000000000000561

33. Huckvale K, Adomaviciute S, Prieto J, Leow M, Car J. Smartphone apps for calculating insulin dose: a systematic assessment. BMC Med. 2015;13:106. doi:10.1186/s12916-015-0314-7

34. Huckvale K, Morrison C, Ouyang J, Ghaghda A, Car J. The evolution of mobile apps for asthma: an updated systematic assessment of content and tools. BMC Med. 2015;13:58. doi:10.1186/s12916-015-0303-x

35. Byambasuren O, Beller E, Glasziou P. Current knowledge and adoption of mobile health apps among Australian general practitioners: survey study. JMIR Mhealth Uhealth. 2019;7(6):e13199. doi:10.2196/13199

36. Larson RS. A path to better-quality mHealth apps. JMIR mHealth uHealth. 2018;6(7):e10414. doi:10.2196/10414

37. Morgan H, McLean K, Chapman J, Fitzgerald A, Hammond Y, Hammoud M. The flipped classroom for medical students. Clin Teach. 2015;12:155–160. doi:10.1111/tct.12328

38. Urs-Vito A, Folta-Schoofs K, Behrends M, von Jan U. Effects of mobile augmented reality learning compared to textbook learning on medical students: randomized controlled pilot study. J Med Internet Res. 2013;15(8):e182. doi:10.2196/jmir.2497

39. Rusatira J, Tomaszewski B, Dusabejambo V, et al. Enabling access to medical and health education in Rwanda using mobile technology: needs assessment for the development of mobile medical educator apps. JMIR Med Educ. 2016;2(1):e7. doi:10.2196/mededu.5336

40. Gordon W, Landman A, Zhang H, Bates D. Beyond validation: getting health apps into clinical practice. NPJ Digit Med. 2020; 3:1–6.

41. Sweet C, Chiguluri V, Gumpina R, et al. Outcomes of a digital health program with human coaching for diabetes risk reduction in a medicare population. J Aging Health. 2018;30(5):692–710. doi:10.1177/0898264316688791

42. Krieger WH. When are medical apps medical? Off-label use and the Food and Drug Administration. Digital Health. 2016;2:1–12. doi:10.1177/2055207616662782

43. Alamoodi A, Garfan S, Zaidan B, et al. A systematic review into the assessment of medical apps: motivations, challenges, recommendations and methodological aspect. Health Technol (Berl). 2020;1045–1061. doi:10.1007/s12553-020-00451-4

Creative Commons License © 2021 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.