Gaming science innovations to integrate health systems science into medical education and practice

Introduction

Global and national attention is on improving individual and population health. Population health initiatives focus on improving health outcomes and the distribution of outcomes in any given population.¹ Simple solutions to improve population health are non-existent. Socioeconomic status, environmental exposures, lifestyle behaviors, and access to care are examples of determinants that contribute to health outcomes and disparities.² The World Health Organization,³ Institute for Healthcare Improvement (IHI),⁴ and National Academy of Medicine⁵ are organizations calling for systematic and community-based innovations to improve health systems and address population health.

Health systems require well-trained teams for delivery of quality services to effectively maximize health outcomes, minimize costs, and meet the needs of diverse populations.³ Systems thinking, multidisciplinary collaboration, and innovative strategies are essential to prepare health professionals for roles in population health.^4–7 Physician leaders and medical educators are promoting health systems science (HSS) integration into education and clinical practice as a catalyst to improve population health.^8–11

HSS is an emerging discipline from the seminal work of the American Medical Association Accelerating Change in Medical Education Consortium, consisting of nearly one-fifth of all allopathic and osteopathic medical schools.¹⁰ The Consortium is working to advance change in medical education to prepare physicians for the needs of individuals and populations now and in the future.¹⁰ The provision of optimal care in complex systems requires an understanding of HSS, the applied science of how care is accessed, delivered, financed, managed, and assessed.¹¹ HSS conceptualizes how to work in collaborative teams for improved quality, value, and safety in the delivery of patient- and population-centric health services.^8–10 HSS aligns with the Quadruple Aim framework for improvement of population health, patient care experiences, work life of providers, and reduction of costs.¹² The study of HSS will give new physicians a broad view of the administrative challenges and societal factors that affect health outcomes and can complicate patient care. Knowledge of HSS can improve providers’ abilities to care for patients and populations.

The Consortium is promoting the field of HSS as the third pillar of medical education, along with the traditional pillars of biomedical and clinical sciences.^8–11 Yet, HSS integration into medical education and practice is not without challenges.¹¹ The difficulty of aligning HSS education with real-world scenarios related to complex delivery systems has been identified.^11,13 In addition, medical educators have been tasked to develop and fit new content into an already heavy curricular load.^11,13 Often, medical educators have limited resources, training, or experiential skills in curriculum development.¹⁴ Opportunities to address these challenges and integrate HSS into undergraduate, graduate, and continuing medical education is possible through gaming science innovations (GSIs), as outlined in this paper. We hope this serves as a tool to help educators incorporate HSS into medical education in interesting and engaging ways.

Gaming science innovations

Gaming science refers to the use of games as training tools to engage learners through motivational, cognitive reasoning, and metacognition components to advance scientific thinking and skill acquisition.^15,16 The historical roots of gaming science lie deep in early civilizations evidenced by war games, chess, and other serious games to promote strategic and critical thinking skills necessary for defense against opposing threats and forces.¹⁷ Military and emergency responders have long recognized the importance and benefits of GSIs.^16–18

Gamification is the application of “game design elements in non-game contexts,”¹⁹ such as education.^20–22 Gamification pertains to the processes of thinking through game elements to solve problems.²³ The gamification term, serious game carries definitions, such as “the use of game principles for the purposes of learning, skill acquisition, and training”¹⁶ and “the voluntary attempt to overcome necessary obstacles.”¹⁶ Existing literature is fundamentally favorable toward gamification uses, benefits, and outcomes.^{16,22,24–31} Educational game innovations are known also as gamified learning interventions,¹⁵ gamified training platforms,²⁵ and the aforementioned serious games.¹⁶ Video games,¹⁸ mobile applications, electronic games,^25,28,32 and simulation^19,32–37 are examples of game designs or learning architectures.¹⁶ In health care, where errors can result in dangerous injuries or loss of life, GSIs hold the promise of safe environments for practice, attainment, assessment, and fine-tuning of skills crucial to improve real-life outcomes.^17,25 Gaming science principles that strengthen scientific and systems thinking²¹ align with the HSS paradigm.

HSS framework

HSS is grounded in Engel’s biopsychosocial model with systems-thinking theoretical roots.^9,38,39 The model includes biological, psychological, and sociological life in a hierarchy of systems spanning to the biosphere.³⁹ The HSS conceptual framework focus is on interdependent elements in a biopsychosocial hierarchy. HSS education will incorporate the physiologic factors that influence disease and health outcomes as well as the social environment, economic environment, and the health care system.³⁹ By incorporating HSS into medical education, learners will know how to address the multitude of factors that influence health and well-being.³⁸

The HSS framework identifies 6 core domains and 5 cross-cutting domains, all linked together by the systems thinking domain.^9,39

Core domains

• Health care structures and processes
• Health care policy, economics, and management
• Clinical informatics and health information technology
• Population health
• Value-based care
• Health system improvement

Cross-cutting domains

• Leadership and change agency
• Teamwork and interprofessional education
• Evidence-based medicine and practice
• Professionalism and ethics
• Scholarship

Linking domain

• Systems thinking^9,39

Core domain: health care structures and processes

The health care structures and processes domain pertains to matters that affect patients, providers, care systems, resources, or processes.^9,39 Broad determinants of care quality include patient satisfaction, nature of interactions with staff, personalized care, accessibility to care, timeliness of care, and teamwork.⁴⁰ Knowledge of this domain is important to promote, care for, and maintain healthy communities. For a generation of health care workers used to video games and social media, training games set in the virtual worlds or virtual reality are cognitively engaging because they allow trainees to construct new knowledge through mediated immersion.⁴¹

The Envision Community Health Center (CHC)© is a gamified health care training platform⁴² to effectively address HSS domains, with a special emphasis on health care structure and processes. Designed to prepare health care teams to provide compassionate, holistic, and evidence-based care to medically underserved populations, medical students complete 26 community-oriented primary care cases related to 10 patient families from diverse ethnicities.⁴³ Gamified elements include dramatic stories set in a virtual world, a scoring system, and virtual preceptors.⁴³ Specific to the first HSS core domain, first- and second-year medical students become oriented with patients, teams, resources, care structures, and processes of Federally Qualified CHCs as they practice care in these settings.⁴³

The process of care can be gamified to provide risk-free practice. Virtual reality simulation is one GSI platform to promote competency development and mastery of processes,³³ such as endoscopy,³⁶ prescribing guidelines,^29,34 clinical evaluations, and surgical decision making.^30,33,37 For example, virtual reality training with endoscopic simulators provides learners opportunities to practice new procedures and advance skills prior to performing procedures on patients.³⁶ Surgical trainees have benefited from reality-based simulation and gamification to gain competency in HSS content areas, such as patient workup and operative skills.³⁰ An iPAD game, PAtient Safety in Surgical EDucation (PASSED), that mimics real-world hospital sentinel events increased medical students’ situational awareness and understanding of patient safety concerns in clinical practice.⁴⁴

Opportunities exist through simulation³⁴ and Internet-based training modules to advance clinical competence⁴⁵ in knowledge acquisition, patient education, and processes, such as prescription guideline adherence to address the opioid crisis³⁴ and other urgent health concerns.²⁹ Simulation stories serve as teaching, learning, and research tools to examine risk evaluation and mitigation strategies for pain and opioid management, in which conflict was minimized when participants were given time to interact with simulated patients.³⁵ Advantages to teaching and learning these skills with GSIs are immediate feedback and increased motivation as players pursue a goal. Games can be designed to challenge learners, so progress to the next level is dependent upon mastering specific competencies.⁴⁶

Core domain: health care policy, economics, and management

This domain encompasses many factors affecting decisions, actions, and plans put forward to achieve specific health care goals.^9,39 Further, this domain includes the components of value, cost, and quality. To understand this domain, students explore policy and health care finance, insurance markets, and forms of reimbursement. Students further evaluate how these factors influence consumer and provider behavior, systems, and health outcomes.^9,39

Audience response systems are GSIs used successfully to engage learners in large group lectures for many topics.²⁸ These can be used for curricular integration of the HSS domain of health policy, economics, and management. Scenarios can be presented where learners choose the next step or determine the optimal response to various inputs. Gamified audience response systems are effective tools to promote active learning, real-time feedback, critical thinking, and problem solving of complex content.²⁸

The British Columbia Patient Safety & Quality Council (BCPSQC) created a province-wide campaign using gamification to encourage clinicians to participate in education and adoption of sepsis identification policies and management tools.²⁶ Success of the gamification learning module was demonstrated when providers, residents, and medical students applied new knowledge and evidence-based policies in the real-life screening of nearly 1000 patients in emergency departments for sepsis. Over 150 lives were saved during the 150-day sepsis screening phase of the campaign. British Columbia reported sustained reduction in sepsis-related mortality after the campaign period. The BCPSQC attributed the campaign success to the use of positive intrinsic and extrinsic game elements as an effective management approach to bring about behavioral changes and policy compliance.²⁶

GSIs have also been effective in engaging learners in role-playing¹⁶ and acquisition of pharmacological management skills^20,29,47 in alignment with health care policy, economics, and management. Games where students and trainees navigate as avatars through virtual scenarios, such as how to manage patient complaints and drug shortages in health centers and other social settings²⁰ are innovative techniques to integrate HSS content into education and practice. A study in the UK of gamified simulation cases found statistically significant decreases in unsafe medication errors when medical students followed care policies to manage medical emergencies.²⁹ The role-playing game, Dr. Fill, includes a tool for learners to self-evaluate the ability to accurately recognize, sort, and use quick-response barcodes for medication distribution. The goal is to increase patient safety by reducing medication errors.⁴⁷ GSIs that promote management skills can also lead to learner awareness of how minimizing errors reduces health costs while improving economics and health outcomes.

Core domain: clinical informatics/health information technology

This HSS domain focuses on informatics sciences, clinical decision-support systems, and electronic health record (EHR) system function and interoperability.^9,39 Reporting of various health data is required at national, state, and local levels.⁴⁸ Measurement of health data is essential to improve patient, public, and population health.⁴⁹ Thus, it is increasingly important to have methods to educate providers to use rapidly changing technology. In a systematic review of gaming science literature, researchers concluded GSIs promote role- and team-based best practices, build confidence, and reduce errors in EHR users.²⁷ In the Envision CHC©,⁴² EHR and clinical decision-support systems mimic real-life review and practice with health information technology and informatics.⁴³ To address this domain, GSI designers can incorporate free EHR and data management resources,^27,50 such as Blue Button⁵¹ and PracticeFusion.⁵² These features provide learners with opportunities to advance understanding and competency in health information and clinical decision support technologies for the documentation, continuity, and delivery of quality care.⁵³

Core domain: population health

The population health domain is inclusive of public health principles for the promotion of equitable health, wellness, and access to quality care for all people.^9,39 Gaming science principles that prompt metacognition and behavioral changes²⁶ can advance HSS education related to population health. The potential for GSI use extends to calls for innovative approaches to address patient safety and population health emergencies, such as the current opioid crisis³⁴ and potential epidemic or pandemic outbreaks.⁵⁴ The POD game,⁵⁵ Outbreak at the Watersedge©,⁵⁶ The Great Flu Game and Medical Mysteries⁵⁷ are interactive educational games that address public health disasters, such as bioterrorism and disease outbreaks.

GSIs that guide learners to explore social determinants of health and population health needs in decision-making have been found to be effective training tools in medical schools,^42,43 athletic training programs,⁵⁸ and nursing programs.^50,59 The CDC Foundation offers a premier serious game kit that addresses socioeconomic, behavioral, environmental and other social determinants of health.⁶⁰ Health and Well-Being for All engages health professions students in collaborative, interactive, case studies. These focus on the social and economic determinants of health. Study topics include navigating the health care system for chronic diseases, such as asthma or obesity in a social environment of poverty, food insecurity, or gang violence. In this GSI, students role play to experience concerns from varying stakeholder perspectives in a community context.⁶⁰

Population health services tailored to meet specific needs might be identified by engaging patients and communities in GSIs.⁶¹ The Wii Fit Plus, for example, tracks activity for improved function of people who have experienced a stroke, multiple sclerosis, or other debilitating health concerns. During this type of gamified care, patients and care teams collaborate in rehabilitation activities to advance HSS content, such as social dynamics for improved compliance and population health outcomes.⁶¹

Core domain: value-based care

Value-based care is an important component of the Quadruple Aim for improvement of population health, patient care experiences, provider work life, and cost reduction.^9,12,39 New approaches are needed to prepare medical students, residents, and physicians to support value-based care.⁶² GSIs are a creative method to promote learner awareness of value as it relates to improved performance for patient and provider experiences, costs, and other quality measures.

The gamification platform, Health eRide: Your Journey to Managing Pain, has served as a useful intervention and measurement tool to assess veterans’ readiness to self-manage chronic pain.³¹ Platforms, such as the medical mobile application, Prognosis: Your Diagnosis©,^63,64 and the simulation-based, DecisionSim^™64,65 provide risk-free deliberate practice for simple and complex clinical presentations where cost-effective decisions are encouraged and value-based care is promoted.⁶⁴ The IHI along with the Scottish National Health Service developed and tested a continuous value management innovation in one of the largest hospitals in Scotland.⁶⁶ The result was a successful value improvement system that included the use of value management boxes and Plan-Do-Study-Act cycles.⁶⁶ GSI design replication of these action-oriented tools and activities is one method to crystallize team performance measures, reduce waste, and promote value management concepts to learners.

Core domain: health system improvement

Health system improvement pertains to the analyses of root causes for effective change management.^9,39 Further, this domain involves processes to identify, analyze, or implement changes in policy, care delivery, or care systems to improve performance.³⁹ As health systems constantly evolve and strive for improvements, GSIs offer opportunities for sustained engagement of all stakeholders in health from patients to provider to administrators.²⁶

The IHI Open School asserts that games are unique tools to learn about improvement.⁵⁴ A number of health system improvement games, such as Emergency Department Flow, Health Care Scattegories, and The Paper Airplane Factory are available for download on the IHI Open School Web site.⁵⁴ Game elements and feedback can be customized for stakeholders to identify, analyze, solve, and improve potential health system scenarios like those in Edugame, which take place in various hospital departments.⁴⁶ A study of the cardiology and nephrology departments in the hospital phase of Edugame demonstrated that the game enhanced student’s motivation and knowledge acquisition⁴⁶ for transformative practice and system improvement.

Cross-cutting and linking domains

HSS cross-cutting domains are knowledge and skill areas that overlap with core content domains.^9,39 For example, leadership and change agency are necessary to effectively adapt to evolving health policies and systems. The cross-cutting domain of teamwork and interprofessional education is a key area where GSIs can be used. GSIs can foster role- and team-based, interprofessional education opportunities.^25,27 Interprofessional education that models how teams behave ethically, learn patient safety protocols, and function efficiently in systems is essential²⁶ to accelerate HSS teambuilding and skill acquisition for quality improvement.

Systems thinking is a way to view, understand, examine, analyze, and improve the quality of system parts and interactions⁶⁷ within and between levels of health systems and the larger social and economic contexts.⁹ The application of systems thinking brings about a scientific habit of mind.⁶⁷ In a proposed theory of work gamification, the type and length of applications translate into systems thinking, an informational pathway, to increase effectiveness and motivation and result in improved performance.⁶⁸ Connections between factors that influence health from micro- to macro-level systems become apparent to learners who navigate through the Envision CHC© cases,⁴² for example. GSIs designed to link HSS with basic and clinical science are effective methods to accelerate learner awareness, systems thinking, and application of Quadruple Aim goals to enhance patient and provider care experiences, improve population health, and control costs.^9,12,38,39

Limitations and implications

Although there are well-documented benefits, GSIs are not without flaws or limitations. All potential human, system, and natural factors cannot be accounted for in gamified learning platforms.¹⁷ Human conditions, such as fear, system decisions outside individual control like warfare strategies, and natural disasters are historical examples of unpredictable wargaming deficits,¹⁷ which also apply to health care simulation. Improved population health involves countless interdependent variables, such as behaviors, systems, and cultures among other social determinants of health.^1,69 Gaming science is rapidly progressing as an andragogical approach to teaching and learning.^22,25,27 Research is required to examine facets of this growing phenomenon.

Advocates and critics of GSIs in health profession education propose further research into the relationships between gaming experiences and translation to real-life clinical practice.^{15,22,25,27,32,45,47,61,70} Research to investigate the perceptions of learners, GSI designers, educators, and administrators is crucial to understand opportunities and challenges to GSI adoption in curricula and training environments.^15,22,70,71 Relevance⁷¹ and cost-effectiveness^32,33,72 as well as learners’ design element preferences, learning styles,^15,16,71 age, and medical predispositions, such as physical disabilities, are GSI research topics.⁶¹ Longitudinal studies of HSS integration and its influence on the application of knowledge, attitudes, and skills in medical students, trainees, and practitioners are important to validate GSI effectiveness in medical education.

Conclusion

With more emphasis on integration of HSS education, providers of the future will be better equipped to improve individual and population health outcomes and experiences while reducing costs and improving care team well-being. Achievement of these Quadruple Aim goals is dependent on transformed health profession education, training, and practice. There is much to accomplish as the Accelerating Change in Medical Education Consortium and other stakeholders work to promote the art and science of medicine and improve the nation’s health. Faculty development, resources, and quality improvement are essential to design, deliver, and evaluate curriculum for an evolving basic, clinical, and HSS triad. Although not a panacea for all scholarly endeavors, gaming science offers educators fresh perspectives and opportunities to advance HSS integration in the continuum of medical education, training, and practice. Historical accounts of serious games and current evidence support the potential use of gaming science for safe learning environments to practice, assess, and gain proficiency for the goal of improving real-life outcomes.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Lewis N. Populations, population health, and the evolution of population management: making sense of the terminology in U.S. health care today. Institute for Healthcare Improvement. Available from: http://www.ihi.org/communities/blogs/population-health-population-management-terminology-in-us-health-care. March 19, 2014. Accessed December 6, 2017.
2.		Magnan S. Social determinants of health 101 for health care: five plus five. National Academy of Medicine. Available from: https://nam.edu/social-determinants-of-health-101-for-health-care-five-plus-five/. October 9, 2017. Accessed December 6, 2017.
3.		Health systems: about. World Health Organization. Available from: http://www.who.int/healthsystems/about/en/. Accessed November 13, 2017.
4.		About us: innovations. Institute for Healthcare Improvement. Available from: http://www.ihi.org/about/pages/innovationscontributions.aspx. Updated 2017. Accessed November 13, 2017.
5.		Dzau VJ, McClellan M, Burke S, et al. Vital directions for health and health care: priorities from a National Academy of Medicine initiative. JAMA. 2017;317(14):1461–1470.
6.		Baciu A, Sharfstein JM. Population health case reports from clinic to community. JAMA. 2016;315(24):2663–2664.
7.		Eggleston EM, Finkelstein JA. Finding the role of health care in population health. JAMA. 2014;311(8):797–798.
8.		Skochelak SE, Hawkins RE, Lawson LE, Starr SR, Borkan JM, Gozalo JD. AMA Education Consortium Health Systems Science. 1st ed. Philadelphia, PA: Elsevier; 2017.
9.		Gonzalo JD, Dekhtyar M, Starr SR, et al. Health systems science curricula in undergraduate medical education: identifying and defining a potential curricular framework. Acad Med. 2017;92(1):123–131.
10.		American Medical Association. Creating a Community of Innovation. Chicago, IL. American Medical Association; 2017.
11.		Gonzalo, JD, Caverzagie KJ, Hawkins R, Lawson L, Wolpaw DR, Chang A. Concerns and responses for integrating health systems science into medical education. Acad Med. Epub 2017 Oct 24.
12.		Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573–576.
13.		Gonzalo JD, Haidet P, Blatt B, Wolpaw DR. Exploring challenges in implementing a health systems science curriculum: a qualitative analysis of student perceptions. Med Educ. 2016;50:523–531.
14.		Thomas PA, Kern DE, Hughs MT, Chen BY, editors. Curriculum Development: A Six-Step Approach for Medical Education. 3rd ed. Baltimore, MD: John Hopkins University Press; 2016.
15.		Buckley P, Doyle E, O’Mahoney A. Individualising gamification: investigating how learning styles impact upon gamification. Comput Educ. 2017;106:43–55.
16.		Cain J, Piascik, P. Are serious games a good strategy for pharmacy education? Am J Pharm Educ. 2015;79(4):47:1–6.
17.		Caffrey Jr. MC. Toward a history-based doctrine for wargaming. Aerospace Power J. 2000;14(3):33–56.
18.		Allen, R. America’s Army and the military recruitment and management of ‘talent’: an interview with Colonel Casey Wardynski. JGVW. 2014(6)2:179–191.
19.		Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining “gamification.” Proceedings for the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. New York: ACM. 2011:9–15.
20.		Yap KY-L, Yap KZ, Yap JYG. A gamification framework for training of patient and medications management skills. JILR. 2015;1(1):47–58.
21.		Morris BJ, Croker S, Zimmerman C, Gill D, Romig C. Gaming science: the “Gamification” of scientific thinking. Front Psychol. 2013;4(607):1–16.
22.		Sanchez-Mena A, Marti-Parreno, J. Drivers and barriers to adopting gamification: teachers’ perspectives. EJEL. 2017;15(5):434–443.
23.		Zichermann G, Cunningham C. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. Sebastopol, CA: O’Reilly Media, Inc.; 2011.
24.		Cudney EA, Murray SL, Sprague CM, et al. Engaging healthcare users through gamification in knowledge sharing of continuous improvement in healthcare. Procedia Manuf. 2015(3):3416–3423.
25.		McCoy L, Lewis JH, Dalton, D. Gamification and multimedia for medical education: a landscape review. J Am Osteopath Assoc. 2016;116(1):22–34.
26.		McKeown S, Krause C, Shergill M, Siu A, Sweet D. Gamification as a strategy to engage and motivate clinicians to improve care. Health Manage Forum. 2016;29(2):67–73.
27.		Gibbs D, Hewitt B, McLeod A. The gamification of electronic health records: a systematic literature review. EPHIIM. 2016;1–12.
28.		Pettit RK, McCoy L, Kinney M, Schwartz FN. Student perceptions of gamified response system interactions in large group lectures and via lecture capture technology. BMC Med Educ. 2015;15:92.
29.		Woodfield G, O’Sullivan M, Haddington N, Stanton A. Using simulation for prescribing: an evaluation. Clin Teach. 2014;11:24–28.
30.		Lin DT, Park J, Liebert CA, Lau JN. Validity evidence for surgical improvement of clinical knowledge ops: a novel gaming platform to assess surgical decision making. Am J Surg. 2015;209:79–85.
31.		Johnson S, Levesque D, Broderick, L. Health eRide: outcomes of a pilot program leveraging game principles of gamification and SMS messaging to help veterans self-manage chronic pain. JMIR. 2016;2(1):29.
32.		McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. Revisiting ‘a critical review of simulation-based medical education research: 2003–2009.’ Med Educ. 2016;50:986–991.
33.		Price Kerfoot B, Kissane N. The use of gamification to boost residents’ engagement in simulation training. JAMA Surg. 2014;149(111):1208–1209.
34.		Boyle, KL. Teaching effective opioid prescribing through a simulation curriculum. Ann Emerg Med. 2015;66(4s):S157.
35.		Heirich M, Mackey S, Newmark J. Use of simulation as a teaching and research tool for opioid management: implications for patient skills. J Pain. 2017;18(4):S49–S50.
36.		Koch AD, Ekkelenkamp VE, Haringsma J, Schoon EJ, de Man RA, Kuipers EJ. Simulated colonoscopy training leads to improved performance during patient-based assessment. Gastrointest Endosc. 2015;81(3):630–636.
37.		Bambakidis NC, Selman WR, Sloan AE. Surgical rehearsal platform: potential uses in microsurgery. Neurosurgery. 2013;73(1):S122–S126.
38.		Borkan JM, Wolpaw TM, Stagg Elliott V. The future of health systems science. In: Skochelak SE, Hawkins RE, Lawson LE, Starr SR, Borkan JM, Gozalo JD, editors. AMA Education Consortium Health Systems Science. 1st ed. Philadelphia, PA: Elsevier; 2017:208–216.
39.		Gonzalo, JD, Starr SR, Borkan JM. What is health systems science? Building an integrated vision. In: Skochelak SE, Hawkins RE, Lawson LE, Starr SR, Borkan JM, Gozalo JD, editors. AMA Education Consortium Health Systems Science. 1st ed. Philadelphia, PA: Elsevier; 2017:10–23.
40.		Cowing M, Davion-Ramaya CM, Ramaya K, Szmerekovsky J. Health care delivery performance: service, outcomes, and resource stewardship. Perm J. 2008;13(4):72–78.
41.		Dede, C. Planning for neomillennial learning styles. Educause Q. 2005;28(1):7–12.
42.		Envision Community Health Center©. Available from: https://vchc.atsu.edu/. Accessed December 7, 2017.
43.		McCoy L, Lewis JH, Bennett T, Allgood J, Bay C, Schwartz FN. Fostering service orientation in medical students through a virtual community health center. J Family Med Community Health. 2016;3(2):1–9.
44.		Kow AWC, Ang BLS, Chong CS, Tan WB, Menon KR. Innovation patient safety curriculum using iPAD game (PASSED) improved patient safety concepts in undergraduate medical students. World J Surg. 2016;40:2571–2580.
45.		Kim H, Heverling H, Cordeiro M, Vasquez V, Stolbach A. Internet training resulted in improved trainee performance in a simulated opioid-poisoned patient as measured by checklist. J Med Toxicol. 2016;12:289–294.
46.		Pesare E, Roselli T, Corriero N, Rossano V. Game-based learning and gamification to promote engagement and motivation in medical contexts. Smart Learn Environ. 2016;3:5.
47.		Bukowski M, Kuhn M, Zhao X, Bettermann R, Jonas S. Gamification of clinical routine: the Dr. Fill approach. Stud Health Technol Inform. 2016;225:262–266.
48.		Health Center Data & Reporting. Health Resources & Services Administration: HRSA Health Center Program. Available from: https://bphc.hrsa.gov/datareporting/. Updated November 2017. Accessed March 14, 2018.
49.		Population & Public Health. The Office of the National Coordinator for Health Information Technology. Available from: https://www.healthit.gov/playbook/population-public-health/. Updated February 28, 2018. Accessed March 14, 2018.
50.		Costello E, Corcoran M, Barnett JS, et al. Information and communication technology to facilitate learning for students in the health professions: current uses, gaps, and future directions. OLJ. 2014;18(4).
51.		Blue Button. HealthIT.gov. Available from: https://www.healthit.gov/patients-families/your-health-data. Accessed December 2, 2017.
52.		PracticeFusion. Practice Fusion Resource Center. Available from: https://www.practicefusion.com/practice-resource-center/. Accessed December 2, 2017.
53.		Kim J. Clinical decision support tools still underused by clinicians. 2014. TechTarget. Available from: http://searchhealthit.techtarget.com/feature/Clinical-decision-support-tools-still-underused-by-clinicians. Accessed March 18, 2018.
54.		Games and Exercises. Institute for Health Care Improvement Open School. Available from: http://www.ihi.org/education/IHIOpenSchool/resources/Pages/Tools/default.aspx. Accessed March 12, 2018.
55.		The POD game. Chicago Department of Public Health and University of Illinois at Chicago School of Public Health Center for the Advancement of Distance Education. Available from: http://thepodgame.com/. Accessed March 16, 2018.
56.		Outbreak at the Watersedge©. University of Minnesota. Available from: http://www.mclph.umn.edu/watersedge/. Accessed March 16, 2018.
57.		Games. Cambridge Infectious Diseases, An Interdisciplinary Research Centre at the University of Cambridge. Available from: https://www.infectiousdisease.cam.ac.uk/schoolzone/games/games-images/Great%20Flu%20Game.jpg/view. Accessed March 16, 2018.
58.		Walker S, Weidner T, Armstrong, KJ. Standardized patient encounters and individual case-based simulations improve students’ confidence and promote reflection: a preliminary study. J Athl Ther Train. 2015;10(2):130–137.
59.		Reime MH, Johnsgaard T, Kvam FI, et al. Simulated settings; powerful arenas for learning patient safety practices and facilitating transference to clinical practice. A mixed method study. Nurse Educ Pract. 2016;21:75–82.
60.		Health and Well-Being for All. CDC Foundation. Available from: https://www.cdcfoundation.org/health-in-a-box. Accessed March 16, 2018.
61.		Hammedi W, Leclerq T, VanRiel ACR. The use of gamification mechanics to increase employee and user engagement in participative healthcare services: a study of two cases. JOSM. 2017;28(4):640–661.
62.		Havyer RD, Norby SM, Leep Hunderfund AN, et al. Science of health care delivery milestones for undergraduate medical education. BMC Med Educ. 2017;17(145):1–6.
63.		Prognosis: Your Diagnosis©. Medical Joyworks. Available from: http://www.prognosisapp.com/. Accessed December 6, 2017.
64.		McCoy L, Pettit RK, Lewis JH, et al. Developing technology-enhanced active learning for medical education: challenges, solutions, and future directions. J Am Osteopath Assoc. 2015;115(4):202–211.
65.		DecisionSim Technology™. Kynectiv. Available from: https://app.decisionsimulation.com/shell/Login.aspx. Accessed December 6, 2017.
66.		10 IHI Innovations to improve health and health care. Cambridge, MA: Institute for Healthcare Improvement; 2017. Available from: www.ihi.org. Accessed March 16, 2018.
67.		Peters DH. The application of systems thinking in health: why uses systems thinking? Health Res Policy Syst. 2014;12:51.
68.		Cardador MT, Northcraft GB, Whicker J. A theory of work gamification: something old, something new, something borrowed, something cool? HRMR. 2017;27:353–365.
69.		Hood CM, Gennuso KP, Swain GR, Catlin BB. County health rankings: relationships between determinant factors and health outcomes. Am J Prev Med. 2016;50(2):129–135.
70.		Rojas D, Kapralos B, Dubrowski A. The role of game elements in online learning within health professions education. Stud Health Technol Inform. 2016:220:329–334.
71.		Galbis-Cordova A, Marti-Parreno J, Curras-Perez R. Higher education students’ attitude towards the use of gamification for competencies development. Je-LKS. 2017;13(1):129–146.
72.		Zendejas B, Wang AT, Brydges R, Hamstra SJ, Cook DA. Cost: the missing outcome in simulation-based medical education research: a systematic review. Surgery. 2013;153(2):160–176.