Evaluation of an Education Strategy versus Usual Care to Implement the STEADI Algorithm in Primary Care Clinics in an Academic Medical Center
Received 4 April 2020
Accepted for publication 13 June 2020
Published 3 July 2020 Volume 2020:15 Pages 1059—1066
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Richard Walker
Kelly Urban,1 Patricia B Wright,1 Amy L Hester,1,2 Geoffrey Curran,3 Martha Rojo,1 Pao-Feng Tsai4
1College of Nursing, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA; 2HD Nursing, Bauxite, Arkansas, USA; 3College of Pharmacy, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA; 4School of Nursing, Auburn University, Auburn, Alabama, USA
Correspondence: Kelly Urban
College of Nursing, University of Arkansas for Medical Sciences, 4301 West Markham Street, Slot 526, Little Rock, Arkansas 72205, USA
Tel +1 501-416-3895
Fax +1 501-686-7942
Email [email protected]
Background: Although falls are the leading cause of morbidity and mortality in the US in the older adult population, there is little information regarding implementation of evidence-based fall prevention guidelines within primary care settings. The objective of this study was to address this gap in the literature by determining the effectiveness of the use of education and written materials as implementation strategies.
Methods: Using a prospective, mixed methods, controlled before-and-after study design, we studied the effect of the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) education and written materials on knowledge and intention to use in primary care clinics as well as test the screening, assessment, and intervention behaviors. This manuscript details the quantitative findings of the study, using STEADI Knowledge Test, Continuing Professional Development (CPD) Reaction Questionnaire, and EMR Reports. We compared data between the study arms (usual implementation versus education implementation) using descriptive statistics, paired t-tests, and factorial ANOVAs.
Results: In total, data from 29 primary care staff, including physicians, APRNs, RNs, and medical assistants, were analyzed. Although we found a statistically significant difference within the education arm between immediate pretests and posttests/surveys mean scores, there was no statistically significant difference between the study arms’ knowledge, intent to use STEADI, or use behaviors. The pre/immediate post education mean knowledge score increased by 1.19 (p= 0.02) and the pre/immediate post education intent to use mean increased by 0.64 (p 0.01). There was no statistically significant change between the study arms over time.
Conclusion: Educational strategies, particularly written materials and an online module, did not increase the long-term use of the STEADI toolkit. Implementation research is needed to identify the strategies that are most effective for promoting the adoption of STEADI in primary care.
Keywords: STEADI toolkit, implementation, primary care providers
In the United States (US), falls are the leading cause of injury-related morbidity and mortality in the older adult population. There are more than 29 million falls annually.1 Fall-related injury and death are projected to increase with the aging of the US population.2 Not only are there a staggering number of injuries due to falls at over three million in 2015,3,4 but the cost of caring for these patients is high, predicted to exceed 100 billion dollars by 2030.5
In response to the devastating public health issue of falls and fall-related injuries, the American and British Geriatric Societies (AGS/BGS) established evidence-based clinical practice guidelines to screen, assess, and intervene on fall-risk factors.6 The CDC utilized these recommendations to create the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) algorithm and toolkit. This algorithm and toolkit were designed for use by providers in primary care practices. The STEADI algorithm is comprised of three core elements: screening, assessment, and intervention recommendations.7 The screening element includes key fall-related questions and evaluation of gait, strength, and balance. The multifactorial assessment includes a falls history and physical exam. The recommended interventions are based on the findings of the assessment, but can include patient education, vitamin D or calcium prescriptions, or referral to PT or community fall prevention program.7
Although STEADI has been in place for several years, there are few studies that have examined implementation strategies designed to support STEADI’s uptake.8,9 Implementation strategies for evidence-based guidelines within the primary care clinics vary in number and the strategy itself. On average, four to five strategies are used when implementing a new guideline.10–17 Strategies for implementation commonly include education, audit and feedback, and a facilitator.10–17 The study results for successful implementation have been mixed, with some studies reporting success with guideline adherence and others with either mixed results or no increase in guideline adherence. Even fewer studies have been performed to measure the success of the implementation strategies, including workgroups, project champions, audit and feedback, training, coaching, and reminders, in fall prevention in primary care.8,18,19 Additionally, there is little information on sustainability. Although there are available evidence-based fall prevention guidelines and screening tools, there continues to be a lack of information regarding the effectiveness of different implementation strategies to implement STEADI8,20,21 The objective of this study was to address the gap in the literature by determining the effectiveness of implementation strategies, specifically education regarding the STEADI toolkit and dissemination of the STEADI written materials, on the implementation of the (STEADI) toolkit. We tested the effect of the STEADI education and written material implementation strategy on the primary care clinic staff’s knowledge and intention to implement the STEADI toolkit at the conclusion of the intervention follow-up compared to usual implementation. We also tested the effect of the STEADI education and written material implementation strategy on primary care clinic staff’s medical record documentation of screening, assessment, and appropriate interventions 30 days after completing the STEADI education implementation strategy compared to primary care clinic staff receiving usual implementation.
Materials and Methods
A prospective, mixed methods study was used to test the effect of the STEADI toolkit education and written materials on knowledge and intention to use in primary care clinics as well as test the effect of the STEADI education and written materials on screening, assessment, and appropriate intervention behaviors compared to usual care. This article describes the quantitative information of the mixed methods study.
Data were obtained from the primary care clinics at a major medical center in the southern US using a convenience sample approach. The sample included primary clinic staff, licensed and unlicensed, since all assist in the STEADI screening, assessment, and intervention. The licensed staff included physicians, advanced practice registered nurses, and nurses. The unlicensed staff included medical assistants. Verbal informed consent was obtained for each participant as approved by the University of Arkansas for Medical Sciences’ IRB. The participants gave verbal informed consent prior to completion of the study documents.
The mixed methods study utilized two study arms, usual implementation (control) and STEADI education implementation strategy. Two clinics were enrolled in each of the study arms using simple randomization. The control arm used the facility’s usual implementation strategy. The data collection methods included the Continuing Professional Development (CPD) Reaction Questionnaire,22 STEADI Knowledge Test,23 and medical record data.
Apriori power analyses were performed prior to the beginning of the study to assess the potential effect sizes based on the anticipated sample size. G*power software was used to perform three priori power analyses due to the need for different calculations for the t-test, factorial ANOVA for knowledge and intervention, and factorial ANOVA for behaviors.24 The effect size (Cohen’s d) of 0.58 for the t-test falls into the medium effect size grouping with each arm containing 20 participants. The Cohen’s f for the factorial ANOVA for both the STEADI Knowledge and CPD Reaction Questionnaire was 0.29 with a sample size of 20 participants per study arm. Finally, the Cohen’s f for the behaviors was 0.3 with a sample size of 12 per study arm.
The two implementation strategies compared in this study were usual implementation and the STEADI education implementation strategy. The usual implementation consisted of the facility’s typical process for implementing new evidence-based guidelines. This included the creation of a reminder in the electronic medical record (EMR) system known as a best practice advisory and a three-page tip sheet outlining the implementation steps. The tip sheet is generated when an older adult has an annual wellness visit. The tip sheet included an introduction of what patient population should be included in the new process, screenshots of the STEADI fall risk screening in the EMR, Staying Independent questions, Timed Up and Go (TUG) exam results, fall risk category, and a documentation tool titled “STEADI High Risk Statement for Providers.” This tip sheet was distributed to the primary care clinic staff by e-mail approximately one month prior to the STEADI go-live.
The STEADI education implementation strategy included written and online educational materials. The use of education and dissemination of written materials as an implementation strategy was chosen because the use of a multimodal approach is essential for effective implementation.25 The principal investigator proctored the online, one-hour, interactive education module. The module titled STEADI Older Adult Fall Prevention Online Course, was provided by the Center for Disease Control and Prevention (CDC).7 The online training included provider education regarding the STEADI algorithm, TUG administration, and description of the written materials included in the STEADI toolkit. The principal investigator administered the online module in person at each clinic to enable participants to ask any questions as they arose during the training.
CPD Reaction Questionnaire
The CPD Reaction Questionnaire was used to evaluate the participant’s intent to implement the STEADI toolkit. This questionnaire is comprised of 12 Likert-scale questions, with scores from 1 to 7. A score of one is less likely to occur and a score of seven is more likely to occur.26 This scale was used to evaluate the participant’s behavioral intention, social influence, beliefs regarding one’s capabilities, moral norm, and beliefs about consequences.26
STEADI Knowledge Test
The STEADI Knowledge Test, available on the CDC Train website, was used following approval from the CDC, to examine the primary care staff’s knowledge of fall risks and prevention. This was a 10 question, multiple choice test. An example of a question is “Which is not a key question when screening older adults for fall risk?”.23
The participant behaviors were examined utilizing a report generated by the EMR for each primary care clinic in the study. The EMR reports consisted of the primary care provider, annual wellness visit date, STEADI screening, screening results, assessment results, and interventions/recommendations. With regards to the STEADI screening performed, this was a “yes/no” answer. If the screening result was identified as “yes”, the assessment and interventions/recommendations were identified. Interventions/recommendations included medication changes, footwear recommendations, use of assistive device, or referrals for vision or physical therapy.
The quantitative data analysis included descriptive statistics, paired t-test, and factorial analysis of variance (ANOVA). The data were analyzed using IBM SPSS 26 software.27 Statistical significance was determined at P<0.05. The descriptive statistics were utilized for demographic information to describe the sample population. The descriptive statistics were collected for the study overall and each study arm as a single entity. Categorical data was described using frequencies and percentages. The between-group analysis of the categorical characteristics was examined using a paired t-test and factorial analysis of variance (ANOVA). A paired t-test was used to compare pre-education and immediate post-education means of the knowledge test and CPD Reaction Questionnaire. The factorial ANOVA was used to test the effects of the independent variables, time (30-day interval between the data points) and treatment (education and written materials versus usual implementation), on the dependent variables (knowledge, intention to use, and use behaviors).
Following approval from the Institutional Review Board of University of Arkansas for Medical Sciences (#228969), the primary investigator (PI) randomized four of the primary care clinics into two study arms, control (usual implementation) and intervention (STEADI education implementation strategy). The randomization was performed using an online software, Random Lists.28 An alternative clinic was used within the intervention arm because of the decision for one of the originally randomized clinics to withdraw from the study. The PI then collected baseline electronic medical record (EMR) data to measure the primary care clinic staff’s STEADI toolkit use behaviors, including two-question screening, STEADI screening, assessment, and appropriate interventions or recommendations.
Following collection of the baseline EMR data, the PI collected baseline STEADI knowledge and intention to use data for the control and intervention arms using the STEADI Knowledge Exam and CPD Reaction Questionnaire. The STEADI education module was then performed at the intervention clinics. Immediately following this education, the STEADI knowledge and intention to use information was collected again. The PI collected the screening, assessment, and appropriate intervention behaviors; STEADI knowledge, and intention to use data 30 days following the initial data collection in the control arm and 30 days following the education in the intervention arm. Once the data were collected, the PI performed data analysis.
A total of 29 participants were included in the study, 13 in the control arm and 16 in the education intervention arm. Professions included physician (13.8%), APRN (6.9%), RN (24.1%), medical assistant (41.3%), and other licensed (13.8%). Figure 1 consists of a CONSORT diagram to describe the study arms. The length of time in the profession was diverse with 20 plus years (20.7%), 15–19 years (13.8%), 10–14 years (20.7%), 5–9 years (20.7%), 3–5 years (6.9%), and 1–3 years (17.2%). None of the participants had been in their profession for less than one year. Participants also had varying lengths of employment at their respective clinics. None of the participants were employed in the clinics 15 or more years. One employee has worked in the clinic 10–14 years (3.4%), two for 5–9 years (6.9%), nine for 3–5 years (31%), 11 for 1–3 years (37.9%), and six for less than one year (20.7%). Table 1 shows the information regarding participants, length of time in profession, and length of time employed in the clinics between the study arms. There were no significant differences between the study arms.
Table 1 Demographic Information
Figure 1 CONSORT diagram to show enrollment of participants in the study.
In the terms of STEADI knowledge, over time, although not statistically significant, at the 0.05 significance level, there was an increase in mean scores between the study arms. A factorial ANOVA was conducted to compare the main effects of the STEADI implementation strategy, education, and the interaction between the education and time on the mean STEADI Knowledge test scores. Table 2 demonstrates the summary of the factorial ANOVA results for the main effects interaction. In the education intervention arm, there was a statistically significant increase in mean scores between the immediate pre-education and immediate post-education test scores. The mean (M) increased by 1.19 (pre-education M 4.31, post-education M 5.5), p 0.02.
Table 2 Factorial ANOVA Summary
STEADI Intent to Use
The participants’ intention to use STEADI mean CPD scores were not statistically different. However, within the education intervention study arm, the mean increase immediately pre-education and post-education was statistically significant. The mean (M) increased by 0.64 (pre-education M 5.47, post-education 6.11), p. 0.01. A factorial ANOVA was conducted to compare the main effects of the STEADI implementation strategy, education, and the interaction between the education and time on the mean scores of the CPD Reaction Questionnaire. Table 2 shows the summary of the Factorial ANOVA results for the main effects interaction.
STEADI Use Behaviors
A factorial ANOVA was conducted to compare the main effects of the STEADI implementation strategy, education, and the interaction between the education and time on the participants’ behaviors of screenings, positive screenings, assessments, and appropriate recommendations. Table 3 outlines the overall behaviors measured and Table 2 shows the factorial ANOVA summary of results. Overall, there was not a difference with statistical significance in fall risk activities, including screening, positive screenings, and appropriate interventions or recommendations between the study arms; however, there was a statistical significance in the interaction between time and education on assessments, p 0.01.
Table 3 Primary Care Clinic Behaviors Summary
Although there was a statistically significant difference in pre-education and immediate post-education STEADI Knowledge Test and CPD Reaction Questionnaire mean scores, there was no significant difference in the longer-term mean scores. There is not sufficient evidence to state that the STEADI education alone was adequate, over time, to increase the healthcare providers’ knowledge or intent to use regarding STEADI. Further studies using other facilitation strategies should be performed to see what the most efficient process for implementation of STEADI will be.
The STEADI Education implementation strategy may not be sufficient to create a significant change in the rate of screenings, positive screenings, assessments, and recommendations for interventions. Further studies using other facilitation strategies should be performed to see what the most efficient process for implementation will be. The individuals implementing the new process should perform a pre-implementation plan to better understand the organization/clinic’s culture and understanding of the use of the new guideline in order to develop the best facilitation strategies to be utilized.29–31
Even after the usual implementation of the STEADI in the EMR, the majority of the clinic staff continued to use the old, two-question fall-risk screening format. This may be in part because there were no de-implementation strategies used. De-implementation is the process of using strategies to prevent the continuation of utilizing old, non-evidence based practices rather than the current practice being implemented.32–34 Further studies should include the use of de-implementation strategies such as actively discouraging prior approaches and removing competing screening tools to determine efficacy in combination with implementation strategies.
The lack of sustainability in both knowledge and intention to use STEADI was identified in the STEADI education implementation strategy arm between the statistically significant increases in means immediately pre and post-education versus the 30-day post-strategy scores. This correlates with the findings in other studies that education alone may not be sufficient to promote long term, sustained change.31,35 Sustainment requires the integration of multiple domains, including political support, funding stability, partnerships, organizational capacity, program evaluation, program adaptation, communications, and strategic planning. A Program Sustainability Assessment Tool for use by facilities has been created and tested.36 Additionally, the Exploration, Preparation, Implementation, Sustainment (EPIS) framework provides guidance on sustainable implementation. Future studies testing program sustainability can use this tool or framework. The EPIS framework takes into consideration elements that have already been identified as important, including buy-in from all levels of the organization, innovation factors including how the primary care staff will use the new EBP, and support from program champions.35
Lastly, a pre-implementation assessment can be utilized to determine the clinic’s interpretation of what their needs are about the program that is going to be implemented.37 This can aid in identifying any barriers that need to be overcome or facilitators that can be used for the implementation. A pre-implementation assessment was not performed for this study because this study was looking to compare the usual implementation process versus the addition of implementation strategies of education and dissemination of written materials.
Conclusion and Recommendations
Educational efforts, particularly the dissemination of written materials and use of an online education module, to increase the use of an evidence-based fall screening tool in primary care did not increase the overall knowledge, intention to use, or use of the STEADI toolkit. Therefore, we recommend that future efforts focus on additional facilitation techniques including feedback, facilitators, or a different evidence-based tool. Facilitators can include clinic-based super users that have received additional education. Lastly, the use of an evidence-based screening tool should incorporate the clinic’s electronic medical record and may automate a fall-risk score based on data that is inputted into the patient’s medical record. Lastly, prior to the implementation process beginning, a pre-implementation survey should be performed to gage the clinic’s barriers and facilitators for implementing the new EBP.
The small sample size due to lack of participation by all primary care clinics in each study arm and use of a single study site could lead to a type II error. This could lead to a lack of the ability to generalize the results. Additionally, the PI did not collect data on the participants’ age or gender. This lack of demographic information could potentially lead to a confounder on the generational differences in the use of electronic medical records or change.
Dr Amy Hester reports personal fees from HealthTech Arkansas, outside the submitted work. Dr. Geoffrey Curran is supported by the Translational Research Institute (TRI), UL1 TR003107, through the National Center for Advancing Translational Science of the National Institutes of Health (NIH). The authors report no other conflicts of interest in this work.
1. Centers for Disease Control and Prevention. Falls are leading cause of injury and death in older Americans; 2016. Available from: https://www.cdc.gov/media/releases/2016/p0922-older-adult-falls.html.
2. Ayoung-Chee P, McIntyre L, Ebel BE, Mack CD, McCormick W, Maier RV. Long-term outcomes of ground-level falls in the elderly. J Trauma Acute Care Surg. 2014;76(2):498–503. doi:10.1097/TA.0000000000000102
3. National Electronic Injury Surveillance System. National Estimates of the 10 Leading Causes of Nonfatal Injuries Treated in Hospital Emergency Departments, United States - 2015. National Center for Injury Prevention and Control; 2015.
4. National Center for Health Statistics. 10 Leading Causes of Injury Deaths by Age Group Highlighting Unintentional Injury Deaths, United States - 2016. Centers for Disease Control and Prevention; 2017.
5. Houry D, Florence C, Baldwin G, Stevens J, McClure R. The CDC Injury Center’s response to the growing public health problem of falls among older adults. Am J Lifestyle Med. 2016;10(1):74–77. doi:10.1177/1559827615600137
6. Panel on Prevention of Falls in Older Persons AGS, British Geriatrics S. Summary of the updated American Geriatrics Society/British Geriatrics Society clinical practice guideline for prevention of falls in older persons. J Am Geriatr Soc. 2011;59(1):148–157. doi:10.1111/j.1532-5415.2010.03234.x
8. Casey CM, Parker EM, Winkler G, Liu X, Lambert GH, Eckstrom E. Lessons Learned From Implementing CDC’s STEADI falls prevention algorithm in primary care. Gerontologist. 2016;gnw074. doi:10.1093/geront/gnw074
9. Landis SE, Galvin SL. Implementation and assessment of a fall screening program in primary care practices. J Am Geriatr Soc. 2014;62(12):2408–2414. doi:10.1111/jgs.13137
10. Ornstein S, Nemeth LS, Jenkins RG, Nietert PJ. Colorectal cancer screening in primary care: translating research into practice. Med Care. 2010;48(10):900–906. doi:10.1097/MLR.0b013e3181ec5591
11. Grunfeld E, Manca D, Moineddin R, et al. Improving chronic disease prevention and screening in primary care: results of the BETTER pragmatic cluster randomized controlled trial. BMC Fam Pract. 2013;14(1):175. doi:10.1186/1471-2296-14-175
12. Harris MF, Lloyd J, Litt J, et al. Preventive evidence into practice (PEP) study: implementation of guidelines to prevent primary vascular disease in general practice protocol for a cluster randomised controlled trial. Implement Sci. 2013;8(1):8. doi:10.1186/1748-5908-8-8
13. Mold JW, Aspy CB, Smith PD, et al. Leveraging practice-based research networks to accelerate implementation and diffusion of chronic kidney disease guidelines in primary care practices: a prospective cohort study. Implement Sci. 2014;9(1):169. doi:10.1186/s13012-014-0169-x
14. Liddy C, Singh J, Guo M, Hogg W. Physician perspectives on a tailored multifaceted primary care practice facilitation intervention for improvement of cardiovascular care. Fam Pract. 2016;33(1):89–94. doi:10.1093/fampra/cmv095
15. Aakhus E, Granlund I, Odgaard-Jensen J, Oxman AD, Flottorp SA. A tailored intervention to implement guideline recommendations for elderly patients with depression in primary care: a pragmatic cluster randomised trial. Implement Sci. 2016;11(1):32. doi:10.1186/s13012-016-0397-3
16. Sanchez A, Grandes G, Cortada JM, et al. Feasibility of an implementation strategy for the integration of health promotion in routine primary care: a quantitative process evaluation. BMC Fam Pract. 2017;18(1):24. doi:10.1186/s12875-017-0585-5
17. Jager C, Steinhauser J, Freund T, Kuse S, Szecsenyi J, Wensing M. A tailored programme to implement recommendations for multimorbid patients with polypharmacy in primary care practices-process evaluation of a cluster randomized trial. Implement Sci. 2017;12(1):31. doi:10.1186/s13012-017-0559-y
18. Baker DI, King MB, Fortinsky RH, et al. Dissemination of an evidence-based multicomponent fall risk-assessment and -management strategy throughout a geographic area. J Am Geriatr Soc. 2005;53(4):675–680. doi:10.1111/j.1532-5415.2005.53218.x
19. Eckstrom E, Neal MB, Cotrell V, et al. An interprofessional approach to reducing the risk of falls through enhanced collaborative practice. J Am Geriatr Soc. 2016;64(8):1701–1707. doi:10.1111/jgs.14178
20. Johnston YA, Bergen G, Bauer M, et al. Implementation of the stopping elderly accidents, deaths, and injuries initiative in primary care: an outcome evaluation. Gerontologist. 2019;59(6):1182–1191. doi:10.1093/geront/gny101
21. Eckstrom E, Parker EM, Lambert GH, Winkler G, Dowler D, Casey CM. Implementing STEADI in academic primary care to address older adult fall risk. Innov Aging. 2017;1(2):igx028. doi:10.1093/geroni/igx028
22. Legare F, Borduas F, Freitas A, et al. Development of a simple 12-item theory-based instrument to assess the impact of continuing professional development on clinical behavioral intentions. PLoS One. 2014;9(3):e91013. doi:10.1371/journal.pone.0091013
24. G*Power: statistical power analyses for windows and mac [computer program]. Version 184.108.40.206: Heinrich Heine Universitat Dusseldorf; 2014.
25. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8(1):139. doi:10.1186/1748-5908-8-139
26. Legare F, Freitas A, Tucotte S, et al. Responsiveness of a simple tool for assessing change in behavioral intention after continuing professional development activities. PLoS One. 2017;12(5):1–13. doi:10.1371/journal.pone.0176678
27. IBM SPSS Statistics for Windows [computer program]. Version 26.0. Armonk, NY: IBM Corp; 2019.
29. Harvey G, Loftus-Hills A, Rycroft-Malone J, et al. Getting evidence into practice: the role and function of facilitation. J Adv Nurs. 2002;37(6):577–588. doi:10.1046/j.1365-2648.2002.02126.x
30. Rycroft-Malone J. The PARIHS framework–a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;19(4):297–304. doi:10.1097/00001786-200410000-00002
31. Powell BJ, McMillen JC, Proctor EK, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–157. doi:10.1177/1077558711430690
32. Upvall MJ, Bourgault AM. De-implementation: A concept analysis. Nurs Forum. 2018;53(3):376–382. doi:10.1111/nuf.12256
33. Upvall MJ, Bourgault AM, Pigon C, Swartzman CA. Exemplars Illustrating De-implementation of Tradition-Based Practices. Crit Care Nurse. 2019;39(6):64–69. doi:10.4037/ccn2019534
34. Wang V, Maciejewski ML, Helfrich CD, Weiner BJ. Working smarter not harder: coupling implementation to de-implementation. Healthc. 2018;6(2):104–107. doi:10.1016/j.hjdsi.2017.12.004
35. Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1. doi:10.1186/s13012-018-0842-6
36. Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The Program Sustainability Assessment Tool: A New Instrument for Public Health Programs. Prev Chronic Dis. 2014;11:E12. doi:10.5888/pcd11.130184
37. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11(1):33. doi:10.1186/s13012-016-0398-2
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.