Ottawa Model of Implementation Leadership and Implementation Leadership Scale: mapping concepts for developing and evaluating theory-based leadership interventions

Background

Leadership is one of the essential conditions for organizational success and is increasingly being recognized as central to ensure positive employee and organizational outcomes.^1,2 Playing a critical role in influencing an organization’s capacity to implement change and innovation,^3–5 leadership in health care is instrumental to creating supportive organizational environments and positive staff attitudes for implementing evidence-based practices (EBPs) to improve patient care and outcomes.^6–9 For example, Gifford et al⁶ found that when leaders prioritized goals for change, engaged with staff to communicate a vision for EBP, and developed action plans to address predetermined barriers and facilitators to implementation, statistically significant increases were observed in EBPs for the care of diabetic foot ulcers in home care nursing. In a large-scale study of EBP implementation to reduce child maltreatment across two US states, leadership predicted sustained implementation and was shown to be different in sites with full, partial, and no sustainment.¹⁰

The leadership of the managers and supervisors who are accountable for staff and work directly with patients has been highlighted as critical to the success or failure of implementing practice change.^11–13 Yet, most are unprepared for their roles in implementing change and are often unaware of effective approaches and strategies for success.¹⁴ Despite growing research in the field, the mechanisms by which leadership influences implementation are not well understood. To stimulate and support additional empirical work in this area, it is essential to identify and understand the aspects of leadership that are critical for implementation, so that capacity-building interventions can be developed and tested to facilitate successful implementation of EBP change.

One way to increase the understanding of what might work or not work in the leadership process is to examine it from a theoretical perspective. Theory can facilitate an understanding of the complexity of factors that influence how and why EBP implementation succeeds or fails in health care.^15–18 Theories of implementation are essentially theories of planned change, providing an understanding of how and why implementation works.¹⁹ They generally describe the process of implementation or explain the factors that influence implementation and predict the planned change.^17,19 Most leadership theories are grounded in the following three perspectives: leadership as relationships, leadership as characteristics, and leadership as behaviors.²⁰ In virtually all of the more dominant theories, leadership is described as a dynamic and complex process that involves influencing a group toward the realization of goals. Implementation leadership has been defined as a multidimensional process of influence that enables clinical staff to use evidence in their clinical decision making and includes the activities and behaviors of unit-level managers and supervisors that influence staff, their environment, and the organizational infrastructures.²¹

Theories of implementation and planned change emphasize the importance of leadership in both the acceptance of new innovations, such as EBPs, and their adoption into practice.¹⁰ For example, Van de Ven et al’s⁴ theory of the innovation journey suggests that leaders can increase the chance of successful implementation by developing skills in leadership, relationships, and concrete strategies for facilitating the process, despite the innovation process being complex. Schein’s²² theory of organizational culture and leadership suggests that leaders influence change through their focus and priorities and by engaging with groups, addressing discrepancies between values and behaviors, and developing plans and strategies. Recent work on the Promoting Action on Research Implementation in Health Services (PARIHS) theoretical framework identifies leadership and management support as key factors of the inner organizational context for influencing the implementation of EBP in health care.²³ Although research has identified many leadership factors that influence implementation, their incorporation into a theory has been incomplete and insufficiently tested. An analysis of 31 planned action theories did not isolate leadership as an explicit construct; however, many of the activities described were consistent with leadership behaviors identified in implementation research, such as identifying the problem, planning the implementation, and evaluating the change.¹⁹ With a growing emphasis on the use of theory in research,^17,19,24,25 the need for theoretical models and frameworks on leadership for implementation is of growing value and interest. Still, there remain a lack of theoretical perspectives that pertain specifically to leadership for implementing EBP in health care and, further still, a lack of validated measures to evaluate the leadership concepts described in leadership conceptual models.

Purpose

The purpose of this study is to demonstrate the alignment of the Ottawa Model of Implementation Leadership (O-MILe) with the Implementation Leadership Scale (ILS) for developing and evaluating leadership interventions for implementing EBP.^11,25 A secondary objective is to describe the methodological process for aligning concepts of a theoretical model with an independently established measurement tool for evaluating theory-based interventions.

O-MILe

The Ottawa Model of Implementation Leadership (O-MILe) is a theoretical model based on leadership theory and empirical research that has been developed, revised, and tested over the past 10 years.^6,25–27 It was initially called the Gifford model and was developed in a qualitative grounded theory study that compared and contrasted the activities of nursing leaders in nine health care organizations (hospitals, home care, and long-term care) that successfully or unsuccessfully implemented and sustained the use of evidence-based recommendations from clinical practice guidelines.²⁷ Based on the analysis, leadership for successful implementation involved the following: 1) facilitating staff, 2) creating a positive climate, and 3) influencing organizational structures and processes. The first-generation Gifford model was then integrated with behavioral leadership theory and planned change theories, and a second-generation model was developed that explicated three meta-categories of effective leadership.^{19,20,26,28,29}

Supported by decades of research,^30–32 the three meta-categories of effective leadership are as follows: 1) relations-oriented, 2) change-oriented, and 3) task-oriented leadership behaviors. Relations-oriented behaviors include supporting, developing skills, and recognizing others and their contributions to increase mutual trust, cooperation among members, and commitment to a unit and organization. Change-oriented behaviors are concerned with integrating a vision, demonstrating commitment, building coalitions to support change, and creating a sense of need. Task-oriented behaviors include planning, clarifying roles, monitoring operations and performance, and efficiently using resources.³² The leadership process hypothesized in the second-generation model influences individuals, the practice environment, and the organizational infrastructure to address barriers and implement EBP.

Based on the second-generation Gifford model, a leadership intervention was developed and piloted in a mixed-methods study with an embedded cluster randomized control trial.³² The purpose of the intervention was to develop a leadership process among frontline managers and clinical leaders to implement evidence-based nursing care for diabetic foot ulcers. The leadership intervention showed promise, with statistically significant increases in EBPs observed in the intervention group.⁶ In the intervention, managers and clinical leaders prioritized which practices needed to change based on evidence/practice gaps in care, and developed detailed leadership action plans with measurable short-term goals of change. Qualitative data expanded the findings and showed that leaders in the intervention group used more relations- and change-oriented leadership behaviors than leaders in the control group. Specific strategies included communicating more with staff, recognizing efforts, demonstrating commitment, and reinforcing goals for change. These findings were then integrated into the third-generation Gifford leadership model.⁶ However, results showed there was still a need for further refinement on how to develop implementation leadership.

In 2013, a group of implementation science researchers and organizational decision makers from the US, Sweden, and Canada convened in Ottawa, ON, for a 3-day planning meeting to clarify and refine the key components of a leadership intervention for implementing EBP. Using principles from the Medical Research Council Complex Interventions Framework,³³ tacit knowledge from the experiences of organizational decision makers was synthesized with findings from the broader research on leadership and implementation science to identify components of an intervention for developing implementation leadership that were salient to both researchers and organizational decision makers. Combined with relations-, change-, and task-oriented leadership behaviors from previous work,^6,26 a fourth-generation model was developed²⁵ and revised as the O-MILe (Figure 1).

Figure 1 The O-MILe. Abbreviation: O-MILe, Ottawa Model of Implementation Leadership.

In addition to specifying the leadership behaviors identified in our previous research,⁶ the O-MILe explicates the knowledge and skills that are necessary for leaders to conduct the behaviors and facilitate implementation. The O-MILe postulates that for successful implementation, front-line leaders require the knowledge of effective leadership practices, site-specific evidence-practice gaps, implementation strategies, and planned change processes, including barriers management. Leaders must additionally have skills to prioritize change, set goals and target outcomes, engage patients and staff, and develop an implementation plan. The knowledge and skills portrayed in the O-MILe are described as the key components of a leadership intervention for participants to develop relations-, change-, and task-oriented leadership behaviors for implementing EBP in health care settings.

Theoretical models are graphic or narrative representations of concepts that infer relationships between components, thereby depicting “what” is involved and “why” and “how” a specific phenomenon works.³⁴ Incorporating primary research with theoretical and empirical evidence from leadership and planned action theories, the O-MILe provides a theoretical foundation for developing and operationalizing unit-level leadership to implement EBP in health care settings, identifying both the knowledge and skills required to develop implementation leadership and the specific behaviors required for practice. Concepts within the O-MILe are consistent with research showing that leaders who focus on individualized consideration³⁵ are proactive and present,³⁶ and address barriers to change have a greater capacity for successfully implementing change.^28,37,38

ILS

The Implementation Leadership Scale (ILS) was developed by a US-based investigative team with support from the US National Institute of Mental Health. The development of the ILS was conducted completely independent from the O-MILe. The ILS is an empirically validated scale that measures unit-level leadership for implementing EBPs. Drawing on strategic climate and leadership theories,^22,39 an initial set of items were developed and tested using exploratory and confirmatory factor analyses with 459 mental health clinicians working in outpatient programs in the US. A 12-item scale of implementation leadership was supported with four subscales that represent proactive leadership, knowledgeable leadership, supportive leadership, and perseverant leadership. The scale demonstrated excellent internal consistency reliability as well as convergent and discriminant validity.¹¹ The ILS has also been validated in child welfare and substance abuse treatment settings.^10,40 While not explicating general leadership behaviors that support implementation, the ILS identifies facets of leadership that are specifically tied to implementation and may be critical to achieve positive implementation results.

Methods

Modified template analysis was used to conceptually map items of the ILS onto concepts of the O-MILe.⁴¹ Template analysis is a technique for structuring and analyzing qualitative data and involves creating a “template” of predefined categories that summarize important themes of the data set and deductively coding data onto the template categories. It can be used to analyze any form of textual data from most methodological and epistemological positions.⁴¹ For this study, concepts of the O-MILe (leadership knowledge, skills, and behaviors) acted as the template and items of the ILS were the textual data. Once the coding template was defined, ILS items were deductively coded or mapped onto the O-MILe template categories.

ILS items were mapped onto all O-MILe categories they corresponded to, and O-MILe categories that did not correspond to any ILS data were left blank. An iterative process was used in which the O-MILe developers (WG, IDG, and BLD) and ILS developers (GAA and MGE) appraised the relevance, conceptual clarity, and fit of each ILS items with the O-MILe concepts through individual feedback and group discussions. Particular attention was paid to “goodness of fit”, and interpretation was managed by corroboration between originators of the two models until consensus was reached.⁴¹ The relevance of this approach was to demonstrate how items of the ILS align with concepts of the O-MILe, a theoretically based conceptual model for developing and operationalizing implementation leadership.

Findings

All ILS items mapped to at least one O-MILe concept, and all but four O-MILe concepts (77%) corresponded to an ILS item (Table 1). With knowledge and skills considered an essential “ingredient” of an intervention for leadership development, all three items of the knowledgeable leadership subscale of the ILS corresponded to the O-MILe concept “develops knowledge of EBP”. The ILS item “develops a plan to facilitate implementation” maps directly to the O-MILe concept “develops an implementation plan”. However, the O-MILe is more specific about what knowledge and skills are required to develop a plan, such as knowledge of current practices in the setting where implementation is to occur and effective implementation methods. While the ILS implies some level of these knowledge and skills, it does not specifically identify them. Rather, it focuses on the behaviors performed during the implementation and not the prerequisite knowledge or skills to perform them. The ILS item “develops a plan” also maps onto task-oriented leadership behaviors of the O-MILe, specifically “clarifies roles and responsibilities” and “modifies documentation forms”, as these behaviors are considered part of an implementation plan. “Leadership theory”, which is part of the core knowledge required for developing leadership in the O-MILe, is not addressed in the ILS. However, investigators of the ILS found that general leadership knowledge was foundational to developing implementation leadership, in particular knowledge on both transformational and transactional leaderships.³⁶ All the relations-oriented leadership behaviors in the O-MILe aligned with items in the ILS – two ILS items from the supportive leadership category and one from the knowledgeable leadership category. Two of these items also mapped onto other O-MILe concepts. For example, the ILS item “is able to answer my questions about evidence-based practice” mapped to both the relations-oriented category of the O-MILe (communicates with staff about clinical practice issues and evidence) and the core knowledge category (knowledge of evidence-based practice).

Table 1 Mapping the ILS onto the O-MILe Note: aILS item mapped onto more than one concept. Abbreviations: EBP, evidence-based practice; ILS, Implementation Leadership Scale; O-MILe, Ottawa Model of Implementation Leadership.

Change-oriented leadership behaviors in the O-MILe are addressed with perseverant and proactive leadership items in the ILS. However, some change-oriented concepts do not explicitly map with ILS items. For example, “demonstrates commitment” and “reinforces vision and goals for change” are implicit within the perseverant items of the ILS. The O-MILe change-oriented leadership behavior “advocates for change internally and externally” did not align with any ILS item as the ILS focuses specifically on what leaders do to influence the immediate team and not people outside or external to the team.

Four of the six O-MILe concepts in the task-oriented leadership category are addressed by ILS items. The ILS item “recognizing and appreciating employee efforts” is captured indirectly in the O-MILe concept “monitors performances and outcomes” as a certain level of monitoring is assumed to be required to “recognize and appreciate employee efforts”. Two concepts from the task-oriented behavioral category of the O-MILe are not reflected in the ILS, and this is attributed to a difference in the degree of specification for each of the tools. For example, the O-MILe task behavior of “conducting regular leadership meetings” can be an antecedent to the ILS item “has developed a plan to facilitate implementation of EBP” and “providing reminders” in the O-MILe can be part of a targeted “implementation plan” in the ILS.

Discussion

Insights from mapping items from the ILS onto concepts of the O-MILe confirm consistencies in the evidence on implementation leadership and offer directions for developing and evaluating theory-based interventions. Findings extend the face validity of the O-MILe and confirm compatibility of using the ILS as a tool to measure leadership knowledge, skills, and behaviors developed from the O-MILe, a theory-based leadership intervention. All ILS items (100%) mapped onto O-MILe, with 13 of 17 (77%) O-MILe concepts corresponding to at least one ILS items, many of them directly, such as the behaviors “recognizing” and “supporting”. Other items mapped more indirectly, using different words but capturing parallel concepts, such as the change-oriented behaviors of “demonstrating commitment” and “reinforcing vision and goals for change” that mapped to “perseverant leadership” items of the ILS, albeit with greater specification and detail. A commitment to change can be considered an antecedent to perseverance, and research on sustainability confirms a relationship between leadership commitment and implementation over time.^42–45 Transformation leadership theory further confirms that the bilateral concepts of perseverance and commitment to a higher level of performance (in this case effective patient care) are interconnected, while aspects of transformational leadership mediate a leader’s ability to foster staff support for innovation and excellence.^46–48 As indicated by their alignment, leaders who are perseverant in confronting obstacles to change demonstrate commitment and reinforcement of the EBP change, both of which are reflected in the ILS and the O-MILe.

In addition to transformational leadership that involves relations- and change-oriented behaviors to inspire and guide change for a shared vision,^2,20,35 transactional leadership is important to the introduction and acceptance of new innovations in organizations.⁴⁹ Transactional leadership is largely reflected in the task-oriented behaviors of the O-MILe that illustrate a leader’s role in providing resources and support for employees. Providing resources and support are equally represented in the ILS proactive and supportive leadership items. While not assumed to be linear, the O-MILe concepts of “clarifying roles and responsibilities”, “procuring resources, education, and policies”, and “modifying documentation forms” are consistent with “establishing clear departmental standards and supports for the EBP” as identified in the ILS.

A greater degree of detail and specificity, not captured in the ILS, is reflected in some of the O-MILe leadership concepts. For example, the ILS item “develops a plan to facilitate implementation of EBP” mapped onto a number of O-MILe concepts that are not represented in the ILS because they are considered prerequisite knowledge or more specific components of the implementation plan, such as clarifying roles and responsibilities. These differences are partly due to the focus of the ILS on implementation at the microsystems team level, whereas the O-MILe includes change management strategies at both the microsystems team level in addition to the organizational level (eg, advocates for change internally and externally). The methods that have been used in the development of the O-MILe and ILS offer a plausible explanation for these differences in specificity. With the first generations of the O-MILe developed through qualitative methods and a cluster randomized control trial,^6,27 the concepts are more contextually detailed describing leadership behaviors that occurred within specific cultural and contextual settings. Whereas the ILS has undergone extensive factor analysis testing with large samples, therefore the emergence of higher ordered latent concepts were able to occur through greater merging of dimensions and concepts. As testing continues with the O-MILe, similar merging may occur resulting in analogous latent concepts and a more parsimonious O-MILe.

The use of template analysis to combine theoretical concepts with psychometrically validated measurement tools presents a novel approach for researchers to confirm that they are capturing key components of a theory-based intervention in their evaluation. This process is also useful to ensure that the evaluation of an intervention is grounded theoretically and the epistemological position is compatible with the measurement techniques being used. Mapping concepts of the O-MILe with the ILS outlines and reinforces the dynamic nature of implementation leadership and illustrates the necessity of unit-level leaders (ie, managers and supervisors) to strategically use a full range of transparent leadership behaviors to achieve EBP as a norm. Leading successful implementation of EBP is complex and multifaceted, and both the O-MILe and the ILS reflect that leaders must use behaviors and approaches that are strategic (change oriented), relational (relations oriented), and functional (task oriented).

While recognizing that the sample size was small, a strength of this study was that the original developers of both the O-MILe and the ILS were involved in the conceptual mapping process and agreed upon the meanings and alignment of concepts and items. However, this is also a limitation because of the inherent bias this approach brings to fit the tools together compatibly. Future research is required with the O-MILe and ILS to test the practical application and validate the alignment of concepts within the tools for intervention development and evaluation.

Another limitation of this study is that both the O-MILe and the ILS are directed toward leadership at the service unit level and have not been tested at a more macro health systems level where problems are more complex, inherently political, and do not typically have evidence-based solutions.⁵⁰ Our findings fit well for microsystems implementation approaches where EBPs have been shown to be effective. However, leadership at senior organizational and systems levels may require different considerations and approaches that cannot be addressed in a single model such as the O-MILe. Furthermore, while both the O-MILe and the ILS were developed and tested in North American settings, concepts may not be generic to other countries and cultures, particularly in countries that show different expressions of individuality, social desirability, and assertiveness.⁵¹ Work is currently in progress using the O-MILe and ILS together to develop and evaluate leadership interventions in Canada and China.

Conclusion

The O-MILe provides a theoretical basis for developing implementation leadership, and the ILS provides an empirically validated tool for measuring the same. The use of template analysis to combine concepts of a theoretical model with an independently developed evaluation tool presents an approach for researchers and decision makers to develop and test interventions with compatible tools. Mapping has affirmed compatibility between the O-MILe and the ILS and provides insights into the leadership behaviors and mechanisms that promote the implementation of EBPs. We suggest that used together, the O-MILe and ILS provide an evidence- and theory-based approach for developing and evaluating leadership for implementing EBPs in health care.

Acknowledgment

IDG is a recipient of a Canadian Institutes of Health Research Foundation Grant (RFN #143237).

Author contributions

WG and GAA conceptualized the study. WG and IDG conducted the initial analysis, and all authors contributed to data analysis. WG wrote the first draft, and all authors contributed to writing and editing. All authors read and approved the final article.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Judge TA, Piccolo RF. Transformational and transactional leadership: a meta analytic test of their relative validity. J Appl Psychol. 2004;89:755–768.
2.		Wang G, Oh I, Courtright SH, Colbert AE. Transformational leadership and performance across criteria and levels: a meta-analytic review of 25 years of research. Group Organ Manage. 2011;36:223–270.
3.		Choi JN, Chang JY. Innovation implementation in the public sector: an integration of institutional and collective dynamics. J Appl Psychol. 2009;94(1):245–253.
4.		Van de Ven AH, Polley DE, Garud R, Venkataraman S. The Innovation Journey. New York, NY: Oxford University Press; 1999.
5.		Damanpour F, Schneider M. Phases of the adoption of innovation in organizations: effects of environment, organization and top managers. Br J Manage. 2006;17:215–236.
6.		Gifford WA, Davies BL, Graham IG, Tourangeau AE, Woodend K, Lefebre N. Developing leadership capacity for guideline use: a pilot cluster randomized control trial. Worldviews Evid Based Nurs. 2012;10(1):51–65.
7.		Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255–274.
8.		Aarons GA, Sommerfeld DH. Leadership, innovation climate, and attitudes toward evidence-based practice during a statewide implementation. J Am Acad Child Adolesc Psychiatry. 2012;51(4):423–431.
9.		Cummings GG, Estabrooks CA, Midodzi WK, Wallin L, Hayduk L. Influence of organizational characteristics and context on research utilization. Nursing Res. 2007;56(4 suppl):S24–S39.
10.		Aarons GA, Ehrhart MG, Torres EM, Finn NK, Roesch SC. Validation of the Implementation Leadership Scale (ILS) in substance use disorder treatment organizations. J Subst Abuse Treat. 2016;68:31–35.
11.		Aarons GA, Ehrhart MG, Farahnak LR. The Implementation Leadership Scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45.
12.		Greenhalgh T, Robert G, Bate P, Kyriakidou O. How to spread good ideas: a systematic review of the literature on diffusion, dissemination and sustainability of innovations in health service delivery and organisation. Milbank Q. 2004;82:581–629.
13.		Ovretveit J. Leading improvement. J Health Organ Manag. 2005;19(6):413–430.
14.		Guerrero E, Padwa H, Fenwick K, Harris L, Aarons G. Identifying and ranking implicit leadership strategies to promote evidence-based practice implementation in addiction health services. Implement Sci. 2016;11:69.
15.		Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23.
16.		Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: a consolidated framework for science. Implement Sci. 2009;4:50.
17.		Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53.
18.		Rycroft-Malone J, Seers K, Chandler J, et al. The role of evidence, context, and facilitation in an implementation trial: implications for the development of the PARIHS framework. Implement Sci. 2013;8:28.
19.		Graham ID, Tetroe J; KT Theories Research Group. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2007;14(11):936–941.
20.		House RJ, Aditya RN. The social scientific study of leadership: Quo vadis? J Manage. 1997;23:409–473.
21.		Gifford WA, Davies B, Edwards N, Griffin P, Lybanon V. Managerial leadership for nurses’ use of research evidence: an integrative review of the literature. Worldviews Evid Based Nurs. 2007;4:126–145.
22.		Schein EH. Organizational Culture and Leadership. 4th ed. San Francisco, CA: Jossey-Bass; 2010.
23.		Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11:33.
24.		Brehaut J, Eva K. Building theories of knowledge translation interventions: use the entire menu of constructs. Implement Sci. 2012;7:114.
25.		Gifford W, Graham ID, Eldh AC, Lefebre N. Theoretical foundations of dissemination and implementation leadership: a conceptual model for leadership development. In: 7th Annual Conference on the Science of Dissemination and Implementation: Transforming Health Systems to Optimize Individual and Population Health; 2014; Bethesda, MD.
26.		Gifford WA, Davies B, Graham ID, Lefebre N, Tourangeau AE, Woodend K. A mixed methods pilot study with a cluster randomized control trial to evaluate the impact of a leadership intervention on guideline implementation in home care nursing. Implement Sci. 2008;3:1–10.
27.		Gifford WA, Davies B, Edwards N, Graham ID. Leadership strategies to influence the use of clinical practice guidelines. Can J Nurs Leadership. 2006;19(4):72–87.
28.		Graham ID, Logan J. Innovations in knowledge transfer and continuity of care. Can J Nurs Res. 2004;36(2):89–103.
29.		Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.
30.		Yukl G. An evaluative essay on current conceptions of effective leadership. Eur J Work Organ Psychol. 1999;8(1):33–48.
31.		Yukl G, Gordon A, Taber T. A hierarchical taxonomy of leadership behavior: integrating a half century of behavior research. J Leadership Organ Stud. 2002;9(1):15–32.
32.		Yukl GA. Leadership in Organizations. 6th ed. Upper Saddle River, NJ: Pearson Prentice Hall; 2006.
33.		Craig P, Dieppe P, Macintyre S, Mitchie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council Guidance. Br Med J. 2008;337:979–983.
34.		Miles MB, Huberman AM, Saldana J. Qualitative Data Analysis: A Methods Sourcebook. 3rd ed. Thousand Oaks, CA: Sage Publications Inc; 2014.
35.		Kouzes JM, Posner BZ. The Leadership Challenge. San Francisco, CA: Jossey-Bass; 2003:446.
36.		Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt M. Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implement Sci. 2015;10:11.
37.		Tistad M, Palmcrantz S, Wallin L, et al. Developing leadership in managers to facilitate the implementation of national guideline recommendations: a process evaluation of feasibility and usefulness. Int J Health Policy Manage. 2016;5(8):477–486.
38.		Baker R, Camosso-Stefinovic J, Gillies C, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2010;(3):CD005470.
39.		Ehrhart MG, Schneider B, Macey WH. Organizational Climate and Culture: An Introduction to Theory, Research, and Practice. New York, NY: Routledge; 2014.
40.		Finn NK, Torres EM, Ehrhart MG, Roesch SC, Aarons GA. Cross-validation of the Implementation Leadership Scale (ILS) in child welfare service organizations. Child Maltreat. 2016;21(3):250–255.
41.		King N. Using templates in thematic analysis of text. In: Cassell C, Symon G, editors. Essential Guide to Qualitative Methods in Organizational Research. London: Sage Publications; 2004:256–270.
42.		Davies B, Edwards N. Sustaining knowledge use. In: Straus SE, Tetroe J, Graham ID, editors. Knowledge Translation in Healthcare: Moving from Evidence to Practice. Sussex, UK: Wiley-Blackwell; 2009:165–173.
43.		Maher L, Gustafson D, Evans A. Sustainability Model and Guide. UK: NHS Institute for Innovation and Improvement; 2007.
44.		Ovretveit J, Staines A. Sustained improvement? Findings from an independent case study of the Jonkoping quality program. Qual Manag Health Care. 2007;16:68–83.
45.		Ploeg J, Davies B, Edwards N, Gifford WA, Elliott-Miller P. Factors influencing best-practice guideline implementation: lessons learned from administrators, nursing staff, and project leaders. Worldviews Evid Based Nurs. 2007;4(4):210–219.
46.		Burns J. Leadership. New York, NY: Harper and Row; 1978.
47.		Bass BM, Avolio BJ. Improving Organizational Effectiveness through Transformational Leadership. Thousand Oaks, CA: SAGE Publications Inc; 1994.
48.		Eisenbeiss SA, van Knippenberg D, Boerner S. Transformational leadership and team innovation: integrating team climate principles. J Appl Psychol. 2008;93(6):1438–1446.
49.		Waldman DA, Bass BM. Transformational leadership at different phases of the innovation process. J High Technol Manage Res. 1991;2:169–180.
50.		Grint K. The Hedgehog and the Fox: leadership lessons from D-Day. Leadership. 2014;10(2):240–260.
51.		House RJ, Dorfman PW, Javidan M, Hanges PJ, Sully de Luque M. Strategic Leadership Across Cultures: The GLOBE Study of CEO Leadership Behavior and Effectiveness in 24 Countries. Thousand Oaks, CA: SAGE Publications Inc; 2014.