Characteristics of a successful collaboration in evaluation of a health care innovation: lessons learned from GPS locator technology for dementia clients

Introduction

Dementia is characterized by progressive and irreversible decline in cognitive functioning. According to the World Health Organization, there are about 47.5 million people who have age-related dementias worldwide with 7.7 million new cases annually. It is expected that the number of people with dementia will reach 135.5 million by 2050.¹ In Canada, 1 in 40 Canadians aged 65 years and older and 1 in 3 Canadians of age over 85 years have dementia.² To devastating social effects, the direct and indirect economic costs of dementia are expected to rise from $33 billion per year in 2016 to $293 billion per year by 2040.³ In order to mitigate the social burden on patients and families, as well as the economic implications for health care, an exploration of innovative and integrated solutions is warranted.

Individuals with cognitive impairment may become disoriented and lost when alone, while navigating indoors or outdoors.⁴ Algase et al conducted a review that resulted in an empirically based, standardized definition of wandering: “a syndrome of dementia-related locomotion behavior having a frequent, repetitive, temporally-disoriented, and/or spatially-disordered nature that is manifested in lapping, random and/or pacing patterns some of which are associated with eloping, eloping attempts or getting lost unless accompanied”.⁵ The wandering behavior of people with Alzheimer’s or related dementias impacts not only their independence but also adds to the stress of family caregivers. For example, more than 40% of family caregivers report that the emotional stress of their role is high or very high.⁶

A variety of interventions are currently used to manage wandering, including the use of physical barriers, physical restraints and medications to manage behaviors.⁷ However, physical interventions inhibit independence, while medications may cause unwanted side effects. Recently, a technological solution has shown utility in the management of wandering: a wearable locator device that uses a global positioning system (GPS). This technology may be an effective approach to monitor individuals at risk for getting lost because it enables caregivers to know the geographic location of an individual with dementia while allowing the individual some autonomy.⁸

In spite of the advances and apparent advantages in the use of GPS technology to monitor and locate people with dementia, its use is debated. Little is known about the reliability and acceptance of GPS technology among individuals with dementia and their caregivers.⁹ Majority of the available evidence come from descriptive case studies focused on cognitively intact older persons as well as formal or informal caregivers providing anecdotal comments about their experiences in using GPS.^10–13

From 2013 to 2015, Alberta Health Services led a prospective research study, the Locator Device Project (LDP), which examined the usability of GPS technology with 45 dyads (primary caregiver–client with dementia) in both rural and urban settings. The primary objective was to evaluate uptake and usability of GPS technology for Home Care dementia clients and their caregivers. The data suggest that the devices had high usability, and that they were well accepted and provided peace of mind to caregivers and independence to patients. Detailed results for the primary objective are reported elsewhere.¹⁴ The purpose of this paper is to report on the project planning, implementation and evaluation process within Alberta Health Services related to the LDP. Specifically, we were interested in 1) knowing which factors contribute to the success of the LDP and 2) what lessons can be learned to inform future implementation and evaluation of new technologies or services.¹⁵

Background

In health care, the literature recognizes three realms that have a direct impact on the successful adoption of innovation: characteristics of the organization or external environment, individuals and the innovation itself (Figure 1). Evaluation of a market-ready technology is viewed as the first phase in implementation.

Figure 1 Factors that have a direct impact on the knowledge transfer of innovations.

Organization/external environment

Organization and external environment factors significantly impact the acceptance of a technology and its adoption.¹⁶ Collaboration between community networks, regulations and legislation, peer pressure and competition are the key environmental factors for successful translation of an innovation from idea into practice.¹⁷ Collaboration among stakeholders is an important facilitator of innovation adoption.¹⁷ Additionally, an organization’s social role, formal and informal, influences the effectiveness of implementation. Organizations that make staff feel welcomed, promote open feedback, peer collaboration and clear communication and offer stability and solidarity among teams are more likely to achieve successful adoption.¹⁸ Not surprisingly, encouragement of staff members to reach beyond their roles and expand boundaries enables quicker translation of new practices.¹⁸ Interdepartmental organizational cohesion¹⁹ is also the key for successful technology transfer and implementation. Scott et al found that excessive external control negatively impacts implementation by restricting creativity and novelty, which are optimized when an organization has open processes and worker autonomy.²⁰

Finally, successful and efficient implementation is contingent on the effectiveness and support of senior management and clinical leadership, credible evidence, innovation infrastructure for the translation of research into practice and the extent to which organizational culture change is required.²¹

The individual

Individual attributes are the second factor of successful implementation. Personal features such as leadership skills, acceptance of change, degree of autonomy, informed decision-making ability and how one attracts necessary resources to sustain the changes and improvements in practice are the key traits.^16,22 An individual’s professional and social networks also play a strong role in individual decision making,¹⁷ while translation of knowledge into practice benefits from a sustained interaction between researchers and practitioners.²³ Barriers to innovation implementation on an individual level include limited autonomy, insufficient time and lack of administration support for applying evidence-based decisions.¹⁷

The innovation

Innovations perceived by users to have a clear advantage (effectiveness or cost-effectiveness) over other existing solutions are more readily implemented.^20,24 Rogers’ Diffusion of Innovation Theory posits five characteristics of an innovation that influence its adoption and implementation: relative advantage, compatibility, complexity, trialability and observability.¹⁷ Of the five characteristics, relative advantage and observability have been identified as the most influential factors for implementation.²⁰ This is supported by previous findings where an innovation that meets a recognized need and is perceived to be compatible with organizational and individual beliefs and values is more likely to be implemented.^24,25,26 Implementation is also catalyzed by innovations that are simple, straightforward and demonstrate reduced risk.¹⁹ Lastly, innovations piloted on a smaller scale to gauge impact and produce positive, observable outcomes are more readily accepted and implemented.²⁰

Methods

As part of Alberta’s Continuing Care Strategy: aging in the Right Place, Alberta government ministries of Health and Innovation and Advanced Education, along with Alberta Health Services partnered to examine innovative technologies that could help seniors to remain in their community as they age. Alberta Health Services had previously piloted three market-ready technologies in the Continuing Care Technology Innovation (CCTI) Project from 2010 to 2012; however, locator technologies were not commercially available in Alberta at that time.

Project development – LDP steering committee

Following similar methodology to the CCTI project and utilizing the learnings from similar recent technology projects in Canada, Alberta Health Services and researchers from the University of Alberta assembled a steering committee of technology and continuing care champions. Chaired by former CCTI leadership, membership of the LDP Steering Committee included Alberta Innovation and Advanced Education and the Alberta Health Continuing Care Branch, stakeholders within Alberta Health Services: Emergency Medical Services, Seniors Health Strategic Clinical Network, Home Care, Addiction and Mental Health, Seniors Health, Health Technology Assessment and Innovation (HTAI), Allied Health; as well as external stakeholders: the Alzheimer Society of Calgary, Alzheimer Society of Alberta, University of Alberta – Faculty of Rehabilitation Medicine (Department of Occupational Therapy), Calgary Police Service and Royal Canadian Mounted Police in Grande Prairie.

Trial design

The LDP used a prospective mixed-methods approach including a pre and post paper-based questionnaire, focus groups and individual interviews.

Technology uptake was examined post knowledge transfer using the After Action Review method, which has shown utility in military and in health care settings.^27,28 The experience and acceptance of GPS technology among dementia clients and family caregivers in this project are described in another article.¹⁵

Procurement process

Alberta Health Services identified a locator device vendor through a standardized vendor evaluation and selection process following a Request for Proposal led by Alberta Health Services’ Contracting Procurement and Supply Chain Management department. The vendor selection committee awarded a contract to the highest scoring vendor, SafeTracks. Ultimately, SafeTracks provided three different locator devices that were evaluated: ST200/ST200PRIME, iLoc Bracelet and SmartSoles insoles.²⁹ Each technology consisted of a GPS device and a web-based application which provided real-time coordinates of the patient and outlined their path of travel. SafeTracks became a collaborator in project development and implementation and maintained an ongoing role in technology education and support.

Participants

For stakeholder interviews, eight informants consisting of the project lead, the provincial lead of home care development, a research assistant, a member of the evaluation team and four caregivers who enrolled in the project shared their opinions regarding the technology and overall LDP processes.

Procedures

Structured interviews were conducted by phone using a set of five interview questions (Supplementary material) with additional ad hoc questions based on participant responses. Four additional detailed questions were posed to the project lead who had extensive involvement and knowledge of the project. The duration of the interviews varied, ranging from 10 to 15 minutes.

Ethics approval was obtained from the Health Research Ethics Board at the University of Alberta, and operational approval was obtained from Home Care in Alberta Health Services. All dementia clients signed informed consent forms or assent forms. All caregivers and other stakeholders also signed informed consent forms. Dyads who used a GPS device did not take increased risks associated with wandering. In fact, the use of a device would have enhanced the chances of caregivers locating those who were lost. Caregiver–client dyads who chose to participate in the study were interested in preserving a client’s freedom to go outside, yet be protected from risks to themselves if they become lost. Other ethical issues raised by the ethics board were shared by the evaluation researcher with the team as an approach to explain the evaluation process. For example, while the use of caregivers as proxies may be seen as undermining the autonomy of dementia clients, this makes sense when we see that the proxy responses correlate with the client responses on initial and exit surveys. Further, the number of client responses dramatically declines for the exit surveys because they no longer have the capacity to complete the surveys by the end of the study due to disease progression.

Analysis

Researchers at the University of Alberta conducted an independent evaluation of the LDP. Technology uptake examination analyzed detailed notes taken during the interviews, which were coded according to their relevance to the three themes that emerged in the literature review: the organization, the individual and the innovation.

Ethics approval

Provided by the Health Ethics Research Board, University of Alberta, Edmonton, Alberta, Canada (study ID: MS2_Pro00040454).

Results

Key elements of success and main barriers identified for the initial steps of implementation for locator GPS technology in Continuing Care are arranged into three main categories and are summarized in Table 1.

Table 1 Key elements of success and barriers in the GPS implementation project Abbreviation: GPS, global positioning system.

The organization/external environment

Alignment of the two main Alberta Health Services stakeholders, HTAI and the Seniors Health portfolio, created an organizational environment that supported project goals and recognized the intended benefits for the organization. HTAI provides support for evidence-based decision making in the adoption of new and innovative technologies, while Seniors Health applies an operational focus to that evidence to support changes in health care delivery that allow for improved access, quality and management of health concerns.

While the project goals were unanimous and transparent, the project team faced a challenging year-long process preparing for participant recruitment where project approvals, and an internal purchasing process faced a high staff turnover causing vendor-related decisions to be delayed. The process was complicated by the need for numerous different types of approval within the organization with a poorly outlined and ensiled approval process. However, despite barriers, senior leadership in the organization acted as consistent champions of the project. With strong leadership support and despite heavy external control over the process, all informants asserted they felt autonomous once the project had been granted approval.

Research assistants played a fundamental role in the organization of the LDP. As the primary contact for participants and caregivers, research assistants were responsible for completing initial interviews, setting up and orienting dyads to a device, addressing questions and concerns of the dyad throughout the implementation process and completing exit interviews. With numerous responsibilities that required a considerable time commitment, the research assistants reported significant autonomy in creating their own schedules and managing their participants. At the same time, the project management team was readily available to the research assistants to provide support as needed. Often, research assistants would resolve issues and share information among each other through informal conversations or at weekly rounds, both encouraged by the management team. By promoting these informal networks and the sharing of information, the management team was able to ensure that the large group of research assistants would be informed and have easy access to information when needed. Strong, efficient and sustained communication among all the members involved in the LDP was one of the fundamental organizational factors that contributed to the success of this project. Another key positive element was the clear process for recruitment and referral process in the LDP implementation procedure framework. Additionally, the flowchart format for project team and participant duties was concise and easy to read. By creating clear communication tools, misunderstandings were minimized and the need to seek clarification of project procedures reduced. Stakeholder feedback stressed the importance and utility of the weekly meetings between the members of the team, allowing the opportunity to openly discuss obstacles, problems and successes and receive honest, constructive feedback. By ensuring individuals were not bound by role descriptions, the project management team promoted a culture of collaboration and interdependent work.

Individuals

Data gathered from the dyads and operational stakeholders indicated that the individual commitment of those involved in the LDP project was a key factor in its success. The project management team demonstrated strong levels of leadership, commitment and dedication at the individual level, which helped capitalize on the support from the organization. Another key factor was the assignment of a project lead focused solely on this project’s deliverables with no other competing priorities, allowing her to be accessible to other project team members and address problems and concerns quickly. Many stakeholders recognized the project lead as a key contributor to the smooth implementation of this project and a resource for project information and concerns. With autonomy strongly encouraged at every level, both the project lead and the research assistants had flexibility to make decisions regarding project implementation and scheduling.

Peer network interactions among the research assistants also helped minimizing mistakes and enhanced sharing of successful strategies. Through short weekly meetings, the close network created among all the research assistants became a key source of information transfer and dissemination, as research assistants shared new strategies and solutions to the problems they experienced and helped their peers navigate similar situations.

The innovation

Finally, the innovation itself was the key to the success of this project. Caregiver participants reported that the locator devices significantly decreased stress levels associated with loved ones’ wandering. They unanimously agreed that the devices demonstrated a considerable advantage over conventional techniques to locate loved ones, such as searching the neighborhood or using cell phones, and helped improving the quality of lives for both the patients and caregivers. While caregivers found the technology somewhat complex to set up and learn, educational support was readily available to quickly address any concerns and support smooth operation. Consequently, the complexity of the device did not hinder or discourage its use.

Discussion

This paper aimed to identify the contributions that led to the success of the LDP and to capture learnings that can help shape future evaluations of technology and inform the next step of implementing locator devices into Alberta Health Services care delivery. Through the LDP, informal peer networks, staff autonomy and a supportive organizational environment were identified as the most important ingredients of success. Further learning came from how the challenges were addressed and how this experience can shape successful future trials.

The autonomy of the LDP team fostered ownership responsibility. The findings showed that the creation of a strong, close working relationship was a factor of success. In the literature, organizations that enable social exchange and network of collaboration are more likely to experience effective technology implementation.¹⁹ In spite of many complicated and challenging administrative processes negatively impacting implementation of the study, persistence and encouragement from the organization motivated staff and supported their autonomy. Autonomy is undeniably an important component of successful implementation.¹⁶ At the same time, dedication and continued involvement of project leadership are the key for administrative success, facilitation and team member support.¹⁷ Lastly, educational and troubleshooting support from the vendor was imperative to quickly address concerns and problems. The ability of the LDP team to ensure smooth operation was also a critical positive factor of GPS implementation in the project. This finding confirms literature reports that assert effective vendor support as a key factor of success in implementation.^30,31

Based on the results of the LDP, key recommendations to support successful trials of technology and innovation are highlighted below:

1. Examine and understand current approval processes for projects to identify opportunities for streamlining the process.

2. Ensure leadership and support staff commitment for the duration of the project to avoid delays resulting from staff turnover.

3. Appoint a project leader who is dedicated solely to be the project to minimize competing priorities and allow timely troubleshooting.

4. Continue building individual capacity by encouraging autonomous decision making where suitable.

5. Encourage regularly scheduled project meetings to provide an open space for timely feedback, project discussion, problem solving, inclusion of remote site team members, collaboration and information dissemination.

6. Support peer-to-peer network building and collaboration. Utilize these networks as a tool for communication and information dissemination.

7. Ensure strong senior leadership support for the project. Establish an overarching body of senior leaders that could advise on project management as needed by the project team.

One of the main barriers in beginning the research project was the multilayered ethics approval process. To further complicate the process, the timing of the project coincided with the harmonization efforts within the ethics community and the entire process was under redevelopment. This is a common theme in the literature where complicated administrative approval processes frequently lengthen the time from inception to trialing and subsequent implementation of innovations. Additionally, high levels of control, which are apparent in an organization with complicated approval processes, are associated with delays and other negative outcomes.^18,32 We believe that the innovative nature of the study, that is, use of GPS devices with a dementia population, also lengthened the ethics review process. Understandably, innovations in health care are scrutinized by ethics reviewers who want to ensure that personal privacy and security issues are thoroughly addressed.

Study limitations

There are some limitations to this study. Due to time constraints, senior leaders involved and members of the steering committee were unable to be interviewed, which would have provided additional feedback at an operational level. Similarly, not all research assistants, members of the evaluation team and dyads were interviewed, which may lead to missed information. However, we feel comfortable with the assumption that opinions of the interviewees were representative of their group and any large discrepancies would have emerged at other points during the LDP process.

Conclusion

Overall, organizational support was a very strong component in the success of this project, despite minor barriers in the approval infrastructure. At the individual level, the effectiveness of the peer-to-peer networks to disseminate information and the autonomy to make decisions contributed to the project’s success. While organizational support and individual capacity were important, the features of the innovation itself, including relative advantage, compatibility, complexity, trialability and observability, played important roles.

Acknowledgments

This study was supported by Alberta Innovation and Advanced Education and Alberta Health Services in 2013–2015. We thank the following individuals who assisted with data collection and data entry: Kristen Rabel, Cassandra Greenhough, Julia Mills, Karen Nabuurs, Teanna Matchett, Peyman Azad Khaneghah and Pranshu Arora, as well as the Locator Device Project Steering Committee which provided oversight for the project.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		who.int [homepage on the Internet]. Dementia Fact Sheet. 2015. [Updated April 2016; cited September 15, 2015]. Available from: http://www.who.int/mediacentre/factsheets/fs362/en/. Accessed September 15, 2015.
2.		ASC. Society, Alzheimer Society of Canada. Rising Tide: The Impact of Dementia on Canadian. Canada: Alzheimer Society of Canada; 2010.
3.		Alzheimer.ca [homepage from the Internet]. About Dementia - Dementia Numbers in Canada. 2015 [Updated January 18, 2017; cited September 15, 2015]. Available from: http://www.alzheimer.ca/en/About-dementia/What-is-dementia/Dementia-numbers. Accessed September 15, 2015.
4.		Chang YJ, Chu YY, Chen CN, Wang, T. Mobile computing for indoor wayfinding based on bluetooth sensors for individuals with cognitive impairments.3rd International Symposium on Wireless Pervasive Computing (pp. 623–627). Santorini, Greece: IEEE, 2008.
5.		Algase DL, Moore DH, Vandeweerd C, Gavin-Dreschnack DJ. Mapping the maze of terms and definitions in dementia-related wandering. Aging Ment Health. 2007;11(6):686–698.
6.		ADI. Alzheimer’s Disease International. World Alzheimer Report. Dementia and risk reduction an analysis of protective and modifiable factors. 2014.
7.		Hermans D, Htay UG, McShane R. Non-pharmacological interventions for wandering of people with dementia in the domestic setting (review). Cochrane Database Syst Rev. 2007;(1):CD005994.
8.		Pot AM, Willemse BM, Horjus S. A pilot study on the use of tracking technology: feasibility, acceptability, and benefits for people in early stages of dementia and their informal caregivers. Aging Ment Health. 2012;16(1), 127–134.
9.		Landau R, Werner S. Ethical aspects of using GPS for tracking people with dementia: recommendations for practice. Int Psychogeriatr. 2012;24(3):358–366.
10.		Landau R, Werner S, Auslander G, Shoval N, Heinik J. What do cognitively intact older people think about the use of electronic tracking devices for people with dementia? A preliminary analysis. Int Psychogeriatr. 2010;22(8):1301–1309.
11.		Zwijsen S, Depla M, Niemeijer A, Francke A, Hertogh C. Surveillance technology: an alternative to physical restraints? A qualitative study among professionals working in nursing homes for people with dementia. Int J Nurs Stud. 2012;49(2):212–219.
12.		Chen Y, Leung C. Exploring functions of the lost seeking devices for people with dementia. Work. 2012;41(Suppl 1):3093–3100.
13.		Werner S, Auslander G, Shoval N, Gitlitz T, Landau R, Heinik J. Caregiving burden and out-of-home mobility of cognitively impaired care-recipients based on GPS tracking. Int Psychogeriatr. 2012;24(11):1836–1845.
14.		Liu L, Juzwishin D. Usability of Locator Technology Among Home Care Clients at Risk for Wandering: Evaluation Report. Edmonton: AHS; 2015.
15.		Liu L, Cruz M, Ruptash T, Barnard S, Juzwishin, D. Acceptance of global positioning system (GPS) technology among dementia clients and family caregivers. J Technol Hum Serv. 2017;1–21.
16.		Kitson AL. The need for systems change: reflections on knowledge translation and organizational change. J Adv Nurs. 2009;65(1):217–228.
17.		Dobbins M, Ciliska D, Cockerill R, Barnsley J, DiCenso A. A framework for the dissemination and utilization of research for health-care policy and practice. Online J Knowl Synth Nurs. 2002;9(1):149–160.
18.		Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.
19.		Bradley EH, Webster TR, Baker D, et al. Translating research into practice: speeding the adoption of innovative health care programs. Issue Brief (Commonw Fund)]. 2004;(724):1–12.
20.		Scott SD, Plotnikoff RC, Karunamuni N, Bize R, Rodgers W. Factors influencing the adoption of an innovation: an examination of the uptake of the Canadian heart health kit (HHK). Implement Sci. 2008;3:41.
21.		Bradley EH, Webster TR, Baker D, et al. Translating research into practice: speeding the adoption of innovative health care programs. Issue Brief (Commonw Fund). 2004;(724):1–12.
22.		Cooke M, Walker R. Research, transformational leadership and knowledge translation: a successful formula. Scand J Caring Sci. 2013;27(1):1–2.
23.		Oborn E, Barrett M, Racko G. Knowledge translation in healthcare: incorporating theories of learning and knowledge from the management literature. J Health Organ Manag. 2013;27(4):412–431.
24.		Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.
25.		Berwick DM. Disseminating innovations in health care. JAMA. 2003;289(15):1969–1975.
26.		Scott SD, Plotnikoff RC, Karunamuni N, Bize R, Rodgers W. Factors influencing the adoption of an innovation: an examination of the uptake of the Canadian heart health kit (HHK). Implement Sci. 2008;3:41.
27.		Tami G, Bruria A, Fabiana E, Tami C, Tali A, Limor AD. An after-action review tool for EDs: learning from mass casualty incidents. Am J Emerg Med. 2013;31(5):798–802.
28.		Bliss JP, Minnis SA, Wilkinson J, Mastaglio T, Barnett JS. Establishing an intellectual and theoretical foundation for the after action review process – a literature review. Arlington, VI: 2011Army Research Institute for the Behavioral and Social Sciences.
29.		Safetracksgps.ca. [homepage from the Internet]. Safetracks Solutions. 2015. Available from: http://www.safetracksgps.ca/site/our_products?cat=5. Accessed September 15, 2015.
30.		Palvia P, Jacks T, Brown W. Critical issues in EHR implementation: provider and vendor perspectives. Commun Assoc Inform Sys. 2015;6:707–725.
31.		Parolia N, Chen J, Jiang J, Klein G. Conflict resolution effectiveness on the implementation efficiency and achievement of business objectives in IT programs: a study of IT vendors. Inform Software Tech. 2015;66(1):30–39.
32.		Bradley E, Webster TR, Baker D, et al. Translating research into practice: speeding the adoption of innovative health care programs. Issue Brief (Commonw Fund). 2004;(724):1–12.

Supplementary material

Locater device project (LDP) interview questions

Interviewee:

Role:

Date & time:

Duration:

1. Please describe the implementation process of the LDP project from your perspective and experience (ie, the process of recruiting and setting up participants or being recruited).

2. What factors or circumstances do you think have contributed to the success of the implementation of the LDP?

3. Did you experience any challenges or barriers (eg, competing priorities, organizational challenges, job role changes, technological challenges) that kept you from carrying out your responsibilities?

4. Can you think of anything that could be done differently to address the challenges or barriers that have affected the implementation process?

5. In your opinion, how effective do you think the project has been in improving information flow and knowledge use between everyone involved (participants, caregivers, case managers and research assistants)? Did you feel well informed in regards to your role, responsibilities, and was access to information easily available when you were unsure?

Locater device project (LDP) interview questions

Interviewee: Tracy Ruptash

Role:

Date & time:

Duration:

1. Please describe the implementation process of the LDP project from your perspective and experience (ie, the process of recruiting and setting up participants or being recruited).

2. Have you previously been a part of the implementation of a program/project? If yes, how has the implementation process of this project compared?

3. Was there any specific or different approach taken interacting and working with the participants in this project compared to previous projects?

4. In your opinion, how effective do you think the project has been in improving information flow and knowledge use between everyone involved (participants, caregivers, HC CM, RAs)?

5. What role did your colleagues and peers play in how you carried out the implementation process?

6. Did you feel supported by the organization in carrying out your role and feel that autonomy was encouraged?

7. What factors or circumstances do you think have contributed to the success of the implementation of the LDP?

8. Did you experience any challenges or barriers (eg, competing priorities, organizational challenges, job role changes, technological challenges) that kept you from carrying out your responsibilities?

9. Can you think of anything that could be done differently to address the challenges or barriers that have affected the implementation process?