A Feasibility Study Of Administering The Electronic Research And Development Culture Index To The Multidisciplinary Workforce In A UK Teaching Hospital

Introduction

Current international evidence indicates an association between individual clinician and organizational engagement in research, and improvements in healthcare performance. However, the mechanisms through which research engagement might impact on performance are complex and do not usually work in isolation; their effectiveness is dependent on the prevailing context and research culture within that organization.¹ Furthermore, the healthcare workforce is made up of many different disciplines, each with its own research culture, yet all disciplines need to work together effectively to promote optimum patient care. There may, therefore, be a shared repertoire of intradisciplinary and interdisciplinary resources within multidisciplinary teams to support patient care.^1,2 As a result of variations in the level and type of research training available to members of individual professions, disciplinary research cultures may be at variance with the research culture of the healthcare organization as a whole. This is a complex issue because of the requirement for researchers to justify investment in research spending in healthcare and to implement research findings into clinical practice.¹

Recent evidence points to a need for a “whole of system” approach to research capacity building.³ For practitioners to engage in research, a research and development (R&D) focus needs to be part of an organizational cultural shift, with the simultaneous development of research skilled practitioners, research infrastructure and the development-oriented nature of the immediate work environment.^4,5 Given the multidisciplinary nature of the healthcare workforce, and the importance of patients receiving evidence-based care from all qualified professionals, individuals need to be supported by a positive organizational culture that encourages use and development of the evidence base, and sees this as a driver in improving safe, high quality care, and in building the workforce’s research capacity and capability.⁶

There is a broad international agreement that embedding research within healthcare systems is beneficial for patient care and for healthcare practitioners and the organizations within which they work.⁷ In the United States of America, for example, better outcomes are reported for patients treated at hospitals that participate in clinical trials.⁸ In Denmark, participation in research has positively influenced the management of patients with gastro-oesophageal reflux disease in general practice.⁹ In the United Kingdom (UK), which is the context of this current study, patients first seen at a hospital with a keen interest in clinical trials are reportedly more likely to receive chemotherapy, which is often associated with improved survival.¹⁰ The National Health Service (NHS) in England recognizes research as fundamental to improving care; it is a core function of healthcare organizations and embedded within the NHS constitution.^11,12 The most recent NHS strategy recognizes the important role of research, innovation and digitization in the prevention of illness, earlier diagnosis, more effective treatments, better outcomes and faster recovery.¹³ In order to drive research forwards therefore, professionals need to be supported by a culture that encourages and engages with research.

A systematic review in 2015 evaluated the evidence relating to factors that could inform the allied health professional (AHP) research culture.¹⁴ This culture was considered to be limited because of lack of time, skills, resources, team and organizational support and structure. This review excluded any studies involving nursing or medical professionals, including those involving mixed samples of nurses, doctors and AHPs. Tools used in the reviewed studies included the Australian Research Capacity and Culture (RCC) Tool;¹⁵ the Research Spider (from the UK but developed for use in Primary Care only);¹⁶ the Research Knowledge, Attitudes and Practices of Research survey from North America and the Edmonton Research Orientation survey from Canada;¹⁷ and the Barriers to Research Utilisation Scale from North America.¹⁸ None of the tools reported in this review were designed to assess R&D culture in a multidisciplinary (medical, nursing and AHP) workforce in an acute NHS hospital in the UK; the focus of the current survey. Measures designed for this purpose in other healthcare contexts are not always relevant to the UK NHS because of the very specific organizational culture and structure of the service.¹⁹ It was for this reason that the R&D Culture Index used in this study was initially developed and evaluated in a large NHS Hospital in England (by a team including one of the authors of this paper) as a means of appraising R&D culture specifically within NHS healthcare organizations. The development and evaluation of the index is briefly summarised below and is reported in more detail elsewhere.⁴

The R&D Culture Index (the Index)

The index is a non-standardized, norm-referenced instrument for assessing the strength of an organization’s R&D culture. It was developed to help identify personal and organizational development needs, measure internal influences on research capacity building and promote healthcare professionals’ research engagement within an acute hospital.⁴ The index has since been used in other UK and international settings and contexts, for example, in an integrated trust in Northern Ireland where 277 nurses completed it.²⁰ Several factors were identified as limiting the Trust’s R&D capacity; these included a lack of support for nurses in changing practice, a lack of knowledge of research terminology and a lack of understanding of how to apply findings to practice. An overwhelming desire to overcome these barriers also emerged. The organization introduced strategies such as Trust-wide R&D link nurses. More recently, in the Republic of Ireland, the index was administered to 733 primary healthcare professionals in one Health Region. Although demonstrating an awareness of the importance of R&D, the results showed that respondents (community pharmacists, general practitioners, health managers, practice nurses and public health nurses) were largely unengaged with R&D in practice. The survey findings were used to encourage a shift towards a more positive R&D culture.

The original index had 18 items, but after exploratory factor analysis, this was reduced to 16 items.⁴ The structure and subsequent validation of the revised 16-item index indicate that it effectively appraises the organizational influences on research capacity. Watson et al noted that the development of research capability in the individual, without simultaneous development of the organization and its infrastructures is unlikely to have a significant influence on the research culture and capacity of that organization.⁴

The validated index has since been used to assess R&D culture in a range of contexts including amongst trainee pediatricians in a specialist regional training group in England;²¹ nurses and midwives in an NHS Trust in Northern Ireland,²² and AHPs, doctors, managers and nurses in a Primary Care Trust in England.²³

The index has also been used in non-UK contexts to inform the development of locally relevant strategies to strengthen research designs. For example, it was drawn upon to help understand the relationship between organizational culture, leadership behavior and employee job satisfaction in two hospitals in Taiwan.²⁴ The index was one of 18 measurement tools included for item extraction from 30 that were appraised and categorized within a constructed framework describing the absorptive and receptive capacities of organizations; this study aimed to develop a composite tool to measure the organizational context for evidence-based practice in healthcare.²⁵ It was included in a thematic analysis of factors influencing recruitment to research trials which found that although nurses and midwives typically report a moderate to strong research orientation this does not always translate into research activity or involvement, due to lack of sufficient training and time.²⁶ In addition, the index informed further qualitative research exploring the challenges that nurses faced in undertaking research²⁷ and a study aimed at normalizing research in practice for Australian nurses/midwives.²⁸

To our knowledge, R&D culture (capacity and capability) across the entire multidisciplinary workforce of an individual hospital or a UK Teaching Hospital NHS Trust has not been assessed using the index. The first aim of this study was, therefore, to convert the index into electronic format (eR&DCI) to assess the potential for electronic administration. The second aim was to test the format and assess the feasibility of administering the eR&DCI to the multidisciplinary workforce in a UK Teaching Hospital by trialing it with the multidisciplinary workforce of the tertiary Children’s Hospital within the wider organization. Our definition of feasibility was (i) the ability to administer the electronic survey and (ii) a response rate comparable with other similar studies; for example, the overall response rate among AHPs in one study using the Australian Research Capacity and Culture (RCC) tool was 30%³ and using the same tool with registered podiatrists achieved a minimum response rate of 6%.²⁹

Methods

A Feasibility Study Was Conducted By

• Converting the validated, paper-based index into electronic format (eR&DCI);
• Administering the eR&DCI in a tertiary Children’s Hospital located within the wider Trust to assess the research capacity/capability of AHPs, doctors and nurses.

Participants And Methods

Procedures

Inclusion Criteria

All registered healthcare professionals (AHPs, doctors and nurses) employed in the Children’s Hospital when the survey was distributed.

Exclusion Criteria

All non-registered healthcare workers employed in the Children’s Hospital when the survey was distributed.

A questionnaire based on the validated index was converted to electronic format using the Bristol On-line Survey tool, now marketed as ‘Online Surveys’³⁰ that is specifically designed for academic, research, education and public health organizations. The original R&D index items were transferred to the online format using the tools available within “Online Surveys”. In addition, specific demographic questions were added that sought to determine characteristics of the study population (eg, gender, age, profession, area of work/specialty, years of experience, part-time/full time and previous R&D training and experience). This would enable any recommendations for organizational development of R&D strategies to be targeted accordingly.

Initially, the eR&DCI was circulated to members of the study steering group for piloting, in order to determine whether the questionnaire structure was displayed correctly, gathered meaningful results and complied with the study’s ethical requirements. This piloting stage also tested whether more or different instructions were needed, if the additional demographic questions were easy to read, understand and answer, approximately how long it took to complete (5–10 mins) and whether data collection and analysis ran efficiently. Based on the pilot results we changed the survey settings so that all questions required an answer (ie, a forced response) to avoid missing data. Introductory presentations about the study were delivered at meetings of senior Ward Sisters/Charge Nurses (nurse ward managers); dieticians; occupational therapists; physiotherapists; the speech and language team and at several doctors’ meetings. Flyers advertising the study were displayed throughout the Hospital. On a specified date, a link to the eR&DCI was emailed to all staff by a member of the Children’s Hospital management team via a circulation list of staff members’ “nhs.net” email accounts (individual email addresses provided to all staff); a reminder email was sent after 2 weeks.

Statistical Analysis

Data were analyzed in SPSS version 22. Descriptive statistics were used to describe and summarise participants’ characteristics, research background, experience, and attitudes towards research. Categorical data such as gender of respondents, the index Likert responses, Agenda for Change (AfC – terms and conditions of service pay rates/banding³¹) were summarized using frequencies and percentages. Quantitative data such as age, number of years of experience, and R&D total scores were summarized using means (SD) for normally distributed data or median (range) for ordinal data. The R&D total scores were obtained by adding data from the16 items and the R&D support score (items 1–9); Personal R&D skills/aptitude (items 10–13) and Personal R&D intention (items 14–16). The Chi-square test was used to compare differences between subgroups (doctors, nurses and AHPs) for categorical data. Group R&D index score subgroup comparisons were conducted using Kruskal Wallis tests. All statistical tests were 2-sided and statistical significance was considered at p<0.05.

Results

Sample Description

The overall response rate was 155 out of 908 (17%). The response rate varied by professional group; responses were received from 79 out of 700 nurses (11%); 38 out of 132 doctors (29%) and 38 out of 76 AHPs (50%). Within the AHP sample, the largest proportion were dieticians (11; 29%), pharmacists (8; 21%) and physiotherapists (5; 13%) with a range of other professions including psychology, occupational therapy and speech and language therapy represented in smaller numbers.

Table 1 illustrates the baseline characteristics of study participants (n=155). Of particular note, 46 participants (29.5%) reported that they had received no specific research training; doctors were more likely to have received research training, with only 4 (10.5%) reporting no research training compared to 32 nurses (40.5%) and 9 AHPs (23.7%).

Table 1 Demographic Information And Professional Qualifications For Participants

Table 2 shows the self-reported rate of engagement in research activities by professional groups. Doctors reported the highest level of engagement in research activities and nursing staff the lowest.

Table 2 Research Activity By Professional Group

Results Of The eR&DCI

The 16 statements of the index and the full responses in this study are shown in Table 3.

R&D Support Domain

The R&D support domain is comprised of nine statements. The majority of participants agreed that practice development was valued as part of their job; there were opportunities to develop practice in their area; there were opportunities to reflect on practice; there were people around them to help and support change or developments in practice and there was strong professional leadership. Respondents generally believed that their own discipline worked as equal partners with others in order to change or develop practice; when analyzed by professional group this view was most strongly held by AHPs, with 81.5% (n=31) in agreement, compared to 72% (n=56) of nurses and 63.2% (n=24) of doctors. The statement which received least support related to staff meetings; 67.3% (n=105) of participants agreed that regular staff meetings were held to explore ideas; this response was similar across all professional groups.

Personal R&D Skills/Aptitude

The personal R&D skills/aptitude domain is comprised of statements 10 to 13; responses indicate a high level of awareness of the influence of research on practice. There was less confidence in personal skills, however, with 76.3% (n=119) agreeing that they understand research terminology and 77.6% (n=121) that they had the skills to use the organization’s library and learning facilities; 69.2% (n=108) of respondents said they were confident using research in clinical practice. When analyzed by professional group, these results demonstrate that 20 (52.6%) of the AHPs and 54 (68.4%) of the nurses agreed with the statement compared to 34 (89.5%) of the doctors.

Personal R&D Intention

This domain is made up of statements 14 to 16; the responses indicate that participants would like more opportunities to share practice development ideas, research and information across the organization; they would like to learn about research activity during the next 6 months, and the greatest majority (n=150; 96.1%) were very keen to use research in practice.

eR&DCI Scores

Table 4 shows the R&D scores’ descriptive statistics; broken down into professional groups. The R&D index 16-item questionnaire scores range from 0 to 48. In our sample (n=155) the median score of 32 across the different professional groups indicates a generally positive attitude towards research. The median R&D domain scores were similar across professional groups except for R&D skills and aptitude that was higher amongst doctors. Figure 1 indicates the total R&D Index score by health professional group.

Table 3 Results Of The Research And Development Culture Index

Table 4 R&D Index Domain Scores By Professional Groups

Figure 1 Overall R&D Index score by professional group.

Barriers And Facilitators

Three free-text questions asked about barriers and facilitators to research; these were completed by only five participants. Those who answered highlighted the importance of support from their own team, the clinical research team and management colleagues in facilitating their involvement in R&D. The key barriers identified were insufficient time for research and the pressures of clinical workload.

Discussion

Principal Findings

The aims of this study were (i) to convert the index into electronic format (eR&DCI) and (ii) to test the format and assess the feasibility of administering the eR&DCI to the multidisciplinary workforce by trialing it in a tertiary Children’s Hospital within a wider UK Teaching Hospital. Our criteria for feasibility were the ability to administer the survey in an NHS context and to achieve a response rate that was comparable with other similar studies.

Our main finding is that we successfully converted the index to Online Surveys and then administered it to the entire multidisciplinary workforce in the Children’s Hospital. We achieved a response rate of 17%; based on response rates in similar studies we judged this to be evidence of feasibility. As indicated in the introduction to this paper, the ease of use of an electronic survey to measure research engagement will promote measurement and monitoring leading to evidence of the impact of research on clinical outcomes. However, in future research, the response rate could be improved by also offering the option for completing a paper-based version of the index. Secondary findings describe the perceived support within the multidisciplinary workforce for R&D activity and the individual response to the R&D culture within the organization in terms of individuals’ research skills and aptitudes, and their own research intentions.

The survey was sent to participants using “nhs.net” email accounts, which made it easy to “click through” to respond, but this was dependent on staff accessing those accounts. Front-line staff may not always have the time or motivation to access their individual “work” email address; it is notable that the lowest response rate was amongst nurses (11%), compared to 29% of doctors and 51% of AHPs. Whilst this may reflect a time constraint, it may also reflect the lack of emphasis previously given to research in nurse education programs and in nursing practice. Alternative solutions to encourage a higher response rate in future studies include (i) hosting the eR&DCI on a secure website that staff could if they wished access directly from a mobile device at their convenience without needing to access it via an email account, and (ii) to offer a blended approach to survey completion with optional access to the paper-based index.

Despite the modest response rate the results generally showed similar or more positive attitudes towards research across all three R&D domains compared to examples of studies in other healthcare contexts and countries that administered the index.^4,20–22,32 However, respondents in our study may represent a biased sample of staff who were more likely to engage in the study because they are supportive of research. Furthermore, the launch of a R&D Strategy in the Children’s Hospital in the year before this study may have influenced some individuals’ decisions on whether or not to engage in the study. Alternatively, the electronic version may have restricted some from responding who may otherwise have participated if a paper version had been available.

The statement which received least support in the index related to staff meetings where 67.3% of participants agreed that regular staff meetings were held to explore ideas. The response was similar across all professional groups and is consistent with other international studies assessing research capacity (using the index and other tools) amongst multidisciplinary professionals, where lack of time to engage with research is a consistently identified barrier.^{3,5,20,22,33–35} This finding was reinforced by the fact that 83% of the participants stated that they would like more opportunities to share practice development ideas, research and information across the organization.

Building on these results, members of the multidisciplinary research partnership strategy group comprising senior clinicians from the Children’s Hospital and senior researchers from the local university established a monthly multidisciplinary “Research Forum” to provide a space within the working day for professionals in practice to meet regularly with experienced researchers to specifically explore ideas related to research. This provided an opportunity to highlight research training activities, as well as guidance on some of the research skills that participants reported they were lacking. The Forum is held in the organization’s library suite which provides an opportunity for staff who do not feel confident in using the library to begin developing their research knowledge, skills and confidence, supported by experienced researchers and library staff.

While responses to the personal R&D skills/aptitude domain suggested that all participants had a high level of awareness of the influence of research on practice, there were differences between the responses from doctors, nurses and AHPs regarding confidence in using research in practice. Almost 70% of respondents reported that they were confident using research in clinical practice, but analysis by professional group demonstrated that 20 (52.6%) of the AHPs and 54 (68.4%) of nurses agreed with the statement compared to 34 (89.5%) of doctors. Nurses’ and AHPs’ limited confidence in using research in practice as compared to doctors may impede their engagement in research activities. Doctors reported the highest level of engagement in research activities and nurses the lowest; this may be due to differing normative expectations, development opportunities, and/or educational approaches between the professions. There has, for example, been less emphasis on understanding and/or undertaking empirical research in traditional undergraduate nursing curricula than in medicine. Research is now, however, increasingly recognized as a key nursing role and so has become an essential component of nurses’ educational preparation.^36,37 There is still a need however for integration of research training into preceptorship programs for newly qualified nurses, and opportunities for postgraduate research training. This is reinforced by the 2016 nursing framework for England, ‘Leading Change, Adding Value’³⁸ that highlights the importance of evidence in closing the care-quality gap by practicing in ways which provide safe, evidence-based care that maximizes choice for patients.

A recent review of the literature for best practice for engaging nurses in research recommends a research budget to protect nurses’ time for involvement in research activities and the inclusion of research priorities in job descriptions, annual goals, and employee performance evaluations.³⁹ A better-developed research career structure exists in medicine, with consultants acting as mentors and sponsors to enable junior doctors to access research opportunities. In addition, more post-graduate early career research funding opportunities are available in the UK for doctors than for nurses and AHPs; for example through the National Institute for Health Research (NIHR) program for Academic Clinical Fellows. Despite this, there is concern in the UK medical workforce that pressures facing the NHS mean that doctors’ capacity to undertake research is declining. Indeed, the number of academic pediatricians recorded in the UK workforce census has decreased and time available for conducting research is being reduced in consultant contracts, with 54% of Consultant Pediatricians not currently being research active.⁴⁰

In this current study, all except one of the participating doctors were at consultant grade. Meanwhile, a mixed-methods study to understand the barriers and facilitators to undertaking research among pediatric trainees who will fill consultant posts in the future was undertaken with 136 UK pediatric trainees.²¹ There was a significant relationship between confidence in using research in practice and ability to understand research terminology; this was not related to research experience or training. Lack of time, academic culture and research opportunities provided in the current training scheme were frequently cited barriers. The authors concluded that pediatric research by doctors requires a supportive academic culture with more flexibility in training and immediate attention to a pressing staffing crisis.

Limitations And Recommendations For Future Research

The modest response rate in the current study means that caution is needed if generalizing these results in the broader workforce. Completion of the eR&DCI was voluntary and anonymous (although there was an option to self-identify and five respondents did this); thus there may have been self-selection bias as participants may have been more interested in research, and so more positive in their responses and confident in their skills and aptitudes. Previous studies using the paper-based index had response rates ranging from 34% to 54%; while studies using the Australian Research Capacity and Culture tool⁴¹ yielded response rates of between 6% and 60% in the podiatrist population and 30% in the overall AHP population.³ In the current study, the eR&DCI was circulated to participants via “nhs.net” email accounts, an individual email address provided to all staff within the organization. An electronic circulation list held by a member of the management team was used, but we could not determine if all eligible staff opened the survey link. As staff do not always access employer-provided email accounts (eg, “nhs.net”), the additional option of a paper-based version of the index, or a link to be activated on a mobile device, may increase responses. Although electronic surveys have advantages over paper-based surveys (such as a reduction in cost), there is also a potential for selection bias and lower response rates.⁴² To minimize this potential we targeted the different steps in the survey process by piloting the survey within the research team, ensuring that the survey link was easily accessible to eligible staff and being transparent about the survey length. Nevertheless, there are reported benefits of combining electronic and paper-based surveys to improve overall response rates from clinicians.⁴³ Based on our experience in this study we recommend a combined approach (electronic plus paper-based) in future studies using the index.

The majority of nurse respondents were from the NHS “Agenda for Change” (AfC) Bands 6 and 7.³¹ Within the organization where the study was conducted these bands generally denote a Sister/Charge Nurse, Clinical Nurse Specialist or Clinical Research Nurse role, and so are not representative of the overall nursing workforce where the largest proportion are staff nurses at AfC Band 5. The majority of AHP respondents were also from AfC Band 6 and 7, but this is more representative of the AHP workforce where most roles are at AfC Band 6 or 7. Future research should seek to be more inclusive, specifically targeting both junior and senior grades of staff across all disciplines.

Overall, there was a particularly low response rate amongst nurses (11%) as opposed to doctors (29%) and AHPs (51%). As frontline nursing staff are less likely to be desk-based, completing an e-questionnaire while at work may be difficult. The eR&DCI does not distinguish between the responses of ward-based nursing staff and others on the same pay band who are not ward-based; for example clinical nurse specialists or clinical research nurses. Making this distinction may have yielded different responses.

In our study, research outputs were assessed using conventional indicators (grant applications, authorship of research papers and presentations at conferences) yet respondents not engaging in these activities may still have effective skills in evidence-based practice, and appraising and utilizing evidence during clinical decision-making. As the imperative to use research in practice comes from a broader appreciation of the benefits of research for patients, it would be beneficial to identify better indicators for the use of evidence in practice.^11,44

Practice Implications

This study demonstrated a positive research culture within the multidisciplinary workforce in this tertiary Children’s Hospital. It is encouraging that the overall index score is positive across all professional groups in terms of the level of research culture measured. However, many respondents had not received formal research training. Almost half (40.5%) of nurses indicated that they had no specific training in research compared to 10.5% of doctors and 23.7% of AHPs. Yet nurses make up the largest group of health professionals providing direct clinical care.

National initiatives to create opportunities for nurses and AHP’s to engage with research opportunities are increasing in the UK⁴⁵ and strategies to develop clinical academic careers in the nursing and the AHP workforces⁴⁶ are being implemented across NHS organizations. In the organization where this study was carried out a Trust-wide strategy for nursing and AHPs aims to positively influence the research culture amongst all grades of nurses and AHPs and has been a recent enabler of research training.⁴⁷ Through a newly developed “Research Academy” the organization is now offering research information and training sessions aimed specifically at nurses and AHPs.

In line with the aims of this study, consideration has been given to administering the eR&DCI to the multidisciplinary workforce in the wider NHS Trust within which the Children’s Hospital operates. Future research would need to address the limitations of the current study as outlined above.

Conclusion

The main finding of this study is that it was feasible to convert the index to an online survey and successfully administer it to the entire multidisciplinary workforce in the Children’s Hospital, with a response rate comparable to similar surveys. As this is believed to be the first study to assess the strength of organizational R&D culture across an entire multidisciplinary workforce in the UK NHS, it also provides a unique opportunity to compare and contrast R&D Support, personal R&D skill and aptitude, and personal R&D Intention in different professional groups. Furthermore, our design and methodology may have potential application in other global regions. The study’s limitations emphasize the need for cautious interpretation of the results, but the results are sufficiently encouraging to warrant further research. Building on the current study by combining electronic delivery of the eR&DCI with self-administration of the paper-based Index would be an effective method to assess the strength of R&D culture in a single organization. This could yield effective identification of modifiable barriers to research activity and potential enablers to optimize the strength of an organization’s R&D culture.