Team Functioning And Beliefs About Team Effectiveness In Inter-Professional Teams: Questionnaire Development And Validation

Background

Inter-professional healthcare teams unite providers from different professions (eg, nurse, nurse practitioner, pharmacist, physician) to provide patient care.¹ Inter-professional teams represent a safe and effective strategy to deliver care to patients and families because each professional contributes specific knowledge and skills.^1,2 These teams were introduced in hospitals over 100 years ago but are relatively new in primary care.³ Team performance has been examined using system-level indicators including access to care, wait times, and screening rates.^4,5 Yet, to better understand patient outcomes, it is essential to capture team-level processes.

Processes are defined as interactions that occur at the level of the patient and care provider.⁶ Team processes are the mechanisms that explain how actions unfold over time in response to what occurs in the surrounding context.^7,8 Researchers have often compared processes to a black box where one can observe inputs and outputs without clearly understanding what is happening in between.^8–11 Processes including teamwork, communication and patient engagement are cornerstones in the delivery of effective care.^12–14 In primary care, Hofhuis et al¹⁵ found that team performance was influenced by how frequently team members interacted with each other. However, these authors were unable to measure team-level variance.

Studies have identified that provider judgements or beliefs about team functioning and effectiveness are influenced by processes that include decision-making, communication, cohesion, care coordination, problem-solving, a focus on the needs of patients and families, as well as role clarity and trust among team members.^16,17 The measurement of these team processes has been a critical challenge because of the financial and human resources required and the lack of validated instruments. Thus, important gaps in knowledge related to team processes and team functioning remain.¹⁸

Greater understanding of how healthcare teams function is crucial given the negative consequences of poor team functioning. In the United States, miscommunication was identified as a key issue following a review of more than 23,000 malpractice claims in 4 years.¹⁹ The cost of the incidents was estimated at $1.7 billion US and 2000 lives were lost.¹⁹ Similar issues have been identified in Canada, where 37% of adverse events were deemed to be preventable with improved teamwork and communication.²⁰

Team Functioning

A systematic review of the experiences of health professionals regarding teamwork and collaboration in primary care found that key attributes of teamwork focused on user needs, interdependent actions, negotiations between professionals, decision-making, mutual respect, trust among team members, and a recognition of the role and work of providers in different professional roles.¹³ These authors identified that providers in primary care faced enormous challenges to promote teamwork and collaboration. In primary care, Beaulieu et al²¹ examined organizational structures associated with the delivery of high-quality care and found that provider views of the team’s ability to innovate were moderately associated with outcomes. In acute care, Sidani and Doran²² identified positive associations between care coordination, counselling and education provided by nurse practitioners and patient outcomes related to symptom resolution, improved functional status and satisfaction with care.

Recent systematic reviews have examined the characteristics of instruments designed to measure team functioning and team effectiveness. Important gaps were identified,^23,24 including the lack of inclusion of key dimensions of team functioning consistent with a conceptual framework and the need for patients and families as healthcare team members. Additionally, instruments have focused on specific clinical areas such as acute care or primary care but have not been validated across different settings. Validated questionnaires are needed to examine the inter-professional team members’ views of team processes in different settings.

Objectives

The study aimed to 1) develop a Provider-Perceptions of Team Effectiveness questionnaire (Provider-PTE) to examine processes in healthcare teams from the perspectives of inter-professional team members and to 2) assess the psychometric properties (ie, reliability, face validity, content validity, construct validity) of the Provider-PTE questionnaire.

Conceptual Framework

The study was guided by a conceptual framework developed to support decisions and research related to nurse practitioner roles.²⁵ Nurse practitioners often work in teams that include members from more than one professional group, and the term inter-professional is used in the framework and the current study. The framework includes three central process dimensions: role enactment, boundary work, and perceptions of team effectiveness. Of particular importance to this study is the Perceptions of Team Effectiveness dimension. Key processes in effective teams include improved communication, involvement in decision-making, cohesion, care coordination, problem-solving, and a focus on patient and family needs.^16,25,26

Methods

Research Design

A cross-sectional survey was undertaken. A self-administered questionnaire represents a cost-effective strategy to gather information from a large number of participants in distinct geographic locations.²⁷

Recruitment

We recruited participants using a convenience sample. We aimed to recruit 250 care providers to assess the psychometric qualities of the instrument and identify differences in high and low functioning teams. Our sample size allowed us to detect medium correlations and a 95% confidence interval half-width of 0.1 with 80% statistical power and a 5% alpha level.^28,29 We included in-hospital or specialized ambulatory care in our definition of acute care.³⁰ Primary care was defined as comprehensive healthcare services for common health concerns at the point of entry to the healthcare system.³¹

Data Collection

Following Research Ethics Committee approval by the Comité d’éthique de la recherche du CIUSSS de l’Est-de-l’Île-de-Montréal (Project no 2018-1168, CIUSSS_EMTL_282), data were collected from December 2017 to June 2018. The survey included an information sheet, consent form, a feedback form and a questionnaire. Participants were under no obligation to participate in the study. They were advised that they could withdraw at any time. Online and paper copies of the documents were available to participants to enhance response rates.³² All documents were available in English and French.

Instrument Development

To address Objective 1, the questionnaire was developed using empirical evidence from multiple case studies of processes in inter-professional teams^16,33–35 and a review of the literature describing processes in teams.³⁶

Item Generation

Several steps were undertaken at this stage (Figure 1). To begin, items were generated using examples and descriptions of processes and actions identified by interprofessional team members in cardiac surgery and long-term care.^16,33–35 (Figure 1). An initial list of 69 items was generated. Draft items were reviewed by members of the research team to ensure item clarity and legibility. Researchers (n=8) who specialized in questionnaire development and advanced practice nursing research reviewed the instrument. They made suggestions to clarify some items by providing examples and adding descriptions before the start of each section.

Figure 1 Flowchart of the Provider-PTE questionnaire development.

As a class exercise, 30 graduate-level nursing students attending a role development course worked in small groups to generate key contributions of advanced practice nursing to team functioning and patient care using relevant literature and their clinical experience. The contributions were mapped to the items in the questionnaire. All of the students’ ideas mapped to at least one specific item but some contributions were measured by up to five questionnaire items. This allowed the research team to identify redundancies. At the same time, our team was conducting a study to examine the psychometric properties of the Patient-PTE questionnaire.³⁷ The items in both questionnaires were worded in a similar fashion for consistency. Items from the patient questionnaire were examined to determine if any were problematic. From these results, the number of negatively worded items were limited to three in the Provider-PTE questionnaire.

Instrument

The Provider-PTE questionnaire included 41 items divided into Work Setting, Team Processes, PTE-Overall, and Outcomes presented in Table 1, along with Socio-Demographic Characteristics. More specifically, eight items described the characteristics of the Work Setting (eg, type of healthcare setting, location, caseload, and team size). Team Processes included 14 items. One item specifically measured respondents’ judgements or beliefs about the team’s effectiveness (BE). This score was used to determine high and low functioning teams. Seventeen items measured PTE-Overall and included Team Processes and factors (eg, role clarity) known to influence PTE. Outcomes included seven items (eg, timely care, staff knowledge and skills). Response options ranged from 1 to 6 (ie, strongly disagree, disagree, disagree somewhat, agree somewhat, agree, and strongly agree). A mean value was estimated to obtain a score for each dimension. The average scores ranged from 1 to 6. Improved perceptions of team processes and better team functioning were indicated by higher scores for Team Processes, PTE-Overall, and Outcomes. The socio-demographic characteristics of providers included eight items (eg, age, gender, education, professional group, and employment status). Most response options in this section were categorical. Age and time in the professional role, the organization and the healthcare team were numeric values. One open-ended question gathered any additional comments from respondents. We used single-item measures to measure boundary work (ie, trust), belief about team effectiveness, and cohesion to reduce response burden.³⁸ Two items measured decision-making, coordination, problem-solving, and role clarity. Three items were used to examine communication and a focus on the role of patients and families as team members.

Table 1 Preliminary Items Related To Healthcare Team Processes And Outcomes

Translation

The questionnaire was translated from English by a professional translator whose mother tongue is French. It was back-translated by a second translator whose mother tongue is English.³⁹ Strategies to enhance the cross-cultural adaptation of the questionnaires included cognitive interviews with participants to ensure that items were consistently understood in the same manner and pre-testing of the instrument.⁴⁰ The questionnaires were reviewed by bilingual members of the research team to ensure that the translations maintained semantic and conceptual equivalence with the framework.^40,41

Pilot Testing

Research team members reviewed the draft items. English and French language versions of the questionnaire were pilot-tested with the same advanced practice nursing researchers to identify unclear wording. No revisions were needed following pilot testing. As proposed by Kilpatrick et al,⁴² the same respondents completed the questionnaire 2 weeks later to determine if their understanding of the items had changed. Participants’ responses and understanding of the questions were unchanged after 2 weeks.

To address Objective 2, cognitive interviewing, a feedback form, and statistical analyses were used to assess the psychometric properties of the questionnaire.

Cognitive Interviewing

Participants were asked to review instructions, questionnaire layout, procedures to complete French and English language questionnaires, and item and response options for clarity.⁴³ Cognitive interviewing was used to examine the questionnaire with those who completed the pilot test and students in the graduate-level course.

Feedback Form

After the questionnaire was completed, a feedback form developed previously was administered.^37,42,44 Participants were asked to provide feedback on typical items such as time needed to complete the questionnaire, clarity of instructions; comprehensiveness of the questions related to the topic; readability; and ordering of questions. Participants had the opportunity to provide suggestions to improve the Provider-PTE questionnaire via an open-ended question.

Analysis

Data were analyzed using the IBM Statistical Package for the Social Sciences (SPSS), version 25.⁴⁵ Mean, standard deviation (SD), and range were generated using descriptive statistics. Overall Provider-PTE scores (PTE-Overall) and dimensional scores were calculated. Instrument performance was examined using Cronbach alpha (α) to assess reliability, item-total correlation of each item, and assessment of missing data. Responses did not follow a normal distribution (p values < 0.001),⁴⁶ according to the Shapiro–Wilk test. For variables with two categories, we used the Mann–Whitney U-test, and for variables containing more than two categories, we used the Kruskal–Wallis one-way analysis of variance for nonparametric testing. A Bonferroni correction was applied to correct for multiple comparisons and decrease the likelihood of Type I error. Rank correlation was determined using Spearman’s rho for continuous data.⁴⁷ Responses to open-ended questions were analyzed using content analysis.⁴⁸

Instrument Performance

Reliability

Internal consistency of the subscales (ie, Team Processes, PTE-Overall, and Outcomes) was assessed using Cronbach alpha. Values ranging from 0.7 to 0.9 were considered acceptable to excellent.⁴⁹ Because we did not have a sufficiently large number of participants from the same healthcare team, we did not compare the within-group variance to the between-group variance to determine the consistency of responses among participants in the same team.⁵⁰

Validity

Face validity was assessed using the feedback form that examined the Provider-PTE questionnaire, formatting and instructions. Content validity was assessed prior to the questionnaire roll-out by expert groups including researchers and graduate students. Construct validity was examined using the known-group technique.^51,52 We compared the scores between specific groups, and hypothesized a priori that no differences in scores would be found between 1) men and women; 2) English and French language respondents; and 3) respondents in urban and rural locations. Differences were expected according to these factors:

1. professional group (intra-professional (ie, within the same professional group), inter-professional and managers);
2. team size (small (less than 5 members), medium (5–10 members), large (more than 10 members);
3. teams with and without nurse practitioners
4. employment status (full time and part time).

Nurse practitioners can autonomously evaluate, diagnose, and treat patients for chronic or acute conditions as well as provide advanced nursing care to best meet patient healthcare needs according to the regulated scope of practice in their jurisdiction.^53,54 Teams with and without nurse practitioners were identified in our study because the province of Québec Canada planned to implement 2000 nurse practitioners by 2024 to improve care quality and access to care.⁵⁵

Streiner et al⁵⁶ define responsiveness as a questionnaire’s ability to detect a meaningful change. However, several authors argue that there is no agreed-upon method to measure responsiveness.^56–58 We compared processes in high and low functioning teams and determined a priori that the scores would be different.

In the following section, results including means, correlations, Cronbach α and p values statistically significant at the 5% level or α/n groups for post-hoc analysis are reported by measure and known group.

Results

Overall, 314 participants responded to the survey. Thirty-one questionnaires included only socio-demographic data and were excluded. Thus, 283 questionnaires were included in the analyses. Most questionnaires were completed online (75%, n=211). The mean time to complete the questionnaire was 21 mins. When three outliers were removed (ie, 243 min, 1447 min, and 1753 min), the mean time to complete the questionnaire decreased to 8.9 mins (SD: 8.75).

Socio-demographic characteristics of participants are presented in Table 2. A majority of respondents completed the questionnaire in French (85%, n=241). Most respondents were female (84%, n=216), and a majority worked full time (80%, n=205). The average age of respondents was 39 years of age (SD: 9.1, range 23–67). Providers had worked in their professional role for 13.3 years (ranging from 1 month to 45 years), in their current role for an average of 7.2 years (ranging from 1 month to 45 years), and in their current team for 6.4 years (ranging from 1 month to 45 years).

Table 2 Characteristics Of Respondents

Healthcare team characteristics are presented in Table 3. Respondents worked primarily in large teams (69%) located in an urban setting (82%). Specialties included primary care (56%), oncology (16%), Emergency/Intensive care unit (12%), and medicine (16%). Half of the respondents worked in specialized outpatient settings and in-patient units. Respondents worked in teams who cared for a wide range of patient populations.

Table 3 Characteristics Of The Healthcare Team

Instrument Performance

Reliability was examined using Cronbach α values for Team Processes, PTE-Overall, and Outcomes. Cronbach α values for Team Processes equaled 0.88; PTE-Overall equaled 0.91, and Outcomes equaled 0.72 (see Table 4).

Table 4 Mean Scores For Boundary Work, Perceptions Of Team Effectiveness, Role Clarity, And Outcomes

Face validity was assessed using responses (n=85) in the feedback form (see Table 5). Most (76%) believed that key questions to understand team functioning were included in the questionnaire and no question was unimportant (87%). Positive assessments were noted for the time needed to complete the questionnaire (96%), instructions (99%), formatting (100%), font size (100%), and logical ordering of questions (96%). Some respondents noted that it would be easier to see the negatively worded items if these expressions (eg, not) were bolded.

Table 5 Responses Related To Face Validity

Content validity was assessed using cognitive interviewing and the responses from the open-ended question in the feedback form (n=85). No question was perceived to be redundant. A small number of respondents found it difficult to select a specific patient group if they worked with several patient populations.

Overall, trust and beliefs about team effectiveness (BE) were ranked highest (mean: 5.2/6±0.8, range 1–6). The dimension PTE-Overall rated highest (mean: 4.8/6 ±0.6, range 1.4–6) followed by Team Processes (mean: 4.7/6 ±0.6, range 1.5–6) and Role Clarity (mean: 4.7/6 ±1.0, range 1.0–6). Outcomes were rated 4.8/6 ±0.6 (range 2–6). The ratings for the variables included in the PTE-Overall dimension ranged from 4.4/6 ±1.1 for problem-solving to 5.2/6± 0.8 for the BE score. The rho coefficients (rs) for the variables included in the PTE-Overall dimension ranged from 0.58 to 0.81 (p value< 0.001) (see Table 4).

Construct validity was examined using known groups (see Table 6). As anticipated, no differences were found between men and women, English and French language respondents, and respondents in urban and non-urban locations. As hypothesized, differences were identified according to professional group (p = 0.017), length of time in the team (p = 0.025), and presence of NPs. We had anticipated differences according to team size but no differences were identified between these groups. Differences were noted in Outcomes according to employment status (p = 0.017).

Table 6 Differences In Team Processes And Outcomes By Known Group Comparisons

Responsiveness was assessed using the scores in high and low functioning teams. Significant differences (p<0.001) were noted in all the scores for all the dimensions (see Table 7).

Table 7 Differences In Processes And Outcomes In High And Low Functioning Teams

Discussion

The Provider-PTE questionnaire is a self-reported instrument available in English and French that measures perceptions of team functioning and team outcomes. The questionnaire includes provider and team characteristics and three subscales (ie, PTE-Overall, Team Processes, and Outcomes). Psychometric testing (n=283) included construct validity using known groups. As anticipated, differences were identified by professional group, teams with nurse practitioners, and length of time in the team. Reliability was examined using Cronbach α. Subscale values ranged from 0.72 to 0.91. Rho coefficients (rs) for processes included in the Team Processes subscale ranged from 0.59 to 0.81 (p< 0.001), indicating that these processes were highly correlated with beliefs about team effectiveness. Responsiveness was assessed, and differences were noted in high and low functioning teams. The questionnaire fills important gaps identified in systematic reviews of instruments that measure team processes by integrating a conceptual framework and examining inter-professional team functioning across different settings.^23,24

More recently, one member of the research team (LP) pilot tested several data collection tools, including the Provider-PTE questionnaire, for her doctoral study examining nursing role enactment in intensive care units (ICUs). Participants (n=8) in the pilot test highlighted that one item needed to be clarified and two items needed to be added to the Provider-PTE questionnaire to better capture effective team functioning in the ICU. The item about patient care goals “The plan of care and patient care objectives are clearly outlined in the patient’s health record” was changed to “Healthcare team members know the goals of patients’ plans of care”. This item was reworded because some respondents indicated that unlicensed care providers who do not have access to the patient’s health record would also need to have an understanding of the goals of care. In addition, participants noted the importance of scheduling regular team meetings to plan patient care. We added two items to measure the conduct of regular team meetings in addition to meetings scheduled for emerging issues “Regular inter-professional team meetings are scheduled to plan patient care” and assess if the team could meet quickly to address emerging issues “The healthcare team meets as needed to address complex care issues that arise”. These changes were consistent with the preliminary findings of a systematic review that our team is conducting to highlight the contributions of team meetings to improved team functioning.⁵⁹

The current study generated preliminary data of the validity of the Provider-PTE questionnaire. Further testing is required for the revised 43-item questionnaire (see Table 8). The next steps include recruiting team members in large existing teams to examine whether perceptions within each team are less varied than between teams. This is important to determine our ability to aggregate survey findings from individuals to the team and organizational levels.^50,60 In addition, repeated measurements with the same teams at different time points are needed to generate further evidence of the instrument’s responsiveness and allow for the determination of a minimally important difference in scores that can be used in different context including research (eg, a randomized control trial) or ongoing quality improvement initiatives.^58,61 As proposed by Husebo and Akerjordet,⁶² measurements can be taken before and after a change in practice is undertaken to examine its impact on team functioning. Thus, it will be possible to tease out factors that are known to influence team functioning including trust, role clarity and the conduct of team meetings.⁵⁹

Table 8 Revised Items Related To Healthcare Team Processes And Outcomes*

The questionnaire makes an important contribution to the teamwork literature because it was developed to be consistent with a conceptual framework. The instrument was validated in English and French, in two different Canadian provinces, and across a range of clinical settings. We were unable to determine the extent of non-response bias because we did not have information on non-responders.⁶³ Strategies to reduce the risk of response bias included the use of pilot testing and cognitive interviewing.⁶³ Simple response options were used to reduce cognitive load when answering questions, and the questionnaire was kept as short as possible while cohering with the conceptual framework.⁵⁶ We anticipate that the recruitment of healthcare providers from different professions, working in acute and primary care with different patient populations in two Canadian provinces, will enhance the generalizability of our findings to other inter-professional teams.

Previous studies have focused on teams in specific contexts, including primary care,^64,65 and acute care^62,66,67 or in specialized clinical areas including mental health⁶⁸ and medicine.⁶⁹ The Provider-PTE questionnaire can be administered in intra- and inter-professional teams to examine team functioning. It can be used to assess team functioning before and after changes in skill mix, in different settings from acute to primary care, and in-patient and out-patient settings. Given that higher scores indicate better team functioning, it will be possible to monitor team performance to identify processes that function well in addition to those in need of improvement. Such measurements will provide feedback to decision-makers, team members as well as patients and families.

Limitations

Our study included a convenience sample. Subsequent studies will need to be conducted using a random sample of providers. We did not recruit a sufficiently large number of providers working in the same team or a large enough sample of managers. Future studies will need to be conducted in larger inter-professional teams in different settings to accrue additional evidence of validity and responsiveness. In addition, we administered the questionnaire to members of one healthcare team without direct patient care activities. Some items were difficult to answer for team members without patient contact, which suggests that survey items would need to be adapted for teams not involved in direct patient care. Finally, we did not use factor analysis to examine construct validity because the proposed subscales had fewer than three items per scale.⁷⁰ Future research is needed to test the questionnaire’s structure and confirm the proposed subscales.

Conclusion

The study produced evidence of validity for the English and French language versions of the Provider-Perceptions of Team Effectiveness questionnaire. The revised 43-item instrument is a self-completed questionnaire that takes less than 9 mins to complete. The questionnaire measures perceptions of team processes and outcomes of all members of the inter-professional team. Reliability indices were acceptable to excellent for the subscales. Face validity and content validity were examined, while construct validity was assessed using known groups. Responsiveness was examined by comparing high and low functioning teams. The questionnaire can be used to examine how inter-professional teams function in different contexts.