Evaluating A Multidisciplinary Cancer Conference Checklist: Practice Versus Perceptions

Introduction

Multidisciplinary cancer conferences (MCCs) are meetings of health practitioners to discuss patient pathways at any stage along the cancer care continuum.^1–3 MCCs function to guide treatment planning through input from diverse specialists, which for breast cancer treatment can include, a general surgeon, plastic surgeon, pathologist, radiologist, medical and radiation oncologists, geneticists and nurse specialist, and ideally, palliative and psychology care specialists.⁴ Research suggests that multidisciplinary care improves patient outcomes,^5,6 as well as professional practice including increased guideline concordant care and improved clinical decision-making.^7–11 Conversely, professional non-attendance at multidisciplinary meetings is linked with delayed decision-making and gaps in patient-centered care.¹²

Our previous research showed variability in MCC practices for young women with breast cancer (YWBC) in regards to frequency of meetings, professional attendance, patient referral patterns, and pre- or post-treatment timing of MCC.¹³ A follow-up study showed similar results, as well as physician-driven rather than systematic case selection for presentation at MCC.¹⁴ Similar concerns regarding case presentation and discussion are reported elsewhere.^15,16 There appear to be few standardized approaches to conducting MCCs, even though MCCs were originally conceived as an approach to standardize care processes.¹⁷ Overall, research suggests that practical supports are needed to help prompt presentation of YWBC at MCC, to support good MCC practice, and to track clinical care considerations.

Checklists have been used successfully in other environments to improve consistency and quality of care delivery.^18–21 In oncology care, checklists have primarily shown benefit in supporting team working.^8,12 To date, most of this research has been conducted in health practice settings where presentation of oncology patients to MCCs is mandatory. In Canada, there is no national mandate for presenting breast cancer patients to MCC. Although, a recently released national guideline from the Canadian Partnership Against Cancer contains statements about who should participate in MCC (e.g., surgeons) and who should be presented at MCC (e.g., complex cases),²² and there are provincial standards for the operation of MCC in Ontario.¹

As part of a Knowledge to Action (KTA) cycle²³ and based on our previous research,¹³ we developed an MCC checklist for use with YWBC. The checklist was designed to support quality care of YWBC by prompting presentation of YWBC at MCC as well as important elements of the MCC process (e.g., diverse specialist referral for YWBC). The present research aimed to pilot the checklist with surgical leads across sites participating in a pan-Canadian prospective cohort study aimed at Reducing the bUrden of Breast cancer in Young women (RUBY).

Methods

Study Design And Ethics

Design of the MCC checklist (Figure 1) was informed by a review of the literature including guidelines on multidisciplinary care of YWBC, expert input from RUBY site principle investigators (PIs), and a process of iterative feedback and refinement (Hales et al, 2008).

Figure 1 RUBY multidisciplinary cancer conference (MCC) checklist.

A logic model (Appendix A) was developed as a framework for evaluation of the MCC checklist pilot and documented intended processes and impacts. The logic model also served as a basis for developing the cross-sectional evaluation survey conducted to gauge end-users’ perceptions of the processes and impacts of checklist implementation. The checklist itself provided direct data on some aspects of checklist implementation. St Michael’s Hospital Research Ethics Board (REB# 17-357C) provided approval.

Checklist Launch

The checklist was launched in June 2017 across the 30 active (of 32 total) RUBY sites, comprising a mix of academic and community-based hospital settings in both rural and urban centres. Most sites began recruiting YWBC between the period of late 2015 to mid-2016. Twenty sites have research coordinators (RCs) assisting with participant recruitment and study administrative processes. The other sites are managed by local PIs who are assisted by a virtual RC. Each site was provided with $500 to support checklist implementation.

The launch began with direct discussion with RUBY PIs and site RCs on the purpose of the checklist in supporting quality care of YWBC by prompting important MCC processes including: patient presentation at MCC (preferably at pre-operative MCC); referral to diverse specialists to support the complex care needs of YWBC; cueing importance of attendance at MCC by diverse specialists; and reminding specialists to consider pathology and radiology results in making treatment decisions. Procedures for checklist completion were also discussed and documented in a follow-up email and the checklist launch was featured in a quarterly RUBY study newsletter distributed to site PIs and RCs. Monthly teleconferences with RCs and ad hoc communication also supported checklist implementation.

Where applicable, RCs presented the RUBY checklist to PIs as part of the new patient file. PIs may have completed the checklist on their own or have enlisted the support of the RC to complete the checklist following patient presentation at MCC. In sites with virtual RCs only, PIs were responsible for completing the checklist, but may have enlisted the support of administrative staff. Checklists were completed after each presentation at MCC (i.e., patients may have been presented more than once) or at the end of 1 year from the period of study recruitment if they were never presented to MCC.

Evaluation Participants

Participants were PIs (surgeons and surgical oncologists) and RCs representing the 30 RUBY sites actively recruiting RUBY participants (i.e., YWBC) at the time of the evaluation: Ontario (n=14), British Columbia (n=4), Quebec (n=4), Alberta (n=3), Manitoba, Newfoundland and Labrador, Nova Scotia, Saskatchewan and the Yukon (n=1 each).

Survey Design, Data And Procedure

Evaluation survey questions focused on implementation processes, with specific examination of checklist use (outputs), impacts, and contextual factors that may affect checklist uptake (Appendix A). Survey questions were categorized according to: checklist use (e.g., is the checklist in use or not); implementation processes (e.g., perceptions of: clarity on the purpose and processes; difficulty of implementation; helpful supports during implementation); current use (e.g., who usually completes the checklist, when completed, with which patients, and for what purpose); impacts on care and practice (e.g., perceptions of impact on diversity of specialist referrals, clinical data tracking, awareness of needs of YWBC, the number of YWBC presented, any negative impacts); context of the implementation (e.g., other tools or checklists currently in use, barriers to implementation, supports for ongoing use, etc.); checklist ratings (e.g., overall ratings of ease of use, feasibility of continued use, extent of support for care of YWBC); future use and improvements (e.g., open-ended questions about possible improvements to the checklist and recommendations for other tools that might support MCC of YWBC); and demographics (e.g., participant profession, years in practice). The survey contained a mix of Likert, categorical and open-ended questions. Prior to dissemination, the survey was piloted by clinical leads and research support staff involved in the RUBY study to test the applicability, validity, flow and functionality of survey questions.

An email containing a link to the electronic survey hosted by Survey Monkey™ was sent to all RUBY sites. The survey ran between May and July 2018.

Checklist Data

RCs/PIs electronically forwarded anonymous completed checklists to the research team. Checklists were collected between July 2017 and September 2018. Checklist data were entered into a password-protected server.

Data Analysis

Thematic content analysis was used for open-ended evaluation survey questions.²⁴ Frequency counts and mean scores were tabulated for numerical questions in the survey and MCC checklist. In cases in which a RUBY participant was presented to MCC more than once and duplicate checklists were received, data on MCC presentation and attendance were counted for each checklist, but referral data were counted only once.

Results

Evaluation Survey Response Rate And Sample Characteristics

Twenty-eight evaluation surveys were received; two were incomplete and one was from a site that had not yet recruited RUBY participants. The final survey sample included 25 participants (89% response rate) who represented mostly urban sites (92%) across nine provinces. Most respondents were RCs (52%) (Table 1).

Table 1 Evaluation Survey Sample Characteristics

Checklist Use/Outputs

Fifteen participants (60%) reported using the checklist. Checklists were most commonly completed by the RC (n=9/15, 60%), followed by the presenting clinician (n=3/15, 20%), and rarely by others (i.e., nurse, administrative assistant and RUBY PI; n=1/15, 7% for each). Participants using the checklist reported completing it for “all RUBY patients” (n=11/15, 73%), or with only “some RUBY patients” (n=3/15, 20%), or for “all YWBC” irrespective of participation in the RUBY study (n=1/15, 7%). Variable timing of the checklist along the continuum of care was reported. The majority (n=9/15, 60%) used it at both pre- and post-treatment MCC, four (27%) used it at pretreatment only, and two (13%) used it at post-treatment only. The checklists were completed an average of 47 days post-presentation at MCC (interquartile range=42 days).

Implementation Processes

Seven of 25 participants (28%) reported that the checklist purpose was unclear, including four who did not use the checklist. Procedures for implementing the checklist (e.g., when to administer the checklist) were unclear for four of twenty-five (16%) respondents. According to those reporting checklist uptake, factors that supported checklist implementation included quarterly administrative teleconferences (n=11/15, 73%), documented implementation and administration processes via email/newsletters (n=8/15, 53%), local champions (n=3/15 20%), implementation funds (n=1/15, 7%) and “other” factors (n=5/15, 33%) such as administrative support or making the checklist part of an initial patient assessment package.

Checklist Impacts On Care And Practice

Of the fifteen respondents using the checklist, eight (53%) listed a variety of its impacts on practice including support of clinical data tracking/uploading data to patient file (n=6, 40%), supporting MCC data collection (n=5, 33%), increasing awareness of the complex needs of YWBC, and prompting YWBC presentation at MCC (n=1, 7% each). No negative impacts were reported.

Contextual Factors

Factors that might affect checklist implementation (e.g., rural vs urban centre; barriers and facilitators to MCC) were assessed across both user and non-user sites (n=25). All but one rural site reported that MCCs were part of clinical practice. The most frequent barriers to or problems of checklist use were forgetting to use the checklist and the presenting clinician failing to complete it (n=8/25, 32% each), data to complete the checklist not available (n=7/25, 28%) and “other barrier” (n=7/25, 28%), such as other multidisciplinary checklist/processes already in place, site PI not present at MCC/not the presenting physician, or very few RUBY participants/YWBC presented to MCC. Supports for the ongoing use of the checklist were reported by the 15 sites using the checklist and most commonly included an RC/assistant to support its use (n=12/15, 80%), reminders for use from the RUBY research team (n=10/15, 67%), as well as on-site checklist champions (n=2/15, 13%), perceptions of positive checklist impacts (n=1/15, 7%) and the checklist design (n=1/15, 7%).

Overall Ratings, Future Use And Improvements

Users gave high overall ratings for ease of checklist use (M=4.1; SD=3.2) and feasibility of continued use (M=3.2; SD=1.3), but lower ratings for the extent to which the checklist supports care of YWBC (M=2.7; SD=1.3). Of those using the checklist, two of 15 (13%) reported that the checklist would become business as usual, five (33%) were unsure of its continued use, and eight (53%) reported that it would not be used further. Of the latter, five reported that there were already tools in place to support multidisciplinary care or that the checklist was not needed in addition to existing care processes, one indicated that the checklist would need to be implemented outside of a time-limited research project to have better uptake and another indicated that specialists are just too busy to complete checklists.

Among all respondents (n=25), ten answered a question about what might improve the checklist. All but one said that no changes were needed to the tool itself; a single comment suggested clarification of one of the questions. Process changes proposed to support checklist use included: greater awareness and implementation by surgeons, greater integration of the checklist with existing data tracking processes, and ongoing reminders to use the checklist.

Irrespective of checklist use, all participants were asked what might support local multidisciplinary care of YWBC. Of those who responded (n=24), nine (38%) said that no improvements were necessary and four (17%) were unsure. Others suggested better professional representation at MCC (e.g., pathology, medical oncology, nursing) (n=3/24, 13%), better checklist implementation/modified checklist procedure, awareness raising of complex needs of YWBC, a registry of YWBC cases, patient triage prior to presentation (n=2/24, 8% each), and better IT support for meetings and buy-in to MCC from specialists (n=1/24, 4% each).

Checklist Data

Checklist data are summarised in Table 2. Fifteen of the 30 eligible sites (50%) representing six Canadian provinces (Ontario=7; Quebec=3; Alberta=2; British Columbia=1; Newfoundland=1; and Manitoba=1) completed checklists. Three hundred and fifty-nine checklists were received of the 764 enrolled RUBY participants at the evaluation cut-off date (47% return rate). Overall, data show that referrals to medical oncology were most common (83%) followed by genetics (74%), radiation oncology (71%), social work (46%) and plastics (44%). Forty-nine patients (14%) received “other” referrals, which most commonly included nutritionist (n=11), kinesiologist (n=11), specialised support programs for young adults with cancer (n=8), gynaecologist (n=8), physiotherapy (n=6) and personalized medicine (n=5). Other referrals included rehabilitation oncology, occupational therapy, neoadjuvant treatment, pastoral care (n=2 for all) or dentist and thoracics (n=1 each).

Table 2 RUBY Multidisciplinary Cancer Conference (MCC) Checklist Data

One-hundred and nine patients (31%) were reported presented at MCC. The majority (52%) were presented pre-operatively and mostly on one occasion only (n=90, 84%). MCC attendance was highest for surgeons (92%), radiation oncologists (94%) and medical oncologists (88%) and lowest for nursing (52%) and plastic surgery (15%). Surgeons were also most likely to lead case presentation (67%), followed by medical oncologists (30%) and radiation oncologists (5%). Imaging and pathology were reported reviewed in 60% and 83% of cases, respectively. Initial treatment plan changed after MCC in 41 of the 100 cases (41%) for whom data were provided. Most of these were major changes (n=27, 27%). Detail on what changes were recommended varied, such as cessation of treatment (e.g., because patient experienced toxicity), additional treatment (e.g., adjuvant chemotherapy in addition to previously prescribed chemotherapy), reconsideration of treatment plan (e.g., decision not to go ahead with plan as decided based on pathological evidence) and treatment delay pending further testing (e.g., magnetic resonance imaging prior to initiating chemotherapy).

Discussion

Previous research shows variable professional attendance and referral patterns as well as timing and frequency of MCCs for YWBC.¹³ This study is the first to report on the development, use and evaluation of a checklist to support MCC practice with YWBC and to document data on MCC presentations. Evaluation findings were mixed. There was only moderate uptake of the checklist and some confusion about its purpose and processes for its use. Checklists were used at roughly half the study sites and for about half of all patients enrolled in the study. There were some reports of benefits for clinical and administrative data collection processes, for prompting presentation of YWBC to MCC and for considering appropriate referrals for YWBC. However, many participants indicated that MCC processes at their site were sufficient without the support of the checklist – a finding that contrasts with results of actual checklist data.

Challenges associated with uptake of new clinical tools or health guidelines are well documented in the literature.²⁵ Overall ratings of the checklist were positive for ease of checklist implementation and use and there were almost no recommendations to change the checklist itself. However, there was limited uptake of the checklist and some confusion about and/or a lack of buy-in to its purpose. For the most part, the checklist was used as a data collection tool by RCs, and this was seen as one benefit of the checklist, particularly for sites with less formalized data tracking processes. A few participants also highlighted clinical benefits of increasing presentation of YWBC to MCC as well as increasing the number and kind of referrals for YWBC. Despite some clinical and data tracking utility, the majority of survey respondents signalled that the checklist would not become part of “business as usual” and almost 40% indicated that improvements in MCC practice were unnecessary.

Participant perceptions of MCC practice contrast with actual checklist data, which show a picture of uneven representation of health care providers at meetings and a patient presentation rate well below international standards.²⁶ The finding that only 30% of YWBC were presented to MCC in sites that are specifically working to improve outcomes for YWBC via the RUBY study is surprising. There was also a discrepancy between perceptions of the timing of patient presentation to MCC, with over half of the participants reporting that patients were presented both pre- and post-operatively and checklist data showing that only 5% of participants were presented more than once.

Discrepancies between perceptions of MCC practice and actual practice point to the need to integrate audit and feedback cycles into multidisciplinary care to ensure quality care practices. This is the essence of the KTA approach undergirding our series of research into MCC of YWBC, including this pilot project. It is critical that health care providers are aware of the overall functioning of their clinical practice routines and the impacts these have on patient outcomes. For example, the finding that 41% of patients presented to MCC were recommended treatment changes points to the potential impacts of MCCs, and the importance of supporting their optimal functioning. Feedback is critical to increasing clinician awareness of gaps in practice and to provide motivation for adopting practice enhancing and quality audit tools. Additionally, clear expectations and targets for practice such as those presented in international guidelines that see MCC as an essential part of the care pathway and which specify targets for patient presentation to MCC^12,26,27 are needed to support local practice.

This pilot study is limited by its observational design and participant population, and findings are indicative only. Checklist data are also limited in that they pertain to sites that self-selected to participate in the evaluation and are relevant only for RUBY participants. It may be that MCC presentation rates are even lower at non-RUBY sites where there is not a push to present YWBC to MCC. Sites with lower rates would have the most room to grow, and therefore, might show more benefit from the use of a checklist tool. However, implementation obstacles would need to be overcome prior to further implementation.

Conclusions

Overall, the MCC checklist shows potential as a data monitoring tool as its findings could help to identify and then aim resources at areas of MCC practice needing improvement (e.g., patient referrals and more consistent attendance including by nurse specialists) as part of an audit and feedback cycle. There is also potential for the checklist to support clinical practice as evidenced at a few pilot sites. However, awareness raising on utility of the checklist in supporting MCC practice and quality monitoring as well the benefits of presenting YWBC to MCC is needed. Awareness raising would help to overcome implementation challenges such as lack of buy-in and may increase motivation to utilise the checklist in real time, which would overcome challenges associated with mining data from incomplete clinical notes.

Beyond the evaluation of the checklist, this study served to highlight important gaps in MCC care. Findings for low rates of presentation of YWBC at MCC raise concern about care equity. Careful consideration of the basis on which decisions are made to present patients to MCC is warranted and these decisions should be well documented. It is unlikely that equity of practice will be realised without policy standards that document expected care practices and minimum targets for practice. However, with the right top down supports to set the expectation for MCCs and audit and feedback cycles that gauge practice, organising tools such as this MCC checklist may help health care practitioners to realise standards in practice.