Evaluation of System Modelling Techniques for Waste Identification in Lean Healthcare Applications

Introduction

Lean is a reengineering philosophy¹ that aims to improve process and system efficiency by reducing non-value-added activities and wastes.² Lean has gained significant attention in the healthcare industry to develop safe, efficient and waste-free operational processes.^3–5 Recent studies reported positive results in various healthcare applications, such as reducing patient waiting time and length of hospital stay,^6,7 improved resource utilization and staff satisfaction.^8,9

An important aspect of lean is the waste identification to reduce non-value-added activities and provide standardized solutions for better performance and operational excellence.^1,10 Lean aims to identify the following eight types of waste: transportation, inventory, motion, waiting, overproduction, over-processing, defect and human potential.^11,12 Earlier studies showed that stakeholder engagement is imperative in the waste identification efforts in lean.¹³ Therefore, visual tools, such as value stream map (VSM), have been used to help stakeholders understand the overall system and identify wastes.

The VSM helps identify value-added and non-value-added activities in producing specific outputs for users by describing the “current state” of operations.¹⁴ Earlier research demonstrates positive results with the use of VSM.^15–18 Further, studies showed that the VSM catalyse staff engagement and transformational improvement.^14,19 Despite these benefits, the VSM may not always represent important support functions and system attributes in the patient flow. Therefore, the VSM may not help identify all types of wastes in healthcare effectively.²⁰ Further, it may not well represent the system behaviour, e.g., communication between stakeholders and other support functions. Due to these limitations, recent studies investigated possible alterations to the VSM to exploit its use in waste identification.²⁰ In parallel with this, various system modelling techniques (SMTs) can also facilitate visual representation of the system structure and behaviour.

SMTs (also known as process maps, system diagrams, and system modelling approaches) are visual tools that provide a diagrammatic description of processes and systems. While a wide range of SMTs have been used in other safety-critical industries, such as aviation, they have been underused in the healthcare field.^21–23 Earlier studies showed the contribution of SMTs in identifying potential safety issues in complex healthcare systems.^22,24,25

A comprehensive study on the usability of SMTs was conducted by Clarkson and colleagues.²⁶ Considering the suitability of SMTs in capturing different system attributes, they shortlisted six different diagram types that provide fundamental diagrammatic representation on the system structure and behaviour in healthcare settings. Task diagrams, information diagrams and organizational diagrams are used to describe the hierarchy of operations, information and people and their roles within the organization, respectively. Communication diagrams represent the flow of material and/or information between people and process. While flow diagrams represent activities taking place in parallel or sequence, system diagrams are used to show data transfer between activities. While the first three diagrams are structural ones describing the system architecture and defining its boundaries, the last three diagrams are behavioural, expanding the system description and presenting the communication and flow of information and people within the process.²⁷

To comprehensively describe the system structure and behaviour, earlier studies showed the potential contribution of SMTs in representing the patient journey in a visual way^28,29 and guiding caregivers in problem identification and mitigation efforts.³⁰ The strength of SMTs has been recognized in understanding existing systems before implementing improvement suggestions³¹ and identifying patient safety risks.^28,30,32,33 Similarly, visualization of the system through different SMTs can potentially strengthen the waste identification process in lean and help users in the decision-making process.³¹ It is therefore imperative to evaluate the usability and utility of each SMT to better understand its potential role in waste identification for the healthcare process. It is also essential to identify what specific types of waste can be identified using a particular type of SMT compared to VSM.

Methods

This study aims to evaluate the usability and utility of SMTs in waste identification. To achieve this, this research considered six different types of SMTs that were comprehensively shortlisted based on their practicality and suitability in capturing different system attributes.²⁶ Besides these SMTs, the VSM was evaluated to compare its usefulness against the SMTs in the same context. As outputs, eight types of wastes that are articulated in lean applications are included. The SMTs, along with the VSM, are evaluated at the outpatient clinic of the Heart and Vascular Institute (HVI) at Cleveland Clinic Abu Dhabi (CCAD), which is a 364-bed multi-specialty tertiary hospital located in Abu Dhabi, UAE.

Two data sets were obtained throughout the study. The first data set was obtained to fully understand the patient journey in the chosen clinic and represent them in the shortlisted SMTs and VSM. The patient journey in the clinic is segmented into five stations: registration, nurse assessment, physician examination, testing and check-out process. Based on observations, informal discussions with stakeholders and shadowing, the general flow of patients, tasks and communication were presented in relevant SMTs via Microsoft Visio (see Appendix). At this stage, the suitability of the SMTs was also shared with the clinical operation manager to validate that they represent the actual practice in the clinic.

The second data set was obtained to evaluate the usability and utility of SMTs in waste identification through interview-based questionnaires with stakeholders from the clinic who voluntarily participated in this study. This method was mainly adopted with the intent to capture both qualitative and quantitative feedback from participants. Further, this particular method was expected to capture the rationale behind participants’ comments and guide them better if there is any ambiguity in SMTs.

In total, twelve participants volunteered to represent all types of stakeholders or care providers in the clinic: two patient access representatives (PARs), two nurses, two technologists, two managers, three physicians, and one health unit coordinator (HUC). The questionnaire consists of three sections. In the first section, participants were asked about their background, such as their current position, working period in healthcare and familiarity with the system modelled (HVI system) and SMTs.

In the second section, three aspects of the usability and utility of SMTs and VSM were asked as follows:

• This system model is easily understandable with the instruction given (usability)
• This system model is helpful in better understanding and communicating the system’s operations (utility in system understanding and communication)
• This system model is helpful in identifying waste within the system (utility in waste identification)

The questions were presented to the participants in a tabular form together with the seven diagrams (organizational diagram, information diagram, task diagram, flow diagram, communication diagram, system diagram, and value stream map). Participants were then asked to choose the degree of agreement for each diagram. Prior to the evaluation, participants went through a ten-minute briefing about the nature of the SMT by the first author. Following the training. It involved the definition of waste types with examples on each and a brief explanation about the SMTs. Then, participants were guided through the diagrams one by one.

In the third section, participants were asked to state their degree of agreement with each SMT’s strength in identifying particular types of waste.

The rankings were numerically assigned for further analysis following the data collection. Five-point-Likert scales (strongly disagree: 1 – strongly agree: 5) were used to record the responses. The participants were asked to select the one option that closely represents their opinion regarding their experience with each SMT. Due to the small sample size as a nature of this study, a test of significance for normality was limited; therefore, descriptive statistics were obtained to summarize the evaluation results.

Results

Twelve participants from the clinic volunteered to participate in this study. Table 1 presents their job titles, their previous experience in the clinic and in using SMTs: The majority had self-reported limited experience with SMTs.

Table 1 Participants in SMT Evaluation

In the second section of the questionnaire, the usability and utility of the SMTs were captured. The summary statistics of the evaluation can be found in Table 2.

Table 2 Summary Statistics of SMT Evaluation (n = 12)

Figure 1 shows the average grade for each question collected from the participants.

Figure 1 The general usability and utility of SMTs in waste identification.

As shown in Figure 1, the SMTs with the structural link (organizational diagram, information diagram, and task diagram) were rated more easily understandable than the ones with the behavioural link (flow diagram, communication diagram, and system diagram). On average, the organizational diagram was the easiest model to understand. Participants’ comments during the interview also reflected their ratings. For instance, they pointed out that the structural diagrams are simple and easy to understand. The task diagram was expressed as a useful diagram in reflecting the process details. The flow diagram was described as complicated because of the sequence of the tasks, mainly if multiple procedures occur in parallel. The system diagram was also found to be complicated as it was observed to be too detailed by the participants. Moving to the communication diagram, participants, in general, found it difficult to follow as it showed patient flow and information flow simultaneously. The VSM was also found to be slightly similar to the flow diagram but with less complexity and involves additional information, such as processing time and capacity in each step of the patient journey.

Based on the participants’ feedback during the interview, it was noticed that their previous experience with the diagrams also influenced their responses with ease of understanding the diagrams. For instance, three participants had no experience with the SMTs, while experienced participants were able to make a clear connection between the content of the diagrams and their work area. Also, experienced participants were able to relate the capacity and cycle times given in the value stream map with their key performance indicators (KPIs), demonstrating the effectiveness of clinical targets in achieving its overall objectives.

For the second question in section two in the questionnaire, most participants found the diagrams helpful in understanding and communicating the system operations with little disagreement. Most participants agreed that the VSM is particularly useful in explaining the system operations. The participants also expressed that the flow diagram, communication diagram, and system diagram represent different system functions but not easy to track due to their complex nature. The organizational and information diagram had a lower agreement in their utility in communicating the system, as they were not found effective diagrams in reflecting the operational process.

For the third question in section two in the questionnaire, participants were asked to grade the usefulness of SMTs in identifying the wastes in the processes. The participants found all the diagrams useful for waste identification; however, the organizational diagram received the lowest ranking. Participants agreed that the most helpful diagram in waste identification is the VSM, followed by the system diagram and task diagram.

The third section of the questionnaire was intended to identify particular types of wastes using different SMTs. Figure 2 presents the results for this specific question.

Figure 2 Usefulness of SMTs in Identifying Particular Waste Type.

The VSM was found as the most useful diagram type in identifying overproduction, inventory, motion, waiting and human potential waste types with the highest average scores; 4.2, 4.1, 4.4, 4.7 and 4.3, respectively. For waiting waste in particular, some participants mentioned that the VSM outperforms as cycle time is presented for each task. They highlighted that delays could also be identified via VSM with their possible contribution to the system performance. It was found that SMTs were not very helpful in identifying the system’s defects as the highest average for any type of SMT was scored below 4.0. For over-processing waste type, flow diagram and communication diagram were found as the most useful diagrams with an average score of 4.3. The participants also pointed out that some tasks that require additional communication and interaction between stakeholders may be performed in addition to regular tasks in real practice. Therefore, they highlighted the usefulness of the flow diagram and the communication diagram in ascertaining their impact in the given system. In identifying transportation waste, both flow diagram and VSM were found equally useful with an average score of 4.2.

Discussion

This study provides valuable insight into the usability and utility of SMTs in identifying systemic wastes that are well articulated in lean. The results showed that the diagrams with structural links, in general, are easier to understand compared to the ones with the behavioural links. Regarding usability and utility in waste identification, behavioural diagrams were found to be more useful, although the communication diagram scored lower than information and task diagrams. Further, structural diagrams were not very useful in identifying particular types of wastes.

Based on the average rates given in ranking, the VSM was found as the most useful diagram in waste identification. This is consistent with the previous studies,^14,20 showing that the VSM is a powerful waste identification tool. However, this study also showed that some SMTs, such as flow diagram and communication diagram, are more powerful than VSM in identifying particular waiting waste. However, it can be concluded that using multiple diagrams are more helpful to identify a comprehensive list of wastes during lean exercises, such as kaizen meetings. Even though SMTs have the potential to generate a clear description of the system from different perspectives, limited resources and time pressures may prevent the number of SMTs from being built and utilized for lean applications. To prioritize the use of diagrams for waste identification, the results of this study can be valuable. Table 3 shows the suitability of SMTs in identifying different types of wastes. Based on this table, a particular kind of SMT can be prioritized for waste identification efforts in lean.

Table 3 SMTs and Waste Types

During the interviews, it was validated that the participants’ job types are related to the kinds of waste that can be recognized and identified. For instance, department administrators focused on over-processing issues since it is related to the nature of their work, such as monitoring capacity, cost, and utilization. In contrast, others were more interested in wastes that influence their day to day tasks. Further, the participants’ background and experience in SMTs influenced their feedback and responses. Their general area of specialization and education played a role in their understanding and interpretation of the diagrams. The depth of analysing the SMTs varied from one person to another, each individual had his/her skills, and they looked at the data on SMTs and VSM perceived them differently. This may have influenced their ratings and feedback in the study.

In this study, it was noted that the participants’ familiarity with the HVI system and their role in a specific part of the patient journey is significant. For instance, in general, nurses are integrated caregivers partly or fully involved in all stages of the patient journey. However, PARs and physicians are more concentrated on their confined tasks. This situation potentially has an impact on the types of wastes they notice in the system. Although the diagrams presented the end-to-end patient journey, participants’ familiarity with the particular step and process affected their waste identification ability. On the other hand, exposing stakeholders with SMTs made them more aware of the overall system, appreciate other roles, and see the effect and dependency of their tasks on other stakeholders. For example, during the observation, when PARs and HUCs evaluated the diagrams, they noticed some similarities between their activities. They also pointed out that appropriate coordination between them would improve the clinic’s overall performance.

During the interviews, participants noted that external factors might also affect the system and contribute to waste generation. Expanding the system boundaries by including other departments, such as the call centre that handles booking patient appointments on the phone, or referrals from other clinical units, such as internal medicine, may help the waste identification process. Further, expanding the scope of SMTs by involving higher layers, such as patients, management, financial counsellor, and insurance companies, may give an enhanced picture of the system from different dimensions to understand system behaviour under different circumstances and recognize more wastes that could affect the overall system. As noted in the literature, lean considers both social and technical aspects in systems.³⁴ Therefore, developing and implementing lean along with VSM and SMTs may help facilitate a gradual cultural and behavioural change.

While the results provided significant insights on SMTs and their contribution towards waste identification, the findings should be considered with limitations. For instance, this study was conducted in an outpatient clinic with a limited number of stakeholders; therefore, the findings’ generalizability to other healthcare settings may be limited. It may also be expected that having more participants may have enhanced the discussion and strengthen the findings for validation and further statistical analysis. While this can be identified as a limitation, both qualitative and quantitative results derived in this study can be claimed to serve as a reliable indicator of an accomplishment that can be done within the timeframe of this study.

Conclusions

This study evaluated the usability and utility of SMTs in waste identification efforts in lean, along with the VSM. While the SMTs can be used in conjunction with VSM, they can also be used as stand-alone diagrams during lean activities, such as kaizen meetings, to enhance stakeholders’ understanding of system behaviour and their engagement in system-wide waste identification efforts. Future research may consider applying SMTs in lean efforts in various healthcare settings to obtain further results for validation purposes. Researchers can also assess the training needs and resource intensiveness of SMTs compared to other waste identification tools and methods.