The Design and Evaluation of Clinical Pathway for Disease Management to Maximize Public Health Benefit

Background

The popularization of evidence-based clinical practice guidelines improves the dissemination and application of evidence by physicians, helps improve the clinical and economic outcomes of diseases, and plays an important role in developing plans for future studies and disease management.¹ However, evidence-based clinical practice guidelines may be insufficient for bridging the evidence- or knowledge-to-practice gap. Studies have shown that more than 20 years is required to assimilate good research evidence into daily clinical practice.^2–4 The critical path method, which has been used in the manufacturing industry, was introduced into the medical industry to develop a clinical pathway (CPW) for disease management in the 1980s. Since then, evidence has been integrated into the local medical and health system,⁵ becoming an important tool for governments and health organizations assumed to directly improve medical care quality, standardize medical procedures, and control medical costs.⁶ An intervention is considered a CPW if it meets four criteria: 1) it is a structured multidisciplinary care plan; 2) it is used to channel the translation of guidelines or evidence into local structures; 3) it details the steps in the course of treatment or care in a plan, pathway, algorithm, guideline, protocol, or other “inventory of actions” (ie, the intervention has time frames or criterion-based progression); and 4) it aims to standardize care for a specific clinical problem, procedure, or episode of care in a specific population.^7,8 However, as an implementation strategy, CPW implementation research and improvement involve extensive complexity and numerous challenges. The implementation effectiveness of CPWs in real-world clinical settings is different from the sum treatment effectiveness of interventions involved in CPWs in relatively controlled trial environments.^9,10 When a CPW implementation effort fails, determining whether the failure occurred because one or more critical interventions of the CPW were ineffective in the new setting (intervention strategy failure) or if good interventions were deployed incorrectly (implementation strategy failure) is important. Unfortunately, we lack a theoretical understanding of CPW implementation.^10,11 All prototypes of the existing CPWs of disease management were discussed and developed by multidisciplinary teams (MDTs) and were evaluated by commonly used cohort studies or traditional efficacy- or effectiveness-oriented randomized controlled trial designs. Due to the inability to determine the actual CPW operating process, the effectiveness of CPW management can only be generally measured, reported findings cannot be attributed to the CPW’s implementation, and further exploring or gaining insights on the causality of pathway components based on reported findings is not possible. Valuable feedback cannot be provided on the applicability of evidence and the rationality of resource allocation.^12–15

Following the above CPW design and evaluation ideas, CPW implementation in China has not achieved substantive success.¹⁶ The Chinese government launched national healthcare reform in 2009, and the CPW programme was one of the prominent ongoing initiatives, which included more than 400 diseases and was required at all tertiary hospitals, with 80% secondary cases by 2015.¹⁷ All CPWs were tables listing all involved work steps and details. The nationwide assessment in pilot hospitals between 2010 and 2011 indicated that 90% of patients included in the CPW programme experienced a reduced length of stay, and the average costs accordingly dropped,¹⁸ but further evaluation found little microlevel insights into how CPWs are implemented in hospitals and how the implementation process affects the pilot outcomes,^19,20 and few patients complied with the CPWs in public general hospitals.²¹ The 2019 review of 10 years of healthcare reform progress showed that China’s financial burden of using healthcare had declined slowly and that the relative efficiency of health resource utilization was decreasing; moreover, the contribution of CPW implementation to this progress is unknown.^22,23 Establishing an effective monitoring and evaluation system remains a challenge for Chinese healthcare reform,^24,25 and one unresolved critical issue is to conceptualize and evaluate how well a CPW focused on not only outcomes but also operating processes is implemented. Based on the current situation of China’s CPW implementation, we propose and discuss a design and evaluation model for CPWs through some theoretical lenses, which may help the designer master the actual functional scope of CPWs and provide reliable evidences of health technology assessment for healthcare decision-making and clinical practice (see Figure 1).

Figure 1 Pathway design and evaluation programme. The process is repeated until the clinical pathway management design achieves the maximum public health benefit.

Fixing Critical External Conditions That Affect CPW Implementation in Research, Where at Least Two Dimensions are Required to Define a CPW’s Functional Scope

No large and comprehensive system is applicable to any field. Understanding the scope of a CPW’s function from the whole health system perspective is essential before CPW implementation research and improvement. Changes in healthcare quality can be focused at the individual level, group/team level, organizational level, or the larger system/environment level, efforts to improve healthcare quality correspond to different levels, and the levels of change in healthcare have interdependencies and impact each other. Efforts considering each level in isolation are unlikely to effectively evaluate and significantly improve healthcare quality and clinical efficiency.^26,27 CPWs for disease management represent an organization-level strategy to improve the quality of overall healthcare (see Figure 2), and the functional scope should be evaluated and discussed under fixed external conditions, such as politics and economic policies, cooperation models, and individual knowledge and expertise. A good CPW design may function weakly or be impossible to implement in some context. Before the 2009 national healthcare reform, a Chinese case study revealed that 30% of departments in a pilot hospital refused to implement CPWs because CPWs harmed hospital revenues.²⁸ After the 2009 national healthcare reform, an early case study found that CPW implementation encountered structural barriers and that CPW effectiveness was undermined by an institutional environment unconducive to cost containment efforts, especially unscientific payment mechanisms.¹⁹

Figure 2 (A) Levels of changes in healthcare quality and corresponding improvement efforts.16 (B) Two dimensions to define the actual functional scope of clinical pathways (CPWs).

Under critical fixed external conditions, we must consider the actual functional scope of a CPW in a specific disease management circumstance before designing it, which can be reflected in at least two dimensions according to its target functions. The first is the extent of the organization-level net benefit of disease diagnosis and treatment generated by CPW operation (benefit dimension), which reflects the net benefit of medical cost control and care effects; the second is the degree to which the whole disease diagnosis and treatment process has been standardized at the organization level by the CPW’s operation (process dimension), which reflects the degree of standardization of medical procedures or the degree to which CPWs have been used in disease management. The process dimension has not been measured by quantitative indicators in previous studies for any ready-made method (see Figure 2). Studies have generally focused on measuring costs and patient outcomes after CPW implementation and not on the actual implementation of CPWs, which can provide vital information to recognize the applicability of CPWs and direct continuous quality improvement. If the benefit dimensional effect size of disease management is small where a CPW has been implemented, the small process dimensional effect size suggests that the CPW has not been implemented well or that its contents require revision.²⁹

Identifying Key Issues and Beginning to Design CPWs Based on Limited Key Issues

Contrary to pursuing a large and all-inclusive CPW, we believe that CPW design must begin with the most critical issues that significantly affect the group or population-based cost benefits of specific disease management, gradually adding more steps and improving the design on the basis of the effectiveness of previous pathway design until the maximum public health benefit is achieved. There are three considerations. First, a CPW is a management implementation strategy involving rational people purchasing as much healthcare as possible with limited resources, and efficiency is especially critical in developing countries with low health budgets. The strategy for malaria is a good example. While individual or combined use of insecticidal nets, artemisinin-based combination therapy, indoor residual spraying of insecticide, and intermittent preventive treatment in pregnancy was far lower in most African countries than the target set by the World Health Assembly, malaria cases and deaths were reduced by 50% or more between 2000 and 2006 or 2007 in areas with high intervention coverage.³⁰ Second, no large and comprehensive CPW is applicable to any disease management plan in any conditions. Third, a complete and optimal CPW cannot be designed quickly due to the gap between evidence and practice. In summary, the refinement, implementation, and verification of key issues and the corresponding interventions should become the central idea of CPW design.

CPW design should primarily consider issues that significantly affect the cost-benefit of specific disease management, as shown by some existing valid evidence. The criticality and prioritization of specific issues must be set based on comprehensive consideration of the key public health goals targeted by a country’s healthcare spending²⁵ and research priority-setting,^31,32 the effect sizes of the measures, and the quality of evidence for measured effects. Issues that can significantly increase the benefits or control costs of disease management should be regarded as the key issues in CPW design. A greater effect warrants higher prioritization. Although we did not define a framework for determining the criticality and priority of problems in the design of a CPW in this paper, research priority-setting has yielded more in-depth discussions on this topic.^31–34 Therefore, CPW designers can review a list of prioritized key issues regarding disease management from prioritization research on a given health problem from health organizations’ documents, clinical guidelines, and systematic reviews at all levels and concentrate on the incorporation of issues with effective solutions in the CPW design with a report to state the process and reasons. In addition, prioritization during CPW design is still inevitably carried out in specific local economic and cultural contexts and is affected by value preferences. Therefore, the participation of different stakeholders in the discussion and the transparency of the CPW design process are very important.

Organizing the Core Issues of a CPW and Generating CPW Elements

As elements constituting a CPW, isolated key issues cannot form a disease management system, and together with other core issues of disease management, these issues must be organized and arranged according to the logic of diagnosis and treatment to form a core management plan to control the cost-benefit of a specific disease. We organized the core issues into three categories (see Figure 3):

Figure 3 (A) Generation of pathway elements and pathway design construction. Taking the clinical pathway for neonatal hypoglycaemia management with rooming-in at a hospital as a hypothetical example, “initial feed within 1 hour and screen plasma glucose 30 minutes after 1st feed” generates critical pathway elements because it is the proven key decision-making step in terms of whole costs and benefits for neonatal hypoglycaemia management with rooming-in. For low blood glucose concentrations that may occur in the first hours after birth or persist for up to several days, the duration of screening plasma glucose prior to each feed and its impact on the whole costs and benefits of neonatal hypoglycaemia management are uncertain. Thus, “Continue feeds q 2–3 hours and screen plasma glucose prior to each feed until 24 hours” generates controversial pathway elements. The other interventions are the standard operating procedures; they generate conventional pathway elements. Each pathway element is constructed by population (pathway node) and intervention (pathway node branch), and interlocking pathway elements ultimately form the pathway prototype. (B) Full-chain trajectory tracking and data collection after a patient enters the pathway. The design of pathway nodes is based on core characteristics and clinical outcomes of the population, and each pathway node is given a unique node code, ie, a, b, c … …. After a patient enters the pathway, the trajectory code is the chronological arrangement of the node code corresponding to the core features and the patient’s clinical outcomes after entering the pathway. For example, 20 newborns stop at node c, their trajectory codes are “ac”, 99 newborns stop at node h, and their trajectory codes are “acfh”. Using these trajectory codes, all patients’ diagnosis and treatment procedure data can be collected and integrated to visualize the pathway’s actual operating process.

1) Critical pathway elements: Key issues generate critical pathway elements, which is the key decision-making step shown by evidence to control the costs and benefits of disease pathway management. The ineffective measures shown by evidence should still be displayed in the CPW design if nonimplementation of these measures may significantly affect the cost-benefit of overall disease management unless nonimplementation of these measures has been widely recognized. Critical pathway elements are a valuable part of CPWs and are the focus of CPW evaluation.

2) Conventional pathway elements: The commonly recognized diagnostic and treatment activities or standard operating procedures generate conventional pathway elements. They cannot affect the cost-benefit of disease management but are essential core steps for logical and reasonable disease management.

3) Controversial pathway elements: Controversial core issues in disease diagnostic and treatment activities generate controversial pathway elements. CPW designers need to label and clarify these elements in CPW designs to allow people to identify their uncertainties in clinical decision-making. These elements do not have to be recommended because forced interventions do not have practical significance and may not even be implemented. Controversial pathway elements will not affect the cost-benefit of the overall CPW before they are resolved in the future. Clinical practice can be achieved according to the current operating procedures.

Constructing Pathway Elements and Visualizing the Actual Operating Process of a CPW

The ability to use existing data sets for visualization of the actual CPW operating process and calculation of the process dimensional effect size caused by CPW operation is very limited. A study used key process indicators (KPIs) to examine the extent to which patients with five conditions complied with national CPWs in seven public general hospitals of Pudong New Area in Shanghai in 2013, including community-acquired pneumonia, acute myocardial infarction, heart failure, caesarean section, and type 2 diabetes. The researcher tried to gather information on each step, but the steps of national CPWs are too detailed to be effectively analysed by traditional comparison. The analysis was focused on the more clinically meaningful steps (KPIs) of each CPW, eg, antibiotic use and length of stay for the pneumonia CPW, timely treatment and evaluation of left ventricular function, and the use of medicine (such as aspirin/clopidogrel, β-blockers, angiotensin-converting enzyme inhibitors/angiotensin-receptor blockers, and statins) and thrombolytic therapy in acute myocardial infarction. Compliance rates for the KPIs for CPWs ranged from 0 to 100%. These scattered data can hardly reflect patients’ compliance with CPWs and the applicability of CPWs. An approach has not been established to analyse process changes in health services generated by a CPW operation but also the relation between process changes and the cost-benefit of health services generated by a CPW operation.²¹ Creative primary data collection will be essential for achieving significant progress in this challenge.²⁶

Given that the actual CPW operating process is the basis of evaluating CPWs, we propose a structured CPW design to facilitate visualization, calculation, and quantification of the actual CPW operating process. Construction of pathway elements translates a CPW into a CPW design that can be studied. Each CPW element (core steps of health management) can be constructed by population (pathway node) and diagnostic and treatment measures (pathway node branches).³⁵ Interlocking pathway elements ultimately form an evidence-based management plan for specific diseases. With CPW management proceeding, the solutions of core issues (pathway elements) gradually categorize the population (pathway root node or primary node) into people with increasingly detailed characteristics (hierarchical pathway nodes) until a certain ending (pathway terminal node) appears (see Figure 3). Each patient’s diagnosis and treatment procedure can be described and recorded by her or his detailed characteristics generated by her or his experienced CPW elements that generate her or his trajectory data. Moreover, the actual CPW operating process can be visualized by chain tracing and integrating the trajectory data of patients entering CPW management. Comparing a CPW’s actual operating process with the original CPW design can locate CPW bottlenecks, which is the basis for evaluating and improving the clinical applicability of pathway design (see Figure 3). Thus, each patient’s trajectory data for diagnosis and treatment procedure should be the data used to study dynamic process changes in health services generated by a CPW’s operation. In fact, this is a mature computer and information technology. We believe that the fourth industrial revolution represented by computer and information technology and accelerated innovation in E-Health and digital health will probably change the workflow and quality of health services in the coming decades.³⁶

Calculating and Quantifying the Process Dimensional Effect Size of CPW Operation

A CPW integrates evidence-based interventions into usual care and can be regarded as a standard operating procedure. When a CPW is operating in a hospital, the closer the diagnosis and treatment procedure of patients in the real world is to the CPW design, the more standardized the medical procedure is, and the more applicable the CPW is. In contrast, the greater the lack of standardization of the medical procedure is, the less applicable the CPW is. We believe that the entropy degree can be a measure of the process dimensional effect size of disease management. As the basic theory for methodologies on studying any system operation, both entropy theory and general system theory have changed people’s minds concerning complex systems, which includes not only part and whole but also entropy and order.^37,38 These theories have been widely used in various disciplines and fields.

Based on the above fundamentals, we further propose a computing formula to calculate the entropy degree for the process dimensional effect size of CPW operation in the real world, defined as pathway entropy (PE), which facilitates quantification and comparison of the clinical applicability of a CPW design. The PE is greater than or equal to 0 and less than 1. In the target population, the more patients enter and complete their diagnosis and treatment procedures according to the CPW design, the more orderly the medical procedure becomes, the greater the CPW applicability is, and the closer PE is to 0; in contrast, the fewer patients who enter and complete the diagnosis and treatment procedures according to the CPW design, the more disorderly the medical procedure becomes, the smaller the path applicability is, and the closer PE is to 1.

Note:

indicates that the node is at the -th level of its longest CPW branch starting from the root node (level 1 node).

represents that the node is the -th node of the-th level of its longest CPW branch.

is the probability that the patients will stop at the node after they enter CPW management.

is the sum of the number of remaining nodes after the node.

For example, we can calculate the PE of the clinical pathway for neonatal hypoglycaemia management with rooming-in at a hospital (this is a hypothetical case, see Figure 3):

G=1

a: i=1

G=2

b: i =1

c: i =2

G=3

d: i =1

e: i =2

f: i =3

g: i =4

G=4

h: i =1

i: i =2

j: i =3

k: i =4

PE=1- (S₁₁+ S₁₂+ S₂₂+ S₁₃+ S₂₃+ S₃₃+ S₄₃+ S₁₄+ S₂₄+ S₃₄+ S₄₄) =1-0.9814=0.0186.

At present, the formula is designed only for CPWs of common acute diseases and may not be applicable to other complex clinical conditions. In addition, the premise of meaningful PE is to design a CPW based on key issues. We include this formula here to present our thoughts more concretely, and we plan to try to use it in improving algorithms of our clinical research application system connected to hospital information system, which requires a more in-depth and comprehensive thematic discussion in specific research.

Calculating and Quantifying the Benefit Dimensional Effect Size of CPW Operation

In terms of the benefit dimensional effect size, we first recommend a cost-benefit analysis for evaluation to quantify all outcomes into monetary terms,³⁹ which may guide a policy decision more directly. Using a cost-benefit analysis, costs can be compared directly to the benefits of any programme, including but not limited to health, and benefits from multiple outcomes can be pooled together. However, this ideal analysis method is not frequently used in health service research due to disagreements concerning monetizing health outcomes. Settling by concession in practice, a cost-effectiveness or cost-utility analysis may be sufficient to guide health decisions in contexts where the only important outcome is health, not other outcomes such as crime.^40–42 We can also determine the best CPW design according to the incremental effects and costs generated by CPW operation.^40–42 Here, we mainly elaborate on the thought process, and the specific methods have been elaborated in detail.^41,43,44 One point emphasized here is that costs involve not only the costs of “intervention strategies” but also the costs related to “implementation strategies”, which are needed to deploy and sustain the “intervention strategies”.⁴⁵

Gradually Including Key Issues in Pathway Design According to Prioritization and Adding More Steps Until the Maximum Public Health Benefit is Achieved

When we talk about the quality of health services, we must talk about the economy, including the management of medical resources and consideration of benefits, which is important because resources are scarce and therefore do not allow the highest-standard health services that people aspire to have. We thereby pursue the most efficient health services.⁴⁶ Chinese medical expenditure growth per patient discharged increased from 17.2% between 2005 and 2008 to 22.1% between 2010 and 2013.^47,48 The medical expenditure growth per outpatient visit increased from 15.5% between 2005 and 2008 to 21.7% between 2010 and 2013.⁴⁸ Both the proportion of out-of-pocket payments and the share of drug costs for healthcare expenditure have fallen, but overall hospital expenditure remains increasing.^22,49 These results imply that we must maximize the efficiency of resource utilization.

As a tool to control the efficiency of health services in disease management, an optimal CPW design can be determined according to the marginal benefit (incremental gain) and marginal cost (incremental cost) generated by the CPW’s operation based on the situation (see Figure 4).⁴⁶ For the management effect of preliminary CPW design starting from limited key issues, the marginal benefit (the slope of the total benefit curve) is greater than the marginal cost (the slope of the total cost curve), the increase in the total benefit is faster than the increase in the total cost, and the net benefit of CPW management is positive. As more key issues are included, the CPW design improves, the more developed it becomes, the more resources are invested, and the more other resources are occupied; the marginal benefit begins to decrease when the CPW design is near perfect, as the marginal cost will increase. When the marginal cost and marginal benefit are equal, the difference between the total benefit and total cost is largest, and the net benefit is largest where the maximum public health benefit can be achieved. The CPW design improves and becomes more complex with development. A higher marginal cost is required if the same unit of benefit is desired. When the marginal cost is greater than the marginal benefit, the total cost increases faster than the total benefit. The intersection point of the total benefit curve and the total cost curve indicates a CPW design with a zero net benefit.

Figure 4 Determination of the current optimal pathway design scheme according to the marginal benefit (incremental gain) and marginal cost (incremental cost). (A) The relationship between marginal benefit and marginal cost and their changes as the CPW design improves.20 (B) The relationship between the total benefit and total cost of CPW management and their changes as the CPW design improves.

Gradual Stepwise Evaluation of and Improvement in CPW Design by Stepped Wedge Design Research to Achieve the Maximum Public Health Benefit of CPW Management

Overall, the focus of CPW evaluation is the effect of critical CPW element execution on the overall CPW management efficiency when other CPW elements are fixed. CPW management efficiency is reflected both in the process dimensional and benefit dimensional effect sizes of disease management generated by a CPW’s operation. Each evaluation will lead to pathway design improvement, including adjustment of the previous critical pathway elements or the addition of new critical pathway elements. Each improvement requires further evaluation. External evaluation schemes of a stepped wedge design can realize step-by-step evaluation of and improvements in CPW design to determine the best-optimized CPW design where the net benefit is largest (see Figure 5), which has been used in some implementation science contexts.⁵⁰ In the stepped wedge study, every group (community) provides before-and-after observations. Every group (community) switched from control to being exposed to sequential interventions but not at the same time. The outcome data can be derived from repeat measurements in implementation stages of different CPW designs in the same cohort throughout the study. At the beginning of the study, all groups (communities) included should not receive the intervention. A group (community) should then be randomly selected at regular intervals to implement theCPW; the CPW should be adjusted and improved further according to the effect of the previous implementation. This process should be performed sequentially until all groups (communities) have received sequential interventions.⁵¹ Accordingly, the CPW design can be gradually developed, evaluated, and improved. The time effect of CPW implementation can be observed; thus, the heterogeneity of CPW implementation in different regions can be further analysed, and the effect of CPW implementation can be quantitatively evaluated. The implementation does not need to be fully conducted at once, nor does it require a large amount of human power or material and financial resources, avoiding resource waste due to an invalid CPW design. Under the current demand for healthcare, which always exceeds available resources, the design and evaluation of and improvement in a CPW are processes of continuous development until the related disease is well controlled and disease management reaches the maximum public health benefit, where the difference between the total benefit and total cost of pathway management is largest, or the total benefit of CPW management is largest and can be supported by the specific local economy.

Figure 5 The external evaluation research plan with stepwise evaluations of and improvements in pathway design.

Conclusion

Through CPW design, key knowledge regarding specific disease management can be extracted stepwise from many scattered and isolated systematically reviewed pieces of evidence. We focus on assessing the effect of solving key issues on the overall cost-benefit of CPW management if other steps remain unchanged. On this basis, we gradually add other key steps into the pathway, followed by regular assessment and improvement, to accumulate and refine effective key management steps of a specific disease to form a core management plan and achieve the maximum public health benefit for that specific disease.

Because of the multistage design and evaluation process, this CPW design and evaluation idea are suitable for managing common acute diseases but not long-term, highly heterogeneous, or rare diseases. This concept must also be tested in clinical practice.

Similar to rapidly developing specialties supported by robust science, the generalized theoretical development of CPW design and evaluation is essential for CPW implementation across diverse clinical settings.^52,53 This report aims to bring more attention to improving pathway design and evaluation and to encourage more in-depth theoretical discussions and methodological studies.