Identification and Prognostication of End-of-Life State Using a Japanese Guideline-Based Diagnostic Method: A Diagnostic Accuracy Study

Introduction

Accurately estimating life expectancy for patients at the end-of-life (EOL) is extremely difficult, especially for patients with non-cancer diseases. A previous longitudinal study indicated that the progression of disability in the last year of life did not follow a predictable pattern.¹ Over the last decade, several tools have been developed to predict the prognosis of patients near EOL; however, these tools lacked accuracy.² Therefore, many older people close to death cannot be given an accurate prognosis.³ Moreover, prognostic uncertainty can be a barrier to providing palliative care.⁴

In Japan, EOL-associated issues attracted nationwide attention following several well-publicized cases of ventilator removal in 2006.⁵ In 2007, the Japanese Ministry of Health, Labour and Welfare (JMHLW) developed guidelines to establish a process for determining EOL status and care policies; these guidelines were revised in 2018.^6,7 This JMHLW guidelines emphasized that patients should not be diagnosed as EOL by a single doctor especially when sufficient consent cannot be obtained from patients and their kins; rather, several doctors should be involved, and the diagnosis should always be supported by professionals from a range of fields (Text S1).⁷ Several EOL guidelines have since been published in Japan.^8,9 However, these guidelines do not set out any eligibility criteria for end-stage diseases, unlike the National Hospice Organization Medical Guidelines,¹⁰ and their usefulness has not been validated. The difficulty in EOL diagnosis for non-cancer diseases may arise from prognostic uncertainty, as shown by several Japanese studies.¹¹ This can lead to inappropriate therapy and use of invasive techniques (eg, tube feeding) without adequate evaluation of patients’ conditions.¹²

In our institutions, a rigorous diagnostic process based on Japanese EOL-related guidelines has been used for over a decade to diagnose EOL among older or critically ill patients. This process and the associated EOL diagnostic criteria built on existing guidelines by including a requirement to confirm the irreversibility of the patient’s condition. We conducted this study to retrospectively review cases diagnosed as EOL or non-EOL. The study objective was to evaluate the diagnostic accuracy of EOL diagnosis using our unique method.

Materials and Methods

This study was approved by the Institutional Review Board of Nanto Municipal Hospital (No. Shiminbyouin-96). In accordance with the guidelines of the Japanese Ministry of Education, Culture, Sports, Science and Technology,¹³ the requirement for obtaining participant informed consent was waived. This study involved no intervention for participants and followed the Standards for Reporting of Diagnostic Accuracy (STARD 2015) reporting guideline.¹⁴

Study Setting

This study involved collaboration among three medical facilities (Nanto Municipal Hospital, Nanto Family and Community Medical Center, or Toga Clinic) in Nanto, which is a super-aged city of about 50,000 people in Japan. To respond appropriately to the large number of deaths in this super-aged population, we required a rigorous method for making EOL judgments to shift the emphasis of medical care from acute to terminal care. Using Japanese EOL-related guidelines, we established a unique diagnostic method in 2007. This diagnostic method included the following three processes. (1) The attending physician judges that the patient is in the EOL state. (2) The attending medical team disclose the patient’s medical condition and present treatment options, including life-prolonging methods, to the patient and their family. (3) The patient’s condition and treatment plan are then discussed in an EOL case conference (EOL-CC) involving experts from different medical fields. Finally, a diagnosis is made in the EOL-CC (Figure 1). Most patients discussed at the EOL-CC were being followed up at one of our institutions.

Figure 1 Criteria and final process for end-of-life diagnosis. This figure indicates the final process of EOL diagnosis; process (3) in the Study Setting section. The whole medical history and condition for each patient were confirmed in the EOL-CC. The conference considered three decision options for each patient, EOL, non-EOL, and pending. Non-EOL means that the patient is deemed to NOT be in the EOL state, whereas pending means that EOL or non-EOL state cannot be determined and additional examinations or considerations are needed. Abbreviations: EOL, end-of-life; IVH, intravenous hyperalimentation; JMHLW, Japanese Ministry of Health, Labour and Welfare; PEG, percutaneous endoscopic gastrostomy.

Study Design and Participants

This retrospective longitudinal observational study was conducted to assess diagnostic accuracy in determining whether a critically ill patient was in an EOL state. Participants were consecutive patients presented at an EOL-CC for whom a decision had been made between January 1, 2010 and September 30, 2017. The beginning of the study target period was when the EOL-CC database records started. The end of the target period was set to September 2017 because all patients’ outcomes had to be determined at the start of this observation study. Each patient’s attending physician determined the necessity for discussion of their case at the EOL-CC. Participants were retrospectively extracted from the database that recorded details of the EOL-CC. This database contained a range of information, including patient ID, age, sex, clinical diagnosis, EOL-CC discussions, and EOL-CC decisions. If a patient had never received an EOL diagnosis (non-EOL), the reason was noted. Participants were excluded from the diagnostic accuracy analysis if an EOL-CC decision was pending. For cases that had been discussed repeatedly, only the initial EOL-CC decision was included.

Diagnostic Criteria for EOL State

At each EOL-CC, specific conditions had to be fulfilled for a target patient to be diagnosed as EOL. These conditions were: (1) the patient had received the best medical care for recovery and no curable interventions remained; (2) even if treatment was continued, the patient’s prognosis was poor (irreversible condition) and death was likely to be imminent (estimated survival <6 months); and (3) several doctors, nurses, co-medical staff, the patient, and their family recognized (1) and (2), which were rigorously confirmed based on medical findings and in-depth discussions (Figure 1). In particular, specialists from different fields provided prognoses for patients with malignancy (Table S1). These EOL criteria followed the Japanese EOL-related guidelines.^6–9 Notably, some of these guidelines suggest that the estimated prognosis should be <6 months if a patient is considered in an EOL state. A conclusion that a patient was in an EOL state required unanimous agreement at the EOL-CC. The type of trajectory to EOL (EOL type) in each patient was determined according to presentations by the attending physicians and patients’ medical records, and categorized as terminal illness (patients with advanced cancer), organ failure, frailty (patients with advanced dementia), or unclassifiable (unable to be classified into the first three categories), with reference to a previous study.¹⁵

Data Collection and Follow-Up

Data on baseline characteristics (symptoms, histories, comorbidities, laboratory data, medications, physical and cognitive functions, and others) were obtained from the EOL-CC database and patients’ medical records. We analyzed retrospective scores for the Charlson Comorbidity Index (CCI), the Palliative Performance Scale (PPS), and the Advanced Dementia Prognostic Tool (ADEPT) as comparative controls for diagnostic accuracy.^16–18 The clinical courses and outcomes for all participants were retrospectively reviewed using their medical records.

Outcomes

The primary study outcome was mortality within 6 months and the secondary outcome was diagnostic odds ratio (DOR). Overall survival (OS) was defined as days between the initial EOL-CC decision (EOL, non-EOL, or pending) and death from any cause. True-EOL was defined as OS <6 months, whereas OS ≥6 months was defined as other-than-EOL. This definition followed the concept of EOL in the Japanese guidelines. If a patient had been transferred to another institution outside our local medical area, follow-up was censored at the day of the last medical record made. If a follow-up period was <6 months during survival, the participant was excluded from analysis and recorded as having missing data.

Statistical Analysis

To evaluate the diagnostic accuracy of our method of determining EOL (as an index test), we used OS as the reference standard. To obtain an index value for the diagnostic accuracy of our unique method against the primary outcome (true-EOL or other-than-EOL), we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and DOR.¹⁹ Sensitivity and specificity of our EOL diagnosis were compared with those of other prognostic tools (CCI, PPS, and ADEPT) using the McNemar chi-square test.²⁰ OS was analyzed by survival analysis using the Kaplan–Meier method and statistically compared using a Log rank test. Logistic regression was used to evaluate the effect of potential confounding factors on death within 6 months. Factors were selected for multivariate analysis if the P-value in the univariate analysis was <0.20. In addition, we used the forward-backward stepwise selection method for the logistic regression analysis to minimize the independent variables adapted to the model.

For two-group comparisons, normally and non-normally distributed numerical variables were compared using t-tests and Mann–Whitney U-tests, respectively. To compare more than two groups, post hoc multiple comparisons were made using Tukey’s honestly significant difference test after confirming the heterogeneity of means using one-way analysis of variance. For categorical data, the proportions in each group were compared using Fisher’s exact test. These statistical analyses were performed using XLSTAT version 2021.4.1 (Addinsoft Inc. Paris, France, https://www.xlstat.com/ja/) and EZR version 1.54 (Saitama Medical Center, Jichi Medical University, Saitama, Japan). EZR is a graphical user interface for R (The R Foundation for Statistical Computing, Vienna, Austria, version 4.0.3);²¹ it is a modified version of R commander (version 2.7–1) designed to provide additional statistical functions frequently used in biostatistics. A two-sided P-value <0.05 was considered statistically significant.

Results

Study Population and Baseline Characteristics

From January 1, 2010, to September 30, 2017, a total of 330 cases were discussed at in an EOL-CC. We identified 319 participants who had received a first decision by the EOL-CC. The EOL-CC evaluated 275 (86%) patients as EOL, 24 (8%) as non-EOL, and 20 (6%) as pending. Of these 319 participants, 315 were eligible for the retrospective analysis (Figure 2). The baseline characteristics and EOL types of the study population are shown in Table 1. The median age of all participants was 89 (range 54–107) years. The terminal illness group was significantly younger than the other groups. The frailty group was characterized by severe deterioration in functioning (eg, physical, cognitive, and eating abilities). Most patients (85%) had eating problems, mainly because of reduced oral intake.

Table 1 Baseline Characteristics in the Retrospective Analysis

Figure 2 Flow diagram of participants through this study. This figure is illustrated based on a diagram of the STARD 2015 reporting guideline. Abbreviations: CC, case conference; EOL, end-of-life; OS, overall survival.

Associations Between EOL-CC Diagnosis and Primary Outcome

The results of the survival analysis using the Kaplan–Meier method are shown in Figure 3. Compared with an EOL diagnosis, a non-EOL diagnosis was associated with a higher survival rate at 6 months after the EOL-CC (43.5% vs 6.9%; log-rank P < 0.001). However, a small number of EOL patients were alive more than 1 year later (maximum of 6.3 years) (Figure 3A). In EOL patients, EOL type was associated with a significant difference in survival at 6 months after the EOL-CC (terminal illness 2.2% [95% confidence interval 0.65–5.8%], frailty 16.1% [9.3–24.5%], organ failure 2.9% [0.2–12.7%], unclassifiable cases 6.7% [0.4–26.0%]; log-rank P = 0.019) (Figure 3B).

Figure 3 Survival analysis using the Kaplan–Meier method (A) All patients eligible for retrospective analysis (N = 315) were included in this survival analysis. (B) Patients given an EOL diagnosis (N = 274) were included. Abbreviations: CC, case conference; CI, confidence interval; EOL, end-of-life.

DOR and Other Index Values of Diagnostic Accuracy

Of the included participants, 297 were eligible for the diagnostic accuracy analysis after excluding patients with a decision pending (women 179, median age 89 [range 54–107] years). Participants’ baseline characteristics were similar to those of the survival analysis population (Table S2). The cross-tabulation of EOL-CC decisions (EOL or non-EOL) by the reference standard results (true-EOL or other-than-EOL) is shown in Table 2. The index test and reference standard results by EOL type distribution are shown in Table 3. The frailty group had the lowest proportion of EOL diagnoses (87/98, 89%) and the highest false positive rate (other-than-EOL among the EOL patients: 14/87, 16%) of all EOL types. Estimates of diagnostic accuracy and the precision of an EOL-CC diagnosis are shown in Table 4. Our EOL diagnosis had high sensitivity (0.95 [95% confidence interval 0.92–0.97]) and PPV (0.93 [0.90–0.96]) but low specificity (0.35 [0.20–0.53]). Among the various prognostic prediction tools, the EOL-CC also had high NPV (0.44 [0.23–0.64]), low NLR (0.14 [0.07–0.29]), and high DOR (10.32, [4.08–26.13]). The sensitivity of EOL diagnosis by EOL-CC was significantly higher than any cut-off values for other prognostic tools, whereas the specificity had significant results in some tools (Table S3). However, with a reference to the 95% confidence intervals, the DOR for the EOL-CC decision was significantly higher than some cut-off values for the other tools (PPS ≤20, PPS ≤30, ADEPT ≥15, and ADEPT ≥16) (Table 4). In the subgroup analysis by EOL type, the PPV and NPV in the terminal illness and organ failure groups were extremely high, but those of the frailty group were relatively low (Table 4).

Table 2 Cross-Tabulation of Index Test Results by Reference Standard Results

Table 3 Summary of Index Test Results and Reference Standard Results by End-of-Life Type

Table 4 Statistics for Diagnostic Accuracy

In the logistic regression analysis of predictors of OS more than 6 months, the prognostic factors included a combination of performance on the functional measurement tools to indicate elements of condition severity. Age, CCI, terminal illness, and inability to communicate were significant prognostic factors in the univariate analysis. However, the multivariate analysis showed that only CCI (OR 0.66 [95% confidence interval 0.52–0.82], P < 0.001) and a severe eating problem (oral intake <500 kcal/day) (OR 0.36 [0.15–0.89], P = 0.02) were significant prognostic factors (Table S4).

Discussion

We established a process to diagnose EOL following Japanese EOL-related guidelines,^6–9 because there was a lack of adequate scoring systems and diagnostic criteria to accurately determine EOL.^2,3 Our EOL diagnosis showed high sensitivity, low NLR, and high DOR, but low specificity. Our results confirmed the accuracy of EOL diagnosis using Japanese guidelines. In actual clinical practice, when attending physicians suspect a patient is at the EOL, definitive diagnosis by this process could support subsequent medical treatments and decision making without hesitation. This is because the EOL-CC guarantees that clinical practice was consistent with the Japanese guidelines. Even if the EOL-CC could not give an EOL diagnosis, it could indicate potential next steps for a patient. This diagnosis method required no additional resources, and was associated with a marked decrease in patients in our hospital beginning tube feeding to prolong their prognosis (data not shown).

As comparative controls for evaluating the degree of diagnostic accuracy, we used the CCI, PPS, and ADEPT. These are common scoring systems used to predict the prognosis of patients facing EOL, and their validity has previously been discussed.^2,22,23 The sensitivity of the EOL-CC decision was significantly higher than any cut-off values in those tools, whereas the specificity had significant results in some tools (Table S3). In addition, with reference to the 95% confidence intervals, the DOR for the EOL-CC decision was higher than some cut-off values in other prognostic tools (Table 4). Most of the other cut-off values (CCI, PPS, or ADEPT with no significant difference in DOR) had inadequate sensitivity to use for screening. Therefore, our method appeared to be reliable for EOL diagnosis. The Supportive and Palliative Care Indicators Tool (SPICT) is also a useful tool for detecting EOL patients.²⁴ However, in our study, patients who met SPICT criteria and those diagnosed as EOL by our diagnostic method were consistent, because our patients had severe conditions. Therefore, the SPICT could not be used as a comparative control in this study.

Patients presented at an EOL-CC were bedridden (unable to move independently), had severe dementia (unable to communicate), and reduced oral intake (unable to eat sufficiently), especially those in the frailty group (Table 1). Therefore, these three elements may prompt the attending physician to diagnose EOL. However, fulfillment of these three elements is not a prognostic factor (Table S3). This may be because an EOL diagnosis by the EOL-CC depended on the reversibility of the patient’s condition as well as these three elements (Figure 1). Hui et al identified a key defining feature of EOL as life-limiting disease with irreversible decline.²⁵ However, such irreversibility was not considered and incorporated in previous research.^2,16–18 Therefore, the novelty of our study is the use of the criterion of irreversibility, which may have improved diagnostic accuracy. Several Japanese EOL-related guidelines described the need to consider irreversibility before diagnosing EOL.^8,9,26 However, the criteria for irreversibility remain unclear and require clarification. Severe eating problems (oral intake <500 kcal/day), which is one of the three elements of EOL diagnosis, had the greatest effect on participants’ prognosis in our study (OR 0.36 [95% confidence interval 0.15–0.89], P = 0.02). Similar findings have been reported by previous studies.^2,3,27,28 Accurate evaluation and appropriate intervention for eating problems is a challenging issue in this field.²⁹ However, addressing this issue would facilitate more accurate evaluation of irreversibility.

In the frailty group, the survival rate at 6 months after the EOL-CC was significantly higher than that in the terminal illness group (16.1% vs 2.2%, log-rank P < 0.001). Moreover, approximately 10% of them were alive more than 1 year later (Figure 3B). This may indicate that accurate prognosis prediction of non-cancer patients with severe conditions is difficult, although our study population was small and lacked power to provide clear evidence. Recent studies have shown similar findings; some improvement of prognostication has been achieved in cancer fields,^30,31 but not in non-cancer fields.^2,3,32,33

Study Limitations

This study had some limitations. First, the cause of death was unknown for many participants because death certificate information could not be confirmed. It is possible that some patients died from causes other than the pathological conditions discussed at the EOL-CC. Second, some baseline data were missing because this was a retrospective study. The CCI, PPS, and ADEPT were scored retrospectively; therefore, their scores might not have been completely accurate. Third, individual differences in interventions such as nursing care, social resources, and medical treatments might have affected prognosis, but these factors were not investigated in this study. Patients with an estimated poor prognosis are less likely to receive burdensome interventions.³⁴ Similarly, some patients diagnosed as EOL may not be given the opportunity to recover from severe conditions. To prevent these problems, we designed a rigorous process by which to determine EOL based on several EOL-related guidelines. Therefore, the risk for misdiagnosis was minimized. In contrast, too rigorous process for EOL diagnosis may take away the opportunity to receive appropriate palliative care earlier. We recognize these points as a major problem which should be improved by future studies. Fourth, the high probability and high DOR results may reflect attending physicians’ selection of patients to present at the EOL-CC. It is possible that physicians avoided presenting some patients owing to advance directives or advance care planning (ACP). It may have been assumed that EOL-CC decisions interfere with treatment strategies already determined by ACP, even though the usefulness of ACP at the EOL has not been proven.^35,36 Fifth, because of physician unawareness, there might be significant under-diagnosis among all patients at true EOL. The actual number of candidates who should have been referred to the EOL-CC was unclear. However, the patient who could not undergo completely our diagnostic process had no index test results. Therefore, sensitivity, specificity, and DOR could not be calculated among them. Because EOL conclusion could guarantee attending physicians’ policy, we believe that most candidates were referred to the EOL-CC.

Conclusion

The diagnostic process we developed following Japanese EOL-related guidelines was associated with accurate identification and prognostication of EOL. However, the study population was limited to patients with severe conditions. Therefore, it is necessary to collect more data from clinical practice and attempt to improve the accuracy of diagnosis.