Development of the ProPal-COPD tool to identify patients with COPD for proactive palliative care

Introduction

COPD is a progressive life-threatening lung disease.¹ During the disease course, the need for disease-oriented care remains unchanged while the need for proactive palliative care increases for patients with COPD.² Proactive palliative care, which includes advance care planning (ACP) conversations, is intended to improve the quality of life and quality of care.^3,4 However, for patients with COPD, such care is not yet common.^2,5 An important barrier is the identification of patients with COPD who might benefit from proactive palliative care.²

Identification for proactive palliative care is complicated by the unpredictable disease course of COPD.⁶ Stable phases in the disease course are interrupted by acute exacerbations; any one of which may be fatal.⁶ Although, general tools to identify patients for palliative care exist, they are not specifically intended or validated for the identification of patients with COPD.^7,8 In addition, models of survival in stable COPD have been developed to be able to adjust and optimize care.^9,10 Unfortunately, these population models are of limited value to predict survival for individual patients.¹¹ This prognostic difficulty has led to a discussion among clinicians whether or not a transition point for the initiation of proactive palliative care exists.^12–15 Some clinicians argue that more specific criteria of end-stage COPD need to be explored.¹³ Others argue that, since such criteria may not exist, searching further may lead to prognostic paralysis.^14,15 Instead, they promote early integration of palliative care according to needs.¹⁵ The focus in this discussion has been on whether or not we can accurately predict mortality instead of the intended objective, identifying patients with COPD in need of proactive palliative care. Since palliative care needs increase during the disease course of COPD,² the prediction of mortality can be used as a proxy for the need of palliative care. To ensure minimal miss out of patients in need of such care, this tool should have a sensitivity near 100 with specificity as high as possible.

Hospitalizations for an acute exacerbation COPD (AECOPD) are associated with significant mortality and therefore create an opportunity to identify patients with poor prognosis in need of palliative care.¹⁶ Recently, potential relevant variables have been identified to predict posthospital mortality in patients hospitalized for an AECOPD.^16,17 This is the first prospective study to incorporate these variables with the objective to develop a multivariable prediction tool (the ProPal-COPD tool) to identify patients with COPD in need of proactive palliative care. Our objective was to assess the discriminating power of a set of indicators for proactive palliative care in predicting death for any cause within 1 year. This outcome was used as a proxy for the potential need of proactive palliative care in patients with COPD.

Patients and methods

Source of data

Data for prognostic accuracy testing in patients hospitalized for an AECOPD were obtained from the PROLONG study.¹⁸ This prospective pragmatic cluster controlled trial was performed in six general hospitals in the Netherlands; of those, three were in the intervention condition and three in the control condition. All recruited patients hospitalized for an AECOPD received usual care, while only patients in the intervention condition with poor prognosis additionally received proactive palliative care. In this trial, poor prognosis was defined as meeting two or more predictors of a set of 11 predictors of poor prognosis. For the prognostic accuracy study, described in this study, we only used data of those patients recruited from the three hospitals in the control condition.

Participants

We screened for potential participants in three general hospitals in the Netherlands from January 14, 2014, to December 29, 2014. All patients with the clinical diagnosis of COPD and hospitalized for an AECOPD who were 18 years or older were eligible. An AECOPD was defined as “an acute worsening of the patient’s condition from the stable state, which is sustained and may warrant the patient to seek additional treatment.”¹⁹ Since the severity of airway obstruction and spectrum of pathogens in pneumonic and non-pneumonic exacerbations are similar,²⁰ patients with COPD who had been hospitalized for an acute pneumonia accompanied by increased sputum and dyspnea were also included. Patients who did not speak Dutch or had a severe cognitive disorder were excluded from participation. All patients received usual care for an AECOPD and the period thereafter.

The study was approved by the Medical Ethics Committee (METC) of the Radboud University Medical Center, Nijmegen (METC protocol number 2012/260). Thereafter, we requested and received research approval from all three participating hospitals. All participants gave written informed consent before enrollment.

Outcome

The outcome measure was death for any cause within 1 year. This outcome measure was used as a proxy for the potential need of proactive palliative care. To ensure minimal miss out of patients in need of such care, we were looking for a tool with a sensitivity near 100% and a specificity as high as possible.

Predictors

All 11 predictors are presented in Table 1. Each of them is a categorical variable indicating whether or not the participant meets the specific criteria for this predictor. References with the rationale to include the predictor in the multivariable prediction model are also presented in Table 1. Patients filled out demographics, the Medical Research Council dyspnea questionnaire (MRC dyspnea) and the Clinical COPD Questionnaire (CCQ) on the sixth day of hospital stay or in case the patient was discharged earlier on the day before leaving the hospital. The MRC dyspnea is a questionnaire used for grading the effect of breathlessness on daily activities.²⁷ The patient’s dyspnea score is rated from 1 to 5, and higher scores represent a more severe dyspnea. The patient was asked to rate the dyspnea severity as it was 2 weeks before hospital admission. The CCQ day version was used to measure the health status of patients.²⁸ The CCQ consists of 10 questions covering three domains (symptoms, functional status and mental state). The maximum total CCQ score is 6 and higher scores represent a worse health status. The treating pulmonologist thereafter filled out a checklist consisting of the remaining nine predictors. A year after the last patient was included, the medical files of all participants were explored. Variables looked for were as follows: if applicable, date of death and missing scores of predictors on the checklist. In case of uncertainty about death and/or date of death, we contacted the general practitioner (GP) of the patient for information.

Table 1 Set of 11 predictors of poor prognosis for patients hospitalized for an AECOPD and the references with the rational to include them in the prediction model Notes: PaCO2, arterial partial pressure of carbon dioxide; PaO2, arterial partial pressure of oxygen. Abbreviations: AECOPD, acute exacerbation COPD; BMI, body mass index; CCQ, Clinical COPD Questionnaire; CHF, chronic heart failure; FEV1, forced expiratory volume in 1 second; GFR, glomerular filtration rate; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation.

Sample size

In this prospective study, we included all eligible patients willing to participate during the year 2014 to maximize the power and generalizability of the results.

Missing data

Missing data were handled using complete case analysis. Participant data were excluded from the analysis if a participant had missing information on any of the potential predictors. Possible reasons for missing data were not or incomplete filled-out questionnaire by the patient and/or an incomplete filled out checklist by the pulmonologist.

Statistical analyses

Baseline characteristics were described using frequencies for noncontinuous data, mean with SD for continuous variables normally distributed and median with interquartile range (IQR) for continuous variables not normally distributed. The analysis plan consisted of two steps. The first step had a descriptive nature and was aimed at exploring the discriminating power of individual predictors in predicting death for any cause within 1 year. For all 11 predictors, the area under the curve (AUC), sensitivity and specificity were calculated. The goal of the second step was to build an optimal prediction model and to assess its predictive performance using internal validation. The model was built using a least absolute shrinkage and selection operator (Lasso) logistic regression model, with 20-fold cross-validation for optimal shrinkage.^29,30 By using the Lasso method, the risk of over-fitting was reduced, and at the same time a relatively simple prediction model was obtained. Because the performance of our optimal prediction model was validated in the same population used to construct it, its predictive performance was overestimated (optimism). To estimate this optimism, the procedure proposed by Steyerberg³⁰ was followed. In short, using bootstrapping, 500 new populations were created, and a prediction model was built for each population in a completely analogous way. The predictive performance of the model, as expressed by the AUC, in both the bootstrapped population and original population was calculated. The mean difference between these two AUC values is an estimate of the optimism of our prediction model.

Risk groups

No risk groups were created.

Results

Participants

The participant flow diagram is shown in Figure 1. Of all 174 participants, 155 participants were included in the analyses. The characteristics of the study populations are presented in Table 2.

Figure 1 Participant flow diagram. Abbreviation: Pt, patient.

Table 2 Characteristics of the study population Notes: Study population (N=155). *Because of missing values, some numbers do not add up to 100%. •GOLD stage according to the last known data in medical file; inclusion in the study was based on the judgment by pulmonologist. Abbreviations: AECOPD, acute exacerbation COPD; BMI, body mass index; CCQ, Clinical COPD Questionnaire; FEV1, forced expiratory volume in 1 second; GOLD, Global Initiative for Chronic Obstructive Lung Disease; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation; prev hosp, previous hospitalization; SQ, surprise question.

Model development

The numbers of participants who met the criteria of each predictor are presented in Table 2. In the study population, 19.4% of the participants (30 of 155) died within 1 year.

The discriminating power of individual predictors in predicting death within 1 year is presented in Table 3.

Table 3 The discriminating power of individual predictors in predicting death within 1 year Note: Study population (N=155). Abbreviations: AECOPD, acute exacerbation COPD; AUC, area under the curve; BMI, body mass index; CCQ, Clinical COPD Questionnaire; FEV1, forced expiratory volume in 1 second; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation; Se, sensitivity; Sp, specificity; SQ, surprise question.

Model specification and performance

The optimal prediction model, build using the Lasso method, and its predictive performance are presented in Table 4. This logistic regression model includes an intercept and the weighted sum of the value of each predictor. The weights are the regression coefficients (B). The value of each predictor is 1 when present and 0 when absent.

Table 4 Optimal prediction model and its predictive performance of death within 1 year Notes: Study population (N=155). B, weight in the model. /, not incorporated in the optimal prediction model. Abbreviations: AECOPD, acute exacerbation COPD; AUC, area under the curve; BMI, body mass index; CCQ, Clinical COPD Questionnaire; CI, confidence interval; FEV1, forced expiratory volume in 1 second; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation; prev hosp, previous hospitalization; SQ, surprise question.

A patient can be considered as in need of proactive palliative care when the total sum of the logistic regression models exceeds a certain cutoff point of the model. The choice of this cutoff is essentially a trade-off between sensitivity and specificity. A popular choice is to choose the cutoff that considers sensitivity and specificity to be of equal importance, so maximizing the sum of sensitivity and specificity. In the setting of this study, a cutoff that prioritizes sensitivity above specificity does make sense. Therefore, we also present the cutoff that maximizes the sum of specificity and 1.5 times the sensitivity. Possible cutoffs for both choices are presented in Table 5.

Table 5 Possible cutoffs considering different trade-offs between Se and Sp of the optimal prediction model Abbreviations: Se, sensitivity; Sp, specificity.

The receiver operating characteristic (ROC) curve for the prediction of death within 1 year with the optimal cutoffs considering different trade-offs between sensitivity and specificity is shown in Figure 2.

Figure 2 The ROC curve for the prediction of death within 1 year with the optimal cutoffs considering different trade-offs between Se and Sp. Abbreviations: ROC, receiver operating characteristic; Se, sensitivity; Sp, specificity.

Discussion

This study shows that we were able to develop a promising multivariable prediction tool to identify patients with COPD in need of proactive palliative care, the ProPal-COPD tool. Since palliative care needs increase during the disease course,² and hospitalizations for AECOPD are associated with significant mortality risk,¹⁶ these hospitalizations create an opportunity for identification. This is why the prediction of death was used as a proxy for the identification of patients in need of palliative care. In our study population, 19.4% of participants died within 1 year. This number is in line with the 1-year mortality rate in other studies investigating patients hospitalized for AECOPD.^31,32 Our optimal prediction model of death within 1 year had good discriminating power; the true AUC was 0.82 (95% CI 0.81–0.82), and the estimated optimism of 0.05 indicated good internal validity. To ensure minimal miss out of patients in need of palliative care, we proposed a cutoff in the prediction model that prioritized sensitivity above specificity. This optimal cutoff was −1.36, with a sensitivity of 0.90 and a specificity of 0.73. This implies that, when used, only 10% of patients in need of palliative care would be missed out and that of all patients identified for such care 84% would die within 1 year.

The ProPal-COPD tool is the first which is using specific predictors of mortality during AECOPD requiring hospitalizations. This tool relies on seven predictors: the surprise question (a general predictor), five markers of COPD severity and the presence of specific comorbidities proven to independently predict posthospital mortality in AECOPD.²⁴ Through the surprise question, the tool incorporates clinical judgment of severity. The surprise question has shown to be a proven predictor of mortality in patients with advanced chronic diseases³³ and is recommended to be used in combination with other disease-specific predictors as screening tool for proactive palliative care.²³ Those disease-specific predictors in the ProPal-COPD tool are five markers of COPD severity being the CCQ (health status), MRC dyspnea, FEV₁% of predicted value, body mass index (BMI) and previous hospitalizations for an AECOPD. The presence of one of the following comorbidities, non-curable malignancy, cor pulmonale, chronic heart failure (CHF) or diabetes mellitus with neuropathy or renal failure, is also incorporated in our tool. These comorbidities, except non-curable malignancy, are conditions liable to acute decompensation especially during AECOPD, which may explain their association with an increased mortality risk.¹⁶

Most predictors of the ProPal-COPD tool are variables already used in clinical practice and do not need extra measurement.³⁴ Only the surprise question, the CCQ and MRC dyspnea, which are short easy-to-use questionnaires, are often not documented and need to be specifically asked for.³⁴ Besides, each predictor in our tool has an own weight (B). Those weights take into account the correlation with other predictors of the tool and therefore should not be interpreted as the individual importance of each predictor in the tool. Moreover, the Lasso method used for model development is looking for the most efficient model.³⁰ In the trade-off between efficiency and effectiveness, four predictors (hypoxemia or hypercapnia, noninvasive ventilation [NIV], professional home care and age) were excluded from our optimal prediction model. This does not mean that those predictors were not predictive of death within 1 year. If we look at the predictor NIV, for example, it was noticeable that only 11 patients met the criteria of this predictor. Besides, NIV as an individual predictor had high sensitivity (0.93) and low specificity (0.07), meaning that almost all patients who were treated with NIV died within 1 year but patients who were not treated with NIV also had a considerable risk of dying. The exclusion of the predictor NIV can therefore be explained by the limited number of patients involved and the high correlation with predictors incorporated in our optimal prediction model. The other three predictors, hypoxemia or hypercapnia, professional home care and age, were merely excluded because of correlation.

The ProPal-COPD tool shows promise for the identification of patients with COPD in need of proactive palliative care. The ROC analysis suggests that it has good prognostic performance with a prognostic capacity superior to the CODEX index (comorbidity, age, obstruction, dyspnea and previous exacerbations).³⁵ The CODEX index has an AUC of 0.68 in predicting death within 1 year and is the only other multivariable model developed to evaluate posthospital prognosis in patients with an AECOPD.³⁵ However, the main objective of both tools is different. Our tool was developed to identify patients in need of proactive palliative care, whereas the CODEX was developed to predict mortality and/or readmissions. In addition, there are several substantive differences. First, our tool incorporates disease-specific predictors of mortality in AECOPD requiring hospitalization, whereas the CODEX index was developed using disease-specific predictors of mortality in stable COPD. Second, our tool contains specific comorbidities proven to independently predict posthospital mortality in AECOPD, whereas the evidence of the Charlson index (comorbidity burden) used in the CODEX index in predicting posthospital mortality in AECOPD is less consistent.¹⁶ Finally, each predictor in our tool has an own weight (B) in contrast to the predictors in the CODEX index. Those weights do not represent the individual importance of each predictor since they take into account the correlation with other predictors in the tool. Still, the own contribution of each predictor is better represented in our tool. It is possible that the abovementioned substantive differences between the ProPal tool and the CODEX index have contributed to the better prognostic performance of our tool.

Our study has several limitations that should be acknowledged. First, predicting posthospital mortality for patients with COPD is complex and this is reflected by the number of potential predictors incorporated in the original set of predictors. We only used already known predictors in this set and it is uncertain if all possible predictors were accounted for. In addition, it is possible that certain predictors were underrepresented in the set of potential predictors. Anxiety and depression, for example, were represented as a subscale of the predictor CCQ. Second, to facilitate implementation in clinical practice, we used dichotomized variables. Each potential predictor was given a clinical cutoff value based on literature. Such dichotomization of continuous predictors may lead to loss of information and reduction in power.³⁶ Third, the use of the ProPal-COPD tool may require collection of data that are not always captured routinely (surprise question, CCQ, MRC dyspnea). Fourth, the ProPal-COPD tool was developed in the Netherlands. International differences in hospital admission policy for patients with an AECOPD may affect the usefulness in other countries. Fifth, it is unknown whether the ProPal-COPD tool is also applicable in academic hospitals since it was developed using a population of patients in general hospitals. Finally, the ProPal-COPD tool was developed in a single population without validation in a different one. The external validity of our findings is therefore unknown and needs to be addressed in further research. Hence, other participant data need to be collected in a new prospective study performed in general and/or academic hospitals to evaluate the performance of the ProPal-COPD tool.

Conclusion

The ProPal-COPD tool is a promising tool to identify patients with COPD for proactive palliative care with good discriminating power (AUC =0.82). It relies on seven predictors: the surprise question (a general predictor), five markers of COPD severity and the presence of specific comorbidities. The prediction of death within 1 year was used as a proxy for the potential need of proactive palliative care. To ensure minimal miss out of patients in need of proactive palliative care, we proposed a cutoff in the prediction model that prioritized sensitivity over specificity. This optimal cutoff has a sensitivity of 0.90 and a specificity of 0.73. Each predictor in the tool has an own weight and the patient can be considered in need of proactive palliative care only when the total sum of the model exceeds the specific cutoff point. This prognostic equation can be easily integrated in the electronic patient record (EPR) system. However, before clinical use of the ProPal-COPD tool, the external validity should be addressed in further research.

Acknowledgments

We thank the patients who participated in this study. We thank the clinical professionals and the administrative staff of the participating hospitals for their commitment and cooperation. We thank Anne Ebenau en Klasminda Verhagen for her assistance with data entry and Aisha de Vries and Charlotte Gradussen for their assistance with the medical file study of participating patients. This study was funded by the Netherlands Organization for Health Research and Development-ZonMw, The Hague. Project number: 80-82100-98-080. The funding source did not play any role in planning and implementing this study, in interpreting its results or in writing the paper.

Author contributions

All authors contributed toward data analysis, drafting and critically revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Vestbo J, Hurd SS, Agusti AG, et al. Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease: GOLD executive summary. Am J Respir Crit Care Med. 2013;187(4):347–365.
2.		Vermylen JH, Szmuilowicz E, Kalhan R. Palliative care in COPD: an unmet area for quality improvement. Int J Chron Obstruct Pulmon Dis. 2015;10:1543–1551.
3.		World Health Organization (WHO) [webpage on the Internet]. WHO Definition of Palliative Care; 2012. Available from: http://www.who.int/cancer/palliative/definition/en/. Accessed December 15, 2016.
4.		Patel K, Janssen DJ, Curtis JR. Advance care planning in COPD. Respirology. 2012;17(1):72–78.
5.		Beernaert K, Cohen J, Deliens L, et al. Referral to palliative care in COPD and other chronic diseases: a population-based study. Respir Med. 2013;107(11):1731–1739.
6.		Giacomini M, DeJean D, Simeonov D, Smith A. Experiences of living and dying with COPD: a systematic review and synthesis of the qualitative empirical literature. Ont Health Technol Assess Ser. 2012;12(13):1–47.
7.		Maas EA, Murray SA, Engels Y, Campbell C. What tools are available to identify patients with palliative care needs in primary care: a systematic literature review and survey of European practice. BMJ Support Palliat Care. 2013;3(4):444–451.
8.		Dalgaard KM, Bergenholtz H, Nielsen ME, Timm H. Early integration of palliative care in hospitals: a systematic review on methods, barriers, and outcome. Palliat Support Care. 2014;12(6):495–513.
9.		Dijk WD, Bemt L, Haak-Rongen S, et al. Multidimensional prognostic indices for use in COPD patient care. A systematic review. Respir Res. 2011;12:151.
10.		Marin JM, Alfageme I, Almagro P, et al. Multicomponent indices to predict survival in COPD: the COCOMICS study. Eur Respir J. 2013;42(2):323–332.
11.		Coventry PA, Grande GE, Richards DA, Todd CJ. Prediction of appropriate timing of palliative care for older adults with non-malignant life-threatening disease: a systematic review. Age Ageing. 2005;34(3):218–227.
12.		Epiphaniou E, Shipman C, Harding R, et al. Coordination of end-of-life care for patients with lung cancer and those with advanced COPD: are there transferable lessons? A longitudinal qualitative study. Prim Care Respir J. 2014;23(1):46–51.
13.		Crawford EJ, Moudgil H, Srinivasan K, Naicker T, Ahmad N. Coordination of end-of-life care for patients with lung cancer and those with advanced COPD: a letter of response. NPJ Prim Care Respir Med. 2014;24:14030.
14.		Kendall M, Buckingham S, Ferguson S, Hewitt N, Pinnock H. We need to stop looking for something that is not there. NPJ Prim Care Respir Med. 2014;24:14031.
15.		Epiphaniou E, Shipman C, Harding R, et al. Avoid ‘prognostic paralysis’ – just get ahead and plan and co-ordinate care. NPJ Prim Care Respir Med. 2014;24:14085.
16.		Steer J, Gibson GJ, Bourke SC. Predicting outcomes following hospitalization for acute exacerbations of COPD. QJM. 2010;103(11):817–829.
17.		Singanayagam A, Schembri S, Chalmers JD. Predictors of mortality in hospitalized adults with acute exacerbation of chronic obstructive pulmonary disease. Ann Am Thorac Soc. 2013;10(2):81–89.
18.		Duenk RG, Heijdra Y, Verhagen SC, Dekhuijzen RP, Vissers KC, Engels Y. PROLONG: a cluster controlled trial to examine identification of patients with COPD with poor prognosis and implementation of proactive palliative care. BMC Pulm Med. 2014;14:54.
19.		Burge S, Wedzicha JA. COPD exacerbations: definitions and classifications. Eur Respir J Suppl. 2003;41:46s–53s.
20.		Lieberman D, Lieberman D, Gelfer Y, et al. Pneumonic vs nonpneumonic acute exacerbations of COPD. Chest. 2002;122(4):1264–1270.
21.		Chu CM, Chan VL, Lin AW, Wong IW, Leung WS, Lai CK. Readmission rates and life threatening events in COPD survivors treated with non-invasive ventilation for acute hypercapnic respiratory failure. Thorax. 2004;59(12):1020–1025.
22.		Curtis JR. Palliative and end-of-life care for patients with severe COPD. Eur Respir J. 2008;32(3):796–803.
23.		Murray S, Boyd K. Using the ‘surprise question’ can identify people with advanced heart failure and COPD who would benefit from a palliative care approach. Palliat Med. 2011;25(4):382.
24.		Divo M, Cote C, de Torres JP, et al; BODE Collaborative Group. Comorbidities and risk of mortality in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2012;186(2):155–161.
25.		Kocks JW, van den Berg JW, Kerstjens HA, et al. Day-to-day measurement of patient-reported outcomes in exacerbations of chronic obstructive pulmonary disease. Int J Chron Obstruct Pulmon Dis. 2013;8:273–286.
26.		Tsimogianni AM, Papiris SA, Stathopoulos GT, Manali ED, Roussos C, Kotanidou A. Predictors of outcome after exacerbation of chronic obstructive pulmonary disease. J Gen Intern Med. 2009;24(9):1043–1048.
27.		Bestall JC, Paul EA, Garrod R, Garnham R, Jones PW, Wedzicha JA. Usefulness of the Medical Research Council (MRC) dyspnoea scale as a measure of disability in patients with chronic obstructive pulmonary disease. Thorax. 1999;54(7):581–586.
28.		van der Molen T, Willemse BW, Schokker S, ten Hacken NH, Postma DS, Juniper EF. Development, validity and responsiveness of the Clinical COPD Questionnaire. Health Qual Life Outcomes. 2003;1:13.
29.		Tibshirani R. The lasso method for variable selection in the Cox model. Stat Med. 1997;16(4):385–395.
30.		Steyerberg E. Clinical Prediction Models, A Practical Approach to Development, Validation, and Updating. New York, NY: Springer; 2009.
31.		Almagro P, Calbo E, Ochoa de Echaguen A, et al. Mortality after hospitalization for COPD. Chest. 2002;121(5):1441–1448.
32.		Martinez-Rivera C, Portillo K, Munoz-Ferrer A, et al. Anemia is a mortality predictor in hospitalized patients for COPD exacerbation. COPD. 2012;9(3):243–250.
33.		Gomez-Batiste X, Martinez-Munoz M, Blay C, et al. Utility of the NECPAL CCOMS-ICO(c) tool and the Surprise Question as screening tools for early palliative care and to predict mortality in patients with advanced chronic conditions: a cohort study. Palliat Med. Epub 2016 Nov 4.
34.		Duenk RG, Verhagen SC, Janssen MA, et al. Consistency of medical record reporting of a set of indicators for proactive palliative care in patients with chronic obstructive pulmonary disease: a pilot study. Chron Respir Dis. 2017;14(1):63–71.
35.		Almagro P, Soriano JB, Cabrera FJ, et al; Working Group on COPD, Spanish Society of Internal Medicine. Short- and medium-term prognosis in patients hospitalized for COPD exacerbation: the CODEX index. Chest. 2014;145(5):972–980.
36.		Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25(1):127–141.