Do patients and carers agree on symptom burden in advanced COPD?

Plain language summary

Patients with COPD are affected by many symptoms such as breathlessness and tiredness in their day-to-day life. Family and friends (informal carers) often have to help manage these symptoms and decide when to get medical help. We wanted to find out whether patients and their carers agree in their judgments of the patient’s symptoms and what factors are associated with better or worse agreement as this may influence how well symptoms are managed, given the carer’s role in this. One hundred and nineteen patients and their carers separately rated how much the patient was bothered by six common symptoms in COPD. Agreement between patients and carers was worse for less obvious symptoms such as depression, fatigue, anxiety and breathlessness than constipation and diarrhea. Carers who said that their patient had more problematic symptoms than the patients themselves tended to have a non-spousal relationship with and live apart from their patient, be symptomatically anxious or depressed themselves and have more unmet support needs. Patients reporting a greater symptom burden than their carer reported for them were younger and had COPD for longer. Our results suggest clinicians should seek opinions about symptoms from both the patient and their carer, promote shared understanding between patients and carers, and screen for mental health problems in both and unmet support needs in carers.

Introduction

COPD carries a high symptom burden, including psychological impacts such as anxiety and depression. Accurate symptom assessment by informal carers, typically family and friends, is likely to be important for their own decision making in supporting symptom management in the home setting and in deciding when to seek medical help.^6,7 Inaccurate assessment may negatively impact on the quality of the patient–carer relationship and the psychological health of both, and may lead to overtreatment or inadequate symptom control.^1,6,8–10

The extent of agreement between patients and carers is debated. There is widespread consensus in cancer that, at the group patient and carer cohort level, carers consistently overestimate patient symptoms,^{2–6,8,9,11–16} although the magnitude of the bias is small.^2,11,13,17 At the individual patient–carer dyad level, strength of agreement on symptoms ranges from fair to substantial.^{2,4,11,12,14–19} It is well established in cancer populations that there is greater agreement between patients and carers for more observable symptoms (typically physical symptoms) compared to less observable symptoms (typically psychological symptoms).^{3–5,8,13,15} Group-level carer bias toward over-reporting is small for physical symptoms but moderate for psychological symptoms, with overestimation primarily for fatigue, pain, anxiety and depression.^1,5,11,13 Individual-level agreement is substantial on physical symptoms compared to only fair on psychological symptoms, with substantial agreement in nausea and vomiting, moderate agreement in pain, poor sleep, poor appetite and constipation, and poor to fair agreement in fatigue, anxiety and depression.^{2,3,11–13,15,18}

Previous studies have investigated patient and carer characteristics associated with symptom disagreement in cancer, with inconsistent links to carer age,^{3,9,10,12,18,20,21} sex,^{3,10,12,13,21} patient–carer relationship^3–5,11,20 and cohabitation,²² moderate disease severity,^1,21 poor patient and carer psychological health,^{1,13,20,23,24} greater subjective caring burden,^{1,3,8–10,20,23} lack of carer support and low carer self-esteem.^3,12

Agreement between patients and carers has been extensively investigated in cancer (including lung cancer),^6,12 but few studies have examined this in COPD¹⁹ or in population-based samples. We assessed agreement on how bothered the patient was by symptoms of breathlessness, fatigue, constipation, diarrhea, anxiety and depression. We took a holistic approach in considering the experience of those living with advanced COPD; thus, we included a spectrum of physical and psychological symptoms resulting from the condition, side effects of treatments, symptoms of conditions commonly comorbid with advanced COPD and symptoms which are difficult to manage in advanced COPD as the action of managing them leads to breathlessness, for example, walking to and using the toilet. We aimed to determine agreement between patients and carers on these symptoms, and factors associated with disagreement in a population-based sample of patients with advanced COPD.

Patients and methods

Study design and participants

We conducted a cross-sectional analysis of data from patients and carers participating in baseline interviews for the Longitudinal Interview Study component of the prospective Living with Breathlessness Study.²⁵ Primary care patients with well-characterized advanced COPD were recruited, if they met at least two of six inclusion criteria: forced expiratory volume in 1 second <30% predicted, at least two exacerbations requiring prednisolone or antibiotics in the previous year, at least one hospital admission for COPD in the previous 2 years, long-term oxygen therapy, cor pulmonale and Medical Research Council Dyspnea scale 4+. Patients were excluded if they had a serious mental health problem, serious learning difficulty, active cancer or active alcoholism. Patients were asked to identify their primary informal carer (defined as the person who provided the most help and support and was not a health care professional) and these carers were also invited to participate. Carer exclusion criteria were as follows: under 18 years of age and unable to give informed consent. Care provided by carers could encompass personal care, for example, walking, dressing, washing, toileting as well as help with practical tasks, including shopping, cooking, housework, transport and paperwork. Six hundred and fifty-two patients with advanced COPD were identified by 63 primary care practices in the East of England (targeted to maximize variability and representativeness), and were mailed a recruitment pack. Two hundred and thirty-five patients consented and participated (36% participation rate), from whom 117 carers were recruited to the study (50% participation rate; not all patients identified a carer). The study received ethics approval from the National Research Ethics Service Committee East of England – Cambridge South (reference number 12/EE/0163). All participants gave written informed consent.

Data collection

Data were collected by conducting separate face-to-face interviews with the patients and carers. Sociodemographics of participants and measures of patients’ disease severity (% predicted forced expiratory volume in 1 second, Medical Research Council dyspnea scale, COPD Assessment Test score, number of hospital admissions, number of exacerbations at home, long-term oxygen use) were obtained.^26,27 Psychological morbidity was identified for both patients and carers using the 14-item (two 7-item subscales) Hospital Anxiety and Depression Scale (HADS), with caseness for anxiety and depression defined as HADS-A and HADS-D score ≥11 (on a scale of 0–21).²⁸ HADS was used as it is a well-validated and reliable self-report screening tool encompassing symptoms of both anxiety and depression, and has been shown to have reliability, convergent/discriminant validity and responsiveness to change in the primary care population.²⁹ Carers identified unmet support needs on the 14-item Carer Support Needs Assessment Tool (CSNAT). CSNAT covers 14 broad support domains in two groupings: support to enable the carer to provide care for the patient (these “enabling” domains include understanding the illness, knowing what to expect in the future, knowing who to contact if concerned, talking with the patient about the illness, managing symptoms, providing personal care, equipment to help care) and support for carers’ own health and well-being (these “direct” domains include looking after own physical health, dealing with feelings and worries, support with beliefs or spiritual concerns, support with financial, legal and work issues, practical help at home, having time to oneself in the day, having a break from caring overnight). The need for more support is rated on a 4-point scale from 0 (none) to 3 (very much more). There is an “anything else” section enabling the carer to write in any other support need they may have that is not covered by the other 14 items.³⁰ The CSNAT tool has been validated in a multicenter UK cohort of carers of palliative patients, demonstrating good face, content and criterion validity, and is being used in populations in Australia, USA, Canada, Europe and China.³¹ The CSNAT was used as it is the only validated screening tool which directly measures carers’ support needs (as opposed to the multiple tools that measure carer burden or distress, which are only indirect indicators of need). Patients and carers rated patient symptoms of breathlessness, fatigue, constipation, diarrhea, anxiety and depression (patients and carers were asked: “how much were you/was the patient bothered by the symptom in the past week”) on 4-point scales: 1 (not at all), 2 (a little), 3 (quite a bit) and 4 (very much).³²

Statistical analysis

Wilcoxon signed-rank test was used to determine whether differences between patient and carer scores were significant at the group level. The direction and magnitude of mean differences (mean scores of carers minus patients) were standardized to Cohen’s d effect size (statistical magnitudes of the observed bias), which were interpreted as 0.2= small, 0.5= moderate and 0.8= large.³³ Agreement on symptoms between patients and carers on the individual level was evaluated by weighted Cohen’s kappa, with 0.81–0.99 defined as excellent agreement, 0.61–0.80 as substantial agreement, 0.41–0.60 as moderate agreement, 0.21–0.40 as fair agreement and ≤0.20 as poor agreement.³⁴ Spearman’s correlation coefficients were used to identify the strength of association between difference in symptom burden estimates by patients and carers (carer score minus patient score) and variables of interest; positive correlation represents greater carer estimation relative to the patient as the variable of interest increases. Mann–Whitney U tests identified categorical variables significantly associated with difference between patient and carer scores. Where variables were significant in univariate analysis, ordinal logistic regression was performed to determine significance when adjusted for patient and carer age and sex. Statistical significance was set at p≤0.05. All statistical analyses were carried out in SPSS Version 23 (IBM Corp., Armonk, NY, USA).

Results

Mean patient and carer age was 71.39±8.69 and 64.16±14.50 years, respectively. Also, 62.4% of patients and 27.4% of carers were male; 80.9% of patients and carers were spouses and 87.2% lived together. Patient and carer characteristics are shown in Table 1.

Table 1 Sociodemographic and clinical characteristics of patients and carers Note: All variables are expressed as mean (SD) unless otherwise stated. Abbreviations: CAT, COPD assessment test; FEV1, forced expiratory volume in 1 second; MRC, Medical Research Council.

Mean group-level differences and individual dyad-level agreement between patient and carer scores for all symptoms are shown in Table 2.

Table 2 Mean group-level differences and individual dyad-level agreement between patient and carer symptom ratings Notes: aMean group-level difference. bIndividual dyad-level agreement.

The most bothersome symptoms were breathlessness and fatigue, followed by anxiety and depression. There was no significant difference between mean patient and carer scores for any symptom at the group level. Agreement between patients and carers on symptoms at the individual dyad level was moderate for constipation, just below moderate for diarrhea and fair for depression, fatigue, anxiety and breathlessness.

Factors associated with disparity in dyad-level patient and carer scores are shown in Table 3.

Table 3 Factors associated with disparity between individual dyad-level patient and carer symptom ratings Notes: aMann–Whitney U test was used for these variables. Spearman’s correlation was used for all other variables. *p≤0.05. Abbreviations: CSNAT, Carer Support Needs Assessment Tool; FEV1, forced expiratory volume in 1 second; HADS, Hospital Anxiety and Depression Scale.

Carers being more likely to estimate greater symptom burden in patients relative to patients themselves was associated with non-spousal patient–carer relationships, non-cohabitation of patients and carers, carer caseness for anxiety and depression on HADS and unmet support needs in more CSNAT domains, specifically more direct carer support domains. Greater symptom burden estimation by the patient relative to the carer was associated with younger patients and longer self-reported duration of COPD. However, the strength of correlation between patient–carer disparity and carer unmet support needs, age and duration of COPD was weak. In multivariate analysis (Table 4), the association between unmet support needs in more direct carer support domains and greater estimation of breathlessness by carers remained when adjusted for patient and carer age and sex (odds ratio 1.250, 95% CI 1.031–1.516), as did younger patient age and greater patient estimation of depression (odds ratio 1.090, 95% CI 1.018–1.167).

Table 4 Multivariate analysis of factors associated with disparity between individual dyad-level patient and carer symptom ratings Note: *p≤0.05. Abbreviations: CSNAT, Carer Support Needs Assessment Tool; FEV1, forced expiratory volume in 1 second; HADS, Hospital Anxiety and Depression Scale; OR, odds ratio.

Discussion

Unlike previous studies in cancer which have found a bias toward carer overestimation, there was no significant difference in this population-based sample of carers and patients with advanced COPD between their ratings of any symptom at the group level. However, agreement between patients and carers at the individual dyad level did not exceed moderate for any symptom, indicating lack of agreement between patients and carers within individual dyads which are masked at the group (cohort) level. What is not known is whether the patient’s or the carer’s reporting reflects the “true” symptom experience, and previous studies have suggested that the patient’s view cannot be considered to be the gold standard. However, it could be that there is no “true” experience and that alternative viewpoints are valuable.³⁵

There are several possible explanations for disagreement. First, it is known that patients engage in protective buffering, concealing symptoms in order to minimize the burden on their carer.^12,21,24 Response shift may occur in patients, with those with advanced disease reporting less symptom burden possibly because patients who are living with advanced COPD simply accept and consciously do their best to minimize their symptoms or subconsciously expect, and therefore experience, less symptom burden.^36,37 Carers may find it less easy to accept symptoms, and their impacts, leading to estimation of greater symptom burden by the carer. Conversely, limited carer presence at patient–clinician contacts may result in lower carer awareness of symptoms and, therefore, lower estimation of symptom burden. Interestingly, there was a tendency toward lower carer estimation for five of the six symptoms, possibly reflecting longer disease trajectories in COPD and slower disease progression, compared to previous studies in cancer, leading to carer compassion-fatigue and response shift.

We found higher agreement for more observable physical symptoms such as constipation and diarrhea than less observable psychological symptoms such as anxiety and depression and those with emotional valence such as fatigue and breathlessness. This can be attributed to greater difficulty appraising psychological symptoms due to a lack of overt and non-verbal cues and, therefore, the need for greater interpretation of cues, which carers may not have the knowledge or skills to do,^3,5,12 but which could be gained. It may also be easier for patients to conceal psychological symptoms, and carers may be more prone in these circumstances to make assessments colored by their individual frame of reference.^6,12,24 In addition, constipation and diarrhea were less-frequent and episodic symptoms, which may have increased carers’ awareness of them as a notable change in contrast to the more daily-present symptoms such as breathlessness and fatigue.

Estimation of greater patient symptoms of anxiety and depression by carers was seen in non-spousal and non-cohabitating patient–carer dyads, probably because these carers have less face-to-face contact with their patients,^5,22 so have less opportunity to observe particularly subjective symptoms. Carer anxiety and depression caseness on the HADS scale were also associated with estimation of greater patient symptoms of anxiety and depression by carers, probably due to projection of their own experiences and feelings, thus creating a more negative view of the situation.^1,12 Higher carer rating of breathlessness was associated with unmet carer support needs across a wider range of carer support domains (number of CSNAT domains) and interestingly only across more domains of direct support for carer health and well-being rather than support to enable the carer to provide care for the patient, which persisted when adjusted for patient and carer age and sex; this may suggest that carer perception of the patient’s breathlessness is related to the carer’s own physical, psychological and spiritual well-being, more than difficulties in managing it.³⁸ Estimation of greater depressive symptoms by relatively younger patients found in univariate and multivariate analyses may reflect the greater difficulty of psychologically adjusting to disease limitations at an earlier stage in life, although given that the cohort was, in general, relatively old (reflecting being population based with advanced COPD), even those who were “younger” may not be “young” in the broader COPD population.

These findings have implications for clinical practice and research. Given the only moderate agreement between patients and carers, clinicians and researchers should, in the context of COPD, seek the perceptions of both.³⁵ Clinicians may be well placed to open a dialogue between patients and carers within dyads,⁸ helping them to gain a shared understanding of each other’s position and address any differences in perceptions. One approach that has been found to be of benefit is perspective taking, which involves imagining the patient’s perspective rather than assuming their feelings (the cognitive, rather than the affective, dimension of empathy).³⁹ In addition, we need to find ways of helping patients express themselves and ensure that we respond to their needs without forcing them to acknowledge or experience their illness more than they want to. Carers can have a role in this, helping to draw attention to symptoms that patients do not emphasize and considering whether patients need help in being a little more demanding about some of their needs. To help promote agreement, screening for and addressing psychological morbidities in both patients and carers and unmet support needs in carers is required. However, patient assessment for anxiety and depression is itself not widespread in respiratory clinics (or indeed for any chronic disease and among all health professionals), so it may be challenging to implement carer assessment of these in a clinical setting. Finally, further investigation of the impact of patient–carer disagreement is warranted.

Limitations

One of the limitations of the study is that the cohort was limited to patients with advanced COPD and, therefore, may not represent the majority of those with COPD. Nevertheless, disease severity was not found in this study to be associated with agreement between patients and carers, so it is possible that similar patient–carer agreement relationships may be found in general COPD populations. Additionally, data were not available for other symptoms associated with COPD, such as cough. We used the HADS screening tool to identify caseness for anxiety and depression in patients and carers, which is not a formal diagnosis of anxiety or depression, but the stringent definition of caseness of HADS score ≥11 (moderate to severe depression) may have improved accuracy. Finally, the cross-sectional analysis could establish association, but not causality.

Conclusion

Agreement between patients and carers was fair to moderate and was poorer for less-observable symptoms. Greater symptom burden estimation by the carer was associated with less patient–carer contact, carer symptoms of anxiety and depression and a wider range of unmet carer support needs; greater estimation by the patient was associated with being younger and having COPD for longer. There is a need to encourage open dialogue between patients and carers to promote shared understanding of patients’ symptom burden. This may encourage carers to explore the impact of symptoms that patients do not emphasize. Screening for and addressing psychological morbidities in patients with advanced COPD and their carers, and identifying and responding to unmet support needs in carers may identify opportunities for supportive intervention. There is a potential to develop a useful therapeutic dialogue addressing the complimentary viewpoints of patients and carers, and clinicians.

Acknowledgments

We are grateful to Silvia Mendonca for providing statistical support. The Living with Breathlessness Study program thanks all participating patients, informal carers and health care professionals; the former Primary Care Research Networks and all recruiting practices (East of England and South London); the British Lung Foundation Breathe Easy support groups and additional patient and carer representatives for their Patient and Public Involvement roles; Kevin Houghton for administrative support; Sam Barclay for additional data entry; and the programme funders Marie Curie and the National Institute for Health Research. The Living with Breathlessness Study program is an independent research supported by Marie Curie (grant number: C28845/A14129) and the National Institute for Health Research (grant number: CDF-2012-05-218). The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health or other funders. Funding sources had no involvement in the study design, collection, analysis and interpretation of data, writing of the report or decision to submit the article for publication. This paper was presented at the British Thoracic Society Conference as a poster presentation with interim findings. The poster’s abstract was published in “Poster Abstracts” in Thorax: http://thorax.bmj.com/content/71/Suppl_3/A258.1.

Author contributions

All authors contributed toward data analysis, drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.		Sneeuw KC, Aaronson NK, Sprangers MA, Detmar SB, Wever LD, Schornagel JH. Comparison of patient and proxy EORTC QLQ-C30 ratings in assessing the quality of life of cancer patients. J Clin Epidemiol. 1998;51(7):617–631.
2.		Sneeuw KC, Sprangers MA, Aaronson NK. The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease. J Clin Epidemiol. 2002;55(11):1130–1143.
3.		Lobchuk MM, Degner LF. Patients with cancer and next-of-kin response comparability on physical and psychological symptom well-being: trends and measurement issues. Cancer Nurs. 2002;25(5):358–374.
4.		Tang ST, McCorkle R. Use of family proxies in quality of life research for cancer patients at the end of life: a literature review. Cancer Invest. 2002;20(7–8):1086–1104.
5.		McPherson CJ, Addington-Hall JM. Judging the quality of care at the end of life: can proxies provide reliable information? Soc Sci Med. 2003;56(1):95–109.
6.		Lyons KS, Lee CS, Bennett JA, et al. Symptom incongruence trajectories in lung cancer dyads. J Pain Symptom Manage. 2014;48(6):1031–1040.
7.		Al-Gamal E, Yorke J. Perceived breathlessness and psychological distress among patients with chronic obstructive pulmonary disease and their spouses. Nurs Health Sci. 2014;16(1):103–111.
8.		Broberger E, Tishelman C, von Essen L. Discrepancies and similarities in how patients with lung cancer and their professional and family caregivers assess symptom occurrence and symptom distress. J Pain Symptom Manage. 2005;29(6):572–583.
9.		Winters-Stone KM, Lyons KS, Bennett JA, Beer TM. Patterns and predictors of symptom incongruence in older couples coping with prostate cancer. Support Care Cancer. 2014;22(5):1341–1348.
10.		Kristjanson LJ, Nikoletti S, Porock D, Smith M, Lobchuk M, Pedler P. Congruence between patients’ and family caregivers’ perceptions of symptom distress in patients with terminal cancer. J Palliat Care. 1998;14(3):24–32.
11.		Lobchuk MM, Degner LF. Symptom experiences: perceptual accuracy between advanced-stage cancer patients and family caregivers in the home care setting. J Clin Oncol. 2002;20(16):3495–3507.
12.		Wennman-Larsen A, Tishelman C, Wengström Y, Gustavsson P. Factors influencing agreement in symptom ratings by lung cancer patients and their significant others. J Pain Symptom Manage. 2007;33(2):146–155.
13.		McPherson CJ, Wilson KG, Lobchuk MM, Brajtman S. Family caregivers’ assessment of symptoms in patients with advanced cancer: concordance with patients and factors affecting accuracy. J Pain Symptom Manage. 2008;35(1):70–82.
14.		Silveira MJ, Given CW, Given B, Rosland AM, Piette JD. Patient-caregiver concordance in symptom assessment and improvement in outcomes for patients undergoing cancer chemotherapy. Chronic Illn. 2010;6(1):46–56.
15.		Yeşilbalkan OU, Okgün A. Patients’ self reports and caregivers’ perception of symptoms in Turkish cancer patients. Eur J Oncol Nurs. 2010;14(2):119–124.
16.		Poort H, Peters ME, Gielissen MF, et al. Fatigue in advanced cancer patients: congruence between patients and their informal caregivers about patients’ fatigue severity during cancer treatment with palliative intent and predictors of agreement. J Pain Symptom Manage. 2016;52(3):336–344.
17.		Jones JM, McPherson CJ, Zimmermann C, Rodin G, Le LW, Cohen SR. Assessing agreement between terminally ill cancer patients’ reports of their quality of life and family caregiver and palliative care physician proxy ratings. J Pain Symptom Manage. 2011;42(3):354–365.
18.		Liu XY, Shen J, Ye ZX, et al. Congruence in symptom assessment between hepatocellular carcinoma patients and their primary family caregivers in China. Support Care Cancer. 203;21(10):2655–2662.
19.		Janssen DJ, Spruit MA, Wouters EF, Schols JM. Symptom distress in advanced chronic organ failure: disagreement among patients and family caregivers. J Palliat Med. 2012;15(4):447–456.
20.		Miaskowski C, Zimmer EF, Barrett KM, Dibble SL, Wallhagen M. Differences in patients’ and family caregivers’ perceptions of the pain experience influence patient and caregiver outcomes. Pain. 1997;72(1–2):217–226.
21.		Low G, Gutman G. Couples’ ratings of chronic obstructive pulmonary disease patients’ quality of life. Clin Nurs Res. 2003;12(1):28–48.
22.		McCusker J, Stoddard AM. Use of a surrogate for the Sickness Impact Profile. Med Care. 1984;22(9):789–795.
23.		Rothman ML, Hedrick SC, Bulcroft KA, Hickam DH, Rubenstein LZ. The validity of proxy-generated scores as measures of patient health status. Med Care. 1991;29(2):115–124.
24.		Lyons KS, Jones KD, Bennett RM, Hiatt SO, Sayer AG. Couple perceptions of fibromyalgia symptoms: the role of communication. Pain. 2013;154(11):2417–2426.
25.		University of Cambridge Department of Public Health and Primary Care Primary Care Unit. Living with Breathlessness Study. Available from: http://www.phpc.cam.ac.uk/pcu/research/research-projects-list/living-with-breathlessness-study/. Accessed June 20, 2016.
26.		Stenton C. The MRC breathlessness scale. Occup Med (Lond). 2008;58(3):226–227.
27.		Jones PW, Harding G, Berry P, Wiklund I, Chen WH, Kline Leidy N. Development and first validation of the COPD Assessment Test. Eur Respir J. 2009;34(3):648–654.
28.		Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–370.
29.		Cameron IM, Crawford JR, Lawton K, Reid IC. Psychometric comparison of PHQ-9 and HADS for measuring depression severity in primary care. Br J Gen Pract. 2008;58(546):32–36.
30.		Ewing G, Grande G; National Association for Hospice at Home. Development of a Carer Support Needs Assessment Tool (CSNAT) for end-of-life care practice at home: a qualitative study. Palliat Med. 2013;27(3):244–256.
31.		Ewing G, Brundle C, Payne S, Grande G; National Association for Hospice at Home. The Carer Support Needs Assessment Tool (CSNAT) for use in palliative and end-of-life care at home: a validation study. J Pain Symptom Manage. 2013;46(3):395–405.
32.		Ewing G, Todd C, Rogers M, Barclay S, McCabe J, Martin A. Validation of a symptom measure suitable for use among palliative care patients in the community: CAMPAS-R. J Pain Symptom Manage. 2004;27(4):287–299.
33.		Cohen J. Statistical Power Analysis for the Behavioural Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
34.		Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360–363.
35.		Nekolaichuk CL, Maguire TO, Suarez-Almazor M, Rogers WT, Bruera E. Assessing the reliability of patient, nurse, and family caregiver symptom ratings in hospitalized advanced cancer patients. J Clin Oncol. 1999;17(11):3621–3630.
36.		White P, White S, Edmonds P, et al. Palliative care or end-of-life care in advanced chronic obstructive pulmonary disease: a prospective community survey. Br J Gen Pract. 2011;61(587):e362–e370.
37.		Pinnock H, Kendall M, Murray SA, et al. Living and dying with severe chronic obstructive pulmonary disease: multi-perspective longitudinal qualitative study. BMJ. 2011;1(2):174–183.
38.		Farquhar M. Supporting Informal Carers. In: Bausewein C, Currow DC, Johnson MJ, editors. Palliative Care in Respiratory Disease. ERS Monograph; 2016:51–70.
39.		Lobchuk MM, McClement SE, Daeninck PJ, Shay C, Elands H. Asking the right question of informal caregivers about patient symptom experiences: multiple proxy perspectives and reducing interrater gap. J Pain Symptom Manage. 2007;33(2):130–145.