Recall Bias in the Assessment of Cough for Patients Discharged from Lung Surgery

Introduction

The increase of people’s awareness about the importance of physical examinations and the wide application of chest low-dose spiral computed tomography¹ enables the increased detection, diagnosis, and treatment of early-stage lung cancers. Currently, surgery is one of the primary treatment modalities for patients with lung cancer; however, it may bring severe symptom burden to patients, such as pain, cough, and shortness of breath, seriously affecting their quality of life.^2–4 Cough, as one of the most common symptoms,^2–5 has been reported in approximately 24.4–79.8% of patients postoperatively,^5–10 even after video-assisted thoracoscopic surgeries (VATs),¹¹ and usually lasting for more than 6 months after discharge.^2,12,13 Three completely different trajectories of cough in patients undergoing lung surgery after discharge have been found,¹⁴ suggesting that care plans, including monitoring schedules for patients at home, should be individualized to balance the burden of outpatient management and inadequate care of patients’ quality of life.

Recently, patient-reported outcomes (PROs) have been widely used as primary or secondary outcomes in clinical research to assess patients’ symptom burden and quality of life.^11,15,16 It is generally accepted that the PRO data collected in a prospective and ecological momentary assessment (EMA) manner are standard and accurate reflections of patients’ conditions.¹⁷ The Food and Drug Administration recommends items with short recall periods or those describing current states based on the following logic: a shorter recall period incurs in less bias, and higher quality.¹⁷ However, because of feasibility issues, not all data can be collected in real-time, which leads to data having to be retrospectively collected. Currently, patient-self-reported cough score is currently more commonly used as an outcome measure than before, particularly in postoperative clinical research of patients with lung cancer.^10,18 Frequently used postoperative cough assessment tools for lung cancer include cough-specific questionnaires (Leicester questionnaire),¹⁹ lung cancer-specific questionnaires (EORTC QLQ-LC13),²⁰ and the M. D. Anderson Symptom Inventory-lung cancer.²¹ Although the validity of these cough assessment tools has been proven and they have been widely used in clinical research,^2,18,22 it is worth noting that these tools require participants to recall symptoms over a certain period of time, such as the past one to two weeks or the last 24 hours.

One issue that should be considered when using retrospective data is the recall bias,²³ which is caused by inaccurate or incomplete recall information when respondents are asked to recall their past exposure status.²³ Recall bias mostly occurs in retrospective studies, such as case-control studies; however, it may also occur whenever the assessment with a certain recall period is used, such as in prospective cohort studies and even randomized controlled trials.²⁴ Recently published evidence of PROs suggests that this retrospective approach is flawed by by the presence of recall bias.^25–28 In these studies, a recall bias is usually defined as the difference between the prospective measurement results representing the real value and retrospective evaluation results, and its existence is determined by statistical inference of the difference.

Currently, the evaluation of recall bias mostly focuses on pain assessment studies,^26–28 data on the prevalence and magnitude of such bias in evaluating postoperative cough after lung surgery is lacking. Therefore, this study primarily aimed to evaluate the prevalence and directions of recall bias by comparing prospective and retrospective cough scores within four weeks after discharge and further exploring the factors affecting recall bias based on qualitative analyses. Ultimately, we aimed to provide empirical evidence for clinicians to accurately monitor cough experienced by patients after discharge from the lung surgery.

Materials and Methods

Participants

Eligible patients were those aged 18 years or older who signed informed consent, were clinically diagnosed with lung cancer and scheduled for surgical treatment and could complete the electronic questionnaire (e-questionnaire) or the paper questionnaire alone or with the help of family members. After signing the informed consent form, trained and qualified data collectors introduced patients to rate the severity of their cough using a web-based application on a smartphone or a paper questionnaire. The Ethics Committee of the Sichuan Cancer Hospital approved this longitudinal cohort study (No. SCCHEC-02-2018-043).

Study Design and Symptom Assessment

Since we planned to evaluate the patients’ worst cough over time, we used the worst cough score in the past 24 hours to represent the actual score. Studies have shown that 24-hour recall corresponded well with the EMA.²⁹ We used the worst cough score in the past 7 days for the retrospective weekly score. The Perioperative Symptom Assessment for Patients Undergoing Lung Surgery (PSA-Lung)³⁰ scale was used to assess cough severity. Both actual (24-hour) score and retrospective (7-day) evaluations were performed using an 11-point numerical rating scale (NRS) that ranged from 0 to 10, with 0 indicating no cough and 10 representing the worst cough experienced.³¹ For each patient, the cough severity was collected on a 0 to 10 NRS once a day (score in the past 24 hours) for 4 weeks after discharge. On the last day of each week, patients were asked to retrospectively evaluate the most severe cough they had experienced during the past 7 days on a 0 to 10 NRS (score in the past 7 days). Moreover, daily and weekly assessments performed on the same day were conducted at least 3 hours apart to minimize the potential impact of the first assessment on the last evaluation. The daily assessment used a smartphone application or a “phone-to-paper” approach by the researcher (patient’s cough score reported by telephone follow-up after discharge and entered into the database by the investigator), according to the patient’s preference, whereas the weekly assessment comprised only the phone-to-paper approach. Figure 1 shows the schematic of the study design.

Figure 1 Schematic of study design.

Demographic information, including age, sex, body mass index, race, level of education, marital status, work status, disease-specific health information (including smoking history, previous complications, Eastern Cooperative Oncology Group (ECOG) performance status), and surgical information, were collected at enrollment.

All data were stored on an account-password-protected online data-management platform.

Outcome Definitions

The primary outcome of this study was the prevalence, magnitude, and direction of recall bias in retrospective measurements. A recall bias was considered present if the weekly measured cough score significantly differed from the maximum daily score. The magnitude of the difference represented the magnitude of the bias, and the sign of the difference represented the direction of the bias (overestimation or underestimation). A positive difference score referred to an overestimation of the actual cough severity, whereas a negative score referred to an underestimation of such. Furthermore, a larger absolute value of the difference score indicated a more severe recall bias. We defined the prevalence as the proportion of measurements that overestimate or underestimate. According to the related literature,³² we defined the time interval as the number of days between the time point of the last measured maximum daily cough score and the time point of the retrospective assessment of cough, and its value ranged from 0 to 6.

The following aspects were considered in the analyses. First, since the retrospective assessment asked about the worst cough in the past week, we used the maximum value measured daily during the corresponding period instead of the average value to represent the actual level of cough at its worst during the past 7 days. Second, with the rise and promotion of the concept of enhanced recovery after surgery and minimally invasive surgery, postoperative hospital stays have been continuously shortened to 3–5 days,³³ and a 7-day evaluation was only used after discharge.

To explore the possible factors involved in recall bias, we analyzed baseline data, including age, sex, education level, and associated preoperative symptoms.

Statistical Analysis

Patient’s demographics and clinical characteristics were presented using descriptive statistics. For each week, the maximum prospective cough score and the retrospective cough score are shown as mean, standard deviation (SD), and 95% confidence interval (CI). Furthermore, paired t-tests were used to identify whether the recall bias was statistically significant. We used 30% of the maximum prospectively assessed score as the minimal clinically important difference (MCID)³⁴ to determine whether the differences (recall bias) between the actual score and prospective measures reached clinical significance.

Subgroups of patients with distinct cough trajectories were identified based on the cough scores at 31-time points (preoperative, postoperative day 1, day of discharge, and each of the first 28 days after discharge) using the group-based trajectory model (GBTM).³⁵ Based on the optimal Bayesian Information Criterion value and clinical significance, we categorized all patients into three subgroups (low, medium, and high).³⁶ Similarly, we used paired t-tests to examine whether retrospective ratings were biased toward different cough trajectories.

Moreover, to analyze the relationship between recall bias and the length of the elapsed time interval, we used the time interval as the abscissa and the mean of the difference between the two measures at the corresponding time points as the ordinate to draw the trend of recall bias over time. Based on the previous analyses, with recall bias as the dependent variable, we used the univariate analysis method and generalized estimating equation (GEE) model to evaluate the factors influencing recall bias. Analyses of the influencing factors were performed separately for overestimation and underestimation.

Results

Patient Characteristics

Overall, 199 eligible patients planning to undergo lung surgery participated in this study and completed on-site enrollment. During the follow-up period, a total of 17 patients were lost to follow-up, 2 patients for week 2, 6 patients for week 3 and 9 patients for week 4.

Table 1 shows the participants’ baseline demographic and clinical characteristics. The median age was 55 (48, 63) years, 57.3% of the patients (114/199) were female, 50.5% (100/198) had completed high school or a higher educational level, and 93.5% (186/199) were married. All patients had a good ECOG performance status (0 or 1) preoperatively. All eligible patients were included in this study, and more than half of them were staged with tumor node metastasis 0-II (65.5%). The surgical approaches included open surgery (9.3%) and VATs (90.7%).

Table 1 Baseline Demographic and Clinical Characteristics of Participants

Trajectories of Postoperative Cough

Based on the prospective cough scores at the 31-time points, we used the GBTM to generate a three-group trajectory model (Figure 2). The “high” group comprised 21.1% (42/199) of all patients with a high level of cough scores, which gradually worsened after discharge and remained at a high level (linear coefficient = 0.01, P =0.02). The “medium” group accounted for 58.3% (116/199) of all patients, with a moderate cough score that gradually decreased slightly from discharge (linear coefficient = - 0.02, P < 0.01) and remained at a relatively low level. The “low” group included 20.6% (41/199) of all patients, who presented with low cough scores, which decreased fairly rapidly (linear coefficient = - 0.07, P < 0.01) over 4 weeks after discharge (Table S1).

Figure 2 Postoperative cough trajectory in patients after lung cancer surgery.

The Presence and Direction of Recall Bias

In all patients, no statistically significant recall bias was found between the two measures in the first week after discharge. From the second week of discharge, a statistically significant difference was observed between the actual (24 h) score and the retrospective (weekly) score, and the retrospective scores were lower than the actual scores (Table 2). The results of subgroup analyses found a significant recall bias in week 2 for the high-trajectory patients (6.26 vs 5.10, P<0.01) and in week 3 for the medium-trajectory patients (2.88 vs 2.60, P=0.01).

Table 2 Differences and Effect Size Between the Prospective and Retrospective Scores of Coughing

Figure 3 shows the trend of the actual cough score from preoperative to 4 weeks after discharge and that of the retrospective cough score 4 weeks after discharge. From the preoperative period to the first week after discharge, the cough gradually worsened, reached a maximum in the first week, and then gradually decreased. Simultaneously, we clearly demonstrated that the retrospective score after discharge was always lower than the prospective score, with statistical significance after the second week.

Figure 3 Prospective and retrospective scores of cough severity over the first month after lung surgery.

The underestimation of the actual score was found in 40.9%, 42.9%, 43.6%, and 39.6% patients for weeks 1, 2, 3, and 4. Whereas the overestimation of the actual score was found in 34.2%, 23.3%, 13.8%, and 15.4% patients for weeks 1, 2, 3, and 4, respectively (Table 3).

Table 3 Direction of Bias in Measuring Cough Severity After Lung Surgery

Does Recall Bias Reach Clinical Significance?

Within 4 weeks after discharge, the proportion of patients who retrospectively underestimated the actual cough score over the MCID increased from 23.3% in the first week to 32.4% in the fourth week. In contrast, 26.4% of patients overestimating more than the MCID in the first week presented a gradual decrease in number, dropping to 9.9% by the fourth week (Table 4).

Table 4 Direction of Bias in Measuring Cough Severity After Lung Surgery (Defined by MCID)

Factors Associated with Recall Bias

During the entire follow-up period, the univariate analysis results showed that patients in the “high” trajectory were likely to have an underestimated cough score on the retrospective (7-day recall) assessment. For all patients, Figure 4 and Table S2 show the trend of recall bias over time intervals; the longer the interval between the daily and weekly measurements, the greater the recall bias.

Figure 4 The trend of recall bias over time interval (a) The trend of recall bias over time interval in the first week after discharge; (b) The second week after discharge; (c) The third week after discharge; (d) The fourth week after discharge; If the 95% CI of recall bias contains 0, it means that the difference is not statistically significant, and on the contrary, it is statistically significant. Time interval was defined as the number of days between the time point of the last measured maximum daily cough score and the time point of retrospective assessment of cough, and its value ranged from 0 to 6.

The GEE results showed that the proportion of overestimation of the cough score (cases in which the retrospective score was greater than the prospective score) gradually decreased along the weeks after discharge and decreased with the prolongation of the interval time. Compared with patients in the “low” cough trajectory group, neither the high nor the medium cough trajectory groups were influencing factors of recall bias (Table S3). In contrast, the proportion of underestimation of cough increased with the interval of time elapsed (β=0.36, P <0.01) and did not change over the weeks after discharge. Patients in the “high” trajectory group were more likely to underestimate the severity of their cough than those in the “low” cough trajectory group (β=1.14, P <0.01) (Table S4). No statistically significant factors affecting recall bias were found other than the above-mentioned variables.

Discussion

This study used longitudinal data to evaluate the recall bias of PROs in assessing the post-discharge cough severity scores in patients after lung surgery. Over one-third of the patients demonstrated recall bias when a commonly used weekly assessment schedule with a 7-day recall period was performed. Patients tended to underestimate the severity of cough, particularly when they were in a “middle” or “high” trajectory group, suggesting that a more frequent follow-up should be designed for patients who reported higher cough scores on the day of discharge for an accurate evaluation of their cough status.

Cough has a significant impact on patients’ daily life, not only regarding physical functioning, but also on the psychological and social performance of patients.^37–39 Therefore, tools for accurately assessing the severity of cough in patients are helpful for postoperative symptom management, which is further beneficial for the function recovery and improvement in quality of life of patients. The magnitude and directions of the recall bias identified by our study will have positive significance for the development of a reasonable cough assessment plan.

Analyzing the direction of recall biases and whether they possess clinical significance is necessary. The recall bias poses a great impact in the clinical environment.^26,27 Patients’ over or underestimation of symptoms during follow-up could affect physicians’ treatment decisions. With overestimation, overly aggressive treatment modalities may be employed; on the contrary, with underestimation, the necessary measures to relieve cough may not be employed, with consequent reduction of the patient’s quality of life. This study found that the retrospective scores are more likely to underestimate the actual cough score of patients, especially when those are in a high-trajectory cough, suggesting that a more frequent assessment using shorter recall periods should be considered in the future for this subgroup of patients.

Notably, the results of this study are inconsistent with those of the recall bias analyses of retrospective PROs in other fields, such as pain in knee osteoarthritis and hip arthroscopy. These studies showed that retrospective assessments overestimate the actual status of patients.^26,40 This disparity may be related to the differences in the research design. We used longitudinal data as the research goal to analyze recall bias more comprehensively; a great advantage of the longitudinal design is that we found that patients with lung cancer had different cough trajectories after discharge, which is consistent with previous findings.¹⁴ Based on this finding, we were able to exam how the patients’ symptom burden impacts the accuracy of PRO assessments within a certain recall period. In contrast, the above-mentioned two studies were cross-sectional, and the patients retrospectively evaluated the pre-treatment symptom scores a long period of time after the end of treatment. Furthermore, patient’s perceptions of treatment effects and changes in symptoms were not considered in these studies.

The subjective assessment of cough may be affected by multiple factors, such as demographic characteristics, disease factors, and elapsed time interval.²³ The results of the GEE model showed that interval of time, weeks after discharge, and postoperative cough trajectory were independent risk factors for the presence of the recall bias. However, for recall bias in different directions, the same factors presented different effects on different recall biases, sometimes even the opposite, such as in interval of time; this is not entirely consistent with previous findings on recall bias²³, (Tables S3 and S4). As previously mentioned in similar studies, the analysis of recall bias should be more refined and cannot be generalized.²⁹ Therefore, we explored the directions of bias and its influencing factors separately. We are confident that achieving this will lead to a better understanding of recall biases. Interestingly, in addition to interval of time, which is a well-known key factor affecting recall bias, different cough trajectories, which were not considered in previous studies,^26–28,40 are also important culprits leading to different recall biases. Therefore, in practice, when retrospective assessment is performed, different measurements should be taken according to the patient’s specific situation to prevent the occurrence of different recall biases. Interestingly, other studies on recall bias have found that individual characteristics such as age are also important influencing factors,^41,42 however, in this study, we did not find that demographic information was an important factor causing recall bias.

Limitations

Our study had some limitations. First, the score for the worst cough in the past 24 hours was used as the result of the prospective evaluation. Undoubtedly, this is not strictly a prospective evaluation, which may impact the entire study. However, previous studies have shown that recalling the past day’s situation does not produce recall bias compared to momentary assessments,²⁹ and this study emphasizes that assessing the worst cough of the past day further reduces possible bias. Second, we only qualitatively judged whether a recall bias was present, the trend and direction of the recall bias, and the factors influencing it throughout the study. A “real” recall bias was not evaluated; in our study, the value of the recall bias is represented by the difference between the prospective and retrospective scores, which is not a real quantitative measure of recall bias. This may cause a certain deviation in the accuracy of the description of recall bias, suggesting that the results should be interpreted cautiously when the difference is small. Third, we used 30% of the maximum 24-hour score as the MCID value when evaluating the clinical significance of the recall bias. Currently, research on the MCID value of cough is lacking; 30% as an effective indicator for the wide application of clinical symptom monitoring was applied to this study, however, its reliability may require further verification.

Conclusion

Retrospective assessment of post discharge cough in patients that underwent lung surgery will introduce a recall bias and mostly incur in symptom’s underestimation. The high-trajectory group, interval of time, and time since discharge are important influencing factors for the occurrence of recall bias. Thus, strategies, such as more frequent assessments with shorter recall periods, should be applied for clinical research or routine care and should be individualized, considering patients’ severity of symptoms, as well as their trajectory over time.