The Impact of Missing Postoperative Longitudinal Pain Data on Study Results Investigating Analgesia from Transversus Abdominis Plane Block on Postoperative Day One Following Abdominal Surgery: A Single-Center Retrospective Chart Review

Introduction

The transversus abdominis plane (TAP) block may provide significant analgesia following abdominal surgery.^1,2 The expected area of analgesia is the abdominal skin around and below the umbilicus.³ Proposed advantages of TAP blocks have included prolonged analgesia, reduced postoperative opioid consumption, and decreased pain scores.^4–8 However, the benefit of TAP blocks compared to subcutaneous local anesthetic infiltration of incision sites has been questioned.⁹

Postoperative pain is commonly reported as median (interquartile range) or mean score for each patient group and may be further broken down for each postoperative day (POD).¹⁰ A retrospective cohort study investigating pain following open colorectal surgery reported lower average pain scores with TAP block compared to thoracic epidural (median (IQR): 4.68 (1.90) versus 5.09 (2.69), p = 0.275).¹¹ Another retrospective study investigating pain following a renal transplant reported lower mean pain scores with continuous TAP block compared to patient-controlled intravenous analgesia at 48 hours postoperatively: 2.49 (95% CI 1.93 to 3.06) versus 2.94 (95% CI 2.39 to 3.50), p value = 0.26.¹² In a retrospective study investigating pain following laparoscopic Roux-en-Y gastric bypass and sleeve gastrectomy, the average pain score during hospitalization was greater for patients who received TAP block (3.7, IQR 2.6 to 4.5) compared to those who did not (3.4, IQR 2.5 to 5.5).¹³ The median pain burden, or area under the Defense and Veterans Pain Rating Scale curve, was also reported in number of hours during the following time intervals: 0–6 hours, 6–12 hours, 12–24 hours, 24–48 hours, and 48–72 hours.¹³ It is unknown if these retrospective complete case analyses yield conflicting results when missing data, which is defined in this study as the discrepancies in the number of recorded pain scores between patients, are replaced with statistical estimations.

Statistical methods that address the discrepancy in pain recordings between patients may lead to significantly different results. Complete case analyses or multilevel modeling, which can bypass discrepancies in pain recordings between patients, may foster inaccurate conclusions and are vulnerable to overrepresentation of patients with the greatest number of pain recordings, and thus may not adequately represent the study population or allow the results to be generalized to other populations.¹⁴ Furthermore, it is plausible that substitution of missing pain scores with imputed values from mean substitution, last observation carried forward, regression imputation or multiple imputation will lead to significantly different results.¹⁴ Therefore, the statistical method used to address incomplete pain data may persuade clinicians to reach inappropriate conclusions.

Because the discrepancy in the number of pain recordings between patients may be substantial, this issue should command deliberate, a priori, attention in retrospective pain research. This topic extends beyond pain research and attempts to illustrate how to attenuate the overrepresentation of patients with the most recorded data in retrospective study results.

Methods

This retrospective chart review was approved by the Institutional Review Board (IRB) at the University of North Carolina at Chapel Hill. The need for informed consent from patients to review their medical records was waived by this IRB because this study imparts no greater than minimal risk to patients’ privacy, stored data does not include identifying information, and contacting patients would lead to unnecessary stress. In compliance with the Declaration of Helsinki, the well-being of the human subjects took precedence over the interest of science and society in considering this research study.

The primary aim is to retrospectively analyze the association between the number of recorded pain scores and TAP block analgesia, baseline pain on postoperative day 1, and patient demographics. The secondary aim is to compare the available statistical methods to address discrepancy in the number of recorded pain scores in the setting of a retrospective review. These aims seek to test the following null hypotheses: (1) the number of pain recordings will be independent of TAP block analgesia, baseline pain on postoperative day 1, and patient demographics; and (2) there will be no significant differences in pain scores with and without TAP block analgesia following complete case analysis, mean substitution, last observation carried forward, regression imputation, multiple imputation, and mixed models.

Population

Adult patients were selected for this research because of the assumed ability of adults to understand the pain scale and communicate pain, when experienced. Abdominal surgeries were selected because TAP blocks are mostly offered to patients undergoing abdominal surgeries and there is inconsistent evidence to support TAP blocks, thus necessitating additional research. The date of hospital discharge was recorded in the dataset so that patients discharged on the same day of surgery would not be included in subsequent analyses. Pain scores were already recorded in the chart, most likely from nursing staff, and not for the specific purpose of this research.

Intervention & Control

This is a retrospective chart review with no specific intervention or control groups made for the sole purpose of this research. The only difference in patients studied in this review is the presence or absence of TAP block as part of the postoperative analgesia regimen. An equal number of patients with and without TAP block were included prior to the elimination of outliers. After identifying the number of patients and surgeries with TAP blocks, the patients who did not receive TAP block were randomly selected based on computer algorithm from the surgeries present in the TAP block group. Therefore, the surgery type was recorded in the dataset so that the same surgeries are represented in both the “intervention” and “control” groups.

Outcomes

The date, time, and value for all pain scores on the first postoperative day were extracted from the medical record. The following demographic data were also recorded: age, gender, BMI, race, and ethnicity. The date and type of surgery were also recorded.

Definitions

Outliers were defined as those values that are 1.5 times the interquartile range for the lower and upper quartiles. For the purposes of this dissertation, missing pain scores are defined as the difference between the greatest number of recorded pain scores in the dataset and the number of pain scores recorded for each patient:

Inclusion & Exclusion Criteria

All adult (age 18 years or older) patients who underwent an abdominal surgery between 2014 and 2022 and were not discharged on the date of surgery were eligible for inclusion. Patients who underwent abdominal surgery before 2014 were not included because the Electronic Medical Record (EMR) at our institution was not implemented until 2014. Patients requiring postoperative mechanical ventilation or postoperative admission to the Intensive Care Unit (ICU) were not specifically excluded. Patients with a primary language other than English were not excluded, and patients with intellectual disabilities were also not specifically identified or excluded. The primary aim of this study is not to analyze pain with and without TAP block but instead to merely present different methods for handling unequal numbers of pain recordings. Therefore, it was not necessary to develop strict inclusion criteria.

Statistical Analysis Plan

Linear regression will quantify the unadjusted association of TAP block analgesia on a number of pain recordings. This linear regression model will be adjusted for demographic data and the first recorded pain score. The statistical analysis plan will further include mean substitution utilizing the mean pain score for each patient; last observation carried forward where the last recorded pain score is substituted for subsequent missing scores; regression imputation; multiple imputation; and mixed models. The results will then be compared. RStudio and STATA will be used for statistical analyses.^15,16

Potential Pitfalls

Patients who remain intubated postoperatively may not be able to verbally communicate a pain score. Patients admitted to the ICU may be asked to report pain more frequently than patients not in the ICU. Patients who may struggle verbally communicating pain on a 1–10 scale (English is not the primary language or intellectual disability) were not specifically excluded.

Justification of Methods

Extracting data that is already present within the EMR is a more practical approach to analyzing interventions that are already being performed. TAP blocks are part of our institution’s Enhanced Recovery After Surgery (ERAS) protocols, and eliminating these interventions for patients may be unethical or impractical as this would directly contradict current policies. The chosen methods are generalizable for investigators analyzing the impact of analgesic interventions already being offered at their sponsoring institutions. Although prospective randomized controlled trials with pain scores recorded at pre-determined intervals would generate stronger hypothesis tests, this was not plausible at our institution in the setting of perioperative protocols.

Postoperative pain is an insufficient outcome measure to evaluate the utility of TAP blocks as components of multimodal analgesia protocols. Data were limited to the first postoperative day given that TAP blocks would not be expected to provide analgesia beyond the first postoperative day. Comprehensive measures to best evaluate TAP blocks were not included in this study, such as opioid consumption, postoperative respiratory distress, nausea, vomiting, or PACU discharge time. The primary and secondary outcomes of this study were not to evaluate the effectiveness of TAP blocks at improving patient care. Instead, this study focuses on the statistical methods of missing longitudinal pain data in the investigation of therapeutic interventions.

Results

Five subjects were identified as outliers given the number of pain scores beyond 1.5 times the interquartile range of the dataset. These outliers were removed from subsequent analyses. A total of 486 patients were included in this study including 2562 pain scores. Baseline characteristics for the included patients are shown in Table 1.

Table 1 Baseline Characteristics

An unadjusted linear regression model demonstrated no significant effect of TAP block on the number of pain scores recorded (0.509 95% CI −0.078 to 1.10, p = 0.089). The linear regression model was adjusted (Adjusted Model 1) for the effect of the first recorded pain score on postoperative day 1. There was no significant association between the first recorded pain score and the total number of recorded pain scores on postoperative day 1 (0.052 95% CI −0.0450 to 0.150, p = 0.292). The linear regression model was further adjusted (Adjusted Model 2) with the following variables: first recorded pain score, age (continuous), ethnicity (categorical), race (categorical), gender (binary), TAP block analgesia (binary), and BMI (continuous). Less pain scores were recorded for black patients, on average, compared to white patients (−0.763 95% CI −1.49 to −0.039, p = 0.039). Age was also associated with lower pain scores: −0.042 (95% CI −0.060 to −0.025, p < 0.001). However, the association between number of recorded pain scores and age is not clinically significant. All linear regression results are displayed in Table 2.

Table 2 Association of TAP Block on Number of Pain Scores Recorded Adjusted for First Pain Score and Demographic Factors

Missing Data

Table 3 displays the percentage of patients with the number of missing pain scores.

Table 3 Percentage of Patients by Number of Missing Pain Scores

These missing data were substituted with the results from complete case analysis, mean substitution, last observation carried forward, regression imputation, multiple imputation, and mixed models.

Complete Case Analysis

A complete case analysis demonstrates the effect of ignoring the discrepancy in recorded pain scores between patients. Results from a complete case analysis indicate that patients who received TAP block reported a median pain score of 4.44 (95% CI 4.11 to 4.78) compared to a median pain score of 3.52 (95% CI 3.15 to 3.88) for patients who did not receive TAP block, and this difference was statistically significant (p = 0.0003).

Mean Substitution

When missing data are replaced with the mean pain score for each patient, those who received TAP block reported a median pain score of 4.38 (95% CI 4.09 to 4.68) compared to a median pain score of 3.62 (95% CI 3.30 to 3.94) for patients who did not receive TAP block, and this difference was statistically significant (p = 0.0006).

Last Observation Carried Forward

When missing data are replaced with the last recorded pain score for each patient, those who received TAP block reported a median pain score of 6.13 (95% CI 5.80 to 6.47) compared to a median pain score of 5.27 (95% CI 4.87 to 5.66) for patients who did not receive TAP block, and this difference was statistically significant (p = 0.0009).

Comparison of Complete Case Analysis, Mean Substitution, and Last Observation Carried Forward

Results comparing the methods of substitution for missing data demonstrated significant differences in the datasets created with each method. Figure 1 presents box and whisker plots for each created dataset by method of substitution for missing data and analgesia.

Figure 1 Box and Whisker Plots for Pain Scores by Method for Handling Missing Data.

Regression Imputation

When missing data are replaced with results from regression imputation, those who received TAP block reported a median pain score of 6.89 (95% CI 6.77 to 7.0) compared to a median pain score of 6.76 (95% CI 6.60 to 6.92) for patients who did not receive TAP block, and this difference was not statistically significant (p = 0.188).

Multiple Imputation

When missing data are replaced with results from multiple imputation, those who received TAP block reported a median pain score of 6.75 (95% CI 6.67 to 6.83) compared to a median pain score of 6.66 (95% CI 6.56 to 6.77) for patients who did not receive TAP block, and this difference was not statistically significant (p = 0.1985).

Mixed Models

When a mixed model is used to analyze pain scores by patient ID, patients who receive TAP block, on average, report a pain score that is 0.554 points higher (95% CI 0.124 to 0.984) compared to patients who do not receive TAP block, and this value is statistically significant (p = 0.012). Notably, the difference in pain scores between patients who did and did not receive TAP block was significantly associated with the time that the pain score was recorded. Patients reported significantly higher pain scores with TAP block between 1600–1959 and 0400–0759 as shown in Table 4.

Table 4 Results from Mixed Model by Time Category

Summary of Results

A summary of the results for each dataset to account for the missing longitudinal pain data is presented in Table 5.

Table 5 Summary of Results by Method to Account for Missing Longitudinal Pain Data

Discussion

Age was significantly associated with the number of pain scores recorded such that older patients, on average, have less pain scores recorded. This result may support current knowledge of older patients being more sensitive to opioids, which may lead to more pronounced analgesia and lower incidence of reporting pain.^17,18 However, the association of age on the number of recorded pain scores is likely not clinically significant. The association of black race with recorded pain scores is both clinically and statistically significant. Black patients, on average, received almost one less pain score compared to white patients in the dataset. This result supports known racial disparities in pain assessment and treatment.¹⁹

TAP blocks were not associated with greater analgesia in any of the methods to account for unequal pain recordings. TAP blocks are unlikely to cause pain. Therefore, other factors may be confounding the results. For example, patients with TAP blocks may receive lower doses of opioids and other medications given the expected analgesia from TAP block and the subsequent desire to lessen the dose of additional analgesics. This discrepancy in medication administration for patients with and without TAP block may lead to differences in reported pain scores. This theory requires further investigation. Alternatively, surgeons may preferentially request TAP blocks for patients they suspect will have pain that is difficult to control postoperatively, such as patients with chronic pain conditions or chronic opioid use. Therefore, selection bias may be contributing to the significantly different results.

The dataset created with last observation carried forward yielded greater pain scores compared to mean substitution and complete case analysis. In clinical practice, patients are asked to report their pain prior to the administration of opioids or other analgesic medications. Therefore, carrying forward the last pain score and substituting that score for all remaining data may falsely inflate the results.

When analyzing the data with mixed models, statistically significant differences were only appreciated between 0400–0759 and 1600–1959. The reasons for these results are unclear from a clinical perspective and may serve as the focus for future investigation.

The results presented within this manuscript highlight the statistical concerns already present within the medical literature, but are not commonly addressed among studies investigating postoperative pain. Research analyzing pain scores may not interpret discrepancies in the number of pain recordings between patients as missing data. As shown in the article by Calkins et al, different analyses for missing pain data may not significantly impact the results, but should be presented and compared as they are based on different assumptions.²⁰ Understanding how missing data is handled in each study becomes particularly important when combining results for meta-analyses. It may be inappropriate to combine studies that approach missing data differently. Schober and Vetter declare that missing data “should be appropriately described and addressed in any research report.”²¹ The present study highlights the importance of that statement.

Limitations

The definition of missing data used in this dissertation may be overly sensitive. One patient may have a substantially high number of pain scores and overly impact the number of missing pain scores. It might be more appropriate to consider only a percentage (such as 95% or 0.95) of the discrepancy in pain scores to be truly missing:

Alternatively, as was done in this study, outliers were defined as those patients with pain scores 1.5 times the lower or upper quartiles, and these patients were excluded from further analyses.

Pain was not specifically defined as abdominal pain. Although it is assumed that a patient’s reported pain number following abdominal surgery would specify the abdomen, it is possible that a patient’s pain has multiple foci and may describe chronic conditions in addition to the acute postoperative pain. This would be difficult to correct in retrospective studies.

Conclusion

Missing longitudinal pain data following abdominal surgery is significantly associated with age and black race. Statistical methods to account for missing pain data yield significantly different results. For outcomes with substantial discrepancy in recorded pain scores, a priori determination for the most appropriate statistical analysis is critical.