Comments on Randomized Trial of Intestinal Obstruction Catheter Combined with Modified Dachengqi Decoction [Letter]

Dear editor

We read with interest the randomized controlled trial by Liu et al¹ evaluating the combination of an intestinal obstruction catheter with modified Dachengqi Decoction in elderly patients with early postoperative inflammatory intestinal obstruction. The authors are commended for addressing a clinically challenging condition. However, we wish to raise several methodological and statistical considerations that, in our view, substantially affect the validity and interpretability of the reported findings.

First, the sample size calculation warrants attention. The authors calculated that 95 patients per group were required based on their assumptions, yet only 31 per group were ultimately enrolled – less than one-third of the target. This severe underpowering is not merely a limitation but a fundamental issue affecting the reliability of the findings. Reito et al demonstrated that many published RCTs remain underpowered to detect clinically relevant effect sizes, compromising the reproducibility of research findings.² In small underpowered trials, observed significant P-values may largely reflect Type I error inflation or effect overestimation. Lee similarly cautioned that underpowered studies risk discouraging clinicians from using potentially effective treatments due to erroneous negative findings.³ We suggest the authors explicitly reclassify their findings as exploratory hypothesis-generation, rather than using confirmatory language such as “significantly improved” throughout the abstract and conclusion.

Second, no adjustment for multiple comparisons was applied despite the testing of over a dozen endpoints (recovery times, inflammatory markers, barrier function, electrolytes, nutritional indices) at multiple time points (7 and 14 days). This omission greatly inflates the family-wise error rate. Even if all null hypotheses were true, the probability of observing at least one significant result by chance alone is substantial. Khan et al found that among cardiovascular RCTs with multiplicity, only 28.3% performed appropriate adjustments, highlighting this as a widespread concern.⁴ Odutayo et al further demonstrated that strategies to control Type I error are inconsistently employed, with important analytical differences between planned analyses and subsequent publications.⁵ The authors acknowledge this limitation in the Methods section but continue to interpret P-values as confirmatory evidence in the Results and Discussion, creating an internal inconsistency. We recommend that all P-values be reported as descriptive only, and that effect sizes with 95% confidence intervals be provided for primary endpoints.

Third, there are notable discrepancies in baseline characteristics that raise concern. Serum sodium in the experimental group was 125.69 ± 5.24 mmol/L, while in the conventional group it was 133.86 ± 6.02 mmol/L – a clinically meaningful difference of over 8 mmol/L. Although the P-value was not significant, this baseline imbalance could profoundly affect recovery, especially in elderly patients with fluid-electrolyte disturbances. Moreover, the proportion of patients with intestinal obstruction as the primary diagnosis differed markedly between groups (25.81% vs 12.90%). These imbalances suggest that randomization may not have produced fully comparable groups, and the observed benefits may partly reflect baseline heterogeneity rather than treatment efficacy. Lauzon et al emphasized that controlling baseline covariate imbalance at the design stage is critical for preserving Type I error rates and ensuring valid treatment comparisons.⁶ Additionally, the authors performed a per-protocol analysis rather than an intention-to-treat analysis. Tripepi et al noted that ITT and PP analyses serve different purposes, and selective reporting of PP results without ITT may introduce bias when protocol deviations occur.⁷ We also note the absence of any covariate adjustment (eg, ANCOVA) for baseline imbalances or prognostic factors, which is critical in small trials to reduce confounding.

Fourth, the conclusions regarding safety and mechanisms appear to extend beyond what the data can support. The authors conclude that the combined therapy is “safe” and “improves intestinal barrier function”, yet no adverse event grading system (eg, CTCAE) was used, and no direct histological or molecular evidence (eg, tight junction proteins, endotoxin levels) was provided. With only 31 patients per group, the study is underpowered to detect even moderate safety signals. The statements regarding anti-inflammatory and barrier-protective mechanisms are speculative and not directly supported by the data presented.

In summary, while the clinical question is undoubtedly relevant and the integrated approach appears conceptually promising, the methodological limitations outlined above – particularly the small sample size relative to the calculated requirement, the absence of multiplicity correction, the baseline imbalances between groups, and the limited statistical modeling – suggest that the findings should be interpreted with appropriate caution. We respectfully suggest that the authors consider framing their conclusions as preliminary and hypothesis-generating, rather than definitive, and that future large-scale, multicenter, double-blind randomized trials with prespecified statistical analyses would be valuable to confirm and extend these observations before any clinical recommendations are made. We hope these comments contribute constructively to the ongoing refinement of research in this important area.