The Bonferroni Paradox: When Secondary Analyses are Held to a Stricter Standard Than the Primary Endpoint [Letter]

Dear editor

We read with great interest the randomized controlled trial by Han et al comparing remimazolam with propofol for anesthesia in patients undergoing renal allograft transplantation, published in Drug Design, Development and Therapy¹ The study addresses an important clinical question, and the pharmacokinetic substudy is particularly welcome. However, we have identified several methodological concerns that may affect the interpretation of the findings and warrant clarification from the authors.

Inadequate Handling of Repeated-Measures Hemodynamic Data and an Unintended Statistical Double Standard

The authors measured mean arterial pressure (MAP) and heart rate (HR) at 8 predetermined time points and compared these at each time point using two-sample t-tests, applying a Bonferroni-corrected threshold of 0.003125 (0.05/16). While we commend the authors for attempting to account for multiplicity, this approach presents a critical logical inconsistency. The primary endpoint – incidence of hypotension during induction (P = 0.010) – was not subjected to any multiplicity correction, whereas the secondary hemodynamic comparisons were held to a far stricter standard (P < 0.003125). Had the primary endpoint been evaluated against this same Bonferroni threshold, it would not have achieved statistical significance, fundamentally undermining the study’s central conclusion. This double standard, wherein secondary endpoints face a more stringent evidentiary bar than the primary endpoint, represents a reversal of standard statistical prioritization and severely weakens the interpretability of the findings.

Beyond this logical paradox, the Bonferroni correction itself is fundamentally inappropriate for correlated repeated measurements. The correction assumes independence among comparisons, yet repeated hemodynamic measurements from the same patient are inherently correlated, rendering the method excessively conservative and markedly inflating the risk of Type II error. Analyzing longitudinal data through a series of pairwise t-tests further fails to exploit the within-subject correlation structure and cannot model time-by-group interaction effects. Repeated-measures ANOVA or linear mixed-effects models would have been the statistically appropriate methods, as they account for intra-individual correlation, handle missing data more robustly, and provide a unified test of the treatment effect across the entire perioperative trajectory without requiring such untenable ad hoc corrections. We would welcome the authors’ rationale for this analytical choice.

Absence of an Operational Definition for the Primary Endpoint

The primary endpoint was the “incidence of hypotension during anesthesia induction,” reported as 33.3% in the propofol group versus 6.7% in the remimazolam group (P = 0.010). However, the article does not specify the threshold or criteria used to define hypotension. Was it defined as an absolute mean arterial pressure (MAP) < 65 mmHg? A systolic blood pressure < 90 mmHg? A relative decrease > 20% or > 30% from baseline? The choice of definition profoundly affects incidence rates: Bijker et al demonstrated that applying 140 different published definitions to the same cohort yielded hypotension incidences ranging from 5% to 99%, and Wesselink et al, in a systematic review of 42 studies, confirmed that the relationship between intraoperative hypotension and adverse outcomes varies substantially depending on the threshold and exposure duration employed^2,3 Without a prespecified, clinically justified definition, readers cannot assess whether the observed difference represents a clinically meaningful reduction in significant hypotensive events or merely a shift in borderline measurements Furthermore, reporting only incidence without accompanying data on hypotension duration, cumulative time below threshold, or area-under-the-curve further limits the clinical interpretability of this key finding.

Pharmacokinetic Blind Spot: The Unmeasured Metabolite CNS7054

The authors’ pharmacokinetic analysis is limited to the parent drug remimazolam. While the study demonstrates a rapid decline in remimazolam plasma concentration following discontinuation, it does not measure the main metabolite CNS7054, which is formed via esterase-mediated hydrolysis and excreted primarily by the kidneys.⁴ This omission is particularly relevant for end-stage renal disease (ESRD) patients. Although the product label and prior Phase I studies classify CNS7054 as pharmacologically inactive, accumulating evidence suggests this characterization warrants nuance.^5,6 In a population pharmacokinetic study of remimazolam in children, Gao et al reported a CNS7054 terminal half-life of 321 min – nearly five times longer than that of the parent drug (67 min) – with marked inter-individual variability and evidence of substantial accumulation during continuous infusion⁷ Moreover, data presented at the 2022 International Society for Anaesthetic Pharmacology meeting demonstrated that CNS7054 accumulates progressively during prolonged remimazolam infusion, with the CNS7054-to-remimazolam plasma concentration ratio increasing from 2.9 to 63.4 depending on infusion sequence; a post hoc pharmacodynamic analysis suggested CNS7054 may function as a very weak GABAA receptor agonist rather than being entirely inert.⁸ For ESRD patients with markedly impaired renal clearance of this metabolite, dismissing any potential contribution to sedation or recovery profiles without direct measurement appears premature.

Bispectral Index Interpretability Under Remimazolam Anesthesia

Both groups were titrated to a BIS target of 40–60. However, Choi et al reported that a substantial proportion of patients receiving remimazolam-based total intravenous anesthesia exhibit BIS values ≥ 60 despite clinically adequate anesthesia, suggesting that BIS may be systematically higher during remimazolam than equipotent propofol anesthesia⁹ If the remimazolam group was maintained at a genuinely lighter depth of hypnosis while displaying numerically similar BIS values, this could partially explain the observed hemodynamic stability and shorter airway removal times – not as a unique pharmacological advantage, but as a consequence of comparatively lighter anesthesia. This possibility does not diminish the clinical utility of the findings, but it underscores the need for caution in attributing these differences solely to intrinsic drug properties.

In summary, we commend the authors for investigating remimazolam in this high-risk surgical population, a topic of considerable clinical relevance. However, the absence of a predefined hypotension definition, the suboptimal statistical handling of longitudinal hemodynamic data, and the failure to measure CNS7054 in a cohort of ESRD patients limit the strength of the conclusions. We hope the authors will address these points in their response.