Inappropriate Analysis of a Cluster Randomized Controlled Trial Due to Not Accounting for Nesting and Clustering [Response to Letter]

Dear editor

We appreciate the interest of the readers regarding the analysis of our study entitled “A home-based dyadic music-with-movement intervention for people with dementia and caregivers: a hybrid type 2 cluster-randomized effectiveness-implementation design”.¹ We are writing to respond to the criticism of the validity of the conclusion of our paper. Here, we report the results of the additional analysis to address the potential clustering effects of the intervention on the outcomes as suggested by Hefner et al.²

Our initial published analysis has taken reference to similar studies and relevant guidelines. The objective of our study was to explore the population-average effect (i.e., the impact of the intervention on the participants). Thus, using the generalized estimating equation (GEE) was an appropriate analysis for outcomes variables across the two time points (baseline and follow-up), regardless of the normality of distribution.³ Before conducting the GEE, we confirmed no cluster effect on the major primary outcomes at baseline by using the GLM univariate analysis. The primary outcomes of interest are the anxiety and depressive symptoms of people with dementia, and the perceived stress of caregivers, which showed no significant variation among centres (see Table 1). Hence, we performed the GEE analysis without considering centre as an independent variable. Various teams of authors adopted a similar analytical approach too.^4,5

Table 1 Comparing Outcomes at Baseline across Centres

However, we notice that a sub-scale outcome (Cornell Scale for Depression in Dementia – Ideational subscale) and a secondary outcome (Quality of the Caregiver–Care Recipient Relationship) are significantly different across centres, using an uncorrected p-value cut-off of 0.05, when we revisit the data. Therefore, after discussing with statisticians, we performed a sensitivity analysis by adjusting centres. Similar to the initial published analysis results,¹ the GEE analysis adjusted to affiliated centre results showed that participants with dementia in the intervention group had improvement in anxiety status and depressive symptoms; and caregivers in the intervention group also had lowered level of perceived stress as compared to the control group (Table 2).

Table 2 Group x Time GEE Analysis Results Adjusted the Clustering Effects

Concerning the data sharing request, we have indicated in the Data Sharing Statement section that the datasets are not publicly available due to ethical restrictions. The journal’s data-sharing policy allows the authors to share or not to share the data consistent with the terms of consent signed by study participants. We did not obtain consent from our participants to share the data beyond the research team, so unfortunately, Ms Hefner’s request of the datasets could not be fulfilled. Although we were not asked to provide details about our methods in the email by Ms Hefner in October 2022, as stated in the published Letter to Editor, we trust that this response letter has addressed their query.

There are different ways of handling statistics as long as they are justifiable and help researchers to understand the data, that is why we had never received similar doubts from the reviewers or journal editors. As the late George Box (1976), a British statistician regarded by many as one of the great statistical minds of the 20th century, mentioned, “All models are wrong, but some are useful”.⁶ We are thankful to the journal for letting us publish the additional data analysis results to clarify the concern being raised.