Back to Journals » Clinical Interventions in Aging » Volume 18
Inappropriate Analysis of a Cluster Randomized Controlled Trial Due to Not Accounting for Nesting and Clustering: Comment on “A Home-Based Dyadic Music-with-Movement Intervention for People with Dementia and Caregivers: A Hybrid Type 2 Cluster-Randomized Effectiveness-Implementation Design” [Letter]
Authors Hefner M, Jamshidi-Naeini Y, Vorland CJ, Golzarri-Arroyo L, Shaw BP, Allison DB
Received 10 December 2022
Accepted for publication 3 January 2023
Published 2 February 2023 Volume 2023:18 Pages 141—142
Checked for plagiarism Yes
Editor who approved publication: Prof. Dr. Nandu Goswami
Marleigh Hefner,1 Yasaman Jamshidi-Naeini,2 Colby J Vorland,2 Lilian Golzarri-Arroyo,2 Brian P Shaw,2 David B Allison2
1Department of Nutritional Sciences, Texas Tech University, Lubbock, TX, USA; 2Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Bloomington, IN, USA
Correspondence: David B Allison, Indiana University School of Public Health-Bloomington, 1025 E 7th St, PH 111, Bloomington, IN, USA, Tel +1 812 855 1250, Email [email protected]
View the original paper by Dr Cheung and colleagues
A Response to Letter has been published for this article.
Cheung et al accepted an important challenge by conducting a cluster randomized controlled trial (cRCT) primarily designed to assess the effect of a music-with-movement (MM) intervention on psychosocial wellbeing of patients with dementia and their caregivers.1 Due to identified barriers in applying MM, the secondary aim of their experiment was to determine implementation strategy effectiveness of the home-based, dyadic MM intervention.1
To appropriately assess outcomes using a cluster randomized design requires that intracluster dependency, typically as quantified by the intracluster correlation coefficient (ICC), be accounted for in the statistical analysis.2 Accounting for non-independence allows for comparison between individuals in the clusters assigned to intervention or control conditions because it accounts for potential non-independence of model residuals within clusters.2 Given this, there are concerns within the publication by Cheung et al.
Most importantly, the authors did not explain how clustering was accommodated within the statistical analysis used, suggesting that it was not. Additionally, accounting for nesting in cRCTs is critical, especially with a small number of clusters (K), because if K is small, the available degrees of freedom for estimating effect sizes must also be small.2 Ignoring either non-independence or limited degrees of freedom often inflates type I error rate.2
We also highlight a concern with the article’s reported sample size calculations. The cRCT CONSORT guidelines require that authors report the methods used for sample size calculation, number of clusters (K), size of clusters (m), whether equal or unequal cluster sizes were assumed, assumed ICC, and an indication of the ICC uncertainty. The authors did include the assumed ICC (0.01) for sample size calculation of their trial registration’s statistical analysis plan (NCT03575026). However, they reported a sample size calculation only with respect to the number of individual participants, not accounting for the number of clusters required to reach statistical power. The number of clusters has greater impact on statistical power in cRCTs than does the number of individual participants.2
We note that the authors used the generalized estimating equations (GEE) approach in their article.1 While GEE can be used to account for clustering, as we previously describe Cheung et al appear not to have done so. Further, they randomized a small number of clusters (7 in total)1 for which GEE can produce inflated type I error rates.3 Thus, switching to an appropriate statistical model for reanalysis using a valid approach, such as linear mixed models accounting for the small number of clusters, is warranted. The authors could also report the results of a reanalysis using GEE with a small-sample correction.4
If we are correct in our understanding that clustering and nesting were not accounted for in the analysis, per the Committee on Publication Ethics (COPE) guidelines for handling post-publication critiques,5 a reanalysis of the data using valid statistical procedures for the study design in a published correction, or retraction and republication, seems essential.
Supported in part by NIH grants U24AG056053 and P30AG050886. The assertions expressed are those of the authors and not necessarily those of the NIH or any other organization.
Dr. Allison and his institution (Indiana University) have received payments for consultation, grants, contracts, in-kind donations, and contributions from multiple for-profit and not-for-profit entities (Alkermes, Inc., Amin, Talati, Wasserman for KSF Acquisition Corp (Glanbia), Big Sky Health/Zero, Clark Hill, PLC, Eli Lilly and Company, Kaleido Biosciences, Mars, Incorporated, Medpace/Gelesis, Novo Nordisk Foundation, Ronald Marron, Law Offices, Soleno Therapeutics, Sports Research Corp., and WW [formerly Weight Watchers]) interested in statistical design and analysis of experiments, but not directly related to the research questions addressed in this letter. The other authors report no conflicts of interest in this communication.
1. Cheung DSK, Ho LYW, Chan LCK, Kwok RKH, Lai CKY. A home-based dyadic music-with-movement intervention for people with dementia and caregivers: a hybrid type 2 cluster-randomized effectiveness-implementation design. Clin Interv Aging. 2022;17:1199–1216. doi:10.2147/CIA.S370661
2. Brown AW, Li P, Bohan Brown MM, et al. Best (but oft-forgotten) practices: designing, analyzing, and reporting cluster randomized controlled trials. Am J Clin Nutr. 2015;102(2):241–248. doi:10.3945/ajcn.114.105072
3. Murray D. Design and Analysis of Group Randomized Trials. Oxford University Press Inc; 1998.
4. Leyrat C, Morgan KE, Leurent B, Kahan BC. Cluster randomized trials with a small number of clusters: which analyses should be used? Int J Epidemiol. 2018;47(1):321–331. doi:10.1093/ije/dyx169
5. COPE Council. COPE flowcharts and infographics—Handling of post-publication critiques. ©2021 Committee on Publication Ethics (CC BY-NC-ND 4.0). Available from: https://doi.org/10.24318/o1VgCAih.
© 2023 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.