Back to Journals » Risk Management and Healthcare Policy » Volume 15
Ignoring Clustering and Nesting in Cluster Randomized Trials Renders Conclusions Unverifiable [Letter]
Authors Siddique AB , Jamshidi-Naeini Y, Golzarri-Arroyo L, Allison DB
Received 28 September 2022
Accepted for publication 1 October 2022
Published 11 October 2022 Volume 2022:15 Pages 1895—1896
Checked for plagiarism Yes
Editor who approved publication: Dr Jongwha Chang
Abu Bakkar Siddique, Yasaman Jamshidi-Naeini, Lilian Golzarri-Arroyo, David B Allison
Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington, Bloomington, IN, USA
Correspondence: David B Allison, Indiana University School of Public Health-Bloomington, 1025 E 7th St, PH 111, Bloomington, IN, 47405, USA, Tel +1 812 855-1250, Email [email protected] Abu Bakkar Siddique, Indiana University School of Public Health-Bloomington, 1025 E 7th St, SPH 394, Bloomington, IN, 47405, USA, Tel +1 571 274-5328, Email [email protected]
View the original paper by Dr Siraneh and colleagues
A Response to Letter has been published for this article.
Siraneh et al1 conducted a clustered randomized controlled trial (cRCT) to test the effectiveness of additional counseling and social support provided by women identified as “positive deviants” to promote exclusive breastfeeding (EBF) within a community. However, their statistical methods did not account for clustering and nesting effects and thus are not valid.
In the study, randomization occurred at the cluster level (ie, kebeles), and mothers were nested within clusters. Three of 6 clusters were in the treatment group and 3 were in the control. Of the 260 mothers, 130 were enrolled in the treatment group and 130 in the control, according to kebele. According to the published statistical methods section, the analyses ignored clustering and nesting effects. Because this is a hierarchical modeling environment and individuals within a cluster are typically positively correlated, an individual-level analysis that does not address clustering effects will generate underestimated standard errors and unduly narrow confidence intervals.2 That is, the results will overstate statistical significance. Treating the individual-level observations (eg, mothers) as independent units of analysis inflates the type 1 error rate.3
One alternative is calculating the mean observation by cluster and analyzing the data at the cluster level. In this case, this approach would reduce the sample size to 6, the same as the number of clusters, resulting in low statistical power. A valid alternative would be to use multi-level hierarchical modeling, which recognizes the hierarchy in the data and accounts for both lower and higher levels as distinct levels simultaneously. Such multi-level modeling estimation will estimate residuals at mother- and kebele-level separately. Statistical packages exist for such modeling.
Moreover, Siraneh et al calculated their sample size based on the randomization of individual participants.1 In a cRCT, however, sample size and power calculations should account for the number of clusters per condition, average cluster size, and intra-cluster correlation coefficient (ICC).4
We requested the deidentified raw data and statistical code from the authors to reproduce their analyses. Even though we pledged to limit our analysis to testing the hypotheses tested in the article, and the Editor-in-Chief deemed our request “appropriate and reasonable”, the authors were unwilling to share their deidentified raw data and statistical code. They said they needed time to analyze the “remaining data” for publication and that the dataset contained identifiers. Thus, we were unable to reanalyze the data using a valid statistical approach accounting for clustering and nesting effects.
Given the analytical methods used, the evidence presented by Siraneh et al1 neither supports nor refutes whether a positive deviance intervention affects EBF. The analytical methods were incorrect. All authors have an ethical and professional scientific responsibility to correct non-trivial reported errors in published papers.5 Furthermore, the Committee on Publication Ethics (COPE), in which this journal is a member, states
Editors should consider retracting a publication if they have clear evidence that findings are unreliable, either as a result of major error (e.g., miscalculation or experimental error), or as a result of fabrication (e.g., of data) or falsification.6
ABS, YJ-N, LGA, and DBA are supported by NIH grants R25DK099080, R25HL124208, and the Gordon and Betty Moore Foundation. The opinions expressed are those of the authors and do not necessarily represent those of the NIH or any other organization.
In the last thirty-six months, DBA has received personal payments or promises for same from: Alkermes, Inc.; American Society for Nutrition; Amin Talati Wasserman for KSF Acquisition Corp (Glanbia); Big Sky Health, Inc.; Biofortis Innovation Services (Merieux NutriSciences), Clark Hill PLC; Kaleido Biosciences; Law Offices of Ronald Marron; Medpace/Gelesis; Novo Nordisk Fonden; Reckitt Benckiser Group, PLC; Law Offices of Ronald Marron; Soleno Therapeutics; Sports Research Corp; and WW (formerly Weight Watchers). Donations to a foundation have been made on his behalf by the Northarvest Bean Growers Association. Dr. Allison is an unpaid consultant to the USDA Agricultural Research Service. In the last thirty-six months, Dr. Jamshidi-Naeini has received honoraria from The Alliance for Potato Research and Education. The institution of DBA, ABS, LGA, and YJ-N, Indiana University, and the Indiana University Foundation have received funds or donations to support their research or educational activities from: Alliance for Potato Research and Education; Almond Board; American Egg Board; Arnold Ventures; Eli Lilly and Company; Haas Avocado Board; Gordon and Betty Moore Foundation; Mars, Inc.; National Cattlemen’s Beef Association; USDA; and numerous other for-profit and non-profit organizations to support the work of the School of Public Health and the university more broadly. The authors report no other conflicts of interest in this communication.
1. Siraneh Y, Woldie M, Birhanu Z. Effectiveness of positive deviance approach to promote exclusive breastfeeding practice: a cluster randomized controlled trial. Risk Manag Healthc Policy. 2021;14:3483. doi:10.2147/RMHP.S324762
2. Wears RL. Advanced statistics: statistical methods for analyzing cluster and cluster-randomized data. Acad Emerg Med. 2002;9(4):330–341. doi:10.1197/aemj.9.4.330
3. Murray DM; Murray C of the D of EDM. Design and Analysis of Group-Randomized Trials. Oxford University Press; 1998.
4. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661. doi:10.1136/bmj.e5661
5. Teixeira da Silva JA. An error is an error… is an erratum: the ethics of not correcting errors in the science literature. Publ Res Q. 2016;32(3):220–226. doi:10.1007/s12109-016-9469-0
6. Barbour V, Kleinert S, Wager E, Yentis S. Guidelines for Retracting Articles. Committee on Publication Ethics; 2009; doi:10.24318/cope.2019.1.4
© 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.