Back to Journals » Journal of Multidisciplinary Healthcare » Volume 16

Evaluation of Global Post-Outbreak COVID-19 Treatment Interventions: A Systematic Review and Bibliometric Analysis of Randomized Controlled Trials

Authors Alfaqeeh M , Zakiyah N , Suwantika AA , Shabrina Z

Received 8 November 2023

Accepted for publication 18 December 2023

Published 23 December 2023 Volume 2023:16 Pages 4193—4209

DOI https://doi.org/10.2147/JMDH.S448786

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Scott Fraser



Mohammed Alfaqeeh,1 Neily Zakiyah,1,2 Auliya A Suwantika,1– 3 Zahratu Shabrina4,5

1Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Bandung, Indonesia; 2Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Bandung, Indonesia; 3Center for Health Technology Assessment, Universitas Padjadjaran, Bandung, Indonesia; 4Department of Geography, King’s College London, London, UK; 5Regional Innovation, Graduate School, Universitas Padjadjaran, Bandung, Indonesia

Correspondence: Neily Zakiyah, Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Bandung, 45363, Indonesia, Tel +62 22 84288888, Ext. 3510, Email [email protected]

Purpose: The outbreak of COVID-19 has led to a global pandemic with millions of cases and deaths. Many randomized controlled trials (RCTs) were conducted to establish effective therapies. However, the methodological quality of these trials is paramount, as it directly impacts the reliability of results. This systematic review and bibliometric analysis aim to assess the methodological approach, execution diversity, global trends, and distribution of COVID-19 treatment RCTs post-outbreak, covering the period from the second wave and onward up to the present.
Methods: We utilize articles from three electronic databases published from September 1, 2020, to April 1, 2023. Inclusion and exclusion criteria were applied to identify relevant RCTs. Data extraction involved the collection of various study details. Risk of Bias (RoB) 2 tool assessed methodological quality, while implementation variability was evaluated against registration information. Bibliometric analysis, including keyword co-occurrence and country distribution, used VOSviewer and Tableau software.
Results: Initially, 501 studies were identified, but only 22 met the inclusion criteria, of which 19 had registration information. The methodological quality assessment revealed deficiencies in five main domains: randomization process (36%), deviations from intended interventions (9%), missing outcome data (4%), measurement of the outcome (18%), and selection of reported results (4%). An analysis of alignment between research protocols and registration data revealed common deviations in eight critical aspects. Bibliometric findings showcased global collaboration in COVID-19 treatment RCTs, with Iran and Brazil prominently contributing, while keyword co-occurrence analysis illuminated prominent research trends and terms in study titles and abstracts.
Conclusion: This study offers valuable insights into the evaluation of COVID-19 treatment RCTs. The scarcity of high-quality RCTs highlights the importance of enhancing trial rigor and transparency in global health emergencies.

Keywords: COVID-19, randomized controlled trials, methodological quality, diversity in execution, keyword trends, geographic distribution

Introduction

The outbreak of a novel and highly contagious COVID-19 disease, caused by the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2 virus), began in Wuhan, China in December 2019.1 The disease has put public health systems under pressure because of its rapid and intense transmission.2 SARS-CoV-2 symptoms can include fever, coughing, myalgia, lethargy, pneumonia, and SARS, which can be fatal.3,4 As of September 06, 2023, the total number of COVID-19 cases reported globally has exceeded 770.43 million, and over 6.95 million deaths worldwide.5 Despite the efforts that were taken such as practicing social distancing, staying at home, wearing masks, and regularly washing hands, the world is still actively working to combat the disease.6 However, the effectiveness of treatment interventions currently being used to treat COVID-19 has a high level of uncertainty as there is not enough clinical experience associated with these interventions.7 Thus, further research is needed to evaluate their potential efficacies against the virus.8

Randomized Controlled Trials (RCTs) are frequently regarded as the gold standard for assessing the efficacy of novel interventions or treatments.9 Several RCTs exploring different COVID-19 treatment interventions have been conducted.10 These include antiviral treatments designed to impede viral replication directly, recombinant neutralizing monoclonal antibodies aimed at preventing viral entry into host cells, supplementary therapies targeting the host immune response (such as anti-inflammatory), anti-fibrotic formulated to prevent angiotensin-converting enzyme 2 (ACE2) downregulation, thereby averting the establishment of a pro-fibrotic microenvironment as well as herbal and non-pharmacological interventions.11,12 Antiviral compounds like remdesivir, along with corticosteroids such as dexamethasone, have consistently demonstrated enhanced survival, while the interleukin 6 receptor antagonist tocilizumab and the Janus kinase 1/2 inhibitor baricitinib have also exhibited reduced mortality in COVID-19 patients.13 Moreover, anti-fibrotic medications like Pirfenidone and Nintedanib are emerging as potential treatment options for post-COVID-19 interstitial lung disease, particularly when administered early in the clinical course of the disease.14 By September 2023, the WHO platform had published more than 726,000 articles related to COVID-19, including 86,067 records of RCTs.15 ClinicalTrials.gov has registered 4315 clinical trials, including 2543 interventional studies.16 The COVID-19 pandemic has generated an unprecedented volume of data and knowledge.17 In response to this, certain studies have undertaken systematic reviews. These reviews aim to compare the impacts of different therapeutic interventions for COVID-19,18 or appraise the evidence concerning drugs employed in the treatment of COVID-19.19 However, there has been a scarcity of studies that systematically assess the methodological aspects of RCTs.20 While earlier studies have assessed the methodological aspects of RCTs during the early stages of the COVID-19 pandemic, our study offers a unique contribution by focusing on a more recent time frame. This timeframe encompasses the time from the onset of the second wave and extends to the present as of the writing of this review. This approach significantly augments our understanding of the evolving landscape of RCT methodology in the later phases of the COVID-19 pandemic. Therefore, it is important that we promptly collect, critically review, assess, and effectively implement this information to ensure generating unbiased results that lead to effective and efficient treatments.

To bridge this gap and provide a comprehensive overview, this paper applies a systematic review and bibliometric analysis of RCTs focused on COVID-19 treatment interventions post-outbreak. Our objectives are to assess the methodological quality of these RCTs, analyze the diversity in their execution, and create interactive visualizations for clustering keywords and mapping research distribution across countries. This would help advance the current knowledge on potential treatments and provide a guide for future research, thereby aiding healthcare systems by reducing hospitalizations and Intensive Care Unit admissions (ICU). This, in turn, ensures the allocation of essential resources to other patients’ needs.

Materials and Methods

Eligibility and Search Strategy

We adhere to the guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) during the process of extracting data and subsequent reporting.21 The search for relevant studies was conducted across multiple electronic databases, including Web of Science, PubMed, and ProQuest. Our search strategy involved the use of terms related to coronavirus and RCTs spanning from September 1, 2020, through April 1, 2023 covering the early stages of the COVID-19 pandemic up to the specified end date. We combined these terms using both “OR” and “AND” operators. Full details about searching strategy in each database are provided in Table S1.

Study Selection

We included all studies that met our following inclusion criteria: human participants diagnosed with COVID-19, any treatment intervention studies for COVID-19, with no restrictions on the type of control group, conducted between September 1, 2020, and April 1, 2023, utilizing RCT design, and available in the English language. The exclusion criteria comprised studies focusing on COVID-19 epidemiology, diagnosis, and prevention, as well as any study types other than RCTs (eg, cohort studies, case-control studies, observational studies, theoretical analyses, literature reviews, systematic reviews, or trial protocols), duplicate studies, and those with incomplete or insufficient information. The study selection process involved the use of Mendeley 1.19.8 software to organize collected records, initially eliminating duplicates through both automated and manual methods. Subsequently, we conducted a review of the titles and abstracts of the remaining studies to determine their compliance with the inclusion criteria. Finally, potentially relevant studies were read in full. The selection process involved two researchers (MA and NZ), with any disagreements being resolved through discussion involving all researchers (MA, NZ, AAS, and ZS).

Data Extraction

A digital data form was used to extract relevant information including the following title, interventions, duration of treatment, dose, author/s, country, registration number, ethic approval, level of severity, inclusion criteria, exclusion criteria, sample size, blinding, sites, funding, and date of release. To ensure efficient data management, Microsoft Office Excel (Microsoft Office 2016) was utilized as the tool for organizing and categorizing the extracted data.

Evaluating Diversity in Execution

The evaluation of execution diversity involved examining the congruence between reports of RCTs and their corresponding registration details. For this evaluation, only those studies with assigned registration numbers were considered, as their methodologies could be cross-checked against the initial registration data to establish coherence. A total of eight critical elements were subjected to comparison, encompassing inclusion and exclusion criteria, sample sizes, research outcomes, sites of recruitment, interventions, and procedures related to blinding.

Assessing the Methodological Quality

Evaluating the methodological rigor of the studies included in the analysis was carried out independently by two researchers (MA and NZ). The assessment was performed using the Risk of Bias (RoB) 2 tool, a revised version of the Cochrane risk-of-bias tool developed for randomized trials.22 The evaluation encompassed the following five domains: randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported results.23 Through consensus discussions among researchers (MA, NZ, AAS, and ZS), the domains were assessed and recorded as low risk, high risk, or some concerns.

Co-Occurrence of Keywords

We have conducted an analysis of keyword frequency to delineate the internal composition of our study sample. Keywords offer a suitable representation of the content within articles, and assessing the co-occurrence frequency of keywords is valuable for comprehending the conceptual framework and developments within a particular field.24 This involves quantifying the intensity of relationships among representative keywords in the selected documents, a phenomenon known as “co-occurrence”, which signifies when two keywords appear together within documents. Keyword co-occurrence analysis was conducted using VOSviewer software, version 1.6.19.25,26

Geographic Distribution of Studies

We analyzed the distribution of included studies among various countries using a visualization tool, Tableau (version 8.1.10).27 The visualization map generated through Tableau allows us to gain valuable insights into the geographic representation of the studies in our analysis. Additionally, it provides a comprehensive overview of the countries contributing to our research. This approach not only enhanced the understanding of the geographical diversity of the literature but also facilitated the identification of potential regional trends or disparities within the dataset.28

Data Analysis

RCTs that met the inclusion criteria and were available in the English language published from September 1, 2020, to April 1, 2023, were categorized by their interventions. For each intervention, dose, recruitment site, registration information, ethical approval, level of disease severity, sample size, blinding, and funding were documented. The date of the first online publication was defined as the “time of first release” for each study.

The analysis of RCTs also involved the examination of PICO information, encompassing participant categories, interventions, control groups, and findings. Patients diagnosed with confirmed COVID-19 were described by various groups, including mild, moderate, severe, mild to moderate, or unspecified. Interventions utilized in these RCTs were categorized into broader groups, such as antivirals, antibiotics, etc. Control groups were divided into interventional control and placebo control. Interventional control entailed a controlled group of patients who received treatments potentially effective for treating COVID-19, which included comparisons among different interventions or the same intervention with varying doses. Primary and secondary findings were additionally grouped into more comprehensive categories, encompassing clinical events, laboratory tests, vital signs, and other assessment scales like exercise intensity rating, fatigue, mental state, tolerability, patient satisfaction, and severity of dyspnea.

The assessment of the methodology in the included RCTs involved the utilization of RoB graphs and RoB summary figures. The presentation of consistency in comparisons is expressed through frequency and percentage. Descriptive statistical analysis was carried out utilizing Microsoft Office Excel. The creation of RoB graphs and RoB summary figures was accomplished using the Robvis tool.29 Graphs illustrating the co-occurrence of keywords and the distribution of studies among different countries were generated using VOSviewer and Tableau.

Results

A total of 501 studies were initially identified for review. After removing duplicate records, we screened the titles and abstracts of 466 studies. Subsequently, the full text of 27 studies was carefully reviewed. In the end, twenty-two studies were found to meet our inclusion criteria.30–52 A complete representation of this study selection process is shown in Figure 1.

Figure 1 PRISMA diagram of the study selection process of the randomized controlled trials (RCTs) included in the analysis for the treatment of COVID-19.

Notes: PRISMA figure adapted from Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. Creative Commons.21

Demographic and Clinical Characteristics of the Included Studies

The earliest included RCT was published on October 20, 2020,38 while the most recent study was published on February 17, 2023.30 Notably, more than half (12/22, 55%) of studies enrolled participants from a single location,30,32–35,37,39,40,42,43,47,48 with the highest number of recruitment sites being 56.52 All studies (22/22, 100%) were approved by ethical committees. The sample sizes across the studies varied considerably, ranging from as low as 3346 to as high as 1372.52 The largest proportion of studies (10/22, 46%) had sample sizes between 101 and 200,30,31,33,34,36–40,44 followed by studies (7/22, 32%) with sample sizes in the range of 33–100 participants.32,35,41–43,46,47 However, it is important to mention that only 8 (8/22, 36%) studies implemented blinding in their study design.31,32,34,42–44,50,52 The duration of treatment in the studies spanned from 1 to 14 days. The most common treatment duration observed in the studies was 10 days, which was employed in 7 (7/22, 32%).32,34,37,40,43,50,51 In terms of funding sources, 5 (5/22, 23%) studies received financial support from industry sources,32,35,41,42,50 while 8 studies (8/22, 36%) received grants from public organizations.31,33,34,36,38,44,46,47 Additionally, 3 (3/22, 14%) studies were funded by both private and non-private organizations,49,51,52 and 6 studies (6/22, 27%) reported no funding.30,37,39,40,43,48 The demographic and clinical characteristics of the included studies are summarized in Table 1.

Table 1 The Demographic and Clinical Characteristics of the Randomized Controlled Trials (RCTs) Included in the Analysis for the Treatment of COVID-19

PICO Characteristics

Among the studies included in our analysis, 59% (13/22) of the studies classified patients into various subgroups based on the severity of their condition, encompassing mild, mild to moderate, moderate, and severe cases. However, 9 (9/22, 41%) studies did not provide specific details about the severity level of the patients included in their research.33,36,38,41,43,44,47,49,51 Concerning the interventions utilized in these studies, three studies centered around antiviral medications,47,50,51 and three studies included the antimalarial Hydroxychloroquine as their sole intervention,46,48,52 with one of them combining it with the antibiotic Azithromycin.42 Additionally, two studies employed the anthelmintic Ivermectin in combination with antibiotics,40,44 two studies utilized the corticosteroid Methylprednisolone,41,43 and two studies employed the immunomodulatory agent Tocilizumab.30,38 Only one study focused on immunotherapy involving convalescent plasma.31 Furthermore, three studies incorporated supplementation strategies, including the use of vitamin C, vitamin D, and amino acid L-Arginine,32,39,49 while one study utilized the acid-oxidizing solution Sentinox.35 Additionally, three studies explored herbal medicine interventions, including Nigella sativa oil, Nilavembu Kudineer with Kaba Sura Kudineer, and Lianhua Qingwen,34,36,37 and one study investigated non-pharmacological interventions such as Qigong exercise and acupressure.33 In terms of control groups, the most frequently employed control was standard care therapy, which exhibited variations across studies, accounting for 50% (11/22).30,31,33,35,37–39,44,46,48,51 The interventional control was used in 23% (5/22) of the studies,40,41,43,47,49 which was the same proportion as those employing a placebo control (23%, 5/22).32,34,42,50,52 Approximately 82% (18/22) of the studies reported both primary and secondary outcomes, with additional details available in Table S2, more information about the most commonly outcome measures used are summarized in Table 2.

Table 2 Frequently Used Finding Measures in the Randomized Controlled Trials (RCTs) Included in the Analysis for the Treatment of COVID-19

Methodological Assessment of Included Studies

The assessment of bias in the included studies reveals varying levels of methodological rigor. In terms of randomization processes, a majority of the studies (14/22, 64%) exhibited a low risk of bias in their randomization processes. Nonetheless, certain studies raised some concern due to limited available information regarding the concealment of the allocation sequence.30,39,40,46,51 Furthermore, three studies were deemed to have a high risk of bias because the concealment of the allocation sequence was not implemented.35,38,41 Regarding deviations from intended interventions, the majority of studies (20/22, 91%) showed low risk of bias, suggesting that the interventions were carried out as planned. Nevertheless, two studies raised some concerns, due to the limited information regarding deviations from the intended intervention due to trial context in the study,40 and the intervention revision from what was initially assigned in the study protocol, which could potentially impact the outcome in the study.51 Regarding missing outcome data, nearly all studies (21/22, 96%) were characterized by a low risk. However, one study stood out due to incomplete outcome data caused by patient dropouts, resulting in an imbalance between the experimental and control groups, and this study was noted for having a high risk of bias.44 Most of the studies (18/22, 82%) exhibited a low risk of bias in their measurement of the outcome. However, some studies raised high risk due to differences in the measurement of the outcome between the intervention groups,44,51 and awareness of study participants’ interventions by outcome assessors, as well as the potential influence of knowledge about the intervention received on the assessment of the outcome.33,35 In terms of selection of the reported results, most of the studies (21/22, 96%) were characterized by a low risk. However, one study deviated from this pattern by failing to report one of the secondary outcomes in the results section. Consequently, this study was deemed to have a high risk of bias.43 The summary of findings indicates that among the 22 studies, 11 exhibited a low risk of bias, while 5 studies raised some concerns, and 6 studies were categorized as having a high risk of bias. The RoB graph for all 22 studies is provided in Figure 2, and the RoB summary covering all the studies is available in Figure S1.

Figure 2 Risk of bias assessment of the randomized controlled trials (RCTs) included in the analysis for the treatment of COVID-19.

Evaluating RCT Reports and Registration Data

In the analysis of the 22 studies included, registration information was available for 19 of them. These registrations were associated with the following registries: ClinicalTrials.gov (n = 10),30,35,37,38,41,46,48–52 Iranian Registry of Clinical Trials (n = 3),43,44,47 Clinical Trials Registry of India (n = 2),32,34 Chinese Clinical Trial Registry (n = 2),33,36 and Brazilian Registry of Clinical Trials (n = 1).42 Notably, one study yielded no results when searching for information using the provided registration number.30 Further, about 14% of studies (3/22) did not state whether they were registered.31,39,40 The most frequent inconsistency, observed in approximately 36% (5/14) of the studies, revolved around guideline modifications, including secondary outcomes. Following closely, the discrepancies between the published sample size and the intended target, coupled with reduced exclusion criteria. The third most prevalent disparity was linked to modifications made to the interventions, with further elaboration provided in Table 3.

Table 3 Comparison of Reports with Registration Information of the Randomized Controlled Trials (RCTs) Included in the Analysis for the Treatment of COVID-19

Key Trends in the Included Studies

When analyzing the most frequently utilized keywords within the included studies, as represented in Figure 3, it becomes evident that certain terms, such as “covid-19”, “randomized control trial”, “sars-cov-2”, “azithromycin”, “efficacy”, “hydroxychloroquine”, “clinical trial”, and “ivermectin”, appeared with larger fonts and circles, representing the high frequency of these keywords appearing in the titles and abstracts of the studies. Figure 3 further illustrates the presence of distinct clusters within the various topic areas. For instance, there is a distinct cluster in the red area that includes keywords like “azithromycin”, “ivermectin”, and “doxycycline”. This implies a deep relationship between them.

Figure 3 Keyword co-occurrence network of the randomized controlled trials (RCTs) included in the analysis for the treatment of COVID-19.

Distribution of Studies Among Countries

The studies were categorized based on the locations where they were conducted. The majority of studies originated from Iran,43,44,47 and Brazil,42,50,52 with each country accounting for three studies. This was followed by two studies each from China,33,36 India,32,34 Italy,35,38 and Pakistan.39,48 Additionally, individual studies were conducted in the United States,50 Canada,51 Ecuador,31 Egypt,30 France,49 Mexico,50 Saudi Arabia,37 Switzerland,41 Taiwan,46 and Bangladesh.40 A visual representation of the geographic distribution of the studies conducted in various countries is shown in Figure 4.

Figure 4 The global distribution of the randomized controlled trials (RCTs) included in the analysis for the treatment of COVID-19.

Discussion

Our systematic review and bibliometric analysis of RCTs centered on COVID-19 treatment interventions, which were conducted during the period beyond the initial outbreak. We assess the methodological quality of these trials, elucidating both their strengths and areas of concern. Furthermore, we explore the diversity in the execution of these RCTs, underscoring the vital importance of consistency between study reports and registration details to ensure scientific rigor. We also conduct keyword co-occurrence analysis to pinpoint essential COVID-19 treatment research trends and assess the geographical dispersion of these studies, emphasizing worldwide cooperation in the pursuit of effective COVID-19 treatments.

Assessing the methodological quality of the included RCTs is crucial for understanding the reliability of their findings. Previous research has highlighted the scarcity of well-structured RCTs in publications related to urgent situations. Xie et al noted that RCTs were the most common design choice in 51 pilot studies. However, only 10 of these studies had incorporated blinding into their research methodology.53 It is essential to recognize that RCTs characterized by suboptimal designs may not serve the best interests of patients or healthcare practitioners and can potentially raise significant ethical and practical concerns. Rather than allowing for exceptional cases, researchers should cooperate and strive to maintain the gold standard of RCTs, thereby making a valuable contribution to the ongoing development of the clinical evidence base for COVID-19.54 A notable example of this is the case of hydroxychloroquine, whose utilization surged following the rapid publication of an open-label, non-randomized, small-scale trial claiming its efficacy, only to be later demonstrated as no more effective than a placebo and consequently withdrawn based on a well-designed RCT with large sample size.55,56 This underscores the importance of raising the standard of research and registered clinical research programs while strictly adhering to the regulations for clinical trials.57–59

Considering the methodological assessment conducted in our study, it becomes apparent that there is a range of rigor in the design and execution of research within this field. Although most studies demonstrated a low risk of bias, some raised certain concerns or were categorized as having a high risk of bias in particular domains. These concerns primarily revolved around issues such as concealment of the allocation sequence, incomplete outcome data, deviations from the intended interventions, and discrepancies in reporting secondary outcomes. These findings highlight the importance of transparent reporting and adherence to predefined protocols in RCTs, as deviations from the original plan can introduce bias and affect the validity of study results. Further, blinding, an essential component of RCTs to minimize bias, was implemented in only a minority of the included studies. This raises questions about the potential impact of knowledge about the intervention on outcome assessments in non-blinded studies.6 The sample size represents another crucial aspect of research, as trials without adequate statistical power tend to miss significant effects. Studies conducted with limited sample sizes are predisposed to Type II errors.60 For instance, previous research highlights a prevalent issue where a considerable number of trials with negative outcomes are marred by insufficient participant numbers, leading to inaccurate conclusions and inefficient resource allocation in research.61,62 In our study, we observed that a majority of the included studies had relatively small sample sizes. This underscores the need for researchers to carefully consider and plan for appropriate sample sizes to ensure the reliability and validity of their findings.

The analysis of execution diversity within RCTs is essential for understanding the consistency between study reports and registration details. In our study, registration information was available for most included studies, and most were registered with well-recognized clinical trial registries. However, discrepancies between the published study reports and registration information were identified in a significant number of cases. These discrepancies in registration and reporting may raise concerns about transparency and rigor within the field of clinical research. We believe that obtaining reliable and high-quality evidence in the near future may be challenging. The primary reasons for the low quality of registered clinical trial protocols could include a lack of clinical research expertise among researchers and their limited experience in handling unexpected health events.63

The key trends identified within the included studies reveal a substantial focus on topics directly related to the COVID-19 pandemic. Unsurprisingly, terms such as “covid-19”, “sars-cov-2”, and “randomized clinical trial” prominently featured in the keywords, indicating the prevailing interest in understanding and addressing the novel coronavirus. These keyword trends offer valuable insights into the prevailing themes and priorities in COVID-19 treatment intervention research.64 Moving beyond thematic trends, the distribution of studies among different countries reveals a global effort to contribute to the understanding of COVID-19 treatment. Despite the fact that all countries are working to create successful treatment plans, countries with high incomes have stood out for their audacious efforts in this field of study due to their abundant financial resources, higher experience, and infrastructure.65 Our finding highlights Iran and Brazil, both middle-income nations, as prominent contributors to our study. This is supported by a study that revealed that low- to middle-income countries have published a smaller proportion of research related to SARS-CoV-2 testing, potentially reflecting their emphasis on research areas constrained by limited resources, such as treatment interventions.66

Improving upcoming RCTs requires robust support from diverse stakeholders such as public agencies, research academic institutions, medical companies, and other health partners. The main focus should be on strengthening research methods and developing adaptable clinical trial designs to improve the reliability of therapeutic evidence in rapidly changing healthcare crises.67 Trials with adaptive designs, which can be adjusted in terms of sample size and randomization based on evolving circumstances, hold promise for improving the efficiency of clinical trials, aligning with the complexities and uncertainties encountered in the context of the COVID-19 pandemic.68 Embracing a broader spectrum of research, including observational and interventional studies, in real-world settings is also crucial. Real-world research, which involves patients from medical institutions, households, and communities, offers greater flexibility and can reflect actual treatment processes.69

This study carries significant implications for various stakeholders. Researchers are reminded of the critical importance of methodological rigor and transparent reporting in COVID-19 treatment intervention studies,70 while healthcare practitioners benefit from a heightened awareness of potential biases and methodological issues when assessing specific treatments.71 Policymakers can utilize the study’s insights to target resources, advocate for transparency among researchers, and tailor interventions to regions with varying research trends.72 Future research priorities include addressing trial heterogeneity, exploring clinical significance, and fostering international collaboration. Additionally, transparent reporting and acknowledgment of limitations in public health communication can foster public trust and realistic expectations regarding COVID-19 treatments.73 Together, these implications advance evidence-based care and informed decision-making in the ongoing battle against the pandemic.

This study is not without limitations. Firstly, our research exclusively focused on the analysis of RCTs for the treatment of COVID-19. By limiting our scope to English-language publications, there may be a potential language bias, which could result in the exclusion of valuable research published in languages other than English. Furthermore, a more comprehensive understanding could be achieved by incorporating additional databases and utilizing free-text searches. The identification of inconsistencies between RCT reports and registration information highlights potential transparency and reporting bias issues, though this study does not delve into the reasons behind these discrepancies. Additionally, our study primarily emphasizes methodological aspects and conducts bibliometric analysis, offering a relatively limited discussion on the clinical significance of the interventions under examination. Lastly, we did not explicitly address or analyze the heterogeneity among the included studies.

Conclusion

In conclusion, as the COVID-19 pandemic continues to evolve, the need for high-quality research on treatment interventions remains essential. This study contributes to the understanding of the current landscape of COVID-19 treatment RCTs and identifies strengths and weaknesses in their methodologies through RoB analysis, underscoring the need for improved study design and execution. Additionally, the bibliometric analysis reveals trends and global collaboration in COVID-19 treatment studies, aiding researchers, and policymakers in prioritizing future investigations.

Data Sharing Statement

All the datasets generated during this review are provided within this manuscript.

Ethical Approval

This systematic review and bibliometric analysis incorporated only publicly accessible papers, and as no human subjects were part of the study, there was no necessity for ethics approval.

Funding

This study is supported by a grant from Universitas Padjadjaran (1549/UN6.3.1/PT.00/2023).

Disclosure

The authors declare no conflicts of interest.

References

1. Benita F. Human mobility behavior in COVID-19: a systematic literature review and bibliometric analysis. Sustain Cities Soc. 2021;70:102916. doi:10.1016/J.SCS.2021.102916

2. Utami AM, Rendrayani F, Khoiry QA, et al. Economic evaluation of COVID-19 vaccination: a systematic review. J Glob Health. 2023;13:6001. doi:10.7189/JOGH.13.06001

3. Cheng ZJ, Shan J. 2019 Novel coronavirus: where we are and what we know. Infection. 2020;48(2):155–163. doi:10.1007/S15010-020-01401-Y/TABLES/3

4. Guan WJ, Ni ZY, Hu Y, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. 2020;382(18):1708–1720. doi:10.1056/NEJMOA2002032/SUPPL_FILE/NEJMOA2002032_DISCLOSURES.PDF

5. World Health Organization. WHO Coronavirus (COVID-19) dashboard with vaccination data; 2023. Available from: https://covid19.who.int/. Accessed September 10, 2023.

6. Ang L, Song E, Lee MS. Randomized controlled trials of traditional, complementary, and integrative medicine-based interventions for coronavirus disease 2019 (COVID-19): a bibliometric analysis and review of study designs. Integr Med Res. 2021;10:100777. doi:10.1016/J.IMR.2021.100777

7. Qomara WF, Primanissa DN, Amalia SH, Purwadi FV, Zakiyah N. Effectiveness of remdesivir, lopinavir/ritonavir, and favipiravir for COVID-19 treatment: a systematic review. Int J Gen Med. 2021;14:8557–8571. doi:10.2147/IJGM.S332458

8. Lou Y, Liu L, Yao H, et al. clinical outcomes and plasma concentrations of baloxavir marboxil and favipiravir in COVID-19 patients: an exploratory randomized, controlled trial. Eur J Pharm Sci. 2021:157. doi:10.1016/J.EJPS.2020.105631

9. Hariton E, Locascio JJ. Randomised controlled trials—the gold standard for effectiveness research. BJOG. 2018;125(13):1716. doi:10.1111/1471-0528.15199

10. Fragkou PC, Belhadi D, Peiffer-Smadja N, et al. Review of trials currently testing treatment and prevention of COVID-19. Clin Microbiol Infect. 2020;26(8):988–998. doi:10.1016/J.CMI.2020.05.019

11. Murakami N, Hayden R, Hills T, et al. Therapeutic advances in COVID-19. Nat Rev Nephrol. 2022;19(1):38–52. doi:10.1038/s41581-022-00642-4

12. Nalbandian A, Sehgal K, Gupta A, et al. Post-acute COVID-19 syndrome. Nat Med. 2021;27(4):601–615. doi:10.1038/s41591-021-01283-z

13. Karampitsakos T, Papaioannou O, Tsiri P, et al. Tocilizumab versus baricitinib in hospitalized patients with severe COVID-19: an open label, randomized controlled trial. Clin Microbiol Infect. 2023;29(3):372–378. doi:10.1016/J.CMI.2022.10.015

14. Karampitsakos T, Sotiropoulou V, Katsaras M, et al. Post-COVID-19 interstitial lung disease: insights from a machine learning radiographic model. Front Med Lausanne. 2023:9. doi:10.3389/fmed.2022.1083264

15. World Health Organization. COVID-19 studies from the world health organization database; 2023. Available from: https://search.bvsalud.org/global-literature-on-novel-coronavirus-2019-ncov. Accessed September 10, 2023.

16. ClinicalTrials.gov. COVID-19 studies from ClinicalTrials.gov Database; 2023. Available from: https://clinicaltrials.gov/. Accessed September 10, 2023.

17. Carley S, Horner D, Body R, MacKway-Jones K. Evidence-based medicine and COVID-19: what to believe and when to change. Emer Med J. 2020;37(9):572–575. doi:10.1136/EMERMED-2020-210098

18. Siemieniuk RAC, Bartoszko JJ, Ge L, et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ. 2020:370. doi:10.1136/BMJ.M2980

19. Hernandez AV, Roman YM, Pasupuleti V, Barboza JJ, White CM. Hydroxychloroquine or chloroquine for treatment or prophylaxis of COVID-19: a living systematic review. Ann Intern Med. 2020;173(4):287–297. doi:10.7326/M20-2496/SUPPL_FILE/M20-2496_SUPPLEMENT.PDF

20. Kudhail K, Thompson J, Mathews V, Morrison B, Hemming K. Randomized controlled trials in patients with COVID-19: a systematic review and critical appraisal. Inter J Infect Dis. 2022;122:72–80. doi:10.1016/j.ijid.2022.05.034

21. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/BMJ.N71

22. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898. doi:10.1136/bmj.l4898

23. Risk of bias tools. Current version of RoB 2; 2019. Available from: https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool/current-version-of-rob-2?authuser=0. Accessed September 13, 2023.

24. Ding Y, Chowdhury GG, Foo S. Bibliometric cartography of information retrieval research by using co-word analysis. Inf Process Manag. 2001;37(6):817–842. doi:10.1016/S0306-4573(00)00051-0

25. van Eck NJ, Waltman L. Software survey: vOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84(2):523–538. doi:10.1007/s11192-009-0146-3

26. VOSviewer. Visualizing scientific landscapes; 2023. Available from: https://www.vosviewer.com/. Accessed September 13, 2023.

27. Tableau. Tableau Desktop 8.1.10; 2023. Available from: https://www.tableau.com/support/releases/desktop/8.1.10. Accessed September 13, 2023.

28. Dalavi AM, Gomes A, Javed Husain A. Bibliometric analysis of nature inspired optimization techniques. Comput Ind Eng. 2022;169:108161. doi:10.1016/J.CIE.2022.108161

29. McGuinness LA, Higgins JP. T. Risk-of-bias VISualization (robvis): an R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. 2021;12(1):55–61. doi:10.1002/jrsm.1411

30. Rabie ASI, Salah H, Said ASA, et al. Clinical consequences for individuals treated with tocilizumab for serious COVID-19 Infection. Healthcare. 2023;11(4). doi:10.3390/healthcare11040607

31. Baldeón ME, Maldonado A, Ochoa-Andrade M, et al. Effect of convalescent plasma as complementary treatment in patients with moderate COVID-19 infection. Transfus Med. 2022;32(2):153–161. doi:10.1111/tme.12851

32. Muralidharan J, Kashyap S, P S, et al. The effect of L-arginine supplementation on amelioration of oxygen support in severe COVID-19 pneumonia. Clin Nutr ESPEN. 2022;52:431–435. doi:10.1016/j.clnesp.2022.09.024

33. Liu ST, Zhan C, Ma YJ, et al. Effect of qigong exercise and acupressure rehabilitation program on pulmonary function and respiratory symptoms in patients hospitalized with severe COVID-19: a randomized controlled trial. Integr Med Res. 2021:10. doi:10.1016/j.imr.2021.100796

34. Srivastava A, Rengaraju M, Srivastava S, et al. Efficacy of two siddha polyherbal decoctions, Nilavembu Kudineer and Kaba Sura Kudineer, along with standard allopathy treatment in the management of mild to moderate symptomatic COVID-19 patients—a double-blind, placebo-controlled, clinical trial. Trials. 2021;22(1). doi:10.1186/s13063-021-05478-0

35. Panatto D, Orsi A, Bruzzone B, et al. Efficacy of the sentinox spray in reducing viral load in mild COVID-19 and its virucidal activity against other respiratory viruses: results of a randomized controlled trial and an in vitro study. Viruses. 2022;14(5):1033. doi:10.3390/v14051033

36. Zhang L, Wu L, Xu X, et al. Effectiveness of lianhua qingwen capsule in treatment of asymptomatic COVID-19 patients: a randomized, controlled multicenter trial. J Complement Integr Med. 2022;28(11):887–894. doi:10.1089/jicm.2021.0352

37. Koshak AE, Koshak EA, Mobeireek AF, et al. Nigella sativa for the treatment of COVID-19: an open-label randomized controlled clinical trial. Complement Ther Med. 2021:61. doi:10.1016/j.ctim.2021.102769

38. Salvarani C, Dolci G, Massari M, et al. Effect of tocilizumab vs standard care on clinical worsening in patients hospitalized with COVID-19 pneumonia: a randomized clinical trial. JAMA Intern Med. 2021;181(1):24–31. doi:10.1001/jamainternmed.2020.6615

39. Kumari P, Dembra S, Dembra P, et al. The role of vitamin c as adjuvant therapy in COVID-19. Cureus. 2020. doi:10.7759/cureus.11779

40. Chowdhury AT, Shahbaz M, Karim MR, Islam J, Dan G, He S. A comparative study on ivermectin-doxycycline and hydroxychloroquine-azithromycin therapy on COVID-19 patients. Eurasian J Med Oncol. 2021;5(1):63–70. doi:10.14744/ejmo.2021.16263

41. Welzel T, Atkinson A, Schöbi N, et al. Methylprednisolone versus intravenous immunoglobulins in children with paediatric inflammatory multisystem syndrome temporally associated with SARS-CoV-2 (PIMS-TS): an open-label, multicentre, randomised trial. Lancet Child Adolesc Health. 2023;7(4):238–248. doi:10.1016/S2352-4642(23)00020-2

42. Rodrigues C, Freitas-Santos RS, Levi JE, et al. Hydroxychloroquine plus azithromycin early treatment of mild COVID-19 in an outpatient setting: a randomized, double-blinded, placebo-controlled clinical trial evaluating viral clearance. Int J Antimicrob Agents. 2021;58(5):106428. doi:10.1016/j.ijantimicag.2021.106428

43. Ranjbar K, Moghadami M, Mirahmadizadeh A, et al. Methylprednisolone or dexamethasone, which one is superior corticosteroid in the treatment of hospitalized COVID-19 patients: a triple-blinded randomized controlled trial. BMC Infect Dis. 2021;21(1). doi:10.1186/s12879-021-06045-3

44. Heydari M, Rahimi J, Foroozanfar Z, et al. The efficacy of ivermectin and metronidazole vs. standard treatment protocols on outcomes of COVID-19 in hospitalized patients: a triple-blinded randomized controlled trial. Arch Clin Infect Dis. 2022;17(5). doi:10.5812/archcid-122525

45. Schwartz IS, Boulware DR, Lee TC. Hydroxychloroquine for COVID19: the curtains close on a comedy of errors. Lancet Regional Health. 2022;11. doi:10.1016/j.lana.2022.100268

46. Chen CP, Lin YC, Chen TC, et al. A multicenter, randomized, open-label, controlled trial to evaluate the efficacy and tolerability of hydroxychloroquine and a retrospective study in adult patients with mild to moderate coronavirus disease 2019 (COVID-19). PLoS One. 2020;15(12 December). doi:10.1371/journal.pone.0242763

47. Nojomi M, Yassin Z, Keyvani H, et al. Effect of Arbidol (Umifenovir) on COVID-19: a randomized controlled trial. BMC Infect Dis. 2020;20(1). doi:10.1186/s12879-020-05698-w

48. Kamran SM, Moeed HA, Mirza Z, et al. Clearing the fog: is hydroxychloroquine effective in reducing coronavirus disease-2019 progression? A randomized controlled trial. Cureus. 2021;30. doi:10.7759/cureus.14186

49. Annweiler C, Beaudenon M, Gautier J, et al. High-dose versus standard-dose vitamin D supplementation in older adults with COVID-19 (COVIT-TRIAL): a multicenter, open-label, randomized controlled superiority trial. PLoS Med. 2022;19(5):e1003999. doi:10.1371/journal.pmed.1003999

50. Golan Y, Campos JAS, Woolson R, et al. Favipiravir in patients with early mild-to-moderate coronavirus disease 2019 (COVID-19): a randomized controlled trial. Clin Infect Dis. 2023;76(3):E10–E17. doi:10.1093/cid/ciac712

51. Ali K, Azher T, Baqi M, et al. Remdesivir for the treatment of patients in hospital with COVID-19 in Canada: a randomized controlled trial. CMAJ. 2022;194(7):E242–E251. doi:10.1503/CMAJ.211698

52. Avezum Á, Oliveira GBF, Oliveira H, et al. Hydroxychloroquine versus placebo in the treatment of non-hospitalised patients with COVID-19 (COPE – coalition V): a double-blind, multicentre, randomised, controlled trial. Lancet Regional Health. 2022;11:100243. doi:10.1016/j.lana.2022.100243

53. Xie Y, Zhang P, Zhao H, Suyun L, Li J. Analysis of clinical research characteristics of clinical study on coronavirus disease 2019: based on inter- national clinical trial registration platform. J Traditional Chin Med. 2020. doi:10.13288/J.11-2166

54. London AJ, Kimmelman J. Against pandemic research exceptionalism. Science. 2020;368(6490):476–477. doi:10.1126/SCIENCE.ABC1731/ASSET/590F0371-2E1E-417F-BC49-828C4D4D4DF7/ASSETS/GRAPHIC/368_476_F1.JPEG

55. Horby P, Mafham M, Linsell L, et al. Effect of hydroxychloroquine in hospitalized patients with covid-19. N Engl J Med. 2020;383(21):2030–2040. doi:10.1056/NEJMOA2022926/SUPPL_FILE/NEJMOA2022926_DATA-SHARING.PDF

56. Saag MS. Misguided use of hydroxychloroquine for COVID-19: the infusion of politics into science. JAMA. 2020;324(21):2161–2162. doi:10.1001/JAMA.2020.22389

57. Chan AW, Tetzlaff JM, Gøtzsche PC, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013:346. doi:10.1136/BMJ.E7586

58. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340(7748):698–702. doi:10.1136/BMJ.C332

59. Li J, Hu JY, Zhai JB, et al. CONSORT extension for reporting N-of-1 trials for traditional Chinese medicine (CENT for TCM): recommendations, explanation and elaboration. Complement Ther Med. 2019;46:180–188. doi:10.1016/J.CTIM.2019.08.014

60. Olivo SA, Macedo LG, Gadotti IC, Fuentes J, Stanton T, Magee DJ. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008;88(2):156–175. doi:10.2522/PTJ.20070147

61. Freiman JA, Chalmers TC, Smith HA, Kuebler RR. The importance of beta, the type II error, and sample size in the design and interpretation of the randomized controlled trial: survey of two sets of “negative” trials. Medical Uses Statistics. 2019;357–389. doi:10.1201/9780429187445-19

62. Moher D, Wells GA, Dulberg CS. Statistical power, sample size, and their reporting in randomized controlled trials. JAMA. 1994;272(2):122–124. doi:10.1001/JAMA.1994.03520020048013

63. Zhu RF, Gao YL, Robert SH, et al. Systematic review of the registered clinical trials for coronavirus disease 2019 (COVID-19). J Transl Med. 2020;18(1):274. doi:10.1186/s12967-020-02442-5

64. Mubaroq SR, Abdullah AG, Setiawan A. The evolution of smart working and sustainability in socio-technical perspective: a scientometrics technology analysis. J Eng Sci Technol. 2020;15:1868–1882.

65. CNN. Covid-19 vaccine trial on humans starts as UK warns restrictions could stay in place until next year | CNN; 2020. Available from: https://edition.cnn.com/2020/04/23/health/coronavirus-vaccine-trial-uk-gbr-intl/index.html. Accessed September 20, 2023.

66. Tran BX, Ha GH, Nguyen LH, et al. Studies of novel coronavirus disease 19 (COVID-19) pandemic: a global analysis of literature. Int J Environ Res Public Health. 2020;17(11):4095. doi:10.3390/ijerph17114095

67. Zhao MZ, Zhao C, Tu S, Wei XX, Shang HC. Evaluating the methodology of studies conducted during the global COVID-19 pandemic: a systematic review of randomized controlled trials. J Integr Med. 2021;19(4):317–326. doi:10.1016/J.JOIM.2021.03.003

68. Zuidgeest MGP, Goetz I, Groenwold RH, Irving E, van Thiel GJ, Grobbee DE. Series: pragmatic trials and real world evidence: paper 1. Introduction. J Clin Epidemiol. 2017;88:7–13. doi:10.1016/J.JCLINEPI.2016.12.023

69. Head BW. Reconsidering evidence-based policy: key issues and challenges. Policy Soc. 2010;29(2):77–94. doi:10.1016/J.POLSOC.2010.03.001

70. Alexander PE, Debono VB, Mammen MJ, et al. COVID-19 coronavirus research has overall low methodological quality thus far: case in point for chloroquine/hydroxychloroquine. J Clin Epidemiol. 2020;123:120–126. doi:10.1016/J.JCLINEPI.2020.04.016

71. Hiebert R, Nordin M. Methodological aspects of outcomes research. Eur Spine J. 2006;15(Suppl 1):S4. doi:10.1007/S00586-005-1057-5

72. Garrison LP, Neumann PJ, Erickson P, Marshall D, Mullins CD. using real-world data for coverage and payment decisions: the ISPOR real-world data task force report. Value Health. 2007;10(5):326–335. doi:10.1111/J.1524-4733.2007.00186.X

73. Porat T, Nyrup R, Calvo RA, Paudyal P, Ford E. Public health and risk communication during COVID-19-enhancing psychological needs to promote sustainable behaviour change In review Conflict of interest statement; 2020. Available from: www.frontiersin.org. Accessed September 30, 2023.

Creative Commons License © 2023 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.