Evidence-Based Medicine in Ophthalmic Journals During Covid-19 Pandemic
Received 27 August 2020
Accepted for publication 29 October 2020
Published 14 December 2020 Volume 2020:14 Pages 4369—4372
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 6
Editor who approved publication: Dr Scott Fraser
Eyal Walter,1 Ori Saban,1 Alan Jotkowitz,2 Tomer Batash,1 Jaime Levy1
1Department of Ophthalmology, Hadassah-Hebrew University Medical Center, Jerusalem, Israel; 2Department of Internal Medicine, Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer-Sheva, Israel
Correspondence: Jaime Levy
Department of Ophthalmology, Hadassah-Hebrew University Medical Center, Jerusalem, Israel
Email [email protected]
Background: COVID-19 has erupted into our lives and forced rapid changes in all fields of medicine, causing a rush for publications that inevitably caused a shift away from the paradigm of evidence-based medicine (EBM). The objective of the present report is to assess and quantify this process.
Methods: We compared the levels of EBM of the publications in the ophthalmic literature on COVID-19 at the beginning of the pandemic and compared it to those of articles published the prior year during April 2019 for the three highest ranking journals in the field of comprehensive ophthalmology.
Results: COVID-19 publications ranked significantly lower (p< 0.001). Time between submission and acceptance was significantly shorter for the COVID-19 publications (p< 0.001), and significantly more publications were accepted without revisions (P< 0.001).
Conclusion: Though a shift away from EBM may be unavoidable in the early stages of a pandemic, we suggest that for the benefit of reliable information and informed decision-making, it is time to go back to EBM.
Keywords: COVID-19, pandemic, evidence-based medicine
The suddenness, speed of spread and ferocity of the Coronavirus Disease 2019 (COVID-19) pandemic forced hurried global changes in virtually every field of life, but perhaps none more so than in the field of medicine. Being at the forefront of the battle against the pandemic, clinicians are the soldiers of this war, who struggle for every bit of information regarding this invisible enemy.
There has been an understandable need for the rapid processing and dissemination of information on this new disease entity. This need has inevitably caused us to shift away from the paradigm of evidence-based medicine (EBM). There has also been a rush to publish reports, bypassing or cutting corners with the usual peer review process, which has led even high-quality journals to have to retract important clinical studies that had already influenced medical practice.1
As a test case, we examined the possible shift from the EBM paradigm in the field of ophthalmology.
On April 27, 2020 we conducted a PubMed and Google Scholar search for all COVID-19-related peer-reviewed medical literature in the field of ophthalmology. We used the search string (“ophthalmology” or “ophthalmic” or “eye” or “ocular”) AND (“COVID-19” or “coronavirus” or “pandemic”).
The cutoff date of January 9th, 2020 was used as the date a new coronavirus disease was declared by the World Health Organization.2 We then manually excluded all publications that were not related to both COVID-19 and to the field of ophthalmology. The level of evidence of each publication was then evaluated and categorized using the widely accepted Center for Evidence-Based Medicine (CEBM) Levels of Evidence into five levels of evidence.3 Reviews and randomized-control trials were categorized as Level 1. Cohort studies were categorized as Level 2. Case control studies were categorized as Level 3, and case series were categorized as Level 4. Expert opinion (such as editorials and commentaries) were categorized as Level 5. Publications not fitting in any category such as websites were categorized into “not available” category. Each publication was independently evaluated and assigned a level of EBM by two investigators (EW and JL), and all discrepancies were settled by a third arbitrator (OS).
For each publication, the time (in days) was determined between first acceptance and publication, and whether any revisions were required prior to publishing the manuscript, as provided by the journal.
We then compared the publications on COVID-19 to data published in a comparative period the prior year. We took the April 2019 issues of the three highest ranking journals in the field of comprehensive ophthalmology, as ranked by Web of Science (Ophthalmology, JAMA Ophthalmology, and American Journal of Ophthalmology). The April 2019 issue was chosen as a comparison as it is exactly one year prior to the conclusion of our literature search. All publications in the April 2019 issues were graded in a similar manner described above (see supplementary table with all data in Appendix 1).
We conducted tests for normality using Kolmogorov–Smirnov and Shapiro–Wilk for publication type and, separately, for publication time. A non-normal data distribution was found for time to publication and publication type, and therefore we used Mann–Whitney non-parametric test to analyze the differences. A p-value < 0.05 was used to denote significance in all tests.
Our search yielded 58 publications, of which 45% were correspondences, 26% editorials, 10% scientific articles, 9% reviews, 5% brief reports, and 5% were miscellaneous. These 58 publications appeared in 27 peer reviewed journals, evenly distributed over impact factor quartiles. The publications saw light over an interval of 78 days (first publication was on February 9th, and last was on April 27th). The average time to publication was 6.8 days (median- 6 days), including time for revisions which were required in 67% of publications. Nearly 40% of the references of these articles were categorized as low-quality EBM (i.e.-websites and commentaries).
The April 2019 issues of the three leading journals in the field of comprehensive ophthalmology had a total of 104 publications, of which 47% were scientific articles, 30% correspondences, 8% brief reports, 3% editorials, 1% review articles, and 12% were miscellaneous. The average time to publication was 97 days (median 109 days), including time for revisions which were required in 90% of the publications. Lower quality EBM references (websites and commentaries) accounted for only 3.8% of all references.
Comparing the level of EBM of COVID-19 publications to that of publications in April 2019 using the Mann–Whitney non-parametric test, COVID-19 publications ranked significantly lower (p<0.001). Figure 1 illustrates the level of EBM of the total publications within the two time periods. Time between submission and acceptance was significantly shorter for the COVID-19 publications (p<0.001), and significantly more publications were accepted without revisions (P<0.001).
Figure 1 Level of evidence-based medicine of the total publications in 2019 and 2020 (in percentage of published articles).
Abbreviation: NA, not available.
This new, severe pandemic carries great uncertainty, which explains the rush to gain knowledge rapidly. Clinicians rely on current published literature to influence their practice, and there was very little known about COVID-19 when it erupted. Our report findings suggest that the majority of published data was of low level EBM, with a short time from acceptance to publication and often with no revisions required.
As clinicians, we strive to assimilate data which has been validated and is evidence based.
It is evident that none of the publications on the topic of COVID-19 is of the higher-ranking EBM levels. This is understandable, as higher levels of EBM require time and data, and the world is at a race to gather this data. However, it is important that clinicians be aware that building a body of evidence necessary for valid EBM takes time, and so the vast majority of publications in the beginning of any new phenomenon are inevitably expected to be of low level EBM.
Indeed, it would seem that competition amongst writers and journals to be the first to publish data regarding this emerging topic has lowered the standards that are imperative to keep the field of medicine evidence based. Even leading journals were part of the race to publish, which has resulted in a growing number of publications that careful review after publication found fundamental flaws that were sufficient to warrant retraction.4,5
Present report has some limitations. First, comparison was made to the three leading journals in the field of comprehensive ophthalmology, which might have skewed the results against the COVID-19 publications. Comparison to all journals was not feasible, so we decided to compare the COVID-19 publications to the three leading journals as this set a standard to which publications strive to achieve. However, we included all publications in these issues, irrelevant of the topic, which in our opinion may present a clearer picture of the field of ophthalmology in general.
Another limitation to the study is that all publications of the same EBM level were grouped, thereby disregarding the impact each publication had. A well published commentary surely had far greater impact than a poorly published review; however, we did not seek to find the impact of publications, but simply describe the level of EBM backing a certain publication. In that regard, a review that had passed peer review has a far better body of evidence to back it, compared to a commentary.
Our findings in the field of ophthalmology are not isolated, and there are several publications that have also addressed this shift away from EBM in other fields.6
Current models suggest COVID-19 will be around for a long time, and perhaps become an endemic disease with seasonal fluctuations, much like the flu.7 We therefore suggest that for the benefit of reliable information and informed decision-making, even if punctual cases of published letters may be of tremendous importance,8,9 it is time to get more reliable and actionable data10 and journal editors, despite the understandable rush to publish continue to rely on the time-tested principles of evidence-based medicine and comprehensive peer review.
Patient and Public Involvement Statement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
No financial support was received for the preparation of the article.
The authors declare they have no conflicts of interest.
1. Paliani U, Cardona A. COVID-19 and hydroxychloroquine: let the available data speak for themselves. Eur J Intern Med. 2020;80:121. doi:10.1016/j.ejim.2020.07.017
2. WHO Statement regarding cluster of pneumonia cases in Wuhan, China; 2020;Available from: https://www.who.int/china/news/detail/09-01-2020-who-statement-regarding-cluster-of-pneumonia-cases-in-wuhan-china.
4. Mehra MR, Desai SS, Kuy S, Henry TD, Patel AN. Retraction: cardiovascular disease, drug therapy, and mortality in Covid-19. N Engl J Med. doi:10.1056/NEJMoa2007621.
5. Retracted coronavirus (COVID-19) papers. Available from: https://retractionwatch.com/retracted-coronavirus-covid-19-papers/.
6. Pericàs JM, Arenas A, Torrallardona-Murphy O, Valero H, Nicolás D. Published evidence on COVID-19 in top-ranked journals: a descriptive study. Eur J Intern Med. 2020;79:120–122. doi:10.1016/j.ejim.2020.07.005
7. Kissler SM, Tedijanto C, Goldstein E, Grad YH, Lipsitch M. Projecting the transmission dynamics of SARS-CoV-2 through the postpandemic period. Science 2020;368(6493):860–868. doi:10.1126/science.abb5793
8. Watson J, Adler A, Amaravadi A, et al. Open letter to MR Mehra SS, Desai, F Ruschitzka, and AN Patel, authors of “Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis.”. Lancet. 2020.
9. Mehra MR, Desai SS, Ruschitzka F, Patel AN. RETRACTED: hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet. 2020. doi:10.1016/S0140-6736(20)31180-6.
10. Frieden TR, Drazen JM, Harrington DP, McMurray JJV, Ware JH, Woodcock J. Evidence for health decision making - beyond randomized, controlled trials. N Engl J Med. 2017;377(5):465–475. doi:10.1056/NEJMra1614394
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]