Exploring the characteristics, global distribution and reasons for retraction of published articles involving human research participants: a literature survey

Introduction

Article retraction is defined as the withdrawal of previously published articles, which is a measure taken by journals or authors if there is evidence of misconduct or error, redundancy, plagiarism or unethical research.¹ There has been an increase in the retraction of scientific publications in recent years.^2–4 For instance, one study showed that over the past 15 years, retractions due to fraud and error had increased by ~1.5 and six times, respectively.² Even after accounting for publication inflation, a dramatic rise in annual retraction counts persists.^3,4 This has raised profound concerns about the quality of scientific work and the social and ethical aspects of research.⁴

Major reasons for article retraction can be generally grouped into two categories in the literature: 1) scientific error, which includes mistakes in sampling, procedures, or data analysis, and accidental omission of information and 2) misconduct, which includes fraud and plagiarism.^5–8 There continues to be substantial debate about whether the increased frequency of retraction is a good or bad sign for biomedical research.⁹ Some argue that it could indicate increasing incidence of cases of scientific misconduct, while others claim that it could imply an improved system of measures against misconduct, resulting in more cases being identified.² One study performed a meta-analysis of several surveys that asked researchers whether they had committed or observed misconduct.¹⁰ It was found that 2% of scientists admitted having committed research misconduct and one in three scientists admitted having engaged in questionable research practices, ranging from simple carelessness to scientific fraud.¹⁰

It is not uncommon that article retractions are not well announced, resulting in the continued citation of retracted articles without acknowledgment of the retraction.¹¹ A blog called “Retraction Watch”, launched by two journalists Ivan Oransky and Adam Marcus in August 2010, examines and reports article retractions and advocates transparency in the retraction processes.¹¹ The blog writers report that article retractions are more frequent than previously thought.¹¹

Some characteristics have been reported on the retracted articles. For example, some studies have shown that retractions appear to be more common in journals with higher impact factors (IFs).^8,12,13 It was also found that a positive correlation existed between the male sex of the principal investigator(s) and the likelihood of article retraction.¹⁴ Other reported characteristics of article retraction included the lack of reported source of funding in the published paper.⁵

Despite the significant increase in the number of article retractions in health research, evidence for the characteristics of such phenomenon remains sparse in the literature. In this survey, we aimed to explore the factors associated with article retraction, the global distribution and the reasons for article retraction. We also evaluated how article retractions were handled by journals and how to improve the journal practices toward article retractions.

Methods

Search strategy

We searched PubMed database systematically by 1) using the descriptor “retract*”; 2) limiting results to human, English language, and article type (retracted Publication) and 3) limiting results reported from January 1980 to February 11, 2016. Table S1 lists the detailed search strategy.

Study selection and data extraction

The inclusion criteria comprised retracted publications involving biomedical research on human research participants and indexed by PubMed as “Retracted Publication”. Articles involving plant or animal research, research not directly related to health and articles reinstated after retraction were excluded. Study selection and data extraction were performed by two reviewers (MK and YJ) independently using a selection checklist and data extraction form developed for the purposes of this study. Disagreement was solved by discussion between the two reviewers, and if no consensus could be reached, a third reviewer (GL or MX) was consulted to make a final decision.

Data collected were grouped into three categories determined a priori: publication characteristics, journal characteristics and retraction characteristics. Publication characteristics included number of authors, sex of senior or corresponding author, geographic location, source of funding and conflict of interest reported. Journal characteristics included journal’s IF and eigenfactor score. Eigenfactor score is a measure of the influence of a journal on the scientific community based on the number and origin of incoming citations.¹⁵ Eigenfactor scores for all journals add up to 100, and a higher eigenfactor score indicates a journal’s prestige and importance.¹⁵ Retraction characteristics included reason for retraction, parties calling for retraction, whether or not the retracted article had an indication of the retraction and whether or not the retraction was reported on Retraction Watch. Reasons for retraction were classified into six categories: 1) misconduct, which included fabrication, falsification, fraud, plagiarism and lack of ethical approval; 2) error, which included mistakes in sampling, data analysis or procedures; 3) irreproducible results, which could be due to error or misconduct; 4) duplicate publication, which could be due to an error by the publisher or misconduct by the authors; 5) author disagreement, which included authorship concerns or lack of consent of all authors and 6) unknown, which included cases where the reason for retraction was not stated in the retraction notice and cases where the retraction notice was unavailable. Each retracted article identified from PubMed was manually searched to assess whether the retraction was reported on Retraction Watch.

Information was collected from the article summary pages on PubMed, the abstracts and the full-texts, where available and accessible. Sex of senior or corresponding author was determined through the authors’ names. If sex could not be inferred from the name, a search of the author’s name and affiliation was performed using Google Images. If sex still could not be determined, it was entered as “Unclear”, in order to avoid assumptions resulting in biased findings.

Statistical analysis

We used descriptive statistics including medians and interquartile ranges for continuous variables and counts and percentages for categorical variables to describe the characteristics of the retractions and the journals. IBM SPSS Statistics, Version 20.0 (IBM Corporation, Armonk, NY, USA) was used to perform statistical analyses. Ethics approval was not required for this study.

Results

Study identification

There were 2,310 records retrieved from PubMed and 13 records retrieved from retraction notices that involved multiple articles. Excluded records included articles involving plant or animal research (n = 443), research not directly related to health (n = 540) and articles reinstated after retraction (n = 1). A total of 1,339 studies were included for analyses in this survey (Figure 1).

Figure 1 Flow diagram showing the study selection.

Publication characteristics

For retracted articles with available and accessible author lists within our study sample, the maximum number of authors on a single article was 21 and the minimum number of authors was one (Figure 2). The median number of authors was 4 (interquartile range: 3–6). More than 66% of the retracted articles had male senior or corresponding authors (Table 1). Table 2 stratifies the reasons for article retractions by sex of the senior or corresponding authors. For all reasons of retraction, the percentage of retracted articles with male senior or corresponding authors was substantially higher than that with female senior or corresponding authors.

Figure 2 Number of retracted articles by number of authors (n = 1,339).

Table 1 Summary of the characteristics of all the retracted articles in the study sample (n = 1,339) Notes: aPercentages were not mutually exclusive since an article could be retracted by more than one of the listed parties. bA retraction policy was unknown due to the journal being discontinued or the journal website being in languages other than English.

Table 2 Reasons for retraction stratified by sex of senior or corresponding author or authors (n = 1,339)a Note: aResults are expressed as counts (percentages) by each reason of retraction.

In terms of geographic distribution, at the continent level, 36% of the retracted articles in our study sample consisted of research conducted in Asia, 32% consisted of research conducted in Europe and 26% consisted of research conducted in North America. At the country level, article retraction was most common in the USA (26%), Japan (11%) and Germany (10%). Of all the articles retracted due to misconduct, 19% were authored in the USA, 17% in Japan and 14% in Germany. Approximately 31% of articles retracted due to scientific error were authored in the USA, 8% were authored in China and 8% were authored in Japan.

Among the articles with information on funding available (n = 408), 48% (n = 197) were publicly funded, 17% (n = 71) were privately funded and 17% (n = 71) were funded through both public and private sources. As regards conflict of interests, there were 25% (n = 340) articles with data on conflicts of interest available, among which 10% (n = 34) declared a conflict of interest (Table 1).

Journal characteristics

A five-year journal IF was available for 1,228 retracted articles. The median IF was 2.94 (interquartile range: 1.81–4.69). Only 0.8% (n = 10) of the articles retracted were published in a journal with an eigenfactor score >1, such as Science, Proceedings of the National Academy of Sciences (PNAS), PLoS One and Nature. The journal PLoS One had the highest eigenfactor score of 1.53, followed by Nature (1.50), PNAS (1.42) and Science (1.22). Table 3 summarizes the journal metrics for the five journals with the highest frequencies of retracted articles. The highest number of retractions (n = 37) occurred in the journal Anesthesia and Analgesia that had a 5-year IF of 3.45 and an eigenfactor score of 0.031. All the five journals had a retraction policy.

Table 3 Characteristics of journals with the top five highest frequencies of retracted articles

Retraction characteristics

In terms of reasons of retractions, more than half (n = 685) of the articles in the study sample were retracted due to misconduct and 14% (n = 193) were retracted due to error (Table 1). For the parties calling for retractions, retraction was done by the journal editor(s) in most articles (59%, n = 794). There were 13% (n = 178) retracted articles published in journals with no retraction policy. For the journal practices toward retracted articles, only 14% (n = 181) of retracted articles were completely withdrawn (Table 1). Of the remaining articles that were still available, 34% (n = 395) had no indication of the retraction, 52% (n = 599) had a watermark indicating that they had been retracted, 8% (n = 91) had a note stating that they had been retracted and 5% (n = 63) had both a note and a watermark (Table 1). Large discrepancies in the announcement about article retraction were also observed. Some journals merely provided a retraction statement without any information on the reason or the party calling for retraction, while other journals included a detailed account of the retraction event and some journals even provided correspondence between author(s) and editorial staff.

There were 446 (33%) articles retracted before August 2010 (when Retraction Watch was launched). Of the articles retracted after August 2010, 63% (n = 567) retractions were reported on Retraction Watch (Table 1).

Discussion

In this literature survey, we examined the characteristics, geographic distribution and reasons for retraction of published articles involving human research participants. We provided data on article retractions from three different angles, including publication, journal and retraction characteristics.

We identified a large number of articles fulfilling our inclusion criteria of retracted articles. It was found that the median number of authors on the retracted publications was 4. The number of authors in published articles may reflect the complexity of biomedical research requiring multi-expertise and collaborative centers to increase sample size and improve generalizability of the study.^16,17 Having a large number of researchers may also mitigate scientific errors and omissions and result in better reporting of studies, which may therefore avoid future retraction of such papers.¹⁸ In this study, we have found that the majority of retracted articles had six authors or less (Figure 2). It is therefore an important consideration for peer reviewers, editors, publishers and funding agencies to ensure that researcher teams are composed of individuals with the expertise required to conduct, analyze, interpret and report research studies to avoid potential misrepresentations and inaccurate reporting of the findings.

In this study, we found that the majority of retracted articles were authored by men investigators as the senior or corresponding authors (Table 2). This finding may be explained by sex differences in risk-taking behaviors, including greater tendency for men for intellectual risk taking.¹⁹ It may also be because there are more men leading research groups and authoring studies than women in biomedical research, yielding the frequency of retraction being proportional to the number of articles published by sex.^20–23 Consistent with a previous report,⁵ we found most of retracted articles had no source of funding or conflict of interest available. Funding information may represent a source of bias in reporting and interpreting the study results. The desire to produce “significant” results may be stronger among researchers who had close relationship between industry, conducted research using their own funds or had personal investments in the results, among others.²⁴ The lack or incomplete reporting of conflict of interest or funding by authors may therefore be seen as an act of misconduct, resulting in a potential retraction of the article by the journal.²⁵ It is therefore essential that sources of funding and conflicts of interests should be clearly documented in published articles. Besides, findings from our study showed that countries with the highest frequencies of article retractions were the USA, Japan and Germany, which was in agreement with a previous study.²⁶ The high frequency of retracted papers in these countries is likely explained by the high volume of scientific publications in these countries. For example, the USA has the highest number of scientific publications worldwide.²⁷ The large number of retracted articles in a country may also reflect regulatory authorities’ practices and stringent reporting standards of research funded publicly. A misconduct by investigators using funding from the National Institutes of Health, for example, may result in severe penalties, including criminal charges and convictions.²⁸ Additionally, variation by geographic location may also represent accessibility of resources for publications in general and also for investigating misconduct leading to a retraction.

In terms of journal characteristics, all the top five journals with the highest number of retracted articles are specialty journals (four related to analgesia/anesthesia and one related to obstetrics and gynecology; Table 3). One previous study found that two of the top 15 authors with the highest number of retracted articles were in the field of anesthesia,²⁹ which may partly explain the findings of our study. Nevertheless, more evidence is needed to further explore the high number of retracted publications in analgesia/anesthesia.

Regarding retraction characteristics, the most common reasons for retraction were misconduct, followed by errors and duplication of publication, which was in line with other studies.^30–32 Misconduct can be seen in the conception and design of the study, the implementation of the study methods, the interpretation of the findings or the reporting of the study results in publications.⁷ Identification of misconduct is the collective responsibility of many individuals and agencies, including the investigators, the local institutions, the ethics boards, the funding agencies, the regulatory authorities, journal editors, publishers and peer reviewers. Despite the presence of many codes of conduct in the scientific field and the potential of harsh penalties for intentional misconduct, it is unfortunate that misconduct continues to occur in the scientific literature. More stringent practices should be put in place to ensure that actions involving misconduct are identified in a timely manner and handled appropriately.

For the journal practice toward retracted articles, we found that some articles were completely withdrawn, while some were still available with no indication of the retraction whatsoever (Table 1). We also observed large discrepancies in retraction notices issued by journals to announce article retraction (Table 1). Such a suboptimal practice toward retracted articles may result in the continued use and citation of invalid and unreliable research evidence. It is therefore important to make the retracted articles transparent, visible and clear to readers to avoid the use of such evidence. We recommend that journals adopt a standardized policy for handling and reporting article retractions in a consistent, transparent and informative manner. In addition, journals may consider setting up an honest-retraction or self-retraction system to allow authors to proactively withdraw their works when they discover serious genuine errors.^33,34 Such a system would also help readers easily know whether the retracted articles were due to either a genuine mistake or other deliberately deceptive actions.³⁴ A recent study by Bilbrey et al³⁵ developed a rating system for evaluating the quality of retraction notices. Based on this rating system and recommendations by the Committee on Publication Ethics (COPE),^35,36 we put together a list that outlines the important information about the retracted article that should be provided to readers through retraction notices (Box 1). For example, part of the key elements in a retraction notice should include the specific section(s) retracted and the effect of the retracted section(s) on the rest of the article. Such details would provide readers with a clear picture of the retracted article section(s) and their potential consequence on the whole study.

Box 1 Proposed elements of an informative retraction notice.

Study limitations

The study has some limitations. The major limitation for this literature survey was the lack of a control group, which thus did not allow for the comparison of the observed trends in retracted articles to those in non-retracted articles. Further research using a comparison group of non-retracted articles is needed to observe the comparative trends and to determine the potential factors associated with the likelihood of retractions in a regression model. Likewise, investigation in the role of other factors may be worth considering. For instance, comparing the number of senior or corresponding male authors in the retracted articles with the number of senior or corresponding male authors in general in science, providing statistics adjusting for the ratio of the number of publications relative to the number of inhabitants per country or assessing whether there is an effect of time on retracted articles in different countries or sex, journal IF and journal practices toward retracted publications may be further worthwhile endeavors. Another limitation is that we did not examine the status of globally retracted articles in different fields of medical specialization or by disease. Moreover, only research articles in English language were examined, which could lead to important trends not being accurately captured in the results. No correlation or regression analysis was conducted to explore the association between journal’s IF and the number of retracted articles, because the majority of journals only had one retracted article (n = 703, 57%). Furthermore, we only searched PubMed and probably missed articles that were not indexed in this database. We restricted the inclusion criteria to human studies, which limited the generalizability of our findings to other biomedical research areas.

Conclusion

Article retraction remains a profound issue in the biomedical community. The characteristics, geographic distribution and reasons for retraction of published articles involving human research participants were examined in this survey. More efforts are needed to improve the consistency and transparency of journal practices toward article retractions.

Acknowledgments

We would like to acknowledge Nancy Toma for her unconditional support on data abstraction. This study received no funding.

Author contributions

All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Glasziou P, Altman DG, Bossuyt P, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–276.
2.		Steen RG, Casadevall A, Fang FC. Why has the number of scientific retractions increased? PLoS One. 2013;8(7):e68397.
3.		Kali A, Srirangaraj S. Emerging trends in retraction of publications in PubMed-indexed medical journals. Ind J Pathol Microbiol. 2015;58(2):263–263.
4.		Steen RG. Retractions in the scientific literature: is the incidence of research fraud increasing? J Med Ethics. 2011;37(4):249–253.
5.		Nath SB, Marcus SC, Druss BG. Retractions in the research literature: misconduct or mistakes? Med J Aust. 2006;185:152.
6.		Fang FC, Casadevall A. Retracted science and the retraction index. Infect Immun. 2011;79(10):3855–3859.
7.		Kasenda B, Schandelmaier S, Sun X, et al. Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications. BMJ. 2014;349:g4539.
8.		Budd JM, Sievert M, Schultz TR. Phenomena of retraction: reasons for retraction and citations to the publications. JAMA. 1998;280(3):296–297.
9.		Fanelli D. Why growing retractions are (mostly) a good sign. PLoS Med. 2013;10(12):e1001563.
10.		Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One. 2009;4(5):e5738.
11.		Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360–363.
12.		Cokol M, Iossifov I, Rodriguez-Esteban R, Rzhetsky A. How many scientific papers should be retracted? EMBO Rep. 2007;8(5):422–423.
13.		Fang FC, Steen RG, Casadevall A. Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A. 2012;109:17028–17033.
14.		Fang FC, Bennett JW, Casadevall A. Males are overrepresented among life science researchers committing scientific misconduct. MBio. 2013;4(1):e640–e612.
15.		Hahn S, Williamson PR, Hutton JL. Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. J Eval Clin Pract. 2002;8(3):353–359.
16.		Regalado A. Multiauthor papers on the rise. Science. 1995; 268(5207):25–26.
17.		Christopher King. Multiauthor Papers: Onward and Upward; 2013. Available from: http://archive.annual-report.thomsonreuters.com/2012/_files/pdf/MultiauthorPapers_ChrisKing.pdf. Accessed September 2016.
18.		Nabout JC, Parreira MR, Teresa FB, et al. Publish (in a group) or perish (alone): the trend from single-to multi-authorship in biological papers. Scientometrics. 2015;102:357–364.
19.		Byrnes JP, Miller DC, Schafer WD. Gender differences in risk taking: a meta-analysis. Psychol Bull. 1999;125(3):367–383.
20.		Kaatz A, Vogelman PN, Carnes M. Are men more likely than women to commit scientific misconduct? Maybe, maybe not. Mbio. 2013;4(2):e156–e113.
21.		Neugebauer KM. Keeping tabs on the women: life scientists in Europe. PLoS Biol. 2006;4:e97.
22.		Head MG, Fitchett JR, Cooke MK, Wurie FB, Atun R. Differences in research funding for women scientists: a systematic comparison of UK investments in global infectious disease research during 1997–2010. BMJ Open. 2013;3(12):e003362.
23.		Ceci SJ, Williams WM. Understanding current causes of women’s underrepresentation in science. Proc Natl Acad Sci U S A. 2011;108(8):3157–3162.
24.		Martinson BC, Crain AL, Anderson MS, De Vries R. Institutions’ expectations for researchers’ self-funding, federal grant holding and private industry involvement: manifold drivers of self-interest and researcher behavior. Acad Med. 2009;84(11):1491–1499.
25.		Hirsch LJ. Conflicts of interest, authorship, and disclosures in industry-related scientific publications: the tort bar and editorial oversight of medical journals. Mayo Clin Proc. 2009;84(9):811–821.
26.		Rahman M, Fukui T. Biomedical publication—global profile and trend. Public Health. 2003;117(4):274–280.
27.		Mhaskar R, Djulbegovic B, Magazin A, Soares HP, Kumar A. Published methodological quality of randomized controlled trials does not reflect the actual quality assessed in protocols. J Clin Epidemiol. 2012;65(6):602–609.
28.		Seife C. Research misconduct identified by the US Food and Drug Administration: out of sight, out of mind, out of the peer-reviewed literature. JAMA Intern Med. 2015;175(4):567–577.
29.		Grieneisen ML, Zhang M. A comprehensive survey of retracted articles from the scholarly literature. PLoS One. 2012;7(10):e44118.
30.		Resnik DB, Wager E, Kissling GE. Retraction policies of top scientific journals ranked by impact factor. J Med Libr Assoc. 2015;103(3):136.
31.		Wager E, Williams P. Why and how do journals retract articles? An analysis of Medline retractions 1988–2008. J Med Ethics. 2011;37(9):567–570.
32.		Steen RG. Retractions in the scientific literature: do authors deliberately commit research fraud? J Med Ethics. 2011;37(2):113–117.
33.		Menezes RG, Ram P, Fatima H, Sathian B. Honesty is the best policy: a brief overview of retraction notices. Nepal J Epidemiol. 2016;6(4):603.
34.		Fanelli D. Set up a ‘self-retraction’ system for honest errors: notices should make obvious whether a withdrawal of research is the result of misconduct or a genuine mistake. Nature. 2016;531:415–416.
35.		Bilbrey E, O’Dell N, Creamer J. A novel rubric for rating the quality of retraction notices. Publications. 2014;2:14–26.
36.		Kleinert S. Cope’s retraction guidelines. Lancet. 2009;374(9705):1876–1877.

Supplementary material

Table S1 Search strategy on PubMed