Effect of Funding Source on “Spin” in Studies of Ocriplasmin Therapy for Vitreomacular Traction and Macular Hole
Authors Hubschman S, Venincasa MJ, Kuriyan AE, Sridhar J
Received 9 October 2019
Accepted for publication 28 November 2019
Published 13 January 2020 Volume 2020:14 Pages 81—88
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Scott Fraser
Sasha Hubschman, 1 Michael J Venincasa, 1 Ajay E Kuriyan, 2 Jayanth Sridhar 1
1Department of Ophthalmology, Bascom Palmer Eye Institute, Miller School of Medicine, University of Miami, Miami, FL, USA; 2Department of Ophthalmology, Flaum Eye Institute, University of Rochester Medical Center, Rochester, NY, USA
Correspondence: Jayanth Sridhar
Bascom Palmer Eye Institute, 900 NW 17 th Street, Miami, FL 33136, USA
Email [email protected]
Purpose: To examine the relationship between industry funding and “spin” in randomized controlled trials (RCTs) and meta-analyses investigating use of ocriplasmin for patients with vitreomacular traction (VMT) and macular hole (MH).
Methods: In this study, we examined all PubMed and Ovid MEDLINE RCTs and meta-analyses published in journals with impact factor ≥ 2 investigating effectiveness of ocriplasmin use for VMT and MH. The main outcome measure was correspondence between the studies’ main statistical outcome and their abstract conclusion wording. Each article was reviewed by three independent observers and was evaluated for source of funding, industry co-authorship, study methodology, statistical significance of main outcome measure, correspondence between results of main outcome measure and abstract conclusion, and journal impact factor. Funding was determined by public disclosure. Discrepancies were resolved by consensus.
Results: Twelve studies met inclusion criteria, of which 11 were industry funded and 1 was non-industry funded; 11 (91.67%) showed correspondence between outcome and abstract conclusion, without difference between industry-funded and non-industry funded publications or between publications in journals with high impact factor (≥ 3) versus low impact factor (≥ 2 and < 3).
Conclusion: In RCTs and meta-analyses of ocriplasmin for VMT and MH, our results suggest that neither industry funding nor journal impact factor affected the rate of “spin” in study conclusions. This study helps physicians understand what challenges they face when learning about a newer, less-established drug.
Keywords: ocriplasmin, microplasmin, vitreolysis, vitreomacular traction, macular hole, spin
Plain Language Summary
Pharmaceutical industry funding of medical research has been continuously growing over the last decades, which may increase the risk of bias and reporting of results in a pro-industry manner due to conflict of interest. Physicians do not always have time to review financial disclosures or possible bias when reading research articles. They often rely on the abstract portion of peer-reviewed articles, likely in scientific journals with high impact factor, to stay informed about new medications. The goal of this paper was to examine the relationship between industry funding and outcome reporting bias in RCTs and meta-analyses investigating use of ocriplasmin, a novel drug used for patients with vitreomacular traction and macular hole. In order to represent the articles that physicians are most likely to read, we evaluated all high qualities studies about ocriplasmin in journals with high impact factor, and assessed whether the articles’ main outcome results matched their abstract conclusion. Our results suggest that neither industry funding nor journal impact factor affected how study conclusions were “spinned”. This study helps physicians understand the challenges they face when learning about a newer, less-established drug.
Over the last few decades, the funding of medical research by pharmaceutical companies has dramatically increased.1,2 Although industry sponsorship of medical research has been found to improve methodologic quality,2 the potential conflict of interest may cause authors to inappropriately display their findings in a positive light and thus cause discordances between study results and conclusions.3 The practice of misleading readers by reporting results in a distorted manner so that findings are viewed more favorably is known as “spin”.4
Because of the large time limitation in daily practice and the overwhelming amount of new information available, many physicians rely on the abstracts of published research and assume that the concluding statements parallel the results of the study.5
Studies investigating the effect of these industry ties have yielded mixed results: many reports have shown that studies funded by private industries were more likely to present results in a way that was in the sponsor’s interest,1 while others found no link between industry support and reporting bias.3 Regardless, the simple possibility of bias requires that physicians carefully review the literature and not blindly trust article conclusions.
When Alasbali et al pioneered the evaluation of bias in the field of ophthalmology and found that industry-funded studies were more likely to positively portray study results compared to their non-industry funded counterpart, they analyzed meta-analyses and randomized controlled trials (RCTs) along with other study methodologies.1 A recent study by our group went further by examining the connection between industry funding and reporting bias in anti-vascular endothelial growth factor (anti-VEGF) studies, but only included RCTs and meta-analyses published in journals with impact factors of 2 or more.6
Our group’s previous study evaluated anti-VEGF therapies, which unlike ocriplasmin, are drugs that have been extensively studied and found to be extremely efficacious in the treatment of macular edema secondary to retinal vein occlusions.6 For a drug as therapeutically successful as anti-VEGF agents, it may be less likely to find discordance between study results and abstract conclusions.6 To the authors’ knowledge, there are no studies investigating potential spin in newer, less established drugs that physicians may not be familiar with.
The safety and efficacy of ocriplasmin (Jetrea, ThromboGenics, Leuven, Belgium), a treatment recently indicated for patients with symptomatic vitreomacular traction (VMT) and vitreomacular adhesion (VMA) including when associated with macular hole (MH), have been evaluated in multiple trials, but the vast majority of these trials were industry funded,7 which could have introduced bias in the results of these studies. Additionally, the use of ocriplasmin still raises some controversies and is the subject of many ongoing and planned clinical trials.8
The goal of this study was to examine the relationship between industry funding and outcome reporting bias in RCTs and meta-analyses investigating use of ocriplasmin for patients with VMT and MH, and assess what challenges physicians face when reading literature about a newer, controversial drug.
The methods in this study were previously described by our group.6 In this study, we examined published randomized clinical trials or meta-analyses (N=12). Institutional Review Board approval and informed consent were not obtained as there were no human subjects and research involved only analysis of available literature.
We completed PubMed and Ovid MEDLINE searches in April 2018 for studies that evaluated the efficacy of ocriplasmin for patients with VMT and MH (Figure 1). There were no cutoffs imposed on date of publication. Articles chosen were limited to English studies with an IF of 2.0 or more based on information provided by the journal website. In both PubMed and OVID, the search terms were: ocriplasmin AND (macular hole OR vitreomacular traction). The screening for study inclusion, removal of duplicates, and removal of studies because of study type or journal impact factor was performed by one of three independent observers (SH). If there was uncertainty in these steps, consensus was reached between all observers (SH, JS, AEK) on a case-by-case basis.
Figure 1 Selection of randomized clinical trials and meta-analyses investigating ocriplasmin use for patients with vitreomacular traction and macular holes.
SH, AEK, and JS each performed a full-text review and evaluation of included publications using a standardized data sheet detailing study methodology, statistical significance (p < 0.05) of the study’s main outcome measure, abstract conclusion, and correspondence between significance of main outcome measure result and abstract conclusion (Table 1). Study methodology was assessed with the same scoring scale used by Alasbali et al (Table 2).1
Table 1 Summary of Full-Text Study Assessments
Table 2 Criteria Utilized for Grading of Study Methodology
Correspondence between significance of main outcome measure result and abstract conclusion was assessed by surveying: 1) whether the abstract conclusion addressed all aspects of the main outcome measure, 2) whether all results relevant to the main outcome measure were summarized in the conclusion, and 3) whether the wording of the abstract conclusion matched the statistical analysis of the results. The identified main outcome measure was based on the authors’ stated main outcome measure. If there was more than one main outcome measure, we evaluated all outcome measures for correspondence between study results and conclusion. Discrepancies in assessed correspondence were settled based on discussion and unanimous agreement of the three observers, while discrepancies in assessed study methodology were settled based on a simple majority vote. One observer (SH) also collected objective data on each publication including sample size, source of funding (industry versus nonindustry), and whether the publication included an industry co-author. Funding status was based on written disclosure within the article. The main outcome measure was correspondence between the studies’ main statistical outcome and their abstract conclusion wording.
The original search yielded 258 publications between both databases, reduced to 153 after duplicates were removed. Of these, 75 publications were included for full-text review. At this point, sixty-three additional articles were excluded based on main outcome measure, study design, or lack of randomization, and 12 publications were ultimately included in our study (Figure 1). Of the 12 studies included, there were 11 RCTs and 1 meta-analysis.
Correspondence Between Main Outcome Measure and Abstract Conclusion
Statistically significant main outcome measures were present in 7 of 12 (58%) publications, and the wording of the abstract conclusion corresponded with these results in 11 of 12 (92%) publications. Non-correspondence in the Benz et al study was due to treatment superiority claims not supported by study design. Although the study’s abstract states that “patients receiving microplasmin were significantly more likely not to require vitrectomy surgery”, the secondary outcome measure of posterior vitreous detachment (PVD) progression at the highest tested dose of microplasmin was the only significant result.
All 12 publications included statements regarding industry-funding status. Eleven of 12 (92%) studies received industry funding – all sponsored by Thrombogenics – and 1 of 12 (8%) studies received no industry funding. There was an industry co-author in 8 (73%) of the 11 industry-funded studies. Study characteristics and reviewer assessments are summarized in Table 1.
Comparing Industry Funded versus Non-industry Funded Publications
Statistically significant main outcome measures were reported in 7 of 11 (64%) industry-funded publications and in 0 of 1 (0%) non-industry funded publications. Correspondence between abstract conclusion and significance of main outcome measure was present in 10 of 11 (91%) industry-funded publications and in 1 of 1 (100%) non-industry funded publications.
Comparing Higher-Impact versus Lower-Impact Publications
When publications were stratified by journal IF into a “high impact” group (N=11) with IF ≥ 3 and a “low impact” group (N=1) with IF ≥ 2 and <3, statistically significant main outcome measures were reported in 7 of 11 (64%) high-impact publications and in 0 of 1 (0%) low-impact publications. Correspondence between abstract conclusion and significance of main outcome measure was present in 10 of 11 (91%) high-impact publications and in 1 of 1 (100%) low-impact publication. Eleven of 11 (100%) high-impact publications received industry funding, while 0 of 1 (0%) low-impact publication did so.
As biomedical industry funding of biomedical research increases,1 many studies examining the industry’s influence across a wide range of medical specialties and drugs have found that there is significant bias in favor of biomedical companies in publications that are industry-funded.21 More germane to ophthalmology, Alasbali et al found that in industry-funded studies assessing the efficacy of topical prostaglandins for reducing intraocular pressure, the wording of abstracts in industry-funded studies was more likely to not correspond with statistical results.1 On the other hand, other studies found no correlation between industry funding and industry-favoring results, or “spin”, in the biomedical literature.2,3,6 Although only one of the papers in this study showed non-correspondence between abstract conclusion and main outcome measure, there still is an impact on the literature as the paper was cited by 20 PubMed central articles in which authors could be mislead and misrepresent the paper’s results.
Our sample size was limited by the relatively small number of RCTs and meta-analyses dedicated to measuring the efficacy of ocriplasmin. In 2012, ocriplasmin was approved by the Food and Drug Administration for the treatment of vitreomacular adhesions, making it the first injectable drug to treat VMAs.8 Although promising, the use of ocriplasmin is controversial.8 We still lack evidence for use of ocriplasmin in multiple clinical situations, and the reports of its deleterious effects may have deterred some ophthalmologists from pursuing further clinical trials.8
It is crucial for physicians whose priority is to care for patients to inform themselves about advances in their field. When trying to obtain knowledge about a novel, less established drug, it is very difficult for physicians to navigate all potential biases in the medical literature, from impact of industry funding to minimal amount of research about a controversial drug. Due to the lag between FDA approval of a drug and non-industry led clinical trials, most trials available about novel drugs are industry-funded, which explains the low number of non-industry-funded articles in this study. Therefore, physicians learning about a drug in its early life on the market have mostly industry-funded research available to them. Therefore, readers must be aware of these potential biases, and must be cautious about journal quality, research quality, and funding source in order to get the most accurate knowledge of the safety and efficacy of a new treatment.
The aforementioned main limitation of this study is the low number of articles included: 11 articles were industry-funded and found in high IF journals, and only 1 article was not industry-funded and from a lower IF journal. This limited sample size is also partly due to our search criteria, which excluded non-randomized controlled trials, case series, and cohort studies. However, even randomized controlled trials are not without bias. For example, a review of neurology RCTs found 180 inconsistencies in reporting of outcome across 180 RCTs, all of which were biased toward statistically significant results.22 Still, we chose these inclusion criteria because high-quality RCTs are considered to provide the best level of evidence, and meta-analyses play an enormous role in evidence-based medicine by limiting most biases.23
Additionally, we excluded all articles found in journals with impact factors below 2, and thus potentially excluded studies of ocriplasmin use from lower IF journals. Physicians may be more familiar with high impact factor journals, and thus we wanted to focus on manuscripts most likely to be read by physicians. Moreover, Mimouni et al found that impact factor may also be affected by bias, as journals of higher impact factor are more likely to publish articles with statistically significant results rather than “negative” results.24 In the present study, 64% of the studies published in high impact factor journals had significant main outcomes; however, comparisons with lower impact factor articles are limited given our low sample size (n=1).
Although neither funding nor impact factor appeared to affect “spin” in RCTs and meta-analyses of ocriplasmin use, this study helps physicians understand the challenges they face when learning about a newer, less-established drug. Future studies involving new drugs would benefit from expanding search criteria to include articles from lower IF journals, carrying out searches in additional databases (ie, Embase), and with methodologies other than solely RCTs and meta-analyses. This strategy may help increase sample size, and allow researchers to better evaluate whether impact factor or study methodology plays a role in “spin”.
This work was presented as a poster at the ARVO 2019 Annual Meeting and the ASRS 2019 Annual Meeting.
JS is a consultant for Alcon, Allergan PLC, and Alimera Sciences, Inc. JS reports personal fees from Alcon, Alimera Science, and Oxurion, outside the submitted work. AEK is a consultant for Regeneron; Alimera Sciences, Inc.; Valeant; and Allergan PLC. AEK reports grants and personal fees from Genentech, grants from Second Sight, personal fees from Allergan, Alimera Sciences, Regeneron, and Bausch Health, outside the submitted work. The authors report no other conflicts of interest in this work.
1. Alasbali T, Smith M, Geffen N, et al. Discrepancy between results and abstract conclusions in industry- vs nonindustry-funded studies comparing topical prostaglandins. Am J Ophthalmol. 2009;147(1):33–38 e32. doi:10.1016/j.ajo.2008.07.005
2. Brown A, Kraft D, Schmitz SM, et al. Association of industry sponsorship to published outcomes in gastrointestinal clinical research. Clin Gastroenterol Hepatol. 2006;4(12):1445–1451. doi:10.1016/j.cgh.2006.08.019
3. Chiu K, Grundy Q, Bero L. ‘Spin’ in published biomedical literature: a methodological systematic review. PLoS Biol. 2017;15(9):e2002173. doi:10.1371/journal.pbio.2002173
4. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058–2064. doi:10.1001/jama.2010.651
5. Barton S. Using clinical evidence. BMJ. 2001;322(7285):503–504. doi:10.1136/bmj.322.7285.503
6. Venincasa MJ, Kuriyan AE, Sridhar J. Effect of funding source on reporting bias in studies of intravitreal anti-vascular endothelial growth factor therapy for retinal vein occlusion. Acta Ophthalmol. 2018;97(2):e296–e302.
7. Tsui I, Pan CK, Rahimy E, Schwartz SD. Ocriplasmin for vitreoretinal diseases. J Biomed Biotechnol. 2012;2012:354979. doi:10.1155/2012/354979
8. Mec-Slomska AE, Adamiec-Mroczek J, Kuzmicz E, Misiuk-Hojlo M. Intravitreal ocriplasmin: a breakthrough in the treatment of vitreomacular traction? Adv Clin Exp Med. 2017;26(3):527–531. doi:10.17219/acem/62122
9. Lescrauwaet B, Duchateau L, Verstraeten T, Jackson TL. Visual Function Response to Ocriplasmin for the Treatment of Vitreomacular Traction and Macular Hole: The OASIS Study. Invest Ophthalmol Vis Sci. 2017;58(13):5842–5848.
10. Birch DG, Benz MS, Miller DM, et al. Evaluation Of Full-Field Electroretinogram Reduction After Ocriplasmin Treatment: Results of the OASIS Trial ERG Substudy. Retina. 2018;38(2):364–378.
11. Dugel PU, Tolentino M, Feiner L, Kozma P, Leroy A. Results of the 2-Year Ocriplasmin for Treatment for Symptomatic Vitreomacular Adhesion Including Macular Hole (OASIS) Randomized Trial. Ophthalmology. 2016;123(10):2232–2247.
12. Varma R, Haller JA, Kaiser PK. Improvement in Patient-Reported Visual Function After Ocriplasmin for Vitreomacular Adhesion: Results of the Microplasmin for Intravitreous Injection-Traction Release Without Surgical Treatment (MIVI-TRUST) Trials. JAMA Ophthalmol. 2015;133(9):997–1004.
13. Jackson TL, Verstraeten T, Duchateau L, Lescrauwaet B. Visual function response to ocriplasmin for the treatment of vitreomacular traction and macular hole. Acta Ophthalmol. 2017;95(8):e740–e745.
14. Dugel PU, Regillo C, Eliott D. Characterization of Anatomic and Visual Function Outcomes in Patients With Full-Thickness Macular Hole in Ocriplasmin Phase 3 Trials. Am J Ophthalmol. 2015;160(1):94–99 e1.
15. Gandorfer A, Benz MS, Haller JA, et al. Association between anatomical resolution and functional outcomes in the mivi-trust studies using ocriplasmin to treat symptomatic vitreomacular adhesion/vitreomacular traction, including when associated with macular hole. Retina. 2015;35(6):1151–1157.
16. Haller JA, Stalmans P, Benz MS,et al.Efficacy of intravitreal ocriplasmin for treatment of vitreomacular adhesion: subgroup analyses from two randomized trials. Ophthalmology. 2015;122(1):117–122.
17. Stalmans P, Delaey C, de Smet MD, van Dijkman E, Pakola S. Intravitreal injection of microplasmin for treatment of vitreomacular adhesion: results of a prospective, randomized, sham-controlled phase II trial (the MIVI-IIT trial). Retina. 2010;30(7):1122–1127.
18. Benz MS, Packo KH, Gonzalez V,et al. A placebo-controlled trial of microplasmin intravitreous injection to facilitate posterior vitreous detachment before vitrectomy. Ophthalmology. 2010;117(4):791–797.
19. Stalmans P, Benz MS, Gandorfer A, et al. Enzymatic vitreolysis with ocriplasmin for vitreomacular traction and macular holes. N Engl J Med. 2012;367(7):606–615.
20. Chatziralli I, Theodossiadis G, Xanthopoulou P, Miligkos M, Sivaprasad S, Theodossiadis P. Ocriplasmin use for vitreomacular traction and macular hole: A meta-analysis and comprehensive review on predictive factors for vitreous release and potential complications. Graefes Arch Clin Exp Ophthalmol. 2016;254(7):1247–1256.
21. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326(7400):1167–1170. doi:10.1136/bmj.326.7400.1167
22. Howard B, Scott JT, Blubaugh M, Roepke B, Scheckel C, Vassar M. Systematic review: outcome reporting bias is a problem in high impact factor neurology journals. PLoS One. 2017;12(7):e0180986. doi:10.1371/journal.pone.0180986
23. Haidich AB. Meta-analysis in medical research. Hippokratia. 2010;14(Suppl 1):29–37.
24. Mimouni M, Krauthammer M, Gershoni A, Mimouni F, Nesher R. Positive results bias and impact factor in ophthalmology. Curr Eye Res. 2015;40(8):858–861. doi:10.3109/02713683.2014.957777
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]