Back to Journals » Orthopedic Research and Reviews » Volume 11

Does Patient Blinding Influence Clinical Outcomes After Annular Closure Device Implantation? A Propensity Score-Matched Analysis

Authors Bouma GJ , van den Brink W , Miller LE , Wolfs JFC, Arts MP

Received 24 May 2019

Accepted for publication 25 October 2019

Published 8 November 2019 Volume 2019:11 Pages 177—182

DOI https://doi.org/10.2147/ORR.S216980

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Professor Clark Hung



Gerrit J Bouma,1 Wimar van den Brink,2 Larry E Miller,3 Jasper FC Wolfs,4 Mark P Arts5

1Department of Neurosurgery, OLVG and Amsterdam University Medical Centers, Amsterdam, The Netherlands; 2Neurochirurgisch Centrum Zwolle, Zwolle, The Netherlands; 3Miller Scientific Consulting, Asheville, NC, USA; 4Haaglanden Medical Center Westeinde Antoniushove, Leidschendam, The Netherlands; 5Department of Neurosurgery, Haaglanden Medical Center Westeinde, The Hague, The Netherlands

Correspondence: Gerrit J Bouma
Department of Neurosurgery, OLVG and Amsterdam University Medical Centers, P.O. Box 22700, Amsterdam 1100 DD, The Netherlands
Tel +31-20-5669111
Email [email protected]

Background: Awareness of treatment group assignment in a clinical trial may influence patient behavior and bias outcome reporting. The objective of this study was to compare 2-year clinical outcomes in blinded vs unblinded patients who were treated with lumbar discectomy and a bone-anchored annular closure device (ACD) for prevention of lumbar disc reherniation.
Methods: This was a secondary analysis of a randomized trial comparing lumbar discectomy with (n=272) vs without (n=278) implantation of a bone-anchored ACD. Among patients who received ACD implantation, 35 (13%) were blinded and 237 (87%) were unblinded to treatment allocation. In patients treated with ACD, propensity score-matching (1:1) was performed to account for imbalances in patient characteristics between blinded and unblinded groups. Key clinical outcomes were back pain severity (0–100 scale), leg pain severity (0–100 scale), Oswestry Disability Index (ODI, 0–100 scale), symptomatic reherniation, reoperation at the treated lumbar level, and device- or procedure-related serious adverse events (AEs). Outcomes were reported through 2 years of follow-up, which coincided with the time at which blinded patients were unblinded.
Results: There were no statistically significant differences in 2-year outcomes between propensity score-matched blinded (n=35) and unblinded (n=35) patients treated with the ACD. In blinded vs unblinded ACD patients compared to baseline, back pain severity decreased by 40 vs 37 points (P=0.61), leg pain severity decreased by 75 points in each group (P>0.99), and ODI decreased by 47 vs 43 points (P=0.19). The risks of symptomatic reherniation (5.7% vs 9.1%; P=0.59), reoperation (8.6% vs 12.2%, P=0.62), and device- or procedure-related serious AEs (5.7% vs 8.9%, P=0.63) were comparably low in blinded and unblinded patients.
Conclusion: In patients treated with lumbar discectomy and a bone-anchored ACD, there were no clinically important or statistically significant differences in back pain, leg pain, ODI, symptomatic reherniation, reoperation, or serious AEs over 2 years of follow-up when comparing patients who were blinded vs unblinded to their treatment assignment. The main limitations of this study were the post hoc nature of the analysis and the potential for bias due to surgeon awareness of treatment assignment.

Keywords: annulus fibrosus, blinding, disc, discectomy, elderly, herniation, lumbar, sciatica

Introduction

A randomized controlled trial (RCT) represents the most rigorous study design for determining the efficacy of a medical treatment. Results derived from a well-designed RCT have high internal validity since effective randomization affords protection from the influence of known and unknown confounding factors between treatment groups. Blinding of patients, physicians, and outcome assessors in an RCT provides an additional safeguard from bias. Participants in a clinical trial may behave differently if made aware of their assigned treatment, which could affect their response to therapy, compliance with concomitant interventions, and the decision to remain in the trial.1 Such performance and detection biases may confound study outcomes and interfere with data interpretation.

Blinding is particularly challenging to implement in RCTs of surgical treatments where obvious differences in cosmesis, imaging results, physical function, and postoperative care may inadvertently allow patients to determine which treatment was received with high probability.2 Further, ethics committees sometimes prohibit blinding of patients who receive surgical implants in a clinical trial. Recently, a large RCT was conducted to evaluate the efficacy of lumbar discectomy with or without a bone-anchored annular closure device (ACD) in patients at high-risk of lumbar disc reherniation.3 Through 2 years of follow-up, implantation of the ACD following discectomy was shown to significantly lower the risk of symptomatic reherniation and reoperation compared to patients treated with lumbar discectomy only.4 In this trial, patient blinding was implemented only at sites where permitted by ethics committees. Therefore, it remains unclear to what extent, if any, patient blinding status among study sites influenced these results. The objective of this secondary analysis from an RCT was to compare 2-year clinical outcomes of blinded vs unblinded patients who were treated with lumbar discectomy and a bone-anchored ACD for prevention of lumbar disc reherniation.

Methods

Trial Design

This was a multicenter RCT of lumbar discectomy with or without implantation of a bone-anchored ACD for prevention of lumbar disc reherniation (ClinicalTrials.gov NCT01283438) in patients at high risk for recurrence due to a large annular defect size.5 The protocol was approved by regional ethics committees (Supplement Table 1), all patients provided written informed consent before trial participation, and the study was conducted in accordance with the Declaration of Helsinki. The design3 and 2-year primary endpoint results4 of this trial have been reported previously. Patients enrolled at sites in The Netherlands were blinded to treatment allocation per guidelines set forth in a prospective protocol. Patients were not blinded to treatment at all other participating sites. This analysis focuses on the results from the trial that are relevant to addressing the question of whether patient blinding status may have influenced clinical outcomes in patients treated with a bone-anchored ACD.

Patients

Patients who were eligible to participate in this trial presented with symptomatic, single-level lumbar disc herniation that persisted despite at least 6 weeks of conservative management. Lumbar disc herniation was identified using MRI. Additional study imaging included low-dose computed tomography at the index level and anteroposterior/lateral and flexion/extension x-rays. Clinical corroboration of imaging findings was obtained by a positive straight leg raise or femoral stretch test. Symptom severity thresholds for study inclusion were leg pain severity at least 40/100 and Oswestry Disability Index (ODI) at least 40/100. Patients who met these presurgical criteria were then scheduled for lumbar discectomy.

Procedure And Randomization

Patients were treated on an inpatient basis with limited lumbar microdiscectomy using an interlaminar transflaval approach as described by Spengler.6 Once the discectomy procedure was completed, an interoperative eligibility criterion was applied in which eligible patients had a large (4–6 mm height, 6–10 mm width) postsurgical defect in the annulus fibrosus. Eligible patients were then randomly allocated, 1:1, to receive lumbar discectomy with no additional treatment or lumbar discectomy followed by an implantation procedure which includes a bone-anchored ACD.

Follow-Up And Outcomes

Patients returned for follow-up visits at 6 weeks, 3 months, 6 months, 1 year, and 2 years. Follow-up visits consisted of imaging (MRI, low-dose computed tomography at the index level, and anteroposterior/lateral and flexion/extension x-rays) and clinical evaluations. Key clinical outcomes were back pain severity, leg pain severity, ODI, symptomatic reherniation, reoperation at the treated lumbar level, and device- or procedure-related serious adverse events (AEs). Surgeons and clinical outcome assessors were not blinded to treatment allocation at any site. Safety oversight and AE adjudication were provided by an independent data safety monitoring board.

Propensity Score-Matching

We anticipated that blinded and unblinded patients in the ACD group may differ with respect to baseline characteristics. Using data from eight clinically relevant baseline covariates (age, sex, body mass index, current smoking status, back pain, leg pain, ODI, and treated level), we calculated propensity scores for each patient using binomial logistic regression. Propensity score-matching (1:1) was then performed using a nearest neighbor greedy match technique without replacement to adjust for covariate imbalances. The caliper width was set at 0.1 SDs of the logit of the propensity score.7 This technique matched each blinded patient to the unblinded patient with the closest propensity score provided the scores differed by no more than the caliper width.

Statistical Analysis

We verified the performance of propensity score-matching by comparing the distribution of covariates between blinded and unblinded groups before and after adjustment using the absolute standardized difference statistic, which was calculated as the difference in means or proportions among groups divided by the pooled SD. Absolute standardized difference values of 0.1, 0.2, and 0.5 indicate negligible, small, and moderate group differences, respectively.8 Continuous variables were reported as mean±SD, and categorical variables were presented as counts (percentages). Comparison of baseline characteristics was carried out with independent samples t-test or Fisher’s exact test in the unadjusted sample, and with dependent samples t-test or the McNemar test in the propensity score-matched sample. Patient-reported outcomes over a 2-year follow-up were compared with mixed-model ANOVA in which the baseline value served as a covariate. Time-to-event data were compared with log-rank tests and Cox proportional hazards regression, adjusted for paired samples among matched patients. Two-sided P-values of less than 0.05 were considered statistically significant. Statistical analyses were performed using Stata v14.2 (StataCorp).

Results

Among 276 patients from 21 sites where bone-anchored ACD implantation was attempted, 272 patients received a device and represent the sample of interest in the current study. A total of 237 patients were unblinded (including 3 cases of accidental unblinding) and 35 patients from four sites were blinded to their treatment assignment until the 2-year follow-up visit was completed.

Comparing blinded and unblinded patients, mild-to-moderate baseline differences were identified for male sex, leg pain severity, and treated level as evidenced by absolute standardized difference values larger than 0.2. Patient outcomes in the unadjusted sample are provided in Supplement Table 2. After propensity score-matching, patient characteristics between groups (35 blinded, 35 unblinded) were well balanced with negligible differences for all covariates except for a minor imbalance in the proportion of patients treated at L5/S1 (63% vs 74%; absolute standardized difference = 0.29, P=0.50). Comparisons of baseline patient characteristics in each group before and after propensity score-matching are provided in Table 1.

Table 1 Baseline Patient Characteristics In Unadjusted And Propensity Score-Matched Samplesa

Over 2 years of follow-up, implantation of the ACD following lumbar discectomy was shown to be safe, which effectively prevented reherniation in these high-risk patients. Further, no statistically significant differences in patient-reported outcomes were observed between blinded and unblinded patients treated with the ACD. Following treatment, back pain severity rapidly decreased, with a 33-point reduction in each group observed at the 6-week follow-up interval. At 2 years of follow-up, back pain severity compared to baseline decreased by 40 points in blinded patients and 37 points in unblinded patients (P=0.61) (Figure 1). Comparing blinded to unblinded patients, leg pain severity decreased by 73 points in each group at 6 weeks, and these improvements were maintained through 2 years (75 vs 75 points, P>0.99) (Figure 2). Similar trends were observed for ODI during follow-up, with reductions of 47 and 43 points (P=0.19), respectively, over 2 years (Figure 3). The risks of symptomatic reherniation (5.7% vs 9.1%; P=0.59) and reoperation (8.6% vs 12.2%, P=0.62) over 2 years were comparably low. The risk of a serious AE attributable to the device or procedure was similar in blinded and unblinded patients (5.7% vs 8.9%, P=0.63) (Table 2).

Table 2 Time-To-Event Outcomes Through 2 Years In Propensity Score-Matched Sample

Figure 1 Change in back pain severity over 2 years in blinded vs unblinded propensity score-matched patients treated with annular closure device. Notes: Plotted values are mean change and 95% CI with covariate adjustment for baseline value. P=0.61 for difference between groups.

Figure 2 Change in leg pain severity over 2 years in blinded vs unblinded propensity score-matched patients treated with annular closure device. Notes: Plotted values are mean change and 95% CI with covariate adjustment for baseline value. P>0.99 for difference between groups.

Figure 3 Change in ODI over 2 years in blinded vs unblinded propensity score-matched patients treated with annular closure device. Notes: Plotted values are mean change and 95% CI with covariate adjustment for baseline value. P=0.19 for difference between groups. Abbreviation: ODI, Oswestry Disability Index.

Discussion

In this analysis of patients treated with lumbar discectomy and ACD implantation for prevention of lumbar disc reherniation, there were no clinically important or statistically significant differences in back pain, leg pain, ODI, symptomatic reherniation, reoperation, or serious AEs over 2 years of follow-up when comparing blinded vs unblinded patients. These results were robust since they were observed in the unmatched sample as well as in the propensity score-matched sample that accounted for differences in baseline group characteristics.

The current study is one of the largest RCTs in spine surgery where patient blinding status was applied uniformly at each site but differed among sites. This presented a unique opportunity to investigate whether patient blinding status influenced clinical outcomes within the same trial. We are unaware of other studies in spine surgery with this study design element since most research on the influence of blinding has drawn comparisons between different studies, not within the same study.912 Each of these studies concluded that treatment benefits were superior in unblinded vs blinded patients. In the current study, we observed no such effect as clinically meaningful and comparable treatment benefits were derived regardless of blinding status. A likely explanation for this finding relates to the intended function of the ACD. That is, lumbar discectomy is used to treat the index lumbar disc herniation and is the sole reason for relief from sciatica symptoms. Implantation of the ACD is not intended to treat disc herniation and associated symptoms, but instead serves to reduce the risk of reherniation in patients at high-risk due to a large postsurgical annular defect.5 Therefore, expectation bias may have been inherently minimized in the current study since symptom improvement was expected from the lumbar discectomy surgery, which was provided to all patients. Further, the possibility that patient blinding status influenced reherniation and reoperation rates is unlikely since these outcomes were largely driven by objective imaging evidence of reherniation in combination with patient symptoms, physical examination findings, and diagnostic testing results. Ultimately, the results of the current study were not significantly influenced by patient blinding status.

The conclusions of this study were strengthened by using the largest known sample of patients treated with an ACD, utilization of propensity score-matching to control for differences in measured baseline patient characteristics between blinded and unblinded groups, and 2-year patient follow-up. There were also several limitations of this research that warrant further discussion. First, propensity score adjustment cannot account for unmeasured variables such as possible regional differences in health-care delivery practices. Such unmeasured variables may have influenced patient outcomes and, thus, are important sources of possible bias. Second, the blinded group included only 35 patients who were treated among four sites in the same country. A larger sample distributed across more sites in different regions would provide a more robust sample for identifying the impact of patient blinding. Third, surgeons and outcome assessors were not blinded in this study; therefore, the independent effect of these potential biases on patient outcomes is unclear. Finally, these results are not applicable to all patients undergoing lumbar discectomy, but only to approximately 30% of patients at high risk of reherniation due to a large postsurgical annular defect.5 The ACD is not indicated for use in patients with smaller defects due to the relatively low risk of reherniation.

Conclusion

In patients treated with lumbar discectomy and bone-anchored ACD, there were no clinically important or statistically significant differences in back pain, leg pain, ODI, symptomatic reherniation, reoperation, or serious AEs over 2 years of follow-up when comparing patients who were blinded vs unblinded to their treatment assignment.

Abbreviations

ACD, annular closure device; ODI, Oswestry Disability Index; RCT, randomized controlled trial.

Data Sharing Statement

All authors have agreed to the submission and publication of this manuscript. The authors confirm that requests for data underlying the findings described in this manuscript may be made to the corresponding author starting 1 year following publication of this article.

Acknowledgments

There were no contributors to the article other than the authors. No writing assistance regarding this study was received.

Author Contributions

All authors contributed to data analysis, drafting or revising the article, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.

Disclosure

Gerrit Bouma reports grants from Intrinsic Therapeutics, during the conduct of the study. Wimar van den Brink reports financial support for study conductance from Intrinsic Therapeutics. Larry Miller reports consultancy with Intrinsic Therapeutics. Jasper Wolfs reports nonfinancial support from Intrinsic Therapeutics; Consultancy activities Training and education from Zimmer Biomet, Safe Orthopaedics, Silony, and has a patent with EIT GMBH with royalties paid, outside the submitted work. Mart Arts reports consultancy with Intrinsic Therapeutics, Zimmer Biomet, EIT, Baxter and Silony, outside the submitted work, and has a patent with EIT GMBH with royalties paid. The authors report no other conflicts of interest in this work.

References

1. Miller LE, Stewart ME. The blind leading the blind: use and misuse of blinding in randomized controlled trials. Contemp Clin Trials. 2011;32:240–243. doi:10.1016/j.cct.2010.11.004

2. Farrokhyar F, Karanicolas PJ, Thoma A, et al. Randomized controlled trials of surgical interventions. Ann Surg. 2010;251:409–416. doi:10.1097/SLA.0b013e3181cf863d

3. Klassen PD, Hes R, Bouma GJ, et al. A multicenter, prospective, randomized study protocol to demonstrate the superiority of a bone-anchored prosthesis for annular closure used in conjunction with limited discectomy to limited discectomy alone for primary lumbar disc herniation. Int J Clin Trials. 2016;3:120–131. doi:10.18203/2349-3259.ijct20162794

4. Thome C, Klassen PD, Bouma GJ, et al. Annular closure in lumbar microdiskectomy for prevention of reherniation: a randomized clinical trial. Spine J. 2018;18:2278–2287. doi:10.1016/j.spinee.2018.05.003

5. Miller LE, McGirt MJ, Garfin SR, et al. Association of annular defect width after lumbar discectomy with risk of symptom recurrence and reoperation: systematic review and meta-analysis of comparative studies. Spine (Phila Pa 1976). 2018;43:E308–E315. doi:10.1097/BRS.0000000000002501

6. Spengler DM. Lumbar discectomy. Results with limited disc excision and selective foraminotomy. Spine (Phila Pa 1976). 1982;7:604–607. doi:10.1097/00007632-198211000-00015

7. Austin PC. Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm Stat. 2011;10:150–161. doi:10.1002/pst.v10.2

8. Normand ST, Landrum MB, Guadagnoli E, et al. Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores. J Clin Epidemiol. 2001;54:387–398. doi:10.1016/S0895-4356(00)00321-8

9. Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, et al. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. 2014;43:1272–1283. doi:10.1093/ije/dyu115

10. Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273:408–412. doi:10.1001/jama.1995.03520290060030

11. Savovic J, Jones H, Altman D, et al. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies. Health Technol Assess. 2012;16:1–82. doi:10.3310/hta16350

12. Savovic J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157:429–438. doi:10.7326/0003-4819-157-6-201209180-00537

Creative Commons License © 2019 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.