A Randomized, Controlled Trial Comparing Tearcare^® and Cyclosporine Ophthalmic Emulsion for the Treatment of Dry Eye Disease (SAHARA) [Response To Letter]

Dear editor

We appreciate the interest and comments provided by Chaurasiya et al regarding our recent paper, “A Randomized, Controlled Trial Comparing Tearcare^® and Cyclosporine Ophthalmic Emulsion for the Treatment of Dry Eye Disease (SAHARA)”. Most of the comments seem to be criticism of our statistical design and analysis; these are addressed in the order raised below.

The first point concerns the hierarchical testing of the two primary endpoints, tear break-up time (TBUT), and Ocular Surface Disease Index (OSDI). Hierarchical testing using the fixed sequence method (ie, testing the second null hypothesis or endpoint is dependent on rejection of the first null hypothesis) is an accepted and standard method to control for multiplicity, the inflation of a Type 1 error rate due to multiple hypotheses.¹ Chaurasiya et al suggest that this methodology could “introduce bias and limit the interpretation of results”. While they do not suggest a specific alternative, they do cite a paper by Huque and Alosh describing a flexible fixed sequence method, the chief advantage being that it allows testing of an endpoint even when a previously tested endpoint was not statistically significant by taking into account correlation between the endpoints to minimize the impact of multiplicity adjustments.² We appreciate this suggestion, however if we had included this method a priori in our statistical analysis plan, there would be no difference in our reported results or conclusions for our primary outcomes. The first primary outcome variable tested, the difference between groups in TBUT, was in fact, highly significant (p=0.0006), and it is worth noting that the difference between groups for the second primary outcome, OSDI, did not approach statistical significance (p=0.9843).

The second point raises concerns about the adequateness of the sample size and the fact that the SAHARA sample size was based on TBUT and not also considering OSDI. This is a reasonable point although ICH E9 states that sample size is usually determined by the primary objective of the trial.³ As TBUT was our first primary endpoint, and we did not have a priori knowledge of the relative degree of improvement in OSDI for patients treated with TearCare or Restasis, it seemed (and remains so) a reasonable approach. Moreover, while the improvement in OSDI was numerically better for TearCare patients at the Month 6 endpoint, the small difference (2.35) and the large standard deviations (>17) suggest that, even if this difference was real, it would have required an impossibly large sample size to achieve statistical significance and would remain of questionable clinical significance.

The third point notes that our analysis for the primary endpoints utilized a linear mixed effects model for TBUT and analysis of covariance for OSDI. Chaurasiya et al apparently have no issue with these methods except to caution that there are underlying assumptions regarding the data that must be met.

Chaurasiya et al also question our exclusion of patients on “various topical and systemic medications” and that this could “limit the generalizability of the study findings to a broader population”. The concomitant medication exclusions in the SAHARA eligibility criteria were carefully thought out. As stated in our paper, these exclusions were not made “because they were contraindications to TearCare therapy but because use of these drugs could interfere with or confound response to cyclosporine”. Two points are important here. First exclusion of these medications generally favored the cyclosporine arm of the study. Second, a randomized controlled clinical trial (RCT) is designed to answer a specific question in a specific, well defined population, so that variability is minimized and the treatment effect of a therapy can be estimated with some precision; RCTs have strong intrinsic validity. Real world, “all-comers” studies are, in contrast, designed to show how a therapeutic performs in the broader population. Both designs provide important, but different, information.

Chaurasiya et al emphasize the importance of reporting safety results. We agree which is why we provided the adverse event, visual acuity, and intraocular pressure results. Finally, Chaurasiya et al suggest that the results for our secondary endpoints be interpreted cautiously. Of course, the results of any published study should be reviewed with a critical eye and in light of the strengths and weaknesses of the study design and execution. That said, we believe SAHARA to be an unusually robust RCT in the therapeutic area of dry eye disease and that the findings are credible and compelling. We do not agree with the assertion that there is a significant “gap” or flaw in the SAHARA study design.