Reliability of Time Domain AS-OCT in Measuring the Extraocular Rectus Muscle Insertion-Limbus Distances: A Systematic Review and Meta-Analysis

Introduction

The insertion-limbus distances of the extraocular rectus are important for the preoperative planning of strabismus surgeries.^1,2 Obtaining accurate preoperative measurements of the extraocular muscle (EOM) insertion-limbus distances can decrease intraoperative manipulations and reduce the risk of complications (ie, prolonged time on general anesthesia and bleeding) for patients with strabismus, especially those undergoing reoperations and information regarding their insertion-limbus distances is not readily available to the surgeon.^3,4 Obtaining preoperative measurements of insertion-limbus distances is difficult and therefore is most reliably determined through intraoperative exploration using calipers.⁵ A diagnostic test to determine insertion-limbus distances preoperatively would be invaluable.⁶

Prior studies have shown that time-domain anterior segment optical coherence tomography (TD AS-OCT) as well as other imaging modalities such as ultrasound biomicroscopy can be used to measure the insertion-limbus distances of the EOM.^3,7,8 Obtaining measurements with AS-OCT is advantageous because minimal contact is required and the technology is routinely available during office visits.^5,7 At the moment, the most common TD OCT systems made specifically for anterior segment imaging include Visante (Carl Zeiss Meditec, USA).^9,10 Although TD AS-OCT has a lower resolution than other imaging modalities such as spectral domain AS-OCT, it is still of interest to investigate whether TD AS-OCT can be used as a reliable pre-operative measurement tool for strabismus surgeries given that it has a deeper tissue penetrance and is still used in a variety of ophthalmic settings.⁶

Therefore, in this systematic review and meta-analysis, we assess the reliability of TD AS-OCT in determining insertion-limbus distances preoperatively by investigating prior studies that compare preoperative TD AS-OCT measurements of the insertion-limbus distances to those of intraoperative calipers, which are assumed to be the gold standard. Firstly, we aim to describe and compare the methodologies of these included studies regarding the preoperative TD AS-OCT and intraoperative caliper measurements. Furthermore, we aim to describe in detail how data (ie, success rates in locating muscles, reliability coefficients, etc.) were reported among these studies and any differences in reporting. Another aim of this study was to conduct a meta-analysis of reliability among these studies as well as to ascertain whether the reliability of TD AS-OCT differed based on whether the rectus muscle had undergone prior operation.

Materials and Methods

Literature Search and Study Selection Strategy

We searched the following databases: EMBASE, PubMed, Google Scholar, Science Direct, and Web of Science. We also searched for references of full texts for eligible articles. There were no restrictions on dates. The population of interest included individuals who had both TD AS-OCT measurements conducted preoperatively and then caliper measurement conducted intraoperatively. TD AS-OCT was the intervention, and intraoperative calipers were the comparator. The primary outcome of interest in the present study was measures of reliability such as intraclass and Pearson’s correlation coefficients. A full list of search terms used for each database is available (Supplement 1). Studies that employ both TD AS-OCT preoperatively to measure the insertion-limbus distances of the medial, inferior, lateral, and/or superior rectus (MR, IR, LR, SR) and calipers intraoperatively as a comparator were included. Studies that do not include either TD AS-OCT or intraoperative calipers are not available in English, analyze or report previously published data, and abstracts, conference proceedings, narrative reviews, and studies with no patient data were excluded.

EK and AG independently screened the titles and abstracts of potential studies for full-text screenings and assessed the eligibility of full texts for further synthesis. Studies could only proceed to the subsequent step of the literature screening if both reviewers concurred on a study’s eligibility. Any conflicts regarding eligibility during either the abstract or full-text screening were resolved through meetings and discussions until a consensus was reached.

Data Collection and Reporting

Both EK and AG also independently gathered data from each study using Covidence (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia). Any discrepancies in the collected data were discussed until consensus was reached. Data and outcomes that were extracted from the studies included the following: general characteristics of the studies, methodology of preoperative TD AS-OCT measurements (ie, degree of gaze for image acquisition, calculation of the location of the limbus, etc.), TD AS-OCT machine utilized, and methodology of intraoperative caliper measurements. We also investigated the reporting methods of included articles and, if applicable, whether an article reported image success rates for primary muscles versus previously operated muscles. We also collected the following data for the MR, LR, SR, and IR: sample size (number of muscles), mean and standard deviation of TD AS-OCT measurements, mean and standard deviation of the caliper measurements, and any calculations of reliability. We followed the 2020 Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guidelines.¹¹

Risk of Bias Assessment

To assess the risk of bias for individual studies, the reviewers utilized the ROBINS-I tool, a Cochrane’s collaboration tool used to assess bias in studies that are not randomized.¹² In order to categorize each study as having low, moderate, or high risk of bias, different domains were assessed, including but not limited to confounding, selection, information, and reporting bias. Both reviewers assigned each study a risk of bias rating, and if any discrepancies were present, the reviewers would convene to reach a consensus.

Meta-Analysis

It was of interest to compare the means and variances of the TD AS-OCT and caliper measurements, as well as their correlations (ie, intraclass coefficient and Pearson’s correlation coefficient). Statistically significant differences in means between TD AS-OCT and intraoperative calipers would indicate that TD AS-OCT is not a reliable measurement tool, while differences in variances would indicate differences in precision between TD AS-OCT and intraoperative calipers. The correlation between the two would estimate parallel forms reliability.¹³

Six studies met the inclusion criteria for meta-analysis, however there were some complications with missing data. Two studies reported all the statistics we sought to extract.^14,15 One study reported the means and variances but only reported their association from intraclass correlation coefficient rather than the Pearson cross-product correlation coefficient.³ One study reported intraclass coefficient instead of the Pearson cross-product correlation coefficient and did not report means and variances.¹⁶ Another study provided both the Pearson cross-product correlation coefficient and intraclass correlation coefficient but did not provide means and variances.⁴ Fortunately, however, these three studies provided charts of raw data from which individual data points could be manually extracted, and statistics of interest estimated.^3,4,16 Lastly, there was one study for which statistics were only reported on individual sub measurements (ie, MR, SR, IR, and LR), rather than the “combined” measurement.⁶ The properties of expectations and variances were used to infer the statistics of interest.¹⁷ One study was excluded from the meta-analysis because it did not provide any correlations.¹⁸ The approach to our analysis and derivations used for missing data for means, variances, and Pearson correlation coefficient can be found in Supplement 2.

Results

Selection of Studies

A total of seven studies fit our inclusion criteria for qualitative analysis.^{3,4,6,14–16,18} The details of the literature screening process can be seen in a PRISMA flowchart (Figure 1).¹¹ No additional articles were found to be eligible by searching the reference lists of included articles.

Figure 1 The initial search of five databases yielded a total of 1674 results after 410 duplicates were removed by Covidence. After abstract/title screenings, 63 full texts were assessed for eligibility. Of the 63 full texts, 7 were eligible for qualitative synthesis, and 6 were eligible for meta-analysis.11

General Characteristics of Included Studies and Patients

Table 1 provides a summary of the general characteristics of the seven included studies, including their risk of bias rating. Patient sample sizes ranged from 16 to 74.^{3,4,6,14–16,18} All studies provided, in some form, a comparison of TD AS-OCT measurements and intraoperative caliper measurements. Five of the included articles included patients undergoing primary strabismus surgery as well as patients undergoing reoperations.^3,4,6,16,18 One article included only patients undergoing primary strabismus surgery of the horizontal rectus muscles.¹⁵ The final study included primary angle-closure glaucoma patients, measuring their SR insertion-limbus distances with TD AS-OCT preoperatively and then with calipers intraoperatively.¹⁴

Table 1 Characteristics of Included Studies

Four of the studies’ primary purpose was to assess the reliability of TD AS-OCT in measuring insertion-limbus distances of the EOM by using intraoperative calipers as a comparator.^3,6,15,18 Two of the seven studies’ primary purpose was to compare TD AS-OCT to other measurement tools.^15,16 For example, one study investigated the accuracy of four different OCT machines in measuring insertion-limbus distances.¹⁶ Another compared the measurements of TD AS-OCT to those of ultrasound biomicroscopy.⁴ Both of these studies also included caliper measurements as comparators.^15,16 One of the studies compared the insertion-limbus distances of the SR in primary angle glaucoma patients to those of normal control patients. This study conducted TD AS-OCT as well as caliper measurements for the first 20 primary angle glaucoma patients that were enrolled.¹⁴

Methodology of TD AS-OCT Measurements

Table 2 provides a summary of the methodologies of TD AS-OCT measurements. The studies would calculate the insertion-limbus distances by first identifying a particular anatomical landmark, such as the scleral spur or the iris root insertion.^3,4,14,15,18 Then, the authors would find the distance between the muscle insertion site and the anatomical landmark (ie, iris root) with the TD AS-OCT caliper function. Finally, they would add a constant value to the measured distance (ie, iris root to insertion distance) based on previous anatomical studies that describe the relative orientation between the anatomical landmark and the limbus. For example, four articles determined the insertion-limbus distance by first measuring the distance between the iris root and muscle insertion and then adding 1.0mm.^3,4,15,18 One study acknowledged that this method of determining the insertion-limbus distance could lead to systematic error since the distance between anatomical landmarks can vary across individuals.¹⁴

Table 2 Methodology of Preoperative AS-OCT

Methodology of Intraoperative Caliper Measurements

Table 3 provides a comprehensive summary of the methodologies used for the intraoperative caliper measurements. Among all of the studies, the number of replicates taken for each muscle varied from 1 to 3.^{3,4,6,14–16,18} One study acknowledged that taking replicates of a given muscle insertion may yield a more accurate measurement because the limbus has about a 2mm width.³ Therefore, with each measurement, there will be slight differences in the point marked as the limbus.³ All studies made measurements from the middle of the muscle insertion.^{3,4,6,14–16,18} One study reported that the rationale for measuring from the midpoints of the muscles was to avoid morphological variations that can be caused by curve or slanted insertions.⁴

Table 3 Methodology of Intraoperative Caliper Measurements

Imaging Success Rates and Reliability

Table 4 provides a summary of the data reported (or not reported) regarding rates of successful imaging in different muscle types. Six of the studies reported high success rates in imaging and locating the EOM with no prior operations.^{3,4,6,14,15,18} All six of these studies showed at least a 94.2% success rate in imaging the EOM insertions with no previous operations.^{3,4,6,14,15,18} On the other hand, studies that included both previously operated muscles and muscles with no prior operations showed that the success rates for imaging previously operated muscles tended to be lower. For example, the rate of success in imaging reoperated muscles ranged from 77.8% to 100%.^3,4,6,18 Any study that reported the imaging success rates for both muscle types found that the imaging success rates of previously operated muscles were almost always lower than those of non-operated muscles.^3,4,6,18 Across all of the included studies, the LR and MR muscles seemed to be the most represented.^{3,4,6,14–16,18}

Table 4 Data Reported

Table 4 also provides information regarding the muscles that were imaged successfully. Two studies only imaged muscles with no prior operations.^14,15 In four studies that included both subgroups, the majority of the sample size consisted of primary muscles.^3,4,6,18 In another study that included both types, 20/31 (64.5%) were previously operated muscles.¹⁶

Almost all studies reported quantitative measures of reliability in their analysis, using either the Pearson’s correlation coefficient or intraclass coefficient. Three studies calculated reliability by combining data of all muscle types and then providing both a Pearson’s correlation and an intraclass coefficient for this “combined” sample.^4,14,15 Two studies also conducted their reliability analysis by combining data from each muscle type and then providing a singular, “combined” reliability coefficient.^3,16 However, these two studies provided just a “combined” intraclass coefficient and did not provide a Pearson’s correlation coefficient.^3,16 One of the studies provided neither a Pearson’s correlation coefficient nor an intraclass coefficient.¹⁸ Another study provided Pearson’s correlation coefficients for the MR, SR, IR, and LR muscles, but did not provide a “combined” Pearson’s correlation coefficient.⁶ All the studies that reported a “combined” Pearson’s correlation coefficient and/or an intraclass coefficient found reliability to be high (>0.7).^3,4,14–16 The reliability of TD AS-OCT in measuring the MR and LR insertion-limbus distances was consistently high across studies.^6,15 The reported reliability coefficients for measuring the insertion-limbus distances of SR muscles, on the other hand, showed significant variation.^6,14 One study found the Pearson’s correlation coefficient for SR muscles to be 0.774, while another study found Pearson’s correlation coefficient for SR muscles to be 0.32.^6,14 Of note, the study that reported a Pearson’s correlation coefficient of 0.32 for SR muscles included a greater number of patients undergoing reoperations in its sample, and it has been shown that measurements of previously operated muscles tend to be less accurate.⁶

In fact, several studies also found that the reliability of TD AS-OCT in imaging previously operated muscles tended to be lower.^4,6,18 Three studies found that a smaller proportion of TD AS-OCT measurements of reoperated muscles fell within 1mm of caliper measurements compared to that of non-operated muscles.^4,6,18 In these studies, a difference of 1mm between TD AS-OCT and caliper measurements was defined as clinically acceptable.^4,6,18,19 Unfortunately, none of the studies that included both muscle types conducted a stratified analysis of reliability based on prior operation.^3,4,6,16,18

Meta-Analysis of Reliability

Figure 2 presents the estimated means and variances for each measurement scheme, and Table 5 reports the specific values. There was no evidence of a difference between means of TD-OCT and intraoperative calipers ( = 6.81, 95% CI [6.41, 7.22]; = 6.73, 95% CI [6.18, 7.29]; = 0.08, 95% CI [−0.44, 0.61]). There was also minimal evidence of a difference in variances ( = 0.26, 95% CI [0.13, 3.24]; = 0.40, 95% CI [0.21, 2.16]; = 0.66, 95% CI [0.60, 1.50]). Reliability was estimated to be good ( = 0.91) though the lower limit was slightly below the recommended minimum acceptable level of 0.70 (95% CI [0.65, >0.99]). This reflects a lack of evidence of difference between measurement schemes; however, the null should not be accepted on this evidence especially. With only six studies, and an average of 32 patients per study, power to detect a difference remains a concern. In addition, because we were unable to perform specific stratifications, much caution must be taken in interpreting these results.

Table 5 Meta Analysis Results

Figure 2 Estimated means and variances for each measurement scheme.

Conducting the meta-analysis by utilizing the “combined” measurements, not individual muscle group manner, is not ideal, as it is possible that there are differences in reliability depending on whether the MR, SR, IR or LR is being imaged. As per Table 4, it is difficult to ascertain whether there is a difference in reliability depending on the rectus muscles. The reliability of TD AS-OCT when measuring the insertion-limbus distances for LR and MR is consistently high.^6,15 However, for SR muscles, one study reported high reliability of 0.769, while another study found the reliability of TD AS-OCT in measuring the SR and IR to be low, at 0.32 and 0.47, respectively.^6,14 We were, unfortunately, unable to stratify the analysis in this manner due to inconsistent reporting/analysis of the included articles.

Furthermore, we were unable to stratify our analysis based on whether the muscles had a prior operation. Based on the evidence mentioned above, there is an indication that previously operated muscles are harder to detect and lead to less reliable TD AS-OCT measurements. As per Table 4, none of the studies that include both muscle types reported a reliability coefficient that was stratified in this manner.

The color modality used for the TD AS-OCT was not considered, as all but one of the studies employed grayscale imaging. In addition, we did not take into account eye positioning during TD AS-OCT imaging because eye positioning has been shown to not significantly affect the AS-OCT measurements of insertion-limbus distance.²⁰

Discussion

The primary objective of this systematic review and meta-analysis was to assess the reliability of TD AS-OCT in determining the EOM insertion-limbus distance. We included studies that compared preoperative TD AS-OCT measurements to those of intraoperative calipers, which were assumed to be the gold standard because they are routinely used in various ophthalmic procedures and allow for a direct view of the muscles. We found that while the TD AS-OCT and intraoperative caliper methodologies were homogeneous across all studies, both the reporting and data analysis were inconsistent across the studies.

About 7.7% of patients with strabismus require reoperation, and obtaining accurate preoperative measurements of the EOM insertion sites prior to reoperation can decrease the risk of complications (ie, prolonged time on general anesthesia and bleeding) if information regarding the insertion is not readily available to the surgeon.^3,4,21 Some reoperations can occur years or even decades after the initial strabismus surgery, leading to potential loss of information regarding the insertion site.¹⁶ Even if detailed information regarding insertion-limbus distances is available to the surgeon, secondary healing may cause movement of the muscle insertion by up to 1–2 mm.^16,22 In addition, muscle slippage may displace the insertion by more than 1–2 mm. Unfortunately, directly obtaining accurate preoperative measurements of the insertion-limbus distances is difficult, and measurements are most reliably determined through intraoperatively using calipers.⁵ Slit-lamp examination can give an indication of the general location of a muscle insertion, but it cannot give accurate measurements.¹⁶ Therefore, having a reliable measurement tool, such as TD AS-OCT, to determine the insertion-limbus distances preoperatively is invaluable.⁶ Other imaging modalities, such as ultrasound biomicroscopy, have also been shown to be a potentially reliable tool for strabismus reoperations. In fact, several studies have shown that preoperative ultrasound biomicroscopy shows good agreement with intraoperative calipers in the setting of reoperations.^23,24

TD AS-OCT would be quite useful in the setting of reoperations. However, the included studies in our systematic review included mostly primary muscles.^{3,4,6,14–16,18} In addition, none of the studies that included both muscle types included a stratified analysis.^3,4,6,16,18 Reliability was estimated to be good ( = 0.91) though the lower limit was slightly below the recommended minimum acceptable level of 0.70 (95% CI [0.65, >0.99]). In other words, about 65–100% of the time, TD AS-OCT measurements would yield the same measurements as those of intraoperative calipers. However, our calculated reliability of TD AS-OCT should be interpreted with caution. Of the 189 total muscles included in our meta-analysis, 159 (84.1%) of the muscles had no prior operation. Therefore, our estimated reliability of TD AS-OCT in measuring insertion distances is more applicable to primary strabismus surgeries.

Based on the included studies, there is evidence that there is a difference in reliability of TD AS-OCT between previously operated muscles and non-operated muscles. For example, the percentage of TD AS-OCT measurements for reoperated muscles that fell within 1mm of caliper measurements was smaller than that of non-operated muscles.^4,6,18 We were, unfortunately, unable to stratify our analysis on the basis of prior operation because none of the studies provided a stratified analysis of reliability.^3,4,6,16,18 One study that employed spectral domain OCT also found differences in reliability between previously operated and non-operated muscles.²⁵ We acknowledge that excluding other imaging modalities from our analysis, such as spectral domain AS-OCT and swept source OCT, limits the comprehensiveness of this systematic review and meta-analysis and should be noted as a limitation to the present study.

There are several factors that could have potentially contributed to the smaller reliability coefficients in previously operated muscles. The limbus-insertion distance in previously operated muscles can be difficult to locate due to scar tissue.²⁶ Further studies need to be conducted to investigate the differences in reliability between these two subgroups of muscles.

Conclusion

The present study highlights the need for more future investigations that focus their reliability analysis specifically on reoperated muscles and that aim to further improve the reliability of TD AS-OCT for reoperated muscles. As stated before, having a preoperative measurement tool for previously operated muscles could be highly beneficial. Based on the current evidence, in the setting of reoperations, there is insufficient data to conclude whether TD AS-OCT has an acceptable reliability in the setting of reoperations, and TD AS-OCT may not be as useful as spectral domain AS-OCT in the setting of strabismus reoperations Future studies should aim to improve the reliability of TD AS-OCT in the setting of reoperations. However, if the reoperated muscle can be successfully imaged, TD AS-OCT could still be a useful tool for finding the general location of muscles when no prior data are available. In the setting of primary surgeries, based on our meta-analysis, TD AS-OCT has an acceptable reliability as a measurement tool for determining the insertion-limbus distances of the EOM, but the benefits of using TD AS-OCT preoperatively for muscles with no prior operation are not clear. Future studies should explore whether there are benefits to using TD AS-OCT preoperatively in the setting of primary strabismus surgeries and if doing so can improve post-operative outcomes. In addition, we hope that the findings of the present study inspire future investigations to report any reliability analysis more thoroughly.