Consolidation of Imaging Modalities Utilizing Digitally Assisted Visualization Systems: The Development of a Surgical Information Handling Cockpit

Introduction

Successful vitreoretinal surgery is dependent on the ability to visualize both the pathology at hand and the surgical field. Advances in visualization technology include digitally assisted vitreoretinal surgery (DAVS),^1–3 clinic-based optical coherence tomography (OCT),^4,5 intraoperative OCT (iOCT),^6,7 and ocular endoscopy.^8,9 Each of these have proven useful for vitreoretinal surgeons, but also have entailed logistical challenges including multiple viewing screens and awkward surgeon positioning when viewing multiple images and video feeds during surgery. Novel technical approaches are warranted to leverage the full potential of modern imaging modalities for both the vitreoretinal surgical patient and surgeon.

DAVS has the potential to address these challenges. A stereoscopic, high-definition visualization system displays the surgical field in real-time on a 3DHD flat-panel screen in the operating room, eliminating the need to view through microscope oculars. Since initial reports in 2010 as an additional feature for primary surgical visualization in vitreoretinal surgery,^1–3 DAVS has been reported to afford equivalent or superior high-quality visualization of the surgical field when compared to microscope oculars with added ergonomic benefits for the operating surgeon.^1–3

The purpose of this paper is to report the techniques and initial clinical impressions for viewing preoperative ancillary testing images, real-time iOCT video, and real-time ocular endoscopy video simultaneously on the same 3DHD screen as the primary surgical image.

Materials and Methods

A consecutive case series of 44 eyes of 44 patients is reported. All patients were treated in accordance with the guidelines of the seventh revision of the Declaration of Helsinki (2013) and IRB approval was obtained from the Integ Review IRB (Austin, TX).

A DAVS system (NGENUITY™, Alcon. Fort Worth, Tx) was utilized with the 3D flat-panel positioned approximately 1.5 m from the surgeon sitting “heads up” at the head of the bed. NGENUITY software version 1.1.17 (n=31) and older TrueVision software version 9.8.13 (n=13) were used on an identical NGENUITY hardware unit. The ocular endoscope (EndoOptiks URAM E2, Beaver Visitec, Waltham, MA) was used in 12 eyes. The Zeiss Rescan 700 iOCT (Zeiss’ US subsidiary, Dublin, CA) was used in 12 eyes. The Haag Streit iOCT (Haag Streit USA, Mason Ohio) was used in 6 eyes (gen one unit in 1 eye and gen 2 unit in 5 eyes). The Leica EnFocus Ultra-HD intrasurgical OCT (Leica Microsystems Buffalo Grove, IL.) was used in 6 eyes. A computer monitor feed from an EHR (Next Gen. Irvine, CA) and image viewing system (Merge, IBM Watson Health. Chicago, IL) was used in 8 eyes. All ancillary information was viewed simultaneously with the primary 3D surgical field using the DAVS unit. Surgeon to screen viewing distance was approximately 1.5 m. In all surgeries, DAVS camera viewing was compared to conventional surgical field viewing through the oculars with off-axis viewing of ancillary content on a separate screen. A comprehensive compilation of patient diagnosis, surgical procedure, and imaging software is provided in Table 1 to demonstrate the various combinations of technology utilized for various diagnoses and surgeries.

Table 1 Patient Characteristics and Surgical Details

Specific connections were as follows:

Ophthalmic Endoscope

Analogue outputs of the Endo Optiks/Beaver Visitec endoscope were digitized using an S video to HDMI converter box (HDCVRYW, KanexPr, Brea, CA) to avoid signal degradation along the 5-m cable run from the endoscope to the DAVS unit. Image quality with this configuration was superior to that achieved by directly connecting the endoscope analogue output to the DAVS unit video card with a long composite or S-video cable. The endoscopic HDMI converter box image on the DAVS screen was equivalent to the S-video feed to the dedicated endoscope LCD monitor (AlphaView model AVC-1AOP, AG Neovo, San Jose, CA) but was slightly inferior to the image quality on an older CRT monitor (Trinitron CRT model PVM-14L2MD, Sony Corporation, Irvine, CA). The surgical microscope used for primary visualization and to which the DAVS camera system was connected was the Leica M844 F-40 in 7 cases and the Haag Streit Hi-R NEO900 in 5 cases. The digitized endoscope HDMI output was also connected to the image injection system of the Haag Streit microscope in one case.

Zeiss iOCT

The DAVS unit’s ancillary HDMI video input was used for all the video signals except the Zeiss Rescan iOCT. The microscope injected overlaid iOCT feed was directly imaged as part of the primary surgical image by the DAVS camera. The picture in picture feature of the DAVS unit was not used. The surgical microscope used for primary visualization and to which the DAVS camera system was connected was the Zeiss Lumera in all 12 cases.

Haag Streit iOCT (Gen 1)

The primary microscope iOCT viewing screen was disconnected from the iOCT control unit and fed directly into the DAVS unit HDMI in using a DVI to HDMI cable.

Haag Streit iOCT (Gen 2)

The microscope-integrated MIOS computer system (which allows for internal 2D video recording, internal iOCT video recording, microscope control functions, and iOCT display functions) was connected directly into the DAVS unit HDMI in using a DVI to HDMI cable screen mirroring the output to the MIOS display screen on the microscope by adjusting the Windows (Microsoft, Redmond WA.) display settings of the MIOS computer.

Leica EnFocus Ultra-HD OCT

The primary microscope iOCT viewing screen was disconnected from the iOCT control unit and fed directly into the DAVS unit HDMI in using a DVI to HDMI cable.

EHR Feed

The operating room computer monitor was disconnected from the operating room PC and a DVI to HDMI cable was used to connect this to the AUX HDMI input of the DAVS system. EHR (fundus drawings) and preoperative diagnostic imaging were brought up before the beginning of the case for display during surgery as deemed appropriate by the operating surgeon (CDR).

Once all video input connections were established, dual screen or multi-screen video format is then selected on the DAVS unit allowing for visualization of the microscopic visual field in addition to the ancillary information (iOCT, endoscope or EHR) simultaneously. The surgeon can toggle between viewing panels to enlarge one video signal over the other while still maintaining multi–image viewing capabilities or can use a 50/50 split screen viewing mode (Figure 1).

Figure 1 Screen grabs from a left eye live video demonstrating simultaneous same screen iOCT viewing with the Haag Streit (gen 2) iOCT side by side with the live surgical field during a membrane peel in a patient with vitreomacular traction and ERM. (A) Vitreomacular traction at fovea, (B) Posterior hyaloid attached at optic nerve, C-D) Membrane peel, (E) Vitreomacular traction at fovea with OCT image injection, (F) Macula following membrane peel with OCT image injection.

Clinical Impressions

Ancillary Video Signal Source

DAVS surgery with ancillary video signals was performed on 10 eyes using the Zeiss Rescan iOCT, 12 eyes using the endoscope, 1 eye with the Haag Streit iOCT (gen 1) 5 eyes with the Haag Streit iOCT (gen 2), 6 eyes with the Leica EnFocus Ultra-HD iOCT, and 8 eyes with an EHR feed. Patient characteristics and surgical details are summarized in Table 1. All surgeries were successful.

Overall Clinical Impression

Simultaneous same screen viewing of ancillary content was superior to two screen viewing or surgical viewing through the oculars and separate screen-based viewing of ancillary content in all cases. Surgeons were able to simultaneously monitor the primary surgical field and the iOCT, endoscopy or preoperative medical record and/or imaging at all times. Ergonomics, eye strain, neck strain and lower back strain were notably better with DAVS compared to traditional multisource viewing. Specific initial impressions by type of ancillary content were as follows.

Zeiss Rescan iOCT

The Zeiss Rescan iOCT was the only ancillary video signal that did not use the picture in picture auxiliary video display feature of the DAVS viewing system. The DAVS camera was able to adequately image the microscope image injected iOCT reticles and scout images with no reduction in quality or utility of these images noted. Full resolution iOCT images needed to be viewed on the ancillary screen. We were unable to evaluate taking the Zeiss iOCT monitor signal and placing this onto the DAVS screen using the secondary video input due to technical complexities. The Zeiss image injection is monocular in the right eye. Resolution of the image injected feed is 600 x 800. The image injected iOCT images were of lesser utility due to display resolution limitations when compared to the separate iOCT viewing screen.

Ophthalmic Endoscope

The single screen solution of dual video feed viewing of the endoscopy video feed real time immediately adjacent to the live surgical view was substantially superior to viewing the endoscopic video feed on a separate screen a face turn away from either the microscope oculars or the DAVS screen. The ability to toggle the endoscopic feed onto the DAVS screen while maintaining the live surgical view (in 3D) in a smaller or 50/50 format was especially useful for maintaining orientation and awareness of globe position during more complex endoscopic maneuvers and during instrument exchanges through the trocars and sclerotomies. Subjectively, surgical efficiency and workflow seemed markedly improved.

Haag Streit iOCT & Leica EnFocus Ultra-HD iOCT

Twelve cases were performed with DAVS-based dual image viewing using these units (6 Leica and 6 Haag Streit – one gen 1 and 5 gen 2). Full resolution live viewing of the iOCT image on the same screen as the live surgical view was particularly useful and made iOCT based membrane peeling feel plausible which had not been the case previously with the images separated between oculars and an off-axis viewing screen or image injected at lower resolution into the main surgical oculars. iOCT image acquisition workflow also seemed more streamlined as it was easier to maintain placement of the iOCT scanning beam on the area of interest with single screen dual video feed viewing. Rapidly alternating between the primary surgical field and the iOCT feed was possible with a quick saccade and did not require a face turn. Both the Haag Streit and Leica iOCT units are equipped with image injection into the microscope oculars and – as was the case with the Zeiss iOCT unit – the image injection was well reproduced on the DAVS screen. The utility of the low-resolution image injected iOCT was less than the full resolution iOCT image displayed side by side with the live surgical view on the DAVS unit. The Haag Streit unit has binocular image injection. The Leica iOCT uses monocular image injection. Resolution is 1280 x 1024 for both units. We felt that the monocular image injection was inferior to binocular image injection due to more eye strain.

NextGen EHR and Merge Image Database

Static preoperative images from the medical record were also assessed. The primary benefit was displaying preoperative fundus drawings side by side with the live surgical field during PPV (n=3) or chandelier scleral buckling surgery (n=1) for retinal detachment. Preoperative OCT imaging was also somewhat useful especially when initiating the membrane peel in patients with more severe epiretinal membranes (n=1) or fibrovascular proliferation and early tractional retinal detachment due to proliferative diabetic retinopathy (n=1). Preoperative wide field fluorescein angiography was useful when displayed adjacent to the live surgical view as it facilitated and simplified the workflow of accurate, precise placement of PRP laser to ischemic areas of diabetic retinas (n=2) with proliferative disease. Opening and formatting images from the EHR to match the orientation of the live surgical view were sometimes tedious.

Zeiss vs Haag-Streit vs Leica Image Injection Systems

Subjectively, the quality of the injected images was approximately the same. Qualitatively, the Haag-Streit image injector is binocular and the Zeiss and Leica image injectors are monocular. We found the monocular image injector functional, but induced more eyestrain and headache even for experienced Zeiss users. The Haag-Streit and Leica units have a native resolution of 1280 x 1024 pixels with 12-bit color encoding and the Zeiss unit had a native resolution of 600 x 800 with 10-bit color depth. While the Haag-Streit and Leica optically injected iOCT images were higher resolution, the utility of these images was not clinically different than the Zeiss injected images. All optically injected images were lower resolution than the iOCT scan itself when viewed on the secondary iOCT microscope-based monitor. Connecting the Haag Streit and Leica units to the DAVS system for side-by-side same screen viewing of the high-resolution iOCT image and 3D surgical field immediately solved this problem. We were unable to assess whether a full resolution Zeiss iOCT scan could be displayed on the DAVS unit using its AUX input video feature, and therefore cannot comment on the quality of the full resolution Zeiss when displayed on the DAVS system.

The Haag Streit image injection was used to image inject the digitized endoscope video feed (n=5) as well as the EHR feed (n=3) into the primary surgical view. In all these instances, the injected images were too large, obscured the surgical field, and were consequently aborted. This was equally the case if the image injected content was viewed through the oculars or captured by the DAVS camera. Same screen viewing with the auxiliary video feed of the DAVS system was superior both ergonomically and from the standpoint of accurately displaying clinically relevant information during surgery without blocking the primary surgical field. Integrated same screen viewing also improved the efficiency and speed of iOCT image acquisition and viewing, thus reducing the surgeon’s threshold to utilize the iOCT.

Discussion

The development of various imaging modalities has enhanced our understanding and management of vitreoretinal diseases. The advantages and utility of clinic-based OCT, wide field angiography, iOCT, and endoscopy have all been well described. Since its introduction, OCT’s ability to optimize the diagnosis, management, and observation of vitreoretinal diseases has transformed patient care.^10–12 OCT has become standard of care to aid in clinical decision-making for patients with medical retina disorders such as age-related macular degeneration (AMD),^11,13 diabetic retinopathy (DR),^14,15 and central serous chorioretinopathy,^16,17 as well as surgical retinal disease such as epiretinal membrane (ERM)¹⁸ and macular hole with retinal detachment (MH RD).¹⁹ To date, intraoperative viewing of preoperative OCT images involved taping printed OCTs to the surgical microscope or viewing these on a computer monitor, which may not be in close proximity to the surgeon position at the head of the patient. This challenge is eliminated by DAVS same screen viewing of surgically relevant information alongside the digital 3DHD surgical field video feed although the workflow of formatting these images for display remains an area for future improvement.

Similarly, iOCT can inform surgical management. Operating microscope-integrated OCT technology and OCT compatible surgical instruments have transitioned OCT from the clinic to the operating room, facilitating a greater understanding of vitreoretinal disease in real-time. iOCT has been utilized for the diagnosis and management of many vitreoretinal diseases, often facilitating the observation of subtle structural entities prior to, during, or following surgical manipulation.^6,20-29 iOCT’s impact on surgical decision-making and surgeon understanding of underlying tissue has been reported as 38% and 48% in the DISCOVER and PIONEER studies, respectively.^30,31 Despite well-described advantages, iOCT utilization remains low due in part to challenges with image acquisition and display during fast-paced vitreoretinal surgery. DAVS same screen viewing of iOCT with the 3DHD assists in alleviating this barrier.

Finally, ocular endoscopy, a surgical imaging modality first described in 1934 for the removal of nonmagnetic intraocular foreign bodies, has continued to evolve and assist in vitreoretinal diseases. Modern fiber optics, higher quality light sources, and video-endoscopy have facilitated real-time visualization, with clinically useful resolution and field of view.^8,9,32-35 Ocular endoscopy enables the surgeon to bypass anterior segment opacities and pupillary abnormalities to visualize and manage pathology in the posterior segment.³⁶ The use of ocular endoscopy in vitreoretinal surgeries is far from new and has been reported for a wide variety of indications, including proliferative vitreoretinopathy,^37,38 post-traumatic endophthalmitis,^39,40 intraocular foreign bodies,^39,41 perforating injuries to the globe,⁴¹ penetrating injuries,^39,41-44 and RD.^45–47 Ocular endoscopy has also been utilized for PPV through the custom flexible iris prostheses.⁴⁸ And has been suggested to offer better long-term visual outcomes for placement of a pars plana tube shunt in the setting of complex anterior segment disease.⁴⁹ A recent report describes the successful hybridization of ocular endoscopy with a 3D visualization system to manage vitreoretinal disease.⁵⁰ Despite clearly defined advantages, ocular endoscopy utilization remains low in part due to the ergonomic challenges of multi-screen off-axis viewing away from the main surgical field. DAVS based same screen simultaneous viewing of endoscopy video alongside the 3DHD surgical field video reduces these challenges. Further rotational processing of the endoscope image as a function of position within the eye may also facilitate orientation and more complex dissections during endoscopic vitreoretinal surgery.

Even though iOCT and ocular endoscopy augment the vitreoretinal surgical armamentarium, both imaging modalities are associated with similar logistical and ergonomic challenges. Ancillary screens require the physician to alternate between viewing the primary surgical field and the secondary imaging source. This is both ergonomically problematic for the physician and technically challenging due to the differences in image orientation when switching between image sources.⁵¹ One proposed solution was the addition of injected images into the oculars by some iOCT vendors, but this has not been reported for ocular endoscopy. The resolution of injected images remains far below the resolution of the iOCT source image, which limits its usefulness and still obligates the surgeon to an unergonomic and inefficient workflow dependent on a secondary off-axis screen.

Compared to traditional analog surgery with microscope oculars, reports on DAVS highlight superior surgeon ergonomics, enhanced surgical observation and teaching capabilities, improved depth of field and stereo-acuity, and enhanced visualization with digital signal processing algorithms and digital filters.^1,2,52 Incorporation of ancillary static image and video feeds, onto the live surgical field provided superior ergonomics and simultaneous viewing capabilities which informed surgical decision-making compared to analog viewing of the primary surgical field and ancillary image viewing on a second screen. For cases where image injection was utilized, DAVS viewing of the injected integrated video feed was non-inferior to viewing through the oculars from an image quality standpoint and was superior from an ergonomic standpoint. However, the limited quality of the image injected content reduced its clinical utility and/or obscured the view of the primary surgical field.

When comparing the functionally of DAVS as a function of surgical microscope type (Haag-Streit, Leica, Zeiss), there were no clinically discernable differences. This is interesting as the stereobases for these three microscopes are not the same – 25, 24, and 22.5mm, respectively. Further objective testing is warranted to more carefully study this question. Installing the correct DAVS camera footplate for each microscope system is crucial to ensure centration of the DAVS image. Ergonomically, surgeon comfort was superior with DAVS same screen viewing of the surgical field and ancillary imaging content compared to viewing through the oculars with ancillary screens. A “heads up” position with minimal head shifting for different images provided a streamlined, comfortable and preferable alternative to traditional surgery.

Same screen viewing of full resolution iOCT alongside the 3DHD surgical field video was more user friendly, much more ergonomic, and conveyed more precise information. Surgical decision-making was more informed with integrated DAVS split screen viewing of iOCT images than when viewing image injected scout images into the oculars, than DAVS imaging of injected optical images, or than full resolution iOCT scans viewed on secondary screens.

The surgical paradigm where the DAVS advantage was the least evident was static images from patient’s EHR. Fundus drawings, preoperative OCT, and preoperative wide field FA could have been examined preoperatively by the surgeon without much loss of functionality. In more complex situations, most notably ischemic diabetics with preoperative wide-angle FA imaging, DAVS integrated same screen viewing was deemed superior to a printed image taped to the microscope arm because quick back and forth viewing between the FA and live surgical view without a face turn by the surgeon allowed for rapid and precise titration of laser to areas of retinal non-perfusion. Also viewing a taped printed image required turning on the operating room light, which detracted from viewing the primary surgical field.

Unfortunately, we were not able to fully assess the utility of image injection systems for the viewing of ancillary images other than iOCT. The fact that current image injection systems are not easily configurable to display signals other than the iOCT for which they were designed, does not mean this is not possible if it were set as an engineering design goal.

Overall, our experience with DAVS and same screen viewing of ancillary images found it to be superior to traditional vitreoretinal surgical viewing approaches. DAVS provided a high-quality image while simultaneously providing the ergonomic and workflow benefits of maintaining all imaging modalities on one screen. DAVS eliminated the need for physicians to rotate their heads from the primary imaging modality to the secondary screens, instead requiring quick ocular saccades within one screen. This made near simultaneous tracking and monitoring of two image feeds possible.

The cost of the hardware and cabling necessary to accomplish same screen viewing of multiple information streams side by side with the 3DHD screen primary video feed of the surgical site was very low and varied between $0 and $83 depending on the particular combinations of microscope and ancillary imaging source. The most expensive solution was for the Endoscope where a 20 foot HDMI cable and an S video to HDMI converter ($83) were needed. The least expensive solution was for the Zeiss Rescan unit where no cabling was required however only the image injected scout iOCT images were viewable. Of course, these costs do not include the cost of the DAVS unit itself which can be substantial. The cost-effectiveness of DAVS acquisition is beyond the scope of this paper. FDA approved DAVS units are currently for sale worldwide by two vendors and over 1000 units of this new technology have been placed worldwide to date (personal communication between CDR and Alcon). We believe that the ability to incorporate same screen viewing of ancillary imaging with the primary surgical field enhances the ROI calculus for DAVS acquisition.

This study is fraught with limitations the most notable being its extremely subjective nature. Nonetheless, we believe that the description of our experiences using all FDA approved microscope-integrated iOCT, a next-generation iOCT unit, the ocular endoscope, preoperative EHR images and diagnostic imaging studies, and three different vitreoretinal surgical microscopes both with and without DAVS is new, important, and useful.

All the necessary technology to achieve integrated same screen viewing of DAVS, endoscopy, iOCT, and preoperative imaging is currently available. DAVS’ unique capacity to organize, integrate, and display diverse and relevant visual information on a single surgical screen is non-inferior to multiple screen viewing and may be preferable in selected instances. Integration of diverse ancillary video feeds with DAVS primary surgical viewing represents a new frontier for vitreoretinal surgery. We combined DAVS with three different microscope systems, 4 different iOCT systems, the ocular endoscope, EHR images, and preoperative testing imaging. We highlight several observations regarding visualization quality, system functionality, and surgeon ergonomics compared to traditional vitreoretinal surgery with oculars and separate screen viewing of additional image and video feeds. Our observations were robustly consistent across all iterations of combined technologies and we conclude that this integration is technically feasible, inexpensive, and straightforward across a broad spectrum of surgical microscopes and ancillary imaging/video sources, clinically useful and desirable from a surgical workflow, and surgeon ergonomic standpoint, and we strongly recommend that an integrated surgical information handling “cockpit” functionality be purposefully incorporated into future iterations of DAVS technology. This concept will certainly continue to expand with future iterations of DAVS technology focusing on usability, workflow, and human factor design engineering.

Summary

A new advantage of screen-based vitreoretinal surgery is described. Consolidated same screen viewing of ancillary video feeds with the primary surgical field is technically feasible, cost effective, offers ergonomic advantages, enhanced surgical precision and may improve surgical efficiency.