How augmented reality makes the invisible visible
Augmented reality (AR) is making its way into the operating room: Whenever surgeons wear the HoloLens AR glasses during punctures of the cerebral ventricle, the navigation aid is displayed directly into their field of vision. We are currently working on the user-friendly design of this novelty together with our partners in the HoloMed research project. An initial test run at Günzburg hospital has now been completed successfully.
SAFELY ON TARGET WITH AUGMENTED REALITYVentricular punctures are among the most common routine interventions in neurosurgery. Nevertheless, the catheter that is inserted into the brain is only optimally placed in two thirds of the cases. The problem: The surgeon does not see the target—a cavity in the brain filled with liquor.
In future, the HoloLens will provide support in this procedure: Using CT or MRT data of the patient, HoloMed computes an exact 3D model of the brain before surgery.
The HoloLens projects the model with millimeter precision onto the patient’s head, making the invisible visible.
Moreover, graphical elements help the surgeon to find the right angle and position. The benefits are obvious: a lower error rate and more safety for patients and surgeons.
Before it reaches marketability and perfect user experience (UX) though, we still have a couple of to-dos on our list: The virtual projection has to be perfectly aligned with the patient’s position. In order to achieve this, we use a system of markers that still has room for improvement. What is especially interesting for us to see is how the doctors perceive the holograms in the surgical site. Currently, the lighting conditions in the surgery room greatly influence the holograms’ visibility. And we have learned that some of them obstruct the visibility of the actual medical instruments. Moreover, we are taking existing concerns about hygiene and time requirements seriously.
MAY WE INTRODUCE THE NEW HOLOMED UIWhen the neurosurgeon puts on the HoloLens and starts HoloMed, four menu items are displayed: preparation, Kocher’s point, incision and angle. The order matches the workflow of a ventricular punction. With the MRT & CT Scans button, the surgeons can interactively display any important scans of the patient.
Some holograms for navigation support are animated and pulsate so that they don’t constantly cover the working area and that the doctor can perfectly align the instruments with the visualizations.
HoloMed can be controlled simply via “long gaze”, i.e. by focusing the hologram for a certain period of time, and speech, allowing for hands-free operation during the complex procedure. A welcome side-effect: There are no problems with regard to the high hygiene standards in the surgery room.
PROJECT CERTIFICATION WITHIN REACH
Suitability for use, safety, and hygiene are also important factors for a successful product certification process. We are supporting the research group with our expertise in compiling Usability Engineering Files to pave the way for the product certification of the HoloLens.
MORE INFORMATION ABOUT HOLOMED
The research project “HoloMed – Context-sensitive support for surgeons in the operating room using augmented reality” (funding code 01IS17005) is co-financed by the Federal Ministry of Education and Research and carried out by the following project partners: mbits imaging GmbH, User Interface Design GmbH, Neurochirurgische Klinik der Universität Ulm (neurosurgical clinic of Ulm University) and KIT – Intelligente Prozessautomation und Robotik (intelligent process automation and robotics) (IAR-IPR). It runs until the end of 2019.