AR, VR Technology Gives Clinicians a New View of Complex Surgeries

Lehigh Valley Health Network's chief of neurological surgery is using the innovative technology to educate his patients and improve surgical procedures.

Healthcare organizations are slowly embracing augmented and virtual reality for a variety of uses centering around education and training. But the ability to take a two-dimensional image or concept and give it depth and substance also holds promise for clinical care—even surgery.

As chief of the division of neurological surgery at the Lehigh Valley Health Network, Walter Jean, MD, has been using AR and VR for more than five years, not only to plan delicate surgeries, but to actually do those surgeries as well. The technology helps him to get a better look at a patient's anatomy both before and during the procedure.

"It's basically operating with GPS," he says. "And it's getting better. Every year the technology improves, and we're seeing more nuances. It's gotten quite apparent that we're going to see surgeries become much smaller and less invasive."

Jean is one of the front-runners in using AR and VR in surgery, where he uses the technology to help his patients understand what they're going through and what he can do for them. He is an integral part of the Fleming Neuroscience Institute at LVHN, which supports the use of those and other state-of-the-art tools and platforms and to attract and train top talent.

The US Food and Drug Administration defines the two technologies as such:

"Augmented Reality (AR) is a real-world augmented experience with overlaying or mixing simulated digital imagery with the real world as seen through a camera or display, such as a smartphone or head-mounted or heads-up display (HUD). Digital imagery may be able to interact with real surroundings (often controlled by users). This is sometimes referred to as mixed or merged reality."

"Virtual Reality (VR) is a virtual world immersive experience that may require a headset to completely replace a user's surrounding view with a simulated, immersive, and interactive virtual environment."

Both technologies, the FDA says, "have the potential to transform healthcare, delivering altogether new types of treatments and diagnostics, and changing how and where care is delivered. Central to their potential in diagnosis and treatment is their ability to deliver both standard and entirely new types of content in highly immersive and realistic ways, remotely, and tailored to a variety of clinical contexts. Physicians, patients, and caregivers can enlist AR/VR to help them prepare for, or perform, certain treatments or procedures."

At LVHN, Jean uses the platform first to help his patients understand both their medical conditions and how he treats them. The process is much better, he says, than giving them images and trying to get them to imagine what's going on inside them.

"You're taking 2D images and converting them to multi-colored 3D renderings in VR," he says. "It's much easier for them to comprehend, for example, where a tumor is and what it does. It's also much easier for doctors to explain the intervention."

The platform also gives doctors an opportunity to better understand and map out a surgery before it takes place, an important benefit when one is talking about neurosurgery and the brain. Imagine, Jean says, stepping into the pathology of a patient rather than viewing it in images, and planning your journey before taking it.

Then, he says, imagine taking that technology into the operating room.

Jean says the platform is integrated with technology-aided microscopes that project images onto the patient, guiding surgeons every step of the way. The process enables surgeons to be more precise with their movements, reducing everything from the size of the incision and subsequent surgery to the recovery time, which in turn improves clinical outcomes.

And that, he says, is where the next phase of innovation is likely to target. While the platform is now contained within tech-enabled microscopes, future iterations might separate the technology from the microscope and allow clinicians to use VR goggles or smartglasses, giving them more freedom to move around during the procedure.

In addition, Jean sees the use of AR and VR in surgical procedures expanding to other specialties and other parts of the body as well, especially in surgical procedures that are delicate and complex, like spinal surgery.

And finally, he sees the potential for AR and VT technology in reducing health inequities across the globe. Millions of surgical procedures aren’t being done in underserved parts of the globe because clinicians there lack access to the necessary guidance and resources. AR and VR technology could link those providers to specialists in other countries, such as the US, to teach them how to perform certain procedures and guide them during those procedures.

As an example, Jean points to the pediatric neurosurgery partnership that now connects the University of Alabama at Birmingham (UAB) and the Global Surgery Program at Children's of Alabama with physicians in Vietnam. Using a telemedicine platform called VIPAAR (Virtual Interactive Presence in Augmented Reality) on standard iPads, specialist at UAB and Children's of Atlanta have been guiding doctors in Ho Chi Minh City on certain surgical procedures targeting epilepsy.

"There is a lot of potential here," says Jean. "Yes, there is some nervousness attached to it as it's a new way of using technology, but this is a gift that will keep on giving."