Mixed reality is proving its place in healthcare amid COVID-19

  • Mixed reality, using Hololens, is proving its value for healthcare workers on the front-line of COVID-19
  • The tech enables fewer people to be present in high-risk areas, and reduces strains of PPE supplies
  • Applications aren’t new in healthcare, but the pandemic could highlight benefits for continued uptake

Mixed reality is proving itself a valuable tool for doctors on the front-line of the COVID-19 health crisis, in another example of how technology has is helping us manage the impact of a global pandemic.

In some of the busiest hospitals in the UK, doctors are using Microsoft Hololens headsets while consulting with patients of COVID-19. This allows them to stream feeds of their visits to other specialists in another room, while minimizing the number of workers in high-risk areas.

In a project led by Imperial College London, the Hololens headsets work with Dynamics 365 Remote Assist and Microsoft teams to send a secure live video to a computer screen. At the same time, gesture, gaze and voice controls allow doctors to examine x-rays, scans and test results in their field of view, without any physical contact.

Speaking to the BBC, one patient commented that the use of mixed reality “just seems quite a good thing that you can have that amount of people in the same room, in one person.

“They’re not only saving me, I’m not passing it on to any one of them or their friends.”

Dr Kinross, a consultant surgeon at Imperial College Healthcare, said the potential of the solution – which has been used before in healthcare – was uncovered at a time of immense need owed to the pandemic.

“In one week our hospital trust switched from being a place that delivered acute, elective care and planned treatment into a giant intensive care unit. We weren’t just trying to restructure an entire building, we were trying to redeploy and retrain our staff, while at the same time we had to cope with an ever-growing number of very sick people.”

“We needed an innovative solution. I’ve used HoloLens before in surgery and we quickly realised it had a unique role to play because we could take advantage of its hands-free telemedicine capabilities,” said Kinross.

Source: Imperial College London

According to Imperial College Healthcare Trust, using Hololens has led to a fall in the amount of time staff are spending in high risk areas by up to 83 percent. But it’s also significantly reduced the amount of personal protective equipment (PPE) being used.

With only one worker (wearing the headset) needing to dress in PPE, early estimates suggest that using mixed reality is saving up to 700 items per ward, per week. That’s especially significant since UK hospitals have suffered severe shortages.

Mixed reality technology has proven a game-changing tool on the front-line in the fight against COVID-19, but the success of the deployment could further cement the technology’s role in healthcare applications afterwards.

Doctors are able to hold hands-free meetings with colleagues and experts anywhere in the world, while they view patient notes in front of them in live consultations, which unlike AI healthcare solutions or even telehealth, remain in-person.

“We’re now looking into other areas where we can use HoloLens because it is improving healthcare without removing the human; you still have a doctor next to your bed, treating you,” Kinross said. “Patients like it, too. They are interested in this new piece of technology that’s helping them.”

One of those other areas includes teaching. With academic areas of hospitals closed overnight due to COVID-19, students can access video feeds remotely to watch demonstrations from lecturers wearing headsets.

For many years now, the tech-savvy healthcare sector has been exploring the potential of mixed reality. In 2016, surgeons carrying out an operation in a hospital in Xinjiang used the technology to consult remotely with colleagues at Wuhan and Virginia, US. The specialists were able to simultaneously view a patient’s MRI and CT scans in 3D, and offer guidance to the surgeons in the operating room in real time.

Taiwan tech giant HTC’s Vive headset, meanwhile, has been used for surgery simulation and medical training around the world. At New York University, the University of California San Francisco, and Penn State University, students use HTC Vive along with VR anatomy software produced by Australia’s Medis Media to study human anatomy.

Mixed reality in enterprise

As well as supplementing a headset wearer’s field of view with digital information, one of the real advantages of mixed reality in industry is the ability for multiple people to be ‘present’ virtually, without being on site.

Headsets like Hololens are seeing rapid uptake on the factory floor, especially as new automation technologies and industrial IoT make less need for human workforces on-the-ground.

Instead, a single worker could provide a stream to a remote team of specialists who can provide support, instruction or training, and overlay relevant information – such as a cross section of a machine unit – into the wearer’s view.

But the potential of visual immersion is now crossing boundaries into other industries, including less ‘physical’ ones that we might not readily associate with futuristic headsets.

In areas like financial services, for example, mixed reality could be utilized by people who want to work with their finances in a much more visual way, or people who want to interact with their advisors and visit educational resources.

augmented realityMixed Realityvirtual reality