How a collaboration unleashed areas of impact for Hololens 2 in healthcare

Among the latest in a string of transformative digital innovations is the HoloLens 2 smart glasses from Microsoft. The HoloLens 2 is a "mixed reality" holographic device that combines what you see with a projected image of what you want to share with connected users. The device can be used with apps for better collaboration, and the Remote Assist feature during a Teams session can facilitate content sharing between the HoloLens 2 user and the people connected via a computer or tablet.

Region Skåne, a leading Swedish healthcare provider, was inspired by the possibilities posed by HoloLens 2 in healthcare. In its pursuit to improve healthcare experience for everyone with the support from the European InterReg ÖKS fund, the healthcare provider wanted to understand how the applications of digital solutions such as the HoloLens 2 can be realized to transform the health experience.

EY teams joined hands with Microsoft and Region Skåne to collaborate on a Proof of Concept (PoC) with the objective to verify the value of HoloLens 2 within Swedish healthcare. The goal of the project was furthermore to evaluate:

1. How can HoloLens 2 offer the right competence for patients regardless of place and time?
2. How can it ensure increased availability of specialist competence?
3. How can it help reduce the number of people in contact with infected patients?

To start, teams from Microsoft, Region Skåne and the EY organization conducted an initial analysis to evaluate potential use cases for HoloLens 2. It found that HoloLens 2 can improve patient treatment, increase specialist involvement in home visits, improve remote emergency involvement and increase opportunities for real-time supervision, while reducing infection contacts and the need for specialist transport.

The teams worked in four stages over an eight-week period — starting by stating the objectives and focus areas for testing, going on to develop simulated test scenarios, executing testing for 35 doctors and nurses in 15 test groups, and finishing up with an evaluation of the performed steps.

The test scenarios were developed based on real care cases in simulated environments. The first two scenarios were set in the emergency department, while the last two were set in a geriatric care setting:

1. Consultation via HoloLens 2 during emergency care of a potentially infectious patient in the emergency department.
2. Digital visualization of a checklist by the doctor for the treatment and care of an urgent, life-threatening and rare condition.
3. Enabling the meeting between healthcare professionals and patients via Teams in the patient's home environment (for treatment of a ostomy complication).
4. Facilitating meetings between professionals and patients via Teams in the patient's home environment (for a neurological examination)

The PoC found that there is a great potential associated with the use of HoloLens 2 in emergency care and geriatric care. The following were the most important observations from the PoC:

• Functionality: HoloLens 2 proved to be flexible and comfortable to use in the test cases, with vision and resolution of holograms considered satisfactory for sharing relevant visual information on the patient, shared documents and pictures.
• Collaboration: The collaboration and communication between healthcare professionals who used HoloLens 2 and the consulting doctor worked well. The quality of sound and image in some cases were above expectations and instruction tools worked well.
• Consultation with the remote assistant: Consultation via Remote Assist was tested via several angles, wherein consultation to instruct on the treatment of the patient and installation of medical devices worked particularly well.
• Document sharing and visualization: The functionality to be able to visually share information (blood gases) and documents worked well.

Among the next steps for the project team is to develop a new approach to illustrate the gains and benefits of using HoloLens 2. This includes testing in more realistic clinical environments, integrating HoloLens 2 with the client’s testing environments, and assessing the use of HoloLens 2 from the perspectives of patients, information security and regulations.

Overall, the PoC has sown the seeds for wider and more practical applications for HoloLens 2 in Swedish healthcare. Inspired by the project, Microsoft also plans to work with EY teams to use Minecraft in teaching children about the healthcare of the future.