When people think of virtual and augmented reality as well as other immersive media technologies, many focus on the entertainment aspect, but these tools are becoming an asset in medical care.
In May, the University of Maryland School of Medicine announced a partnership with the University of Maryland, College Park, and the University of Michigan to create the Center for Medical Innovation in Extended Reality (MIXR). Founded through $5 million from the National Science Foundation’s Industry-University Collaborative Research Centers Program, the center aims to accelerate the development of these technologies for use in clinical trials and eventually, more broadly, in medical care.
Companies such as Microsoft, Meta, Google, and others will provide funding and expertise to the team to develop, test and certify these technologies for use in the medical field.
“Virtual reality has many uses in the healthcare environment,” said Amitabh Varshni, dean and professor at the University of Maryland, College Park, School of Computer, Mathematical and Natural Sciences and principal investigator at MIXR. “For training, we’ve actually done studies showing that people can retain information better – about 9% better – than if they were viewing the same information on a 2D desktop screen.”
The staff has also developed virtual reality training prototypes for specialized surgical techniques such as emergency fasciotomy where the fascia is cut to relieve tension or pressure to address the resulting loss of blood circulation in an area of tissue or muscle.
For augmented reality, the team developed a point-of-care ultrasound prototype that projects the information directly to the patient so the clinician doesn’t have to keep looking away at the screen.
“These examples are only the beginning,” says Varshney. “With the additional momentum and synergies that our new center will bring—including working with federal regulatory experts to bring new devices and technologies into clinical settings faster—we anticipate a time in the not-too-distant future when immersive-only headphones will be as common in a hospital setting as the stethoscope.”
Officials note that MIXR is needed due to the rapid movement in the private sector to develop new immersive technologies used in gaming, entertainment, education and training. This has filtered down to scientists and clinicians using these same visualization tools in a clinical setting or for advanced medical training.
“We believe our new center will be a focal point for the industry to collaborate – at the highest level – with academia and healthcare professionals to build, test and certify new devices that can dramatically improve patient care and medical training,” said Varshney.
In 2017, Varshney and Dr. Sarah Murthy launched initial work with the Maryland Mixed Reality Center (MBRC).
“Working with Dr. Sarah Murthy and colleagues in Baltimore has been very exciting and rewarding,” he said. “They represent the best in emergency medicine. Now, with additional input from technology leaders such as Google, Microsoft and others, we believe we have developed a critical mass to move our ideas forward quickly and efficiently. The common theme of using technology to improve patient outcomes has driven our efforts from the start. This is satisfying. Especially for me as a computer scientist.”
MBRC will continue to work on other immersive projects not directly related to medicine and healthcare including tacit bias training and the use of immersive environments to train foreign language professionals at a very high level. They have also collaborated with artists and performers to bring new ideas in classical music and opera to the stage.
“So, while some of the new activities of MIXR may overlap with our work at MBRC, we consider them to be separate, but complementary, entities,” Varshney said.
Some of the new activities that MIXR staff are exploring have not yet been used in a largely medical setting. Murthi works to help patients deal with physical and emotional trauma by immersing themselves in another world with a focus on quadriplegic patients who are hospitalized due to acute spinal cord injury. Another collaborator, Dr. Luana Colloca, is a medical scientist who uses immersive techniques to reduce the need for addictive opioid pain relievers.
Varshney and colleagues at UM are completing a HoloCamera studio that includes more than 300 immersive cameras combined to create 3D visualization images to help train healthcare providers to perform challenging medical procedures.
“We are in the final stages of addressing the technical challenges that arose in integrating 300 immersive cameras,” he said. “The system works but we need it to run smoothly for what we have in mind — advanced training for emergency medical procedures. We expect to work with our partners in Baltimore on user study training scenarios over the next six months.”
The collaborators planned to start three days at College Park in October to bring all MIXR partners together including scientists, clinicians, private technology companies and federal regulation experts. This event is designed to share ideas on their agenda for the near future and the next five years. “We’re definitely excited about what’s coming next,” Varshney said.