Augmented reality has already been demonstrated as a powerful technology for a host of applications - from virtual training opportunities to walkthroughs of buildings that have yet to be constructed. But NASA - ever the innovators - have come up with another use of augmented reality technology: to “holoport” a flight surgeon aboard the International Space Station (ISS).
While we may be most familiar with the practice of telemedicine because of the pandemic, astronauts in space have had to have medical attention from afar since the advent of space travel. In an innovative trial project, NASA has shared that they were able to send NASA flight surgeon Dr. Josef Schmid aboard the ISS using a combination of 3D capture and augmented reality technology.
In the released image of the interaction, Dr. Joseph Schmid, NASA flight surgeon, seems to hover inside the ISS as he flashes Spock’s “Live Long and Prosper” hand signal in greeting to the other remote participants.
According to the press release from NASA, this was accomplished with the help of a Microsoft HoloLens.
“Using the Microsoft Hololens Kinect camera and a personal computer with custom software from industry partner Aexa, ESA (European Space Agency) astronaut Thomas Pesquet had a two-way conversation with live images of Schmid and De La Pena placed in the middle of the International Space Station. This was the first holoportation handshake from Earth in space.”
HoloLens, which is a device that allows for pass-through vision, has been used by design, construction and other professionals to overlay location specific data onto a wearable display, allowing for virtual inspections, walkthroughs to detect clashes in preconstruction, and more.
Using the technology to capture and recreate individuals in real or near real time, however, is a different level of usage - which is what NASA is calling ‘holoportation.’
Holoportation is a type of capture technology that allows high-quality 3D models of people to be reconstructed, compressed and transmitted live anywhere in real time. When combined with mixed reality displays such as HoloLens, it allows users to see, hear, and interact with remote participants in 3D as if they are actually present in the same physical space. Holoportation has been in use since at least 2016 by Microsoft, but this is the first use in such an extreme and remote environment such as space.
Dr. Scmid says that this test represents a ‘complely new manner of communication across vast distances.”
“It is a brand-new way of human exploration, where our human entity is able to travel off the planet. Our physical body is not there, but our human entity absolutely is there. It doesn't matter that the space station is traveling 17,500 mph and in constant motion in orbit 250 miles above Earth, the astronaut can come back three minutes or three weeks later and with the system running, we will be there in that spot, live on the space station.”
NASA is demonstrating this new form of communication as a precursor for more extensive use on future missions. Plans are to use this next with two-way communication, where people on Earth are holoported to space and astronauts are placed back on earth.
This type of interaction goes far beyond the Zoom-style calls we’ve gotten accustomed to during the pandemic era, and could have far-reaching applications if the technology continues to develop.
While this has a very practical application for space travel, there are many situations on earth in which being physically co-located is not possible: offshore oil rigs, remote research stations, or even future space missions. Anywher it is difficult to be together, yet critical to have face-to-face human interaction may be the next arena for this type of technology to appear.