“We believe these types of 3D scanning technologies, will bring 3D acquisition to a new audience, where lightweight capture of the real-world becomes truly ubiquitous.”
This week Microsoft announced MobileFusion, a software pipeline for 3D scanning objects that runs “purely on standard, off-the-shelf mobile phones.”
MobileFusion performs all capture using the device’s existing RGB camera and all computation using the integrated processor. “Everything happens on the phone itself,” said Pushmeet Kohli, a principal research scientist with Microsoft Research in a prepared statement.
In a paper describing MobileFusion, Microsoft calls it superior to other systems for 3D scanning on a mobile device because they require active depth sensors or high end GPUs to function. “This limits object scanning scenarios to PCs or high end laptops and tablets, with custom hardware,” they explain.
Some attempts at solutions integrated into the mobile device, such as Google’s Project Tango, raise cost and power requirements by adding extra sensors and extra processing hardware. Microsoft cites this as a reason that such solutions have failed to achieve ubiquity.
Other current solutions take a different path to make mobile scanning possible, utilizing cloud-based processing to sidestep the limitations of the smartphone processor. However, this type of processing requires an internet connection to function, restricting object scanning to areas with wifi or cell-phone reception capable of communicating large amounts of data.
Microsoft says that since their MobileFusion solution does not require additional sensors or an internet connection, “scientists in remote locations or hikers deep in the woods can capture their surroundings using a regular cell phone.”
To use MobileFusion, a user first points the phone at an object to be scanned and then places a “volumetric grid” around it. As the user circles around the object to capture it from every angle, MobileFusion builds the scan on screen in real time. This helps the user to see their progress and position the phone properly for capture. It also helps MobileFusion to perform complex stereo-matching and six-degree-of-freedom estimation to reduce scanning errors.
Notably, the technology captures 3D surface models rather than point-based models. Microsoft explains their reasoning, saying that “surface models encode connectivity information, allowing real-world geometry to be modeled in a spatially continuous manner. These connected surface models also seamlessly integrate into existing 3D applications and tools e.g. for viewing and editing 3D content, printing or game physics.”
Given the focus on surface models, the MobileFusion system will likely be best at creating visual content where the accuracy is a secondary consideration. In other words, it will be suited for the creation of content that an ordinary consumer would use. As Microsoft explains, “we believe these types of 3D scanning technologies, will bring 3D acquisition to a new audience, where lightweight capture of the real-world becomes truly ubiquitous.”
Researchers are currently working to ensure that the system works on “all types of mobile devices,” and hopes to make it available to the general public “in some form, but there are no firm plans right now.”
