Facebook’s bad year is starting to get at least a little bit better (and definitely not because of its dating service) . For one, the Oculus Go VR headset is getting great reviews in industry rags. More pertinent to SPAR 3D, however, is a series of announcements that shows the company making big steps toward making 3D data an intrinsic part of its platform.
New cameras for 3D video capture
First up, Facebook has partnered with RED, who you probably know as the maker of some frankly insane 8K digital video cameras capable. Together, the two companies are building a 6DoF camera that captures all the data necessary for gen
erating immersive VR experiences in post-production.
Facebook says the device will be powered by its “industry-leading depth estimation technology,” but has remained tight-lipped about what, exactly, that means. We know this much: It’s an AI-powered algorithm that uses photogrammetry to generate 3D models.
Practically speaking, the camera will be able to capture 3D depth information for every object in a scene. Combine that with the 6DoF capture, and it will allow users to shoot “infinite perspectives from any direction within a field of view so your audience can explore the scene freely.”
3D photos/3D point cloud memories
Facebook made another 3D announcement at its recent F8 developers conference: the depth-estimation technology will also be put to work in a feature that generates 3D photos, and in an experimental feature that enables users to recreate “memories” in VR.
The 3D photos feature generates photos that change perspective when the viewer tilts her phone or moves through her news feed. It’s not precisely 3D, but it’s close enough for rock n’ roll.
The VR memories feature is more ambitious. It converts still photographs and videos into low-fi point clouds, which engadget explains as meaning that “you could conceivably transform a photo of your 10th birthday party into a virtual recreation of childhood living room.”
Since Facebook doesn’t have full image coverage of your 10th birthday party (probably), it works with what it can find, resulting in decidedly low-fi point clouds. These low-fi point clouds, however, could offer the ideal aesthetic for VR recreations of indistinct memories. Commentators have described the experience as like being in a pointillist painting dotted with floating photographs.
Here’s engadget again:
“Facebook told me that this pointillism effect is used to help fill in the gaps, and to ‘create a sense of presence in the space.’ But more than that, the fuzzy nature of the environment also evokes a sense of memory. After all, most people don’t dream in HD. Instead, dreams and memories are misty and hazy, and a pointillistic world where objects are blurry and ill-defined contributes to that feeling immensely.”
As “cool” as it all is, this memory feature looks a lot like the point of a wedge. This feature could allow the company to recreate, map, and store the physical spaces where we live and play. If you consider the company’s track record with user data to be dicey—and I certainly do—you might find the potential risks of “recreating your memories” in VR to be too great.
