The leaps forward in the field of photogrammetry have produced some amazing technological advances with any number of potential applications. But have you considered the potentially insidious uses for the technology? Four researchers from Indiana University’s School of Informatics and Computing have.
In a project Robert Templeman, Zahid Rahman, David Crandall, and Apu Kapadia have put together with the assistance of the Naval Surface Warface Center, the four researchers show how photogrammetry technology, in combination with mobile malware, could be used to surreptitiously create 3D models of indoor environments using only a user’s mobile phone’s camera. Or to be precise:
This paper introduces a novel “visual malware” called PlaceRaider, which allows remote at- tackers to engage in remote reconnaissance and what we call “virtual theft.” Through completely opportunistic use of the phone’s camera and other sensors, PlaceRaider constructs rich, three dimensional models of indoor environments. Remote burglars can thus “download” the physical space, study the environment carefully, and steal virtual objects from the environment (such as financial documents, information on computer monitors, and personally identifiable information). Through two human subject studies we demonstrate the eectiveness of using mobile devices as powerful surveillance and virtual theft platforms, and we suggest several possible defenses against visual malware.
Yikes. But it makes sense, obviously. If the guys from Rome in a Day can recreate the major tourist attractions of Rome in 3D using nothing but found photos on Flickr taken by thousands of different people and cameras, surely it can’t be difficult to recreate the interior of an office or another other location using the photos grabbed every time someone takes a phone out of a pocket to check the time, email, or a text.
It looks something like this:
Oh, and these guys have thought of my Flickr objection: “To our knowledge, ours is the first work to apply reconstruction algorithms to data of this type, since most work has studied thoughtfully taken photos (e.g. obtained from consumer photo-sharing sites like Flickr). To facilitate economical computa- tion and reduce storage and network load, we develop heuristics using image quality metrics and sensor data that quickly identify the small fraction of images likely to contain valuable evidence and discard the rest.”
Are we sure these dudes are working for the good guys?
Anyway, I’m sure you can guess the findings of the paper: Essentially, yes, they can put a Trojan Horse-style piece of malware on a mobile device rather easily, quickly obtaining all the necessary permissions from the operating system, and then capture not only a 3D model of the interior, but also a way to zoom in on potentially useful items in the scene, like checks and computer screens with juicy information on them.
Of most interest to us in the 3D data capture world, however, are their findings on how and when you can successfully create the 3D models. There are some very interesting tests in the paper on quality of model in relationship to the size of the image and a smart workaround for discarding photos that aren’t useful to the model creation. But here’s the conclusion that matters:
These results suggest that faithful 3D models of a space can often be generated from opportunistically captured images. This is a somewhat surprising result because most Structure from Motion approaches were designed for use with deliberately composed images. For example, some approaches assume that images have a low in-plane rotation twist angle (i.e. were taken such that the camera was level with respect to the ground). As Figure 1 shows the images used in our study are often of very low quality, with high twist angle, motion blur, poor focus, and random composition. The accuracy of the test models generated from these photographs demonstrates the ability of the feature detection and feature-key-matching algorithms to contribute to model generation despite the poor photography inherent to opportunistic image collection.
Just say we weren’t trying to use this technique for evil purposes. To what other uses could we put technology like this? Certainly undercover police and the like could collect very useful data without attracting too much attention. Is there a search-and-rescue application with robots collecting photos in dangerous locations and streaming them back? It seems like the possibilities are many and varied.
Nice work, guys. Now just don’t tell any criminals what you’re up to…
