Go to Terrapattern.com and try out something you’ve been waiting for: a visual search engine for satellite imagery. It’s still in alpha testing, meaning it’s more of a prototype than a product, and so it only allows you to search New York, San Francisco, and Pittsburgh. Still, it already feels like something important.
Testing it Out
I opened up the Pittsburgh data set and searched for “baseball field” (since it’s summer…).
Next, I clicked on the baseball stadium and the program dug up similar map tiles throughout the data set. It displayed the search results in a “geographic overview,” a gallery of thumbnails, and a “plot of their relative similarity.”
How well did it work? As you can see below, a number of my search results are high school track and football stadiums. Searching for “baseball diamonds” got much better results.
How’d They Do That?
Terrapattern, like a lot of impressive technology these days, is built on top of a “deep convolution neural network,” or (DCNN). Think of it as an AI-brain that has been trained to recognize patterns in visual input.
You might be surprised to find that it’s the training, not the coding, that’s the hard part.
“Terrapattern was only possible due to the astonishing crowdsourced mapping effort of the OpenStreetMap project, which has generously categorized large parts of the world with its Nominatim taxonomy. We trained our DCNN using 466 of the Nominatim categories (such as “airport”, “marsh”, “gas station”, “prison”, “monument”, “church”, etc.), with approximately 1000 satellite images per category. Our resulting model, which took 5 days to compute on an nVidia 980 GPU, has a top-5 error rate of 25.4%.”
After this, the program auto-classified a few million more satellite photos to make them searchable. (You didn’t think the neural net starts up every time you make a search, did you?). When you enter a search term, the engine finds places whose descriptions “are similar to those of the tile you selected.”
This is why it’s impressive but not perfect. When I found that “baseball field” and searched for similar map tiles, Terrapattern found the tiles that had the most similar descriptions. Meaning, I found a lot of stadiums, though not many of them included baseball diamonds. The search engine was doing its job just as it was supposed to.
The results weren’t perfect, I think, because I’ve never used this kind of search engine before and I didn’t quite know what to search for. Remember the first time you used Google? You probably searched in complete sentences rather than loading that search box up with search terms. I bet it will take us a long time to get used to visual search engines, too.
