Accurate, high-resolution 3D city models are increasingly in demand for defense and security applications as well as civilian needs for urban planning and conservation. SPAR 2006 attendees learned how one company uses terrestrial 3D laser scanning and airborne LiDAR to generate digital 3D simulations of urban settings. David Colleen, CEO of Planet 9 Studios, detailed a DARPA project to let soldiers on the battlefield use 3D scene models to control UAVs – the unmanned aerial vehicles deployed in Afghanistan, Iraq and other hot spots to gather intelligence and sometimes deliver strikes. On the home front, Colleen described scanning and modeling Watts Towers – a Los Angeles folk-art landmark on the National Register of Historic Places – to create a real-time 3D visual interface to the structure’s engineering conservation database.
Colleen’s company develops 3D “virtual cities” for architects, engineers, researchers, film producers, game developers and others who need realistic city backdrops, and provides custom 3D modeling, animation, interface design and application development. In the UAV guidance project, Colleen describes how the U.S. Department of Defense “wanted a system where a soldier in the field with a tablet computer or a cell phone could use a 3D interface to point to a particular part of a building – say, a door, window or roof – and task UAVs for reconnaissance. For example, if he thinks there’s a sniper on the roof, he can task a swarm of UAVs to fly around and give him a video of what’s on that roof, or to notify him when somebody walks out of a door. The premise was that a 3D interface would be an easy way for a soldier to do that in the field.”
To this end, the military launched a development program it calls HURT – Heterogeneous Urban RSTA (Reconnaissance, Surveillance and Target Acquisition) Team. The $11.6 million program is funded and managed by the Defense Advanced Research Projects Agency (DARPA), the central R&D organization for DoD, with the Air Force Research Lab serving as technical and contracting agent. Lead contractor is Northrop Grumman‘s Integrated Systems sector; the development team also includes SRI International, Planet 9 and others.
In September 2005, the HURT team gave its first demonstration at the former site of George Air Force Base in Victorville, CA, where a grid of abandoned streets and buildings is used to train military forces for urban warfare. According to Northrop Grumman, the exercise began with four small UAVs – two Ravens and a Pointer (both fixed-wing aircraft) and a RMAX helicopter – deployed at low altitudes under HURT system control. Participants on the ground with handheld computers played the role of military forces in the field. The warfighters viewed surveillance images of the entire area, but could also request specific information about a suspected enemy position by moving a cursor over the subject on the handheld monitor. The HURT system autonomously prioritized each request and directed the most suitable UAV to the location for a closer look while maintaining continuous broad-area surveillance by the other UAVs.
Data collection and modeling work process
To prepare for that demonstration, Planet 9 engaged Airborne 1 Corporation, an aerial LiDAR service provider based in El Segundo, CA, to capture data needed to model the Victorville site. “We don’t own a laser scanner; we may never own a laser scanner,” says Colleen. “We work with people who do in different parts of the country. Sometimes we need ground-based laser scanning at high resolution; sometimes we need it from aerial sources. In this case we worked with Airborne 1. They were tasked with making a LiDAR mesh at spacing of four inches or better at ground level.”
“Also, we gave them a tough task that they had never been asked for before,” he continues. “We wanted to be able to perceive the dimples of windows and doors. But what we found was that their laser scanning rig mounted in an airplane is tilted at 20 degrees, at best. After flying for two days, they had many passes, which they correlated into two meshes: a first-returns mesh for buildings and foliage, and a second mesh scraped of that feature data to give a ground plane. In inspecting the mesh, we found right off that our notion of scanning and getting oblique data on the sides of buildings was useless. We learned that to get that data, we would either have to go from a terrestrial source – somebody driving a truck around – or else [try]mounting the airborne scan heads at a different angle.”
In preparing the model, Colleen reports his company found that “current polygon decimation tools are not adequate for our needs, so LiDAR-derived data points were used as guidelines for traditional polygonal modeling.” What were the limitations? “Our day-to-day production tools are GIS tools and 3D computer animation tools, in this case 3D Studio MAX,” Colleen explains. “The polygonal mesh was brought into 3D Studio MAX, and that’s where our artists began. We would like to have had software for reducing the mesh down to its planes. I’ve seen a number of tools that let users manually fit a plane or a series of points to describe a plane, and reduce the mesh by replacing the scan points with polygonal planes. But in all our experience testing these automated tools, we’ve found it faster and cheaper to do it manually with our existing 3D software. That’s unfortunate because if we had an automated approach to this, we could have saved a lot of time and money.”
“The resulting polygonal mesh [shown in Figure 1]is a high-performance mesh geared for running in the game engines that we use,” Colleen concludes. “In this case, the result was not geared for a Hollywood pretty picture; it was a functional interface to let a soldier click on those doors or windows.”
Urban modeling – unique needs
Colleen enumerated how urban simulation models differ from engineering-grade models created from laser scan data. “Most of the data that I see being collected by the laser scanning community is extremely accurate, extremely dense data,” he says. “We operate at the other end of the spectrum. To get things to work on your grandmother’s computer, we need very small file sizes, very compact data, in what we refer to as low-polygon structures.” For example, Colleen describes a building model created by Planet 9 for a computer game. “It’s a very realistic-appearing structure, but under the hood, the data is very sparse. That’s to get the frame rate up, and to keep the data small so it can fit on a CD or be transmitted easily across the Internet.”
Another difference – “all the data that we work with is polygons with photographs applied to them,” he says – “that is, not point clouds with RGB values attached, but a polygon structure with a texture map attached using texture mapping coordinates.”
“Perhaps the biggest divide between what we do and what most [terrestrial 3D laser scanning practitioners]do,” Colleen continues, “is that we need polygonal meshes that are watertight, with no holes, no occlusions, no data spikes – good, real-world representations of the places we’re trying to depict.”
“And last, we need the data to be geo-registered,” he concludes. “In the earliest days it didn’t matter, but then the Google folks began changing things.”
Part 2 next week – scanning and modeling Watts Towers; lessons learned; next-generation wish list.
About Planet 9 Studios
Planet 9 Studios (San Francisco, CA and Orlando, FL) delivers PC- and Internet-ready solutions for urban and security planning, training, situational awareness, and public relations by implementing geo-specific, real-time, computer-based 3D simulations of urban settings. A leading supplier of 3D city data, the company has produced more than 44 virtual cities, which are licensed by architects, engineers, researchers, film producers, game developers and others interested in using realistic city backdrops in animations and real-time applications. Planet 9 also provides custom 3D modeling, animation, interface design and application development, and has been involved in defense research projects, working with SRI International, the CIA, NASA, Sandia National Labs and Booz Allen Hamilton.