Written by: Dr. Florent Poux
The value in a 3D project has always been past the scan itself, to what you can turn that scan into.
But a “middleman”, overlooked, is a scene a machine can act on: a structured model a system can query, reason over, and change. not a point cloud a person has to handle manually.
This article is about how to make that leap on real projects, and why owning the pipeline beats renting five disconnected tools to get there.
Before this, let me share some interesting point from history. Around 245 BCE, the Library of Alexandria had a problem that will sound familiar. It held hundreds of thousands of scrolls, and almost nobody could locate a specific one.
The knowledge was all there, physically present, and practically useless. A heap of scrolls answers no questions.
A scholar named Callimachus fixed it. He compiled the Pinakes, a 120-scroll catalog that indexed every work by author and subject and cross-referenced them. He turned a warehouse into something you could interrogate.
The scrolls never changed. What changed is that you could finally ask the library a question and get an answer back. I guess you can make a parallel with Google, and now LLMs.
BUT, that is the gap in front of our industry right now.
We have “solved” capture. A crew can walk a building with a terrestrial scanner and leave with a point cloud accurate to a few millimetres, and a drone can georeference a quarry before lunch.
The data is beautiful (especially if you are on the 3D Gaussian Splatting train) but, on far too many projects, it just sits there.
Somebody exports a mesh, somebody else traces a few polylines by hand, and a deliverable that should take an afternoon takes a week.
The bottleneck is not the scan. It is everything that happens to the scan afterwards.
I want to be concrete about “afterwards,” because the distance between a beautiful cloud and a scene a machine can use is where the margin now hides.
From Scan to Knowledge
A raw cloud is our heap of scrolls. A human reads a room in it instantly, but a computer reads nothing.
Nothing in the file states “this surface is a wall, that gap is a door, and the door is 78 centimetres wide.”
Making 3D data useful to a system means climbing a ladder. A lot of workflows stop one rung too early.
The four rungs from a raw cloud to a scene you can question: geometry, then objects, then relationships, then a queryable state. It matters because a workflow that stops at rung two hands the client geometry and calls it a deliverable. (c) F. Poux
Geometry is the bottom rung: the raw cloud, XYZ coordinates plus maybe intensity or colour.
Objects are the next rung. Semantic segmentation labels every point with a class, then instance segmentation cuts those points into discrete things, so one cluster becomes a specific chair and one surface becomes the floor.
Relationships are the third rung. You wire the objects into a scene graph: the chair is on the floor, the door is in the wall, the corridor connects two rooms.
Queryable state is the top. Now you can ask the scene a question in plain terms, how many doors are narrower than 80 centimetres, and get an answer without a human tracing anything.
That top rung is what I mean by a scene a machine can act on. It is not a prettier cloud.
It is a structured model with semantics, instance identities, and relationships attached. That is the whole difference between data you look at and data a system reasons over.
This top rung has a name. It is the smart point cloud, a scan carried all the way to knowledge, where geometry, semantics, and relationships travel together, the idea set out in The Smart Point Cloud (Poux, 2019). The same relationship-first view runs through later work comparing engineered and learned featuring (Poux and Billen, 2019).
Here is that top rung, produced from a real indoor scan.
A raw interior scan carried up the ladder to a semantic scene: every point classified, every object counted, from walls and floor to windows, tables, chairs, and sofas. It matters because a scene you can count is a scene you can query, and counting is the first thing a raw cloud cannot do. (c) Florent Poux, NEURONES 3D
Once every point carries a class and every object carries an identity, the scene becomes countable. You read eleven chairs, four windows, and one sofa off the model without opening a measurement tool.
🪐 Note: This pattern is not new, and it is not really about 3D. Callimachus added no scrolls to Alexandria; he added structure, and that structure was worth more than most of the content. Raw logs become dashboards, loose files become databases, scans become scenes, and every time the value migrates from the measurement to the index built on top of it.
The dead-time tax of renting five tools
I want to share a massive optimization point, a cost nobody puts on an invoice. And this comes from consulting for x3 Fortune 500 companies as well as small surveying and geospatial firms. For all of these, I see a cost ladder by chaining separate tools.
One package handles registration, another denoises, a third meshes, a fourth runs semantic segmentation, and a fifth exports to the client format. Each tool does its slice genuinely well.
But between every pair of tools sits a manual export, a re-import, a format conversion, a coordinate-system check, and a coffee while something loads.
That in-between time is dead time. It is a tollbooth between every pair of tools, and you pay it in full on every project.
It is unbillable. It is invisible on any single tool’s demo. It quietly drains your return.
In a large firm it disappears into overhead. In a small firm it is brutal.
The person doing the exporting is the same person who should be winning the next job. And the time to get properly trained on five separate tools is time those firms do not have.
🦚 Florent’s Note: I have watched this pattern for fifteen years, and what still surprises people is where the hours actually go. It is rarely the algorithm that eats the day, it is the seams between the tools, the little translations nobody scoped and nobody bills. Add them up across a year and the dead time often outweighs the productive time.
So what is the alternative? It is not a bigger single vendor.
Actually, let me put that more plainly: the fix is not one giant suite that promises everything. The fix is owning the pipeline.
When the steps live in one system you control, the dead time between them collapses. The workflow stops being a relay race between products that were never designed to pass the baton.
What “act on the scene” really means
The top of the ladder in action: build a scene graph, test it against a goal, and where the room fails, rework it until it passes. It matters because this is a system doing something a measurement alone never could. (c) F. Poux
Let me make the top of the ladder tangible, because “a machine that acts on a scene” stays abstract until you watch it happen.
Take an accessibility question: can a wheelchair move through this room.
A recent research system builds a scene graph of a scanned interior, every object and every spatial relation between them, and is then handed that objective.
It does not just measure the doorway and print a number. It reasons over the layout, checks the clearances against the rule, and where the room fails, it proposes moving the furniture so the wheelchair path opens.
The scan stops being a record of how the room is. It becomes something the system can test and rework against a goal.
That is the entire shift in one example. The value is not in the measurement, which we already had. It is in the structure underneath it, which is what lets a system act.
The forthcoming study behind that example is worth reading for anyone tracking where this is heading: LLM-Supervised Point Cloud Processing: from Unsupervised 3D Scene-Graph Generation to Interactive Scene Manipulation
One piece of that pipeline is already practical enough to run on your own scans: open-vocabulary segmentation.
The same interior segmented by text prompt rather than a fixed class list: ask for “walls,” “floor,” or “door,” and the regions come back coloured. It matters because it removes the ground-truth bottleneck, labelling categories a model was never explicitly trained on. (c) Florent Poux, NEURONES 3D
Instead of training a network on a closed list of classes, you segment the cloud by text prompt. That sidesteps the usual ground-truth bottleneck, because you are no longer hand-labelling a dataset before you can ask a new question.
🦥 Geeky Note: The quiet hero here is the scene graph, just nodes and edges: objects as nodes, spatial relations as edges (on, in, near, connects). Once a room is a graph, a clearance rule becomes a graph traversal instead of a manual measurement, and “rearrange the furniture” becomes editing node positions and re-checking the edges. Modest data structure, and it is what turns a scan from a picture into something you compute against.
Where it pays, and where it honestly does not
I am not going to pretend this fits every job.
Structuring pays where the environment is repetitive and asset-heavy and the same deliverable returns again and again: building portfolios, plant and utilities, anything you scan on a schedule.
Structuring pays on repetitive, asset-heavy work and does not pay on one-off irregular sites. It matters because forcing automation onto the wrong job turns a profitable contract into an expensive science project. (c) F. Poux
There, the up-front cost of building the pipeline amortises across every project after it.
It does not pay on a one-off capture of an irregular site with no repeating structure.
On that job, a skilled technician tracing the few details that matter beats teaching a system to find them, especially where heavy occlusion and uneven point density make automated segmentation unreliable.
The honest read is that this is a portfolio decision, not a religion. Automate the repetitive, keep humans on the singular, and know which is which before you start.
🌱 Growing Note: If you want to test this without betting a contract on it, pick the single most repetitive deliverable you produce. Structure that one scene end to end, once, and time it against your current chained workflow. That measured comparison tells you more about whether to invest than any vendor demo ever will.
The deliverable of the future is a scene, not a cloud
The firms pulling ahead have understood one reframing. Clients never actually wanted a point cloud. They want the answers a point cloud contains, and a raw cloud makes them dig for those answers by hand.
A point cloud makes the client dig for answers by hand; a queryable scene hands the answers over. It matters because clients never wanted the cloud, they wanted the answers inside it. (c) F. Poux
A structured, queryable scene hands the answers over directly. It is a deliverable you can charge a premium for, because it saves the client the exact dead time it saved you.
A floor plan is the plainest version of that.
A georeferenced floor plan generated straight from the structured scene: room areas, furniture footprints, and dimensions, no draughting by hand. It matters because this is the queryable scene collapsed into a drawing a client can act on immediately. (c) Florent Poux, NEURONES 3D
That drawing came from the scan, not from a draughtsperson. The room areas and furniture footprints are read straight off the structured scene, georeferenced and internally consistent.
Getting there is more within reach than it was, and that is the encouraging part (what you see above is a full automated process from the raw point cloud, under 1 minute).
The building blocks for structuring scenes, visualising them, and letting an AI read and act on them already exist. You extend them rather than write them from scratch.
The methods are taught end to end with the 3D Geodata Academy courses supported by the O’Reilly book 3D Data Science with Python.
What separates the firms that do this is no longer a research budget. It is a week or two of deliberate upskilling, and the decision to own the pipeline instead of renting it in five disconnected pieces.
Callimachus did not write the scrolls. He built the catalog, and the catalog is what everyone remembers.
The deliverable of the future is the same idea: not the cloud, but the queryable scene you build on top of it and own.
About the Author
Dr. Florent Poux is a 3D geospatial AI researcher and educator with fifteen years in 3D geoAI R&D, with first steps in land surveying. He founded the 3D Geodata Academy (learngeodata.eu), where professionals learn to build, run, and extend their own spatial AI pipelines rather than renting a chain of disconnected tools. His work spans a real-time 3D software stack (Neurones 3D) with an accompanying SDK, hands-on certifiable course + training, and fractional 3D development and CTO advisory for firms building internal capability. He is the author of 3D Data Science with Python (O’Reilly).
