2.5-D Decomposition for LLM-Based Spatial Construction

summary

Video file (mp4)

The gist

The paper "2.5-D Decomposition for LLM-Based Spatial Construction" addresses the critical challenge of enabling Large Language Models (LLMs) to perform complex, geometrically consistent spatial

In short

The episode discusses "2.5-D Decomposition for LLM-Based Spatial Construction," a method that improves AI's ability to build in 3D space. The technique separates complex spatial planning from simple height calculations, allowing an LLM to focus only on flat coordinates while code handles the vertical stacking, significantly boosting accuracy.

Key concepts

2.5-D Decomposition
This method improves AI's ability to build in 3D space by separating the task. Instead of asking an LLM to handle all three dimensions, it only plans where blocks go on a flat floor (2D), while separate code handles the height dimension.
Neuro-symbolic approach
This approach combines the creative capabilities of Large Language Models (LLMs) with the reliability and precision of traditional, deterministic code. This combination allows for high-level planning while ensuring accurate execution of specific calculations.
Adaptive prompt enrichment
This technique is used to prevent errors in AI instructions before they occur. The system scans the input instructions for vague or tricky phrases and injects a specific example to guide the model toward clearer understanding.

Terminology used across episodes

This episode discusses

The paper

2.5-D Decomposition for LLM-Based Spatial Construction · Read on arXiv

Rockwell Automation

Autonomous systems that build structures from natural-language instructions need reliable spatial reasoning, yet large language models (LLMs) make systematic coordinate errors when generating three-dimensional block placements. We present a neuro-symbolic pipeline based on 2.5-D decomposition: the LLM plans in the two-dimensional horizontal plane while a deterministic executor computes all vertical placements from column occupancy, eliminating an entire class of errors. On the Build What I Mean benchmark (160 rounds), GPT-4o-mini with this pipeline achieves 94.6% mean structural accuracy across 12 independent runs, within 3.0 percentage points of the 97.6% ceiling imposed by architect-agent errors that no builder-side improvement can address. This outperforms both GPT-4o at 90.3% and the best competing system at 76.3%. A controlled ablation confirms that 2.5-D decomposition is the dominant contributor, accounting for 28.7 percentage points of accuracy. The pipeline transfers directly to edge hardware: Nemotron-3 120B on an NVIDIA Jetson Thor AGX achieves 96.0% mean structural accuracy with the identical pipeline, slightly exceeding the cloud result. Expanding the system prompt by four targeted examples to exceed the model's 8,320-token prefix cache page size, combined with low-effort reasoning, reduces mean per-request latency by 3X to 19.7 seconds at 95.6% accuracy. The underlying principle, removing deterministic dimensions from the LLM's output space, applies to any autonomous construction or assembly task where gravity or other physical constraints fix one or more degrees of freedom. A transfer experiment on 500 IGLU collaborative building tasks confirms the effect generalizes beyond the primary benchmark.

DOI: 10.1109/NAECON70028.2026.11674959

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "2.5-D Decomposition for LLM-Based Spatial Construction".

Jane: The paper was written by Paul Whitten, Li-Jen Chen and Sharath Baddam from Rockwell Automation.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Title: Tom: We're looking at a fascinating new paper called "two point five-D Decomposition for LLM-Based Spatial Construction" by Paul Whitten, Li-Jen Chen, and Sharath Baddam.

Jane: The authors are all from Rockwell Automation, which gives a very interesting industrial context to this research.

Tom: Jane, that title sounds like something straight out of a geometry textbook.

Jane: It does, but it's actually describing a way to help AI understand how things stack up in the real world.

Tom: So, are they saying AI is currently bad at building things?

Jane: They've found that when you ask these models to place blocks in three dimensions, they make constant mistakes with the height.

Meng: That makes sense from an engineering standpoint because LLMs aren't calculators by nature.

Lu: I see it as a massive opportunity to give these digital minds a physical sense of gravity!

Lalam: This research could fundamentally change how we perceive the relationship between language and physical space in our culture.

Tom: Lu, do you think this is just about blocks, or is it something bigger?

Lu: I think this is the first step toward autonomous machines that can interpret a human's dream and build it physically.

Meng: We have to be careful not to get too ahead of ourselves, though.

Meng: I'm more interested in whether this can actually work on the low-power hardware we use in real factories.

Jane: That's actually a huge part of what they're testing, isn't it?

Tom: It really is, and that leads us right into how they actually structured this whole system.

Summary: Tom: To recap, we're discussing "two point five-D Decomposition for LLM-Based Spatial Construction," and they've found a way to stop AI from failing at height calculations.

Jane: The main trick is that they don't ask the LLM to worry about the vertical axis at all.

Tom: They call it two point five-D decomposition, right?

Jane: Exactly, the LLM only plans where the blocks go on a flat floor, and then a separate piece of code handles the stacking.

Meng: So the LLM picks the coordinates on the ground, and the "deterministic executor" does the math for the height?

Jane: You've got it, Meng, because the height is just a result of how many blocks are already in that column.

Lu: This neuro-symbolic approach is brilliant because it combines the creativity of the LLM with the precision of traditional code.

Tom: It's also incredibly effective, since they used the "Build What I Mean" benchmark to prove it.

Jane: They showed that GPT-4o-mini, which is a smaller model, actually beat the much larger GPT-4o.

Tom: That's wild, because the smaller model hit ninety-four point six percent accuracy while the big one only hit ninety point three percent.

Meng: That tells me the architecture is doing the heavy lifting rather than just raw model size.

Lalam: It shows that intelligence can be found in how we organize tasks, not just in how many parameters a model has.

Lu: I love how this proves that we don't need a giant brain for every single tiny calculation.

Tom: It's a much smarter way to use the resources we have, and it sets the stage for some really clever error correction.

Improvements: Tom: We've covered the two point five-D part of "two point five-D Decomposition for LLM-Based Spatial Construction," but the researchers added several other layers to make it bulletproof.

Jane: They implemented something called "adaptive prompt enrichment" to catch errors before they even happen.

Tom: Jane, how does that work in practice?

Jane: They scan the instructions for tricky phrases and then inject a specific example to guide the model.

Meng: I noticed they also mentioned a "peephole plan verifier" to fix things after the fact.

Tom: That's like a quick spell-check for spatial plans, isn't it?

Meng: It's exactly like that, and I'm also impressed by their results on the NVIDIA Jetson Thor AGX.

Jane: They managed to get ninety-six percent accuracy on that edge hardware, which is huge for real-world deployment.

Lu: Imagine a robot in a construction site that can correct its own mistakes in real-time!

Lalam: This level of self-correction will help humans trust autonomous systems in our shared physical environments.

Tom: They even addressed "underspecification," which is when the human instruction is a bit vague.

Jane: Instead of guessing and failing, the agent is programmed to ask a clarification question if the color or count is missing.

Meng: That's a very practical way to handle the messiness of human communication.

Lu: It's like the machine is finally learning to say, "I'm not sure, can you tell me more?"

Tom: It really brings everything together, from the high-level planning down to the tiny hardware optimizations.

Conclusion: Tom: We've reached the end of our look at "two point five-D Decomposition for LLM-Based Spatial Construction."

Jane: This paper really shows that we can make AI much more reliable by simply narrowing its focus.

Tom: By removing the dimensions that are easy to calculate with code, they've unlocked much higher accuracy.

Lu: I'm leaving this session feeling like the gap between digital thought and physical action is closing fast.

Meng: I'll be watching to see how these neuro-symbolic pipelines get integrated into industrial robotics.

Lalam: I believe this will foster a new era of collaborative building between people and machines.

Tom: Thanks to everyone for joining us today.

Jane: We'll see you next time for another deep dive!

Tom: Goodbye, everyone!

More episodes

← Home