Automated Assembly Instruction Generation from CAD Models Using Grounded Large Language Models: A Human-in-the-Loop Framework

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Automated Assembly Instruction Generation from CAD Models Using Grounded Large Language Models".

Tom: The gist The proposed framework formulates CAD-grounded assembly instruction generation by mapping STEP assemblies to a typed ProductGraph, deriving an assembly order through deterministic topological sorting,

Jane: First, who's behind it and why it matters.

Title and authors: Tom: Let's talk about who put this together: Aaron D’Souza, Mohammed Azeez Khan, Ashutosh Mishra, Arshaan Khan, Neha K. Nair, and Amar Kumar Behera. They are from a mix of engineering departments at institutions like NIT Warangal and IIT Kanpur.

Jane: It’s interesting to see engineers from different backgrounds coming together on something that mixes CAD interpretation with language generation. That variety suggests they're trying to build something broadly applicable rather than just a niche tool for one type of part.

Lu: They are tackling the problem of assembly documentation, which is traditionally a manual written by hand based on interpreting CAD models. The paper is focused on how structured product data and large language models can work together in this area.

Meng: The title itself tells us the main components: "Automated Assembly Instruction Generation from CAD Models Using Grounded Large Language Models: A Human-in-the-Loop Framework". That really captures the whole system—automation, grounding, LLMs, and human oversight.

Lalam: It’s about moving away from just letting an LLM wander around and generating text in a vacuum. This framework is specifically designed to ensure that the instruction writing process is constrained by engineering facts extracted directly from the CAD model.

The paper's summary: Tom: So, what’s the actual mechanism here? The authors propose formulating this as CAD-grounded assembly instruction generation. They take an input assembly, say S, which has parts and constraints from a STEP file, and they map it to this typed ProductGraph intermediate representation.

Jane: That ProductGraph is key because it carries all the necessary engineering info—the bill of materials, the parts themselves, and the constraints like mates or torque specifications. It acts as that central source of truth for the system.

Lu: The next step is deterministic reasoning, where they derive an assembly order using something called topological sorting on a directed graph built from those parts. This gives them a fixed sequence to work with before any language generation happens.

Meng: If there's a cycle in that ordering—if part A must come before B and B must come before A—they explicitly log that as an unresolved constraint inconsistency instead of just failing or guessing the order. That makes the diagnostic fallback sequence really useful for troubleshooting.

Lalam: And then, for each step in that determined order, they use a language model, but crucially, it only sees a slice of the ProductGraph and the specific rules relevant to that step. The language generation is conditioned only on that graph context.

The paper's improvements: Tom: Now let's look at what they suggest as improvements over previous work. They are focusing on separating the engineering state from the reasoning, which ensures "product state stays in the graph" and "order stays deterministic."

Jane: They are also using a very strict rule for language generation: it must be conditioned only on that graph slice and rules, meaning it has to use imperative wording, one sentence, and if there's a torque value or part name, it has to use that exact fact.

Lu: The verification architecture is another big improvement. They suggest a two-tier automated review process: first a deterministic tier checks names against patterns and second an AI-assisted tier uses things like FActScore and SelfCheckGPT to check for missing safety info or inconsistent quantities.

Meng: That layered approach to checking is smart because it handles both the structural correctness—the part names and quantities—and the contextual correctness, like making sure a safety warning is actually attached when it should be.

Lalam: They also suggest integrating geometry-aware reasoning by pulling kinematic mates from the STEP files into that ProductGraph representation. This means before generating language, you’re already considering insertion direction or stable poses based on the geometry itself.

Conclusion: Tom: So, to wrap up this paper on "Automated Assembly Instruction Generation from CAD Models Using Grounded Large Language Models: A Human-in-the-Loop Framework," the main implication is that you can reliably generate assembly instructions from CAD without just letting the language model invent facts.

Jane: The framework establishes a clear separation where deterministic reasoning handles the order, and the LLM acts purely as a realization layer for facts already vetted by engineering constraints. The human-in-the-loop control is essential because that's where accountability lies, not just in the text itself.

Lu: It shows how to use a ProductGraph as that stable interface between all these stages, moving from CAD interpretation right into grounded generation. The paper lays out a solid lineage for this type of intermediate engineering representation and how it can be used for manufacturing and inspection later on.

Meng: From an engineering standpoint, the fact that quantity and torque information stays locked in the ProductGraph is huge. It means those critical specs aren't lost or accidentally changed during the writing process.

Lalam: This paper demonstrates that assembly documentation can indeed be treated as a grounded transformation of CAD-derived product structure, proving that structured data controls the language realization rather than letting the language model drive it blindly.

Tom: That’s what we have today with this paper on "Automated Assembly Instruction Generation from CAD Models Using Grounded Large Language Models: A Human-in-the-Loop Framework." It’s a lot of engineering rigor put into making sure the instructions actually match the physical reality of the model.

Jane: It moves us closer to a world where technical documentation is automatically generated but still held accountable by human review. Next up, we’re going to look at some work on how AI models can predict missing data distributions and what that means for forecasting.