Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification".

Jane: Recent advances in Large Language Models (LLMs) have enabled workflows that generate SystemVerilog Assertions (SVAs) from natural-language specifications, with the potential to accelerate Formal Verification (FV).

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Welcome back everyone! We've got a fascinating paper on arXiv today that I think is really going to make waves in the verification space. It’s titled "Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification." Jane, you've been looking over this material—what exactly is the big idea here?

Jane: Thanks, Tom. Well, what this paper does at its core is tackle a real headache in generating SystemVerilog Assertions from plain English specs. They point out that when you just feed an LLM a spec and an RTL block, the connection between the two gets fuzzy, which leads to syntax errors later on. This work introduces a structured Knowledge Graph to fix that by building context from different parts of the design data.

Lu: I find this approach incredibly compelling because it moves beyond just text processing; they are structuring inputs into these specific Intermediate Representations like `spec chunks.json` and `design model.json`, which really forces the AI to see the hierarchy instead of just reading flat documents. This level of structural grounding is what allows for meaningful context retrieval during property generation, which is something I’ve been exploring in my work at Tsinghua lately.

Meng: From an engineering standpoint, I'm interested in how this KG handles the complexity of real RTL code. They are using formal tool analysis outputs to populate the `design model.json` with things like FSM encodings and signal bit-widths, which means the AI isn't guessing what a signal is; it’s getting hard facts about it. That level of precision is what we need to move from experimental results to reliable verification.

Lalam: If I look at this through my lens, the most impactful vision here is how this moves AI capabilities beyond just writing code; it enables a system that understands the entire design ecosystem. This architecture allows for three distinct refinement loops—syntax repair, CEX correction, and coverage augmentation—which means the AI isn't just generating one assertion; it’s iteratively fixing and improving it based on formal results. That level of self-correction is how we build truly reliable verification tools.

Tom: That iterative refinement sounds intense, Lalam, but I see how that solves the problem of messy LLM outputs. So, to break it down further, what is this Knowledge Graph actually constructed from? It sounds like a lot of moving parts.

Title and authors: Jane: The Knowledge Graph is built by taking structured Intermediate Representations or IRs from several sources—the specification chunks, the requirements text, the RTL metadata from formal tools, and even feedback from those formal runs. These IRs are normalized into typed JSON files to make sure everything is deterministic when we feed it into the multi-agent system.

Lu: The construction of the KG is what makes this work verification-centric, meaning every piece of data has a defined role in the overall workflow; they map out things like requirement-to-specification links in `tracelinks.json` and property-to-coverage relationships. This traceability is crucial because it gives the AI a roadmap to know exactly where to look when it needs context for a specific task, like fixing a syntax error or finding a coverage gap.

Meng: That mapping sounds like essential infrastructure for any robust system; I’m thinking about how easy it would be for us to audit the results later if we had this structured context linking every assertion back to its original requirement. It moves the verification from a black box process to something traceable.

Lalam: Precisely, Meng; that traceability is what elevates the output from just code generation to verifiable engineering artifacts. This structure allows the AI agents to perform targeted context retrieval, meaning they don't have to sift through millions of lines of text randomly when they need a specific piece of design metadata or a formal result. That contextual awareness is key for improving our culture around verifiable design.

Tom: So, the paper suggests these structured inputs and mappings are the key to moving from loose text processing to something that has actual engineering context, leading us into what they call the multi-agent orchestration workflow. How does this translate into actual work on generating assertions?

Jane: The workflow breaks down property generation into different specialized agents, starting with the `sva lead` overseeing everything. The `spec analyst` takes the natural language requirements and decomposes them into timing constraints and exceptions, which then gets translated by the `sva author` into actual SVA code.

Lu: I’m particularly interested in the syntax correction agents; they use compilation diagnostics to attribute errors directly back to specific properties using a `syntax analyzer`. This deterministic repair mechanism, guided by the design model IR, seems like a very practical way to handle the messy output that LLMs often produce when they try to write complex syntax.

Title and authors: Meng: That sounds like a necessary layer of defense; having an agent specifically tasked with parsing error messages and linking them back to specific signals in the RTL is exactly what we need when we start scaling these LLM-driven tools. It grounds the correction in reality, not just linguistic pattern matching.

Lalam: And then you have the counterexample correction agents and coverage improvement agents working in parallel. This means if a formal run fails, there’s an agent that analyzes the VCD paths to suggest constraint adjustments, or another agent looking at coverage metrics to find dead code regions to target for new properties. It's a complete feedback loop built into the process.

Tom: That sounds like it handles both the creation and the fixing of properties automatically based on formal results, which is a huge step forward in automation. So, what about those three refinement loops they mentioned—syntax repair, CEX-guided correction, and coverage augmentation? How do they interact with that Knowledge Graph?

Jane: The Knowledge Graph supports these loops by acting like an artifact registry with property-granular updates. When an agent needs to make a repair decision, it queries the KG to find the relevant neighborhoods—for instance, in CEX correction, it retrieves VCD paths from the `formal results.json` entity.

Lu: The way they define these neighborhoods allows for bounded context traversal; an agent doesn't just get a general overview of the design, it gets exactly what it needs for its specific task, which is much more efficient than having the AI process everything at once. This targeted access to information seems like the real innovation here.

Meng: From an operational standpoint, that bounded context retrieval means we can test these agents on specific failure modes with very focused data sets rather than having them wander around trying to figure out the whole system. It makes debugging the LLM workflow much more tractable for real-world applications.

Lalam: This entire architecture moves us toward a highly autonomous verification pipeline where the AI doesn't just follow instructions; it uses its internal, structured knowledge base to navigate complex design problems and self-correct based on formal evidence. It’s about giving the AI a verifiable memory of the design.

Tom: It sounds like we are moving toward a system where the AI can truly act as an autonomous debugging agent, not just a code generator, by using this KG to ground its reasoning in formal facts. Before we wrap up this discussion on "Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification," what do you all think about these potential implications?

Title and authors: Jane: I think the primary implication is that we can generate much higher quality SVAs with significantly lower syntax error rates than we currently see from LLMs alone. This means faster turnaround times for design verification projects.

Lu: The deeper implication, I think, is the improved specification-to-RTL grounding they achieve; it bridges the gap between what a designer writes and what the formal tool understands. It suggests that for complex micro-architectural details, this structured context is not optional but necessary for reliable AI assistance.

Meng: I see this translating into reduced verification cycles, which means faster product development timelines because we spend less time debugging assertion syntax and more time focusing on the actual functional verification. It’s about increasing engineering velocity through better tooling.

Lalam: From a cultural standpoint, this work shows us how AI can be integrated into high-assurance environments by making its outputs auditable and traceable; it shifts the focus from just trusting the output to understanding the provenance of that output.

Tom: That's a lot to take in, but it sounds like we are looking at a system that provides complete, auditable traceability from a high-level requirement all the way down to the formal proof status. It’s really about building trust back into the verification process.

Jane: Exactly; it moves us toward achieving higher formal coverage because these agents are proactively looking for gaps identified in coverage metrics and filling them with targeted properties. We're not just checking what we told the AI to check; the AI is finding what we missed.

Lu: The future work mentioned suggests extending these KG capabilities, which could allow for even more sophisticated reasoning across different domains, maybe integrating temporal dynamics from things like path signatures with static design structure. That points toward much richer, more context-aware AI assistants.

Meng: I wonder if we can apply this KG concept outside of chip design; the structured IR approach seems adaptable to other complex systems where requirements and implementation details are highly coupled. It’s a general framework for grounding AI in engineering realities.

Lalam: Ultimately, this paper, "Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification," shows how to build an environment where the AI uses formal results and design structures as its primary source of truth for generating assertions. It’s about giving the AI a structured understanding of reality.

Title and authors: Tom: Well, that's a lot to wrap up on, but it really highlights how crucial structure is when we try to make LLMs do high-stakes tasks like formal verification. We’ll be looking for more papers like this one as we push these agentic systems further.

Jane: I think the main thing listeners should remember is that by structuring the context into a Knowledge Graph, we can get SVAs that are not only syntactically correct but also deeply connected to the design's actual requirements and formal verification outcomes.

Lu: Indeed, the paper lays out a very clear path for how structured IRs enable these powerful multi-agent systems to perform targeted repairs and augment verification intelligently. It’s a blueprint for next-generation AI assistance in hardware design.

Meng: So, the key takeaway is that adding this KG structure provides the necessary context for an AI to move from generating plausible code to generating formally sound, verifiable assertions. That’s a tangible step forward for practical implementation.

Lalam: This research fundamentally changes how we think about agentic AI in hardware; it shows that the success of these systems relies not just on the language model's size, but on how well we can structure its access to design and formal knowledge.

Tom: Fantastic discussion everyone. We’ve covered a lot about how this Knowledge Graph approach provides the necessary grounding for robust AI-assisted verification, and I think this paper is a very solid contribution to the field.

Jane: It really shows that by formalizing the relationship between requirements, design, and results in this graph, we get much more reliable assertion synthesis than before.

Lu: That's what makes it so interesting; it’s not just about making the AI talk better; it’s about making the AI *know* the design deeply enough to make good decisions.

Meng: For me, the practical implication is seeing tools that can autonomously fix their own errors based on formal feedback, which dramatically speeds up our verification pipelines.

Lalam: This paper on Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification, provides a powerful framework for building highly autonomous and trustworthy AI agents in complex engineering domains.

Tom: That’s all the time we have for this episode, folks. We’ll be right back with more deep dives into the latest research next time!

The paper's summary: Tom: So, we've been talking about how this paper uses Knowledge Graphs to fix assertion generation problems, and now Jane, can you give us a quick summary of what they actually propose?

Jane: Well, the core idea is that instead of just feeding an LLM a big block of text for a requirement and then hoping it writes correct code, this paper builds a structured knowledge graph from various pieces of data—like the spec itself, the RTL code structure, and even feedback from formal verification tools. This graph acts as this central map, giving the AI a deep understanding of how all these design elements relate to each other.

Lu: Exactly! It’s about moving away from just pattern matching in text toward true design grounding. The paper constructs these specific Intermediate Representations, or IRs, which are like standardized digital blueprints for different parts of the design information, making it deterministic for the AI to navigate.

Meng: From my side, I’m interested in how this structure helps with practical implementation; they are creating a system where you can trace every single generated assertion back to its original requirement and even its formal proof status. That level of traceability is something we need when we’re trying to ensure our verification results are reliable.

Lalam: And that traceability is what makes it so powerful for the AI's culture; it gives the AI a verifiable memory of the design, which means when it makes a correction, you can see exactly why and how it made that choice. It moves us toward a system where we don't just trust the output, but we understand its entire lineage.

Tom: That traceability is what really sets this apart from previous methods I’ve seen for assertion generation; it’s not just about getting syntactically correct code anymore, it’s about generating assertions that are provably connected to the design intent. So, if we think about the implications, what do you guys see happening in the wider world with this kind of technology?

Jane: I think the immediate impact is a significant reduction in manual effort for generating verification properties because these agents can handle complex refinement loops autonomously—fixing syntax errors, correcting formal failures, and even finding coverage gaps on their own. This should speed up the entire design verification cycle considerably.

Lu: Looking at the big picture, I see this framework suggesting that we can start treating complex hardware design not just as a sequence of code to be written, but as a living system with interconnected knowledge that an AI can navigate intelligently. The potential is in creating AI assistants that function like true co-design partners rather than just code generators.

Meng: For practical impact, this means we could see verification times drop substantially for complex chips where manually writing assertions is a massive bottleneck; it allows our teams to focus on the harder problems of architectural exploration instead of tedious syntax checks.

Lalam: I think the biggest cultural shift here is in how we view AI's role in high-assurance engineering; this shows that AI can be deployed not just as a suggestion engine, but as an iterative partner that learns from formal evidence and design context to improve its own output.

Tom: That’s a powerful vision—moving the AI from being a passive assistant to an active, self-correcting agent within the design flow. It sounds like this work is laying some really solid groundwork for how we build next-generation verification tools that actually understand the underlying system structure.

Jane: Absolutely, and it's not just about better code; it's about building trust into the entire verification pipeline by providing complete proof of lineage for every single assertion generated.

Lu: It’s a sophisticated architecture because it handles multiple refinement loops—repair, correction, augmentation—all driven by querying this central knowledge graph for bounded context information.

Meng: I just hope the practical implementation scales well; having these agents talk to each other through this structured graph needs to be robust enough for real-world production environments where things get messy quickly.

Lalam: Ultimately, this paper suggests that the future of engineering with AI lies in systems where the AI has a deep, auditable, and context-aware understanding of the physical design itself.

Tom: So we’ve heard that this Knowledge Graph approach is about creating a self-grounding system for AI in verification, which promises higher quality assertions and faster cycles. What's next on our agenda?

The paper's improvements: Tom: So, we've been hearing about how this paper uses Knowledge Graphs to fix assertion generation problems, and now Jane, what are the specific improvements they suggest for making this whole system better?

Jane: The key improvement is moving from a single-step process to a closed-loop refinement cycle driven by those three distinct refinement loops we discussed earlier: syntax repair, CEX correction, and coverage augmentation. The Knowledge Graph makes this possible because agents can query localized context neighborhoods relevant to their specific task automatically.

Lu: That’s where the real power lies; the KG allows for dynamic context retrieval. For example, when an agent needs to fix a syntax error, it doesn't just guess; it uses trace links and design metadata from the IRs to deterministically guide its repair strategy based on actual tool diagnostics.

Meng: From an engineering standpoint, that autonomous debugging capability is huge; it means we can move away from tedious manual correction of LLM outputs toward a system that self-correcting based on formal feedback. This level of automation is what makes the practical impact feel real for our teams.

Lalam: And I see this in terms of AI culture, because it shifts the expectation for how we work with these models; instead of just asking the AI to write code and hoping for the best, we are building an environment where the AI actively uses formal verification data to improve its own reasoning iteratively.

Tom: So, if I understand this correctly, these improvements mean that when a formal tool spits out an error or a counterexample, the AI doesn't just stop; it uses the Knowledge Graph to find the exact piece of context needed—like a specific signal declaration or a requirement text—to fix its mistake.

Jane: Exactly, Tom; it’s about making the correction deterministic rather than heuristic. For CEX correction specifically, they show that agents can adjust constraints or add assumptions by traversing paths stored in the formal results IR entities within the graph.

Lu: The coverage augmentation loop is also a big win because it lets the AI look at aggregated coverage metrics and then traverse backward through the graph to find requirements or RTL statements that are currently unverified, allowing it to generate targeted properties for those specific gaps.

Meng: That proactive approach to filling verification gaps is very appealing; instead of passively verifying what’s already there, the AI is actively hunting for weaknesses based on quantitative metrics like reachability percentages.

Lalam: This capability really elevates the vision of AI in engineering because it shows that the model can learn not just from what it's asked to do, but from the actual failures and successes recorded by formal tools throughout a long verification process.

Tom: So these improvements focus on making the AI smarter about *how* to fix things, using structured context to guide its iterative refinement loops rather than just generating one big assertion and hoping for the best. This moves us closer to truly autonomous verification agents.

Conclusion: Tom: We've covered a lot about how this paper uses Knowledge Graphs to fix assertion generation problems, and now Jane, can you give us a final summary of the main conclusions they draw?

Jane: They conclude that by constructing this verification-centric Knowledge Graph from structured IRs—the spec chunks, RTL models, and tool feedback—we can achieve higher quality SVAs with much lower syntax repair overhead than previous methods. The core finding is that this grounded context provides a necessary structure for the AI to produce reliable code.

Lu: It’s really about establishing a formal link between design intent and generated assertions through that structured graph; they show it works across seven different benchmark designs, which is quite broad evidence.

Meng: From an engineering standpoint, the conclusion is that this approach provides auditable traceability, which is essential for high-assurance environments where you need to prove *why* an assertion looks the way it does. That level of documentation helps immensely with compliance and debugging later on.

Lalam: I think the most profound implication is how this advances our cultural view of AI assistance; it shows that we can build systems where the AI’s output isn't just a creative guess but something deeply connected to verifiable engineering facts, which fosters a more rigorous approach to tool development.

Tom: So, we’re wrapping up on "Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification," and it sounds like this structured knowledge base is what finally gives the AI the necessary context to perform complex verification tasks reliably.

Jane: That’s right; it moves the process from a black box generation task to a traceable, evidence-based reasoning engine. It’s about giving the AI a memory of the design itself.

Lu: The future work they mention points toward extending this KG to handle even more complex temporal dynamics and cross-domain coupling, which suggests we could integrate it with other AI systems for even broader applications in hardware description languages.

Meng: For practical engineering, I think the next step is making sure these IR extraction tools are as robust across different hardware architectures as possible so that this system can be deployed on a wider variety of designs without needing a completely new setup each time.

Lalam: Ultimately, this research shows that the most impactful vision for AI in engineering is one where it doesn't just generate text, but actively builds and navigates a verifiable knowledge structure to solve complex problems.

Tom: What an exciting summary! We’ve heard that the Knowledge Graphs, the Missing Link in Agentic AI-based Formal Verification paper shows us how structured context can make assertion synthesis significantly more reliable and traceable.

Jane: It really highlights how building that structured map allows agents to perform those iterative refinement loops we talked about—fixing errors, correcting counterexamples, and augmenting coverage—all guided by the design's actual structure.

Lu: This paper is a fantastic piece of theoretical work because it lays out a clear blueprint for how to use structured data representations to anchor agentic reasoning in complex engineering domains.

Meng: I just hope the next iteration focuses on making this more automated for rapid prototyping, so we can test these concepts on new designs much faster in our startup environment.

Lalam: This paper is a testament to what’s possible when we combine deep formal verification results with structured knowledge representations to build truly trustworthy engineering assistants.

Tom: Well, that's all the time we have for this session on this research; it’s been a fascinating deep dive into how structure improves AI-assisted verification. Join us next time when we look at how other papers are tackling similar challenges in the field of automated reasoning and agent collaboration!