NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents
summary
In short
NOMOS is a four-pass compiler that transforms natural language policies into static, verified tool-call gates for LLM agents. It addresses the risk of relying solely on LLM extraction by inserting a rigorous static verification layer. This mechanism significantly reduces policy violations and ensures deterministic, safe execution at microsecond speeds.
Key concepts
- Four-Pass Compiler
- NOMOS uses four sequential stages: PARSE, EXTRACT (LLM), RESOLVE (static verification), and MATERIALIZE. This structured approach ensures that the natural language policy is systematically converted into a final, executable rule set, moving from unstructured text to verified code.
- Static Verification Pass
- This crucial intermediate step mechanically checks every candidate rule generated by the LLM extraction against strict criteria like self-blocking reachability and argument consistency. It acts as a safety filter to catch errors that pure LLM generation might miss, ensuring operational correctness before execution.
- Tool-Call Gate
- The final output is a deterministic gate that dictates exactly when an agent can call a specific tool based on verified conditions. This ensures the enforcement mechanism is predictable and operates at high speed without needing further LLM calls during runtime.
- Domain-Dependent Cost
- The cost associated with NOMOS operation varies by task complexity and domain. While simple tasks have low costs, complex ones like slack management incur higher costs. This suggests that the operational benefit of the system is tied to the specific nature of the work being performed.
Terminology used across episodes
This episode discusses
- NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents · Paper Radio
- tau squared-Bench: Evaluating Conversational Agents in a Dual-Control Environment
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
- Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents · Paper Radio
- PolicyGuard: A Dialogue-Grounded Sub-Agent Verifier for Policy Adherence in LLM Agents
- Formal Policy Enforcement for Real-World Agentic Systems
- VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation
- Towards Enforcing Company Policy Adherence in Agentic Workflows
- Autoformalization of Agent Instructions into Policy-as-Code
- Solver-Aided Verification of Policy Compliance in Tool-Augmented LLM Agents
- tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
- Formalizing and Benchmarking Prompt Injection Attacks and Defenses
- Universal and Transferable Adversarial Attacks on Aligned Language Models
- StruQ: Defending Against Prompt Injection with Structured Queries
- SecAlign: Defending Against Prompt Injection with Preference Optimization
- Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents · Paper Radio
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
- Defending Against Indirect Prompt Injection Attacks With Spotlighting
- Defeating Prompt Injections by Design
- IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems
The paper
NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents · Read on arXiv
Min-Young Yu, Tony Kim, Jang Won Choi
Corners Co., Ltd.
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: I'm Nadia, and with me are Elias and Priya, guest researcher.
Elias: Today's paper: "NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents".
Nadia: Detailed Summary of NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents The research presented in the paper "NOMOS:
Elias: First, who's behind it and why it matters.
Paper summary: Elias: So wrapping up this discussion on NOMOS, the authors are essentially pushing for this idea of coupling LLM-driven policy generation with deterministic enforcement through heavyweight formal machinery >
Nadia: They’re arguing that the safety benefit from that verification mechanism is strong enough to hold up even when tested against a second agent model >
Priya: It suggests a general principle: whenever an LLM writes artifacts that a deterministic system will execute, the missing component is this static verifier standing between them >
Elias: So, if you think about how this works in the real world, it’s not just about making the AI behave better on a benchmark; it’s about ensuring that critical tool executions stay within defined boundaries >
Nadia: They are showing that this isn't some abstract theory; they've shown concrete reductions in violations, from sixty-six point three percent down to two point six percent >
Priya: The paper lays out the title NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents as a way to formalize this necessity >
Elias: It’s about turning natural language policies into gates that are mechanically checked, so you don't rely solely on the LLM's ability to write something correct >
Conclusion: Nadia: So we’ve gone through how NOMOS actually builds these gates, but now we need to talk about what this whole project really means for our work here on agent safety and control >
Elias: Yeah, looking at the title itself, "NOMOS," it sounds pretty serious, like a system of laws or rules being applied mechanically >
Priya: It’s about taking policies that an LLM writes in plain English and turning them into something that a computer can check rigorously before it ever runs anything >
Nadia: Exactly. The authors are showing how you don't just trust the LLM to write something safe; you need a formal way to verify those rules >
Elias: They’re proposing this four-pass compiler where the static verification step is the core idea, not just a nice add-on >
Priya: The big implication for us is that we don't have to rely on just scaling up the LLM's ability to write good rules anymore >
Nadia: It’s about making sure that even if the LLM hallucinates something dangerous, the verification layer catches it before it becomes an execution problem >
Elias: I think what’s really interesting is how they frame this as a necessary component for safety, not just a convenience >
Priya: They did show that this static check actually makes the system much more robust against those kinds of errors we see in testing >
Nadia: It moves the problem from being purely about LLM instruction following to being about ensuring deterministic execution at the gate level >
Elias: And if you look at their results, it shows that this structural verification really helps cut down on violations when dealing with state-changing calls >
Priya: I mean, they saw a big drop in errors on those airline policies and retail ones when they used this method compared to just letting the LLM do it >
Nadia: It’s not just about getting better scores on benchmarks; it's about building a system where the cost of being wrong is much lower than what we see now >
Elias: So, while this is a specific technique for policy compilation, the underlying idea seems to be that any LLM artifact going into a deterministic execution engine needs this formal check >
Priya: It suggests that for critical agent tasks, the verification layer isn't optional; it’s a structural requirement for reliability >
More episodes
- 2610.10597-Certified Corruption Budgets: Anytime-Valid Leaderboard Claims under Adaptive Rigging
- 2610.10608-From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage
- 2610.10612-PyCache Trap: The Inspection-Execution Gap in Agent Skill Scanners
- 2610.10644-SoK: Failure Modes in Common Criteria Product Evaluation - A Taxonomy and Design-for-Evaluability Guidance
- 2610.10617-MRCert: Towards Post-deployment Patch Robustness Certification for Adversarially Patched Samples via Type-specific Masking
- 2610.10620-When AI Finds Hidden Messages, Does It Report?
- 2610.10625-Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models
- 2610.10992-The Hint Weight of ML-DSA Signatures Is Key-Dependent: An Empirical Study across the Three FIPS 204 Parameter Sets
- 2610.10659-Applying Security by Design at the Point of Execution: How Governed Security Requirements Affect the Security of AI-Generated Code
- 2610.10735-DITTO: A Context-aware Pickle-based Pre-Trained Model Scanner for Effective Security Audits