Deontic Policies for Runtime Governance of Agentic AI Systems

arXiv:2606.19464 · cs.AI, cs.MA · Submitted 2026-06-17 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Deontic Policies for Runtime Governance of Agentic AI Systems".

Jane: The paper was written by Anupam Joshi, Tim Finin, Karuna Joshi and Lalana Kagal from University of Maryland, Baltimore County and MIT.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: We're looking at a fascinating new paper today called "Deontic Policies for Runtime Governance of Agentic AI Systems." It sounds like a mouthful, but it's hitting on something we've all been worried about lately.

Jane: It really is, Tom, because as these AI agents start doing more things on their own, we need a way to keep them in check. The title mentions "deontic," which is just a fancy way of talking about the logic of duties and permissions.

Tom: So instead of just saying "yes" or "no" to an action, we're talking about what an agent is actually required to do?

Jane: Exactly, and the authors—Anupam Joshi, Tim Finin, Karuna Joshi, and Lalana Kagal from UMBC and MIT—are proposing a much deeper way to handle that.

Lu: I love how they're framing this because it moves us away from simple guardrails toward actual governance. If an agent has the power to move money or access medical records, we can't just rely on a prompt telling it to be good.

Meng: That sounds great in theory, Lu, but I wonder how these researchers from UMBC and MIT actually plan to implement that level of control without slowing everything down.

Jane: That's the big question, Meng, and it leads us directly into the core of their research.

Lalam: We should consider how this changes our relationship with technology. If we can build systems that understand their own obligations, we might finally move toward a culture of true digital responsibility.

Tom: That's a heavy thought to start with, Lalam, so let's look at what the paper actually says they've built to solve this.

Summary: Tom: The paper explains that current tools like Rego or Cedar are too limited because they only handle permissions and prohibitions. They can tell an agent "you can't do that," but they struggle with "you did that, so now you must do this."

Jane: Right, and the authors argue that if an agent installs software, it might have a duty to notify a security officer immediately after. Current systems don't have a native way to manage those kinds of follow-up tasks.

Meng: I noticed they've developed something called AgenticRei to fix this. How does it actually sit between the AI and the real world?

Jane: It uses a three-step process where it extracts the action into a simple triple, evaluates it against a logic engine, and then applies the decision.

Tom: And they're doing all of this using an engine called RDFox that lives entirely outside of the Large Language Model.

Lu: That's the most brilliant part to me! By keeping the decision-making process separate from the LLM, they avoid that "reasoning drift" where a model might misinterpret its own security rules.

Meng: I was looking at their performance numbers, and they claim these decisions happen in under ten milliseconds. If that's true, it could actually work in real-time production environments.

Lalam: It provides a deterministic safety net that doesn't care how much the AI's reasoning fluctuates. This creates a world where we can trust the outcome even if the process is unpredictable.

Jane: That reliability is exactly what's missing, so let's talk about the specific improvements they suggest to make this better than anything we have now.

Improvements: Tom: The paper really pushes for more than just simple rules; they want "meta-policies" that can resolve conflicts when two different rules clash.

Jane: Imagine a situation where a company policy says "don't export data," but a legal subpoena says "you must export this specific data." AgenticRei uses these meta-policies to decide which rule wins based on logic rather than just which one was written first.

Meng: They also talk about something called semantic grounding, which sounds like it could save developers a lot of headache.

Jane: It does, because instead of listing every single type of sensitive file, you can just tell the system that anything belonging to a certain category is off-limits.

Tom: Like if they define "Protected Health Information" as a broad class, the system automatically knows that a "pediatric oncology record" is also protected because it's a subclass.

Lu: That's so much more elegant than the old way of manually updating every single rule whenever a new data type appears! It allows the governance to evolve naturally alongside the data itself.

Meng: I was reading their financial services example where an agent is allowed to make a high-value transaction only if it presents a specific credential and then immediately fulfills an obligation to file a report.

Jane: That's a perfect example of how they combine permissions, credentials, and duties all in one go.

Lalam: This level of detail turns accountability from an afterthought into a built-in feature of the system. It allows us to build complex societies of agents that actually follow the spirit of our laws.

Tom: It's a massive leap forward in how we think about controlling these autonomous systems.

Conclusion: Jane: We've covered a lot today, from the basic idea of deontic logic to the way AgenticRei uses ontologies to handle complex data hierarchies.

Tom: It really seems like the researchers are providing a blueprint for how we can finally move agentic AI from experimental toys to reliable enterprise tools.

Lu: I'm thinking about how this could enable entire ecosystems of agents that can cooperate across different organizations while still respecting everyone's private rules!

Meng: From my side, seeing that sub-ten-millisecond latency makes me think this is actually something we could start implementing in our stacks very soon.

Lalam: If we embrace this, we aren't just building smarter tools; we are building a foundation for a more stable and predictable digital culture.

Jane: Well, thank you all for joining us to discuss "Deontic Policies for Runtime Governance of Agentic AI Systems."

Tom: We'll be back with another paper soon, so stay tuned!

University of Maryland, Baltimore County · MIT

cs.AI, cs.MA

Submitted: 2026-06-17

Updated: 2026-06-17

Comments: 10 pages, 1 figure. To be published in the 2026 IEEE Symposium on Agentic Services which is part of the IEEE Conference on Web Services

Journal ref: 2026 IEEE International Conference on Web Services (ICWS), Sydney, Australia, 2026, pp. 1255-1264,

DOI: 10.1109/ICWS72778.2026.00159

Code: https://github.com/cosai-oasis/ws4-secure-design-agentic-systems

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 88/100

The gist: This paper introduces AgenticRei, a framework for the "runtime governance of agentic AI systems" designed to address security, privacy, and compliance challenges that exceed the capabilities of

Key concepts

Deontic Logic
The logic of duties and permissions. It moves beyond simple "yes" or "no" rules to manage obligations, such as requiring an AI agent to perform a follow-up task, like notifying a security officer, immediately after completing a specific action.
AgenticRei
A governance system that sits between an AI and the real world. It extracts actions into triples and evaluates them using the RDFox engine outside the Large Language Model, preventing reasoning drift while maintaining sub-ten-millisecond latency.
Meta-policies
Rules designed to resolve conflicts when different policies clash. They allow a system to logically decide which rule wins—for example, choosing between a company's data export policy and a legal subpoena—rather than simply following the most recent rule.

Terminology

Summary

This paper introduces AgenticRei, a framework for the runtime governance of agentic AI systems designed to address security, privacy, and compliance challenges that exceed the capabilities of current policy engines. As autonomous agents gain the ability to invoke tools and coordinate across organizational boundaries, they require a deterministic enforcement layer that moves beyond simple permit/prohibit constraints to manage complex duties and regulatory requirements.

The governance gap

Current policy engines such as XACML, Rego, and Cedar are limited because they address only the permit/prohibit subset of this governance structure. They lack the ability to provide obligation lifecycle management, meta-policy conflict resolution, dispensations that waive obligations in specific circumstances, and ontological reasoning over domain class hierarchies. The authors argue that open, multi-agent systems require four specific properties:

  • Obligations: behavioral obligations that arise because access was granted.

  • Principled conflict resolution: meta-policies which are rules about rules to resolve overlapping authorities.

  • Semantic grounding: the ability to reason over a domain ontology so policies apply automatically to subclasses.

  • Dynamic, cross-authority trust: where the policy itself names trusted credential issuers rather than relying on hard-coded identifiers.

How it works

AgenticRei implements an extract–evaluate–apply contract that operates at the action boundary, ensuring enforcement is entirely outside the LLM. This architecture follows a three-step process to intercept tool calls and agent-to-agent messages:

  1. Extract: A TripleExtractor maps outbound actions to a concrete subject, action, resource triple.

  2. Evaluate: A PolicyEngine queries an RDFox-based engine loaded with Rei-encoded deontic rules and domain ontologies to return a verdict of PERMIT, PROHIBIT, or DEFAULT-DENY.

  3. Apply: The middleware either allows execution—appending obligation text to the result—or short-circuits the invocation with a structured policy-violation message.

Deontic logic in practice

The framework utilizes the Rei deontic framework to provide four essential modalities: permissions, prohibitions, obligations (attached via deontic:provision), and dispensations. By using pluggable ontology fragments, the system can express a prohibition once at a class level—such as Protected Health Information (PHI)—and have it apply automatically to all current and future subclasses through OWL/RDFS entailment. This approach addresses authority creep by making the autonomy threshold a governed object itself, where raising permissions is an explicit policy change rather than a silent configuration relaxation.

Threat model and performance

AgenticRei is designed to defend against policy-violating actions resulting from compromised reasoning, adversarial prompt injection, or deliberate misuse. While it does not prevent the LLM's reasoning process from being manipulated, it acts as a deterministic boundary; the authors note this is analogous to mandatory access control in an operating system, where kernel policy constrains what a program can do once it runs. In preliminary measurements, the system achieved sub-10ms end-to-end latency per decision, which is well within the bounds required for intercepting synchronous actions in production environments.

Improvements for AI systems

1. Decoupling Policy Enforcement from LLM Reasoning via a Deterministic Middleware Layer

  • What the improved system can do: The system will intercept every outbound tool invocation and agent-to-agent (A2A) message at the framework boundary, mapping them to subject, action, resource triples. These are evaluated by an external high-performance logic engine (e.g., RDFox) rather than being interpreted from the LLM's context window. This eliminates security reasoning drift and ensures that even if an LLM is compromised via prompt injection, it cannot bypass deterministic runtime constraints.

2. Implementation of Deontic Obligation and Dispensation Lifecycle Management

  • What the improved system can do: Beyond simple permit/prohibit actions, the system can attach mandatory behavioral duties (obligations) to permitted actions (e.g., If you execute this transaction, you must notify the CISO within 60 seconds). It can also manage dispensations, allowing an agent to be legally released from a standing obligation if specific conditions are met, providing a granular control mechanism for complex workflows.

3. Integration of Ontological Reasoning for Automated Policy Scaling

  • What the improved system can do: By grounding policies in OWL/RDFS class hierarchies, the system can enforce rules at a high-level concept (e.g., Prohibit access to all Protected Health Information) and automatically apply those rules to any current or future subclasses (e.g., Genetic Test Results or Pediatric Oncology Records) without requiring manual updates to the policy source code.

4. Application of Semantic Meta-Policies for Conflict Resolution

  • What the improved system can do: In multi-organizational environments where rules overlap, the system will resolve conflicts using formal meta-policies (rules about rules) rather than arbitrary evaluation orders or priority integers. This allows for authoritative, auditable decisions, such as a specific project-level permission taking precedence over an organizational prohibition based on a defined semantic relationship.

5. Enabling Cross-Pillar Credential-Gated Overrides

  • What the improved system can do: The system can link policy decisions to cryptographic identity infrastructure. It can permit an agent to bypass a standard prohibition only if the agent presents a verifiable credential issued by a specific, trusted authority (e.g., Treasury Officer) explicitly named within the policy itself, ensuring that autonomy is granted based on verified authority rather than mere attribute claims.

6. Generation of Machine-Verifiable Audit Records with Policy Versioning

  • What the improved system can do: Every decision will produce a structured audit record containing the matched rule, round-trip latency, credential issuers presented, and a unique hash of the loaded policy Knowledge Base (KB). This allows for forensic reconstruction of exactly which version of a policy was in effect at the moment of any specific action.

Sources

Related papers