When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems
cs.CR, cs.AI, cs.OS
Submitted: 2026-09-20
Updated: 2026-09-20
Comments: 32 pages, 5 figures, 6 tables
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Large language model agents are now privileged principals that take consequential actions: editing code repositories, operating inboxes, completing purchases.
Terminology
Abstract
Large language model agents are now privileged principals that take consequential actions: editing code repositories, operating inboxes, completing purchases. Their authority is kernel-grade, but it comes without what classical systems security requires: a trusted mediator interposed on every access. Operating-system vendors are now rebuilding the platform around this de-facto agent kernel, inheriting complete mediation as a design problem. We systematize the security of such systems around a single distinction: a crossing mediated over provenance admits a deterministic check, while one over content semantics does not. A trust-boundary taxonomy locates where mediation must occur and isolates the central mediation gap at two kinds of semantic judgment: distinguishing data from instruction in untrusted input, and an authorized action from an unauthorized one. We argue that this gap leaves an irreducible residual of undetected attacks wherever inputs and actions are not restricted in advance to an enumerated set. The same distinction makes attack-success statistics actionable, placing each number on a spectrum from deployment debt (a sound deterministic mediator left unused) to a structural gap (no such mediator known). We systematize defenses across runtime monitoring, architectural separation, and authorization, and show that current evaluations tend to overstate deployed security through evaluation-validity failures. Finally, we carry that analysis forward beyond the de-facto kernel, to an architecture in which the model itself becomes the arbitration core, and derive the design constraints, open challenges, and research agenda for a security-first AI-native OS.
Sources
- Toward Securing AI Agents Like Operating Systems
- AI Agents May Always Fall for Prompt Injections
- Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security
- The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents
- AIOS: LLM Agent Operating System
- MemGPT: Towards LLMs as Operating Systems
- AgentOS: From Application Silos to a Natural Language-Driven Data Ecosystem
- LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem
- AgenticOS: An Intent-Oriented Secure Operating System Architecture for Autonomous AI Agents
- InputSnatch: Stealing Input in LLM Services via Timing Side-Channel Attacks
- The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
- Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
- A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
- SoK: The Attack Surface of Agentic AI - Tools and Autonomy
- Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?
- Rewriting the Response Path: Silent Tampering and Provider-Signed Defense in BYOK LLM Agents
- Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain
- Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions
- Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem
- MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs