PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents

This document synthesizes information from two distinct perspectives—a high-level technical overview of the PACE mechanism and a detailed artifact/experiment report—to provide a comprehensive understanding of Provenance-Aware Capability Enforcement (PACE), a novel security framework designed for tool-using Large Language Model (LLM) agents.

The fundamental challenge addressed by PACE is the inherent risk posed by LLM agents that turn generated text into real side effects. This process allows for the poisoning of critical components—such as tool metadata, retrieved pages, memory states, and reusable skills—which can subsequently steer the agent's next actions. Traditional security measures like vetting an artifact before admission are insufficient because they cannot account for the dynamic nature of these side effects. Furthermore, a safe variant and a leaking variant can produce identical admission evidence; therefore, a sound gate cannot relax mediation at the egress site for either case.

PACE moves the critical decision point—the enforcement of tool calls—to the last enforceable tool boundary, where concrete proposals, authenticated authority, runtime provenance, and a working hook are all available simultaneously. It achieves this by mediating every tool call through a rigorous four-phase process:

1. PROPOSE (Phase I): The agent proposes an action. During this phase, the system freezes the current arguments, extends the influence graph to incorporate schema-defined effects, and expands these potential effects based on the proposal.

2. CUT & CERTIFY (Phase II): This is where path confinement and capability verification occur concurrently.

• Path Confinement (P): Proposes an executable cut of represented influence paths, severing potentially malicious or unauthorized influence routes in the agent's execution graph.

• Capability and Effect Verification (C): Checks the proposed effects against an authority compiled from the authenticated request. This distinguishes the certified contract (PACE-C)—the strict set of allowed operations—from the evaluated configuration (PACE-P), which handles restoration and repair behavior.

3. ENFORCE (Phase III): The system installs the necessary manifest based on Phase II decisions and dispatches the call if it passes verification against PACE-C.

4. FINALIZE (Phase IV): The final decision rule, defined by Equation (7) (dec(at) =), commits the outcome for the subsequent call, establishing a certified contract (PACE-C) that preserves the cut and ensures the final call passes effect verification.

PACE is grounded in formal proofs that establish strong security properties:

• Represented-Path Separation (Proposition 1): If a path verifier accepts a certificate (cert t), removing the proposed cut (C t) ensures that the utility (U t) is unreachable from the current state (B t P t). This proves that path confinement effectively severs malicious influence.

• Authority over Schema-Exposed Effect Atoms: The system guarantees authority over all effects exposed in the schema, linking concrete cuts to real execution links via executor grounding.

• Transparent Chains Simulation (Proposition 2): This proposition ensures that if a sequence of calls adheres to specific host emission rules (reading/writing only at declared positions, preserving pass-through identifiers), the resulting state (gamma) can be successfully emitted and simulated in the system graph (G - t).

The framework clearly separates two operational modes:

• PACE-C (Certified Contract): This is the contract that preserves path confinement (P) and final effect authorization. It represents the strict, verified set of allowed operations for a given call.

• PACE-P (Evaluated Configuration): This configuration handles the dynamic aspects of agent behavior, specifically restoration and repair. The combiner associated with PACE-P can restore calls that were blocked by PACE-C and dispatch declared repairs without requiring a full final-call recertification.

The research rigorously validates PACE across eight executable agent-security benchmarks utilizing three target model families.

• Performance: The full combination of P, C, and the final decision rule achieves the strictly lowest attack success on most benchmarks, achieving scores such as PACE 0.42 29.57 10.5 64.64 8.75 81.63 (Table 2).

Here are the specific improvements that can be made to AI systems by implementing the PACE framework, as described in this paper:

1. Improve security against tool-using LLM agents by introducing a multi-phase enforcement mechanism (PACE) that mediates every tool call immediately before execution.

2. Implement Path Confinement to propose executable cuts of represented influence paths within the agent's execution graph, ensuring that unsafe influence cannot reach terminal effects without explicit verification.

3. Integrate Capability and Effect Verification to check schema-defined effects against an authority compiled from the authenticated request, ensuring that only authorized actions are permitted based on the request's capabilities.

4. Distinguish between a Certified Execution Contract (PACE-C) and a Restoration/Repair Configuration (PACE-P), allowing the system to selectively restore blocked calls or apply declared repairs without compromising fundamental security guarantees.

5. Establish a formal guarantee of Represented-Path Separation, ensuring that an attacker cannot reach an unsafe terminal effect if the required influence path has been severed by the enforcement mechanism.

6. Ensure Authorized-Effect Soundness by verifying that every effect atom is covered by a valid capability and satisfies nine specific obligations (e.g., authorization coverage, confidentiality, persistence) before execution.

7. Create a Conditional Grounded Execution Theorem, which guarantees that under specific conditions (current state, current epoch), any causal chain from a protected source to an unsafe effect cannot complete at the call site unless it traverses a path that is explicitly permitted by the enforcement mechanism.

8. Enhance resilience against Trust Inversion and Admission-Time Evidence Confusion by precisely defining when admission-time evidence can or cannot license relaxing an egress site, preventing attackers from using self-written specifications to bypass security checks.

9. Implement a formal, verifiable protocol (MAC chain) for runtime enforcement that ensures freshness, anti-replay capabilities (using one-shot authorization tokens), and durable execution state management across distributed deployments.

By implementing these improvements, the enhanced AI system can:

1. Maintain strong isolation between model reasoning and real-world effects by strictly controlling what the LLM can do with its tools.

2. Prevent prompt injection from steering subsequent tool calls or retrieving sensitive data through poisoned metadata or memory, as the influence paths are actively cut before execution.

3. Guarantee that every action taken by a tool is authorized according to the authenticated request, mitigating unauthorized actions stemming from model-generated text manipulation of schemas or arguments.

4. Provide a verifiable audit trail showing exactly which proposed actions were accepted and how they were modified (restored or repaired), allowing for high-assurance debugging and recovery.

5. Enforce strict causality, ensuring that even if an attacker manages to construct a long sequence of steps, the enforcement mechanism will break the chain at the first point where an unsafe influence is attempted.

Tool-using large language model (LLM) agents turn generated text into real side effects, so poisoned tool metadata, retrieved pages, memory, and reusable skills can steer the next call. Vetting an artifact before admission does not settle this. A safe variant and a leaking variant can produce the same admission evidence, and a sound gate then cannot relax that site for either. We make that condition precise, which leaves the last boundary a deployment can still act on. We present Provenance-Aware Capability Enforcement (PACE), which mediates every tool call immediately before it executes. Path confinement proposes an executable cut of represented influence paths, while capability and effect verification checks schema-defined effects against authority compiled from the authenticated request. We distinguish the certified execution contract from the evaluated configuration, which can restore an authorized call after a proposed block or apply a declared repair. Confinement requires the final action to preserve the certified cut. On eight executable agent-security benchmarks with three target-model families, the evaluated configuration gives strictly lowest attack success in 62 of 79 eligible attack columns and ties in 14; full-benchmark native utility loses at most three points relative to the undefended agent. A complete ablation over 1167 paired cases attributes most security gains to effect verification and refusal control to boundary adaptation. A reduced-scale adaptive search succeeds on 0/30 out-of-authority targets against the defense.