Toward Safe LLM Agents: A Survey of Specification, Verification, and Enforcement

arXiv:2608.14590 · cs.AI · Submitted 2026-06-22 · Read on arXiv

cs.AI

Submitted: 2026-06-22

Updated: 2026-10-17

Comments: 37 pages, 2 figures, 11 tables. Revised version under review at ACM Computing Surveys. Data deposit: https://doi.org/10.5281/zenodo.23376335. PRISMA 2020 checklist included as an ancillary file

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large Language Model (LLM) agents take irreversible real-world actions, yet no system gives formally grounded, task-level safety guarantees for their plans.

Terminology

Abstract

Large Language Model (LLM) agents take irreversible real-world actions, yet no system gives formally grounded, task-level safety guarantees for their plans. We examined this gap in a PRISMA 2020 systematic review of six databases (20-29 May 2026) covering LLM agent specification, verification, and enforcement (2022-2026). We narratively synthesized 38 studies, assessed risk of bias per study, and graded evidence certainty with GRADE. First, specification is the bottleneck: natural-language-to-formal translation reaches only 24.8-34.2% semantic correctness on one Planning Domain Definition Language (PDDL) benchmark. Second, runtime monitoring is the most mature enforcement strategy, reducing unsafe actions by 65.37% in one household-agent study, without complete guarantees. Third, in one τ-Bench study of transactional tool-use agents, blocking 94% of unsafe actions left safe task completion below 5%, as agents found unsafe alternatives (the verifier tax). Fourth, no approach simultaneously achieves soundness, completeness, scalability, semantic grounding, and utility preservation. We contribute a three-level taxonomy and a ten-problem research agenda.

Related papers