Visual Compliance via Executable Safety Rule Entailment
cs.AI
Submitted: 2026-09-16
Updated: 2026-09-16
Comments: Accepted to EMNLP 2026. 36 pages, 20 figures, 24 tables
Project page: https://cityofnewyork.github.io/opendatatsm/publicpolicies.html
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in LLMs and VLMs have enabled safety systems to reason beyond simple risk patterns toward more contextual and semantic safety concerns.
Terminology
Abstract
Recent advances in LLMs and VLMs have enabled safety systems to reason beyond simple risk patterns toward more contextual and semantic safety concerns. However, as risk patterns continue to evolve and safety rules become more complex, existing training-based end-to-end safeguards face persistent challenges in adaptability and explainable reasoning over complex safety rules. To address these challenges, we propose GuardEn (Guarding by Safety Rule Entailment), an executable safeguard framework that decomposes safety policies into atomic propositions through Safety-Rule Compilation, modeling their composition as executable code. At test time, Scene-Grounded Execution instantiates these atomic propositions with contextual visual information derived from scene graphs, enabling rule-grounded and interpretable safety reasoning. Experiments on SafetyVisionBench demonstrate the effectiveness of programmable safeguard for complex visual safety assessment, achieving an average improvement of 9.8 F1 points over the strongest baseline.
Sources
- Qwen2.5-VL Technical Report
- Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- GSPR: Aligning LLM Safeguards as Generalizable Safety Policy Reasoners
- Depth Anything 3: Recovering the Visual Space from Any Views
- Towards Policy-Adaptive Image Guardrail: Benchmark and Method
- Kimi K2: Open Agentic Intelligence
- SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
- OpenAI GPT-5 System Card
- LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratories
- ReAct: Synergizing Reasoning and Acting in Language Models
- ShieldGemma 2: Robust and Tractable Image Content Moderation
- ShieldGemma: Generative AI Content Moderation Based on Gemma
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection