The Uncontrolled Variable: Vision-Language Refusal Is Conditioned on the Image-Attachment Interface, and Not Robust to Irrelevant Image Properties
cs.CR, cs.AI
Submitted: 2026-08-12
Updated: 2026-09-25
Code: https://github.com/vacantfury/llm_guardrail_security
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Text is All You Need for Vision-Language Model Jailbreaking
- Narrow Fine-Tuning Erodes Safety Alignment in Vision-Language Agents
- SAVeS: Steering Safety Judgments in Vision-Language Models via Semantic Cues
- Benign-to-Toxic Jailbreaking: Inducing Harmful Responses from Harmless Prompts
- DUAL-Bench: Measuring Over-Refusal and Robustness in Vision-Language Models
- Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models
- Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shift
- DAVSP: Safety Alignment for Large Vision-Language Models via Deep Aligned Visual Safety Prompt
- Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills
- Token Composition: A Graph Based on EVM Logs