Dynamic Deep Prompt Optimization for Defending Against Jailbreak Attacks on LLMs

arXiv:2609.26185 · cs.CR, cs.AI, cs.CL · Submitted 2026-08-10 · Read on arXiv

cs.CR, cs.AI, cs.CL

Submitted: 2026-08-10

Updated: 2026-08-10

DOI: 10.1609/aaai.v40i42.40887

Code: https://github.com/doniobidov/ddpo

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Embeddings — 25× in this paper · explained in 2 episodes
• Alignment — 7× in this paper · explained in 1 episode
• Dynamic Deep Prompt Optimization — 6× in this paper
• Fine-Tuning — 6× in this paper · explained in 5 episodes
• Benchmark — 5× in this paper · explained in 2 episodes
• Catastrophic Forgetting — 5× in this paper · explained in 5 episodes
• Computational overhead — 5× in this paper · explained in 2 episodes
• Cosine Similarity — 5× in this paper · explained in 3 episodes
• Attack Success Rate — 4× in this paper
• Benign Pass Rate — 4× in this paper
• Core — 4× in this paper · explained in 2 episodes
• Robustness — 4× in this paper · explained in 5 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Adaptability — 3× in this paper · explained in 1 episode
• Adversarial Attacks — 3× in this paper · explained in 1 episode
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Attack Success Rate (ASR) — 2× in this paper · explained in 2 episodes
• Feature Extraction — 2× in this paper · explained in 2 episodes
• Prompt Adversarial Tuning — 2× in this paper
• Reinforcement Learning — 2× in this paper · explained in 4 episodes
• Robust Prompt Optimization — 2× in this paper
• Ablation Studies — 1× in this paper · explained in 1 episode
• Accuracy — 1× in this paper · explained in 1 episode
• Activation Function — 1× in this paper · explained in 1 episode
• Adversarial Examples — 1× in this paper · explained in 2 episodes
• Directed Representation Optimization — 1× in this paper
• Foundation Model — 1× in this paper · explained in 3 episodes
• Gradient-based Distributional Attack — 1× in this paper
• Greedy Coordinate Gradient — 1× in this paper
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Low-Resource Languages — 1× in this paper · explained in 4 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Prompt Automatic Iterative Refinement — 1× in this paper
• Static Deep Prompt Optimization — 1× in this paper
• Supervised Fine-Tuning — 1× in this paper
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes
• Transferability — 1× in this paper · explained in 4 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
• Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
• Multilingual Jailbreak Challenges in Large Language Models
• RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
• Gradient-based Adversarial Attacks against Text Transformers
• Baseline Defenses for Adversarial Attacks Against Aligned Language Models
• Break the Breakout: Reinventing LM Defense Against Jailbreak Attacks with Self-Refinement
• DeepInception: Hypnotize Large Language Model to Be Jailbreaker
• P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
• An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
• HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
• SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
• AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Universal Adversarial Triggers for Attacking and Analyzing NLP
• OpenChat: Advancing Open-source Language Models with Mixed-Quality Data
• GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis
• SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
• A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
• Jailbreak Attacks and Defenses Against Large Language Models: A Survey

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs