Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models

summary

The gist

Machine unlearning aims to remove specific concepts from pretrained text-to-image diffusion models, yet several white and black-box attacks have been introduced to make the model generate such

In short

BEAP is a black-box attack that uses a Large Language Model (LLM) to iteratively generate adversarial prompts against text-to-image models after they have been unlearned. It combines concept detection, image quality, and alignment scores to guide the LLM in creating natural, human-readable prompts that successfully bypass safety filters and recover the removed concepts.

Key concepts

BEAP Framework: This is a black-box adversarial search strategy. It uses an LLM to refine prompts by scoring them against three metrics: concept detection (checking if the unlearned concept is present), image reward (how well the image matches the prompt), and aesthetic quality. A prompt must pass all thresholds to be considered successful.
LLM-Driven Iterative Search: The core process involves an LLM refining prompts over multiple steps. In each step, the LLM takes feedback from concept detectors and image reward metrics to generate new, improved paraphrases. This loop continues until a prompt achieves high scores or a maximum iteration limit is hit.
Embedding-Aware Search Component: This component guides the LLM toward relevant words by using text embeddings. It creates an adversarial vocabulary of words semantically close to the target concept and uses their similarity to bias the LLM's prompt generation, ensuring the generated text remains natural while focusing on the unlearned concept.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Federated Learning — explained in 16 episodes
Retrieval-Augmented Generation (RAG) — explained in 16 episodes
Interpretability — explained in 15 episodes
Sample Complexity — explained in 15 episodes
Generalization — explained in 13 episodes

This episode discusses

The paper

Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models · Read on arXiv

Arian Komaei Koma Seyed Amir Kasaei AmirMahdi Sadeghzadeh, Mohammad Hossein Rohban

Department of Computer Engineering, Sharif University of Technology

Machine unlearning aims to remove specific concepts from pretrained text-to-image diffusion models, yet several white- and black-box attacks have been introduced to make the model generate such unlearned concepts. These attacks, nevertheless, do not assume a realistic threat model, i.e. they either assume access to the model weights, or result in gibberish adversarial prompts that could be easily detected even through naive rule-based safeguarding. We aim to address this gap in this paper. We introduce BEAP, a black-box, embedding-aware adversarial prompting attack that leverages a large language model (LLM) to iteratively generate effective adversarial prompts and exploit such hidden vulnerabilities. BEAP performs an embedding-aware search in text space, combining multiple reward signals: unlearned concept presence, text-image alignment, and image quality, to refine generated prompts. Unlike previous attack methods, BEAP keeps its prompts undetectable to safety filters while producing high-quality images. Across five unlearning methods, BEAP achieves a macro-averaged ASR of 97.8% under the held-out OpenNSFW2 evaluation, exceeding the white-box UDA baseline by 41.6 percentage points (56.2% to 97.8%). Counting unsuccessful searches at the full 100-query budget, BEAP uses 32.1 image-generation queries per evaluated prompt on average under this criterion.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Erased but Exploitable".

Jane: Machine unlearning aims to remove specific concepts from pretrained text-to-image diffusion models, yet several white and black-box attacks have been introduced to make the model generate such unlearned concepts.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, to wrap up what we just heard, this paper "Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models" is essentially proposing BEAP, a black-box adversarial prompting attack designed to uncover concepts that haven't been fully removed from text-to-image diffusion models through unlearning.

Jane: They claim the thesis is that current attacks are limited because they often need internal model access, whereas BEAP works by leveraging an LLM to iteratively generate prompts based on multiple reward signals—concept presence, image alignment, and aesthetic quality <ref:2605.26332#pg0>.

Lu: The paper argues that unlearned models still regenerate concepts through simple prompting or paraphrasing because some of the knowledge remains in their parameters, and they show that existing methods often produce prompts with abnormally high perplexity which makes them easily detectable by standard safeguards <ref:2605.26332#pg1>.

Meng: So, the importance here is that this approach aims to find these residual traces without needing weights or gradients, which addresses a major hurdle for practical security research in this area <ref:2605.26332#pg1>.

Lalam: It matters because it shows that even after supposed erasure, there are still pathways to generate those erased concepts, which gives us a clearer picture of the model's internal state and what remains accessible <ref:2605.26332#pg0>.

Tom: The paper specifically introduces BEAP as a black-box adversarial search that uses an LLM to refine prompts based on feedback from those three signals, and it even includes an embedding-aware constraint to bias the search toward relevant vocabulary <ref:2605.26332#pg2>.

Jane: That embedding guidance is key because it helps keep the search focused on words that are semantically linked to the target concept, which should lead to higher quality and more interpretable adversarial prompts than random searching <ref:2605.26332#pg0>.

Lu: It's interesting how they combine the LLM's natural language generation with these structured feedback signals—concept detection score, text-image alignment, and image quality—to make the search process more intelligent <ref:2605.26332#pg0>.

Meng: I see the practical value in that structured feedback; it means we aren't just getting a long list of prompts to sift through, but a guided refinement process that aims for actual success <ref:2605.26332#pg0>.

Lalam: If this method can consistently find high-quality, interpretable prompts without model weights, it opens up avenues for studying how we can better detect and control the residual knowledge left in these diffusion models <ref:2605.26332#pg0>.

Conclusion: Tom: Looking at the conclusion of "Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Models," it really hammers home that BEAP is a viable black-box method because it successfully recovers unlearned concepts across all models tested, achieving high ASRNudeNet and ASRAll scores <ref:2605.26332#pg2>.

Jane: So, the implication for us is that these unlearning methods aren't as absolute as we thought; there are still ways to generate those erased concepts if you use a method that intelligently searches the text space like BEAP <ref:2605.26332#pg2>.

Lu: What this means for the broader field is that it suggests we need to move beyond attacks that rely on full model access and focus more on developing these kinds of prompt-based methods, even if they are black-box, to understand the limitations of unlearning <ref:2605.26332#pg1>.

Meng: Practically speaking, this means that security researchers shouldn't just focus on cracking the weights; they also need to consider these prompt-based exploits because they are more likely to be deployable in real-world scenarios <ref:2605.26332#pg1>.

Lalam: It gives us a better sense of how persistent these concepts are, which could influence future work on model safety and ensuring that the concepts we want erased are truly gone <ref:2605.26332#pg0>.

Tom: Exactly, so the title itself suggests that while something is erased, it can still be exploited if you use an embedding-aware prompting strategy to find those hidden vulnerabilities <ref:2605.26332#pg0>.

Jane: And the authors have clearly shown that BEAP provides prompts that are not only successful conceptually but also visually coherent and semantically aligned with the original concepts, which is a strong point <ref:2605.26332#pg1>.

Lu: The paper suggests a future direction where we can explore how to use these prompt-based methods not just for attack but also to understand the underlying knowledge structure of these diffusion models <ref:2605.26332#pg0>.

Meng: If this research helps us better map out what kind of knowledge remains, it could inform how we design more effective unlearning techniques in the first place <ref:2605.26332#pg1>.

Lalam: It really shows that understanding the residual traces is crucial for building a safer future with AI, because as long as there are ways to generate what's supposed to be gone, we need better tools to manage that risk <ref:2605.26332#pg0>.

More episodes

← Home