Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation

Backdoor attacks can be implanted in Large Language Models (LLMs) during training, causing unwanted behavior when a trigger appears in the input. This work proposes NEEDLE, a training-free method for targeted backdoor removal that uses sequential weight orthogonalisation to suppress backdoors while preserving refusal-related representations, achieving the lowest mean Attack Success Rate (ASR) among evaluated defenses with minimal changes to model capability and safety.

The gist

NEEDLE is a training-free method for targeted backdoor removal that applies sequential weight orthogonalisation to suppress the backdoor while preventing changes in refusal-related representations.

Key Contributions and Methodology

The paper introduces NEEDLE, which estimates a backdoor direction from triggered and untriggered prompts and a refusal subspace from harmful and benign prompts. The core of the method involves applying sequential weight edits to the model weights to satisfy two constraints: (i) removing the backdoor projection, i.e., b⊤l W⋆l = 0, and (ii) preserving the refusal projection, i.e., R⊤l W⋆l = R⊤l Wl. This is achieved through a process detailed in Figure 2, which involves estimating directions at each layer and applying targeted weight edits.

Backdoor Direction and Refusal Subspace Estimation

The method first estimates the backdoor direction by computing the difference between mean activations from triggered and ordinary samples at each layer, specifically defining bel =δbdl − (µol)⊤δbdl∥µol∥22 to ensure orthogonality to ordinary activations. Concurrently, it constructs a subspace to capture multiple refusal directions. This involves computing differences between mean activations from refused and compliant responses to harmful prompts, and then using Singular Value Decomposition (SVD) on the resulting matrix of vectors (including the primary refusal direction and three additional directions) to form the orthonormal basis Rl for the orthonormal refusal basis, Rl ∈ RM×4.

Sequential Weight Orthogonalisation

The removal process is sequential, applied in increasing layer order from layers 12–34. The first step defines a weight edit direction ul = (I−RlR⊤l)bl, which is the component of bl orthogonal to the refusal subspace. This operation satisfies both constraints when ul ≠ 0. To correct for drift in refusal projections caused by earlier edits, a second update W(2)l = W(1)l + ∆Wl is applied to the MLP output matrix. The optimization problem for Cˆl involves fitting four linear regressions jointly, one for each refusal subspace coordinate, using ridge regularization to find the unique solution: Cˆl = argmin Cl Xnli=1 Clzmlpli −dli2 + λl∥Cl∥2F.

Evaluation and Results

NEEDLE was evaluated across multiple model families (Gemma-3-4B-IT, Qwen3-4B-Instruct-2507) and attack types (sentiment steering, targeted refusal, code injection). The results demonstrate that NEEDLE achieves the lowest mean Attack Success Rate (ASR) among the evaluated defences, including 0% on challenging code injection attacks. Furthermore, it resulted in the lowest KL divergence and minimal changes in capability and safety, with an average relative capability loss of only 0.48% on Gemma and 0.72% on Qwen. The method successfully addresses the trade-off between accurate removal and safety preservation, resulting in a small safety cost (an average of 3.95% on Gemma).

Ablations and Insights

The ablation studies confirm the necessity of both components: ablating only the backdoor direction degrades safety behavior, while preserving a single refusal direction (k=1) is insufficient. The paper shows that Preserving a rank-four refusal subspace is necessary to achieve better results, as the overlap between the backdoor and refusal directions often lies outside the single primary refusal direction. Additionally, testing activation interventions confirms that adding the refusal direction significantly raises its rate from 1.6% to 34.4% on Gemma, supporting the importance of preserving this subspace for safety. The analysis also suggests that the backdoor is expressed across the generated response rather than at a single position.

Conclusion

The findings suggest that LLM backdoor removal is best treated as a targeted model editing problem when trigger information is available, rather than relying solely on broad fine-tuning or inference-time defenses. NEEDLE provides a training-free solution that effectively removes the backdoor while maintaining the safety of the original model.

**(Word count check: Approximately 480 words.

As a fastidious and diligent AI researcher, I have analyzed the core contributions of NEEDLE (Needle: Needle in the Haystack) from this preprint. The primary innovation is a training-free, targeted backdoor removal method based on weight orthogonalization guided by activation subspace estimation to preserve safety mechanisms (refusals).

Here are specific improvements and what the resulting AI system can achieve:

) Improved AI System Capabilities via NEEDLE Application:

The following improvements focus on creating LLMs that are demonstrably more secure against targeted, trigger-based manipulation while maintaining high utility and safety.

A robust, training-free defense mechanism for removing specific backdoor triggers from pre-trained LLMs without requiring access to the original poisoned data or a clean reference model.

The system can perform targeted removal of adversarial behaviors (e.g., generating specific harmful text, inserting malicious code) that are activated by a hidden trigger string or pattern in the input prompt.

The defense preserves the model's crucial safety capabilities, specifically its ability to refuse harmful requests (e.g., refusing to generate instructions for illegal activities or hate speech), ensuring that removing a backdoor does not inadvertently cause the model to become overly compliant or jailbreak-able.

The system achieves superior performance metrics across multiple attack vectors:

• Significantly lower Attack Success Rate (ASR) compared to existing defenses (e.g., achieving near 0% on challenging code injection attacks).

• Minimal degradation in general model capability benchmarks (MMLU, GSM8K), evidenced by KL divergence remaining very low (e.g., 0.01 nats/token on Gemma-3-4B-IT).

• Maintained or even improved safety metrics against harmful prompts (Safety ∆ is minimal).

The system can be deployed as a targeted model editing solution, allowing researchers to surgically remove known vulnerabilities in deployed open-weight models without requiring expensive full fine-tuning cycles or complex inference-time modifications.

) Specific Technical Mechanisms for Improvement:

These improvements are realized through the following technical steps inherent in the NEEDLE methodology:

The system first estimates a specific backdoor direction vector by contrasting activations between inputs containing the trigger and those that do not (using mean activation differences, Eq. 3). This allows for precise identification of how the backdoor manifests in activation space.

Simultaneously, it constructs a multi-dimensional refusal subspace (Rl) by analyzing the difference between activations generated by harmful prompts versus compliant ones. This subspace captures the model's inherent safety mechanisms and refusal logic (Eq. 4).

The core editing operation involves calculating a corrective direction, ul, which is orthogonal to both the identified backdoor direction and the entire refusal subspace: ul = (I−RlR⊤l)bl (Eq. 6). This mathematically ensures that any weight adjustment made along this direction suppresses the backdoor projection while strictly preserving all projections onto the refusal subspace.

The system applies these orthogonalization updates sequentially across model layers (1 to 34), first modifying the output projections (attention and MLP) and then using a ridge regression update (Eq. 9) to correct for any drift in the preserved refusal projections caused by earlier layer edits, ensuring the final weights are optimally tuned for both security and safety constraints.

Backdoor attacks can be implanted in Large Language Models (LLMs) during training, causing unwanted behaviour when a trigger appears in the input. Existing backdoor defences for LLMs attempt to remove the backdoor but inadvertently shift the model's output distribution to benign prompts, which can result in degraded model performance and safety. We propose NEEDLE, a training-free method for targeted backdoor removal. Once a trigger has been identified, our method estimates a backdoor direction and a refusal subspace through activation vectors, then applies sequential weight orthogonalisation to suppress the backdoor while preventing changes in refusal-related representations. NEEDLE requires neither a clean reference model nor the original poisoned training data. Evaluation is conducted across multiple model families and attack types. NEEDLE achieves the lowest mean Attack Success Rate (ASR) among the evaluated defences, including 0% on challenging code injection attacks, while resulting in the lowest KL divergence and minimal changes in capability and safety.