GradSentry: Gradient Spectral Entropy for Backdoor Sample Filtering in Large Language Model Fine-Tuning

arXiv:2605.26574 · cs.CR · Submitted 2026-05-26 · Read on arXiv

cs.CR

Submitted: 2026-05-26

Updated: 2026-08-31

Code: https://github.com/dongdongzhaoUP/GradSentry

Terminology

• Clustering — 34× in this paper · explained in 2 episodes
• Fine-Tuning — 26× in this paper · explained in 5 episodes
• F1 Score — 8× in this paper · explained in 3 episodes
• Robustness — 8× in this paper · explained in 5 episodes
• Accuracy — 6× in this paper · explained in 1 episode
• Parameter-Efficient Fine-Tuning — 5× in this paper · explained in 2 episodes
• Instruction Tuning — 4× in this paper · explained in 2 episodes
• Core — 3× in this paper · explained in 2 episodes
• Spectral Signature — 3× in this paper · explained in 2 episodes
• Adaptation — 2× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Computational overhead — 2× in this paper · explained in 2 episodes
• Feature Extraction — 2× in this paper · explained in 2 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Distillation — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Intrinsic Dimension — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Low-Rank Adaptation (LoRA) — 1× in this paper · explained in 9 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Parameter-Efficient Fine-Tuning (PEFT) — 1× in this paper · explained in 4 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Singular Value Decomposition — 1× in this paper
• Singular Value Decomposition (SVD) — 1× in this paper · explained in 2 episodes
• Supervised Fine-Tuning — 1× in this paper

Sources

• GPT-4 Technical Report
• Lost in Backpropagation: The LM Head is a Gradient Bottleneck
• BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• ProtegoFed: Backdoor-Free Federated Instruction Tuning with Interspersed Poisoned Data

Related papers

• SoK: AI-Augmented Binary Reversing
• Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
• Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
• Efficient Fuzzy PSI under One-Sided Assumptions
• Sealing the Audit-Runtime Gap for LLM Skills
• Token Composition: A Graph Based on EVM Logs