DRIFT: Data Selection for LLM Instruction Tuning via On-Policy Attribution

arXiv:2606.18307 · cs.LG, cs.AI · Submitted 2026-06-16 · Read on arXiv

cs.LG, cs.AI

Submitted: 2026-06-16

Updated: 2026-09-26

Code: https://github.com/huggingface/open-r1

Terminology

• Influence Functions — 25× in this paper · explained in 2 episodes
• Fine-Tuning — 12× in this paper · explained in 5 episodes
• Benchmark — 8× in this paper · explained in 2 episodes
• Distillation — 8× in this paper · explained in 2 episodes
• Self-Distillation — 8× in this paper · explained in 7 episodes
• Supervised Fine-Tuning — 8× in this paper
• Cosine Similarity — 5× in this paper · explained in 3 episodes
• Instruction Tuning — 5× in this paper · explained in 2 episodes
• Core — 3× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 3× in this paper · explained in 6 episodes
• Score Function — 3× in this paper · explained in 4 episodes
• Supervised Fine-Tuning (SFT) — 3× in this paper · explained in 8 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Adam — 2× in this paper · explained in 2 episodes
• External Validation — 2× in this paper · explained in 2 episodes
• Proximal Bregman Response Function — 2× in this paper
• Sparsity — 2× in this paper · explained in 4 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Catastrophic Forgetting — 1× in this paper · explained in 5 episodes
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Data Selection with Importance Resampling — 1× in this paper
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Domain Adaptation — 1× in this paper · explained in 5 episodes
• Fisher Information — 1× in this paper · explained in 3 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Hilbert Space — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Theoretical Guarantees — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Efficient Online Data Mixing For Language Model Pre-Training
• What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions
• IF-GUIDE: Influence Function-Guided Detoxification of LLMs
• Studying Large Language Model Generalization with Influence Functions
• Influence Functions for Efficient Data Selection in Reasoning
• Large-Scale Data Selection for Instruction Tuning
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models
• Do Influence Functions Work on Large Language Models?
• RegMix: Data Mixture as Regression for Language Model Pre-training
• BM25S: Orders of magnitude faster lexical search via eager sparse scoring
• Reinforcement Learning Finetunes Small Subnetworks in Large Language Models
• Olmo 3
• TRAK: Attributing Model Behavior at Scale
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• The Curse of Recursion: Training on Generated Data Makes Models Forget
• Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
• Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
• MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks