Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities

arXiv:2501.12147 · cs.CL, cs.AI, cs.LG · Submitted 2026-08-17 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities".

Jane: The paper was written by Qirun Dai, Dylan Zhang, Jiaqi W. Ma and Hao Peng from University of Illinois Urbana-Champaign.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title and Authors: Tom: Welcome back to the channel, everyone. Today we’re cracking open a fresh one from arXiv, and the title is a mouthful: “Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities.” Jane, I’m going to need you to translate that for me.

Jane: Happy to, Tom. So, when you train a large language model, you don’t just throw all the text on the internet at it. You pick a specific set of examples to teach it how to follow instructions. This paper is about how to pick those examples really well.

Tom: Right, and the key word in that title is “balanced.” It’s not just about picking the most helpful data, it’s about making sure the model doesn’t become a one-trick pony.

Jane: Exactly. The authors are from the University of Illinois Urbana-Champaign, and they’ve noticed that the smartest data-picking methods, the ones that measure how much each example influences the model, have a hidden bias. They tend to favor certain skills over others.

Tom: So, you could end up with a model that’s a coding wizard but can’t do basic math or follow a simple instruction?

Jane: That’s the fear, and their experiments show it actually happens. They found that a method called LESS, which is a popular influence-based approach, would pick data that made the model great at one thing but noticeably worse at others, like logical reasoning.

Tom: And that’s a problem for real-world use. I don’t want an assistant that’s brilliant at Python but can’t help me with my taxes or write a decent email.

Jane: Right. The whole point of a general-purpose assistant is that it’s general. So, this paper is tackling a really practical problem that’s been hiding in plain sight.

Tom: So, they’re not just pointing out the problem, they’re fixing it. I’m guessing that’s where the “improving” part of the title comes in.

Jane: You’d be guessing right. They propose a new algorithm, and I’m excited to dig into how it works, because the solution is pretty clever.

Tom: Alright, I’m hooked. Let’s get into the nitty-gritty of what they actually found and how they fixed it.

Summary and Problem: Tom: So, Jane, we’ve got the title and the problem. Now, let’s get into the meat of the paper. The authors ran a bunch of experiments with models like Llama-three and Mistral, and they found that the standard way of picking data, the LESS method, was really unbalanced.

Jane: Yeah, and it’s a bit counterintuitive. You’d think that picking the most “influential” data would be the best strategy, but it backfires. They showed that some tasks, like world knowledge, just naturally have higher influence scores than others, like logical reasoning.

Tom: So, it’s like the algorithm has a favorite child. It keeps picking data that makes the model better at that one task, but it ignores the others.

Jane: Precisely. And here’s the kicker, Tom. They found that this bias doesn’t even help the model get better at that favorite task. It can actually hurt its performance there too.

Tom: Wait, that’s wild. So, the algorithm is so focused on, say, world knowledge, that it picks redundant data, and the model just gets worse at everything, including world knowledge?

Jane: Exactly. It’s like studying only the first chapter of a textbook for the final exam. You’ll know that chapter really well, but you’ll fail the rest of the test. And because you spent all your time on that one chapter, you might not even get an A on that section.

Tom: That’s a great analogy. So, the authors looked at this and said, “We need to fix the selection process itself.” They didn’t just want to pick the highest-scoring data; they wanted to pick a balanced set of data.

Jane: Right. And their solution is called BIDS, which stands for Balanced and Influential Data Selection. It’s a two-step process that directly addresses the bias they found.

Tom: Okay, so we have the problem and the name of the solution. What’s the secret sauce? How does BIDS actually work?

Jane: Well, the first thing they do is normalize the influence scores. Think of it like comparing test scores from different classes. A ninety percent in one class might be an A, but a ninety percent in another might be a C. You have to put them on the same scale before you can compare them.

Tom: That makes sense. So, they’re leveling the playing field between tasks.

Jane: And then, they do something even more interesting. Instead of just picking the top-scoring examples all at once, they pick them one by one, and they pay attention to what they’ve already picked.

Tom: Oh, so it’s a more thoughtful process. It’s not just a greedy grab.

Jane: Exactly. At each step, they look at the data they’ve already chosen and figure out which task is being neglected the most. Then, they pick the next example that does the most to help that neglected task.

Tom: So, it’s actively trying to fill in the gaps. That’s a much smarter way to build a training set.

Jane: It is. And the results show that this simple change makes a huge difference. We should get into the numbers, because they’re pretty impressive.

Improvements and Results: Tom: Okay, Jane, so BIDS is this two-step process: normalize the scores, then iteratively pick data to help the most underrepresented task. But does it actually work?

Jane: It does, and the results are pretty compelling. They tested it on seven different benchmarks covering five capabilities: coding, math, logic, world knowledge, and instruction following.

Tom: And the headline result?

Jane: BIDS consistently beat the standard LESS method and other baselines on the average score across all those tasks. It wasn’t just a small win either; it was a clear improvement.

Tom: So, it’s better at the overall goal of being a generalist.

Jane: Right. But the most surprising result, Tom, is that training on just fifteen percent of the data selected by BIDS was better than training on the entire dataset.

Tom: Wait, you’re telling me that using less data gave them a better model?

Jane: That’s exactly what they found. The fifteen percent subset, when trained for four epochs, outperformed the model trained on one hundred percent of the data. It’s a huge deal for efficiency and performance.

Tom: That’s incredible. It really shows that data quality is more important than data quantity. You can have a huge pile of data, but if it’s not the right data, it’s just noise.

Jane: And it’s not just about the average. They also showed that BIDS makes the performance more balanced across the different tasks. The model isn’t just good at one thing; it’s good at many things.

Tom: So, we’re not sacrificing performance on one task to get a better average. We’re actually getting a better model across the board.

Jane: Exactly. They even did an ablation study where they removed parts of their algorithm to see what was helping. They found that both the normalization step and the iterative selection step were important. Each one contributed to the overall improvement.

Tom: So, it’s not a magic trick. It’s a well-engineered solution with two key components that work together.

Jane: And the analysis of the selected data shows why it works. The data picked by BIDS has a much more even influence distribution across all the tasks. It’s not lopsided anymore.

Tom: So, they’ve essentially found a way to make the data selection process fair. It’s not just about picking the most influential data, but about ensuring all the capabilities get a fair shot.

Jane: That’s a perfect way to put it. And it has huge implications for how we train models in the future. Let’s bring in Lu and Meng to get their take on what this means for the field.

Lu: I think the most exciting implication is that it challenges the assumption that more data is always better. This paper provides a clear, evidence-based path to train a better model with less data, which is a massive win for efficiency.

Meng: And from an engineering standpoint, the computational cost of BIDS is minimal. You’re just doing some column-wise normalization and a simple iterative loop. It’s not a heavy lift to add this to an existing training pipeline.

Tom: So, it’s not just a theoretical idea. It’s something that could be adopted pretty easily.

Meng: Exactly. It’s a practical upgrade that can be implemented without needing a massive amount of extra compute.

Jane: And that makes it a really powerful tool for anyone training LLMs, from big labs to smaller startups.

Conclusion: Tom: Alright, let’s wrap this up. We’ve been talking about “Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities,” and it’s been a fantastic discussion.

Jane: It really has. To recap, the paper identifies a critical flaw in current influence-based data selection methods: they’re biased towards certain tasks, which leads to unbalanced and even worse performance.

Tom: And their solution, BIDS, fixes this with two clever ideas: normalizing the influence scores and iteratively selecting data to help the most underrepresented tasks.

Jane: The results are clear. BIDS not only beats other selection methods but also allows you to train a better model on a fraction of the data. That’s a huge win for both performance and efficiency.

Lu: It really is a significant step forward. It shows that we can be much smarter about how we train our models, moving away from brute-force data collection towards more thoughtful, balanced curation.

Meng: And it’s practical. The algorithm is simple and doesn’t add much overhead, so it’s something that can be adopted by the wider community.

Lalam: From a cultural perspective, this is about creating AI systems that are more well-rounded and reliable. An assistant that can code but can’t hold a conversation is less useful than one that can do both well. This research helps move us toward AI that is a more capable and trustworthy partner in our daily lives.

Tom: I love that. It’s not just about technical benchmarks; it’s about making AI more useful for everyone.

Jane: So, with that, we’re going to say goodbye to this paper. It’s been a great one, full of practical insights and exciting results.

Tom: Thanks for joining us, everyone. We’ll be back soon with another paper to break down. Until then, keep learning and stay curious.

Qirun Dai, Dylan Zhang, Jiaqi W. Ma, Hao Peng

University of Illinois Urbana-Champaign

cs.CL, cs.AI, cs.LG

Submitted: 2026-08-17

Updated: 2026-08-18

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 68/100

The gist: Selecting appropriate training data is crucial for effective instruction fine-tuning of large language models (LLMs), which aims to (1) elicit strong capabilities, and (2) achieve balanced

Key concepts

Influence-based Instruction Tuning
This process involves selecting specific examples from a dataset to teach a large language model how to follow instructions. Instead of using all available text, the model is trained on carefully chosen examples to improve its ability to perform tasks.
LESS method
LESS is a popular, standard approach for data selection that measures how much each example influences the model. The hosts note that this method can be biased, favoring certain skills (like world knowledge) over others (like logical reasoning).
BIDS (Balanced and Influential Data Selection)
BIDS is a proposed algorithm designed to fix the bias in data selection. It works in two steps: normalizing influence scores and then iteratively picking data to help the most underrepresented task.
Instruction Tuning
This is a method of training large language models where they are given specific instructions or prompts (rather than just general text). The goal is to make the model a more effective, general-purpose assistant that can follow diverse commands.

Terminology

Summary

Selecting appropriate training data is crucial for effective instruction fine-tuning of large language models (LLMs), which aims to (1) elicit strong capabilities, and (2) achieve balanced performance across a diverse range of tasks. Influence-based methods show promise in achieving (1) by estimating the contribution of each training example to the model’s predictions, but often struggle with (2). Our systematic investigation reveals that this underperformance can be attributed to an inherent bias where certain tasks intrinsically have greater influence than others. As a result, data selection is often biased towards these tasks, not only hurting the model’s performance on others but also, counterintuitively, harms performance on these high-influence tasks themselves.

As a remedy, we propose BIDS, a Balanced and Influential Data Selection algorithm. BIDS first normalizes influence scores of the training data, and then iteratively balances data selection by choosing the training example with the highest influence on the most underrepresented task. Experiments with both Llama-3 and Mistral-v0.3 on seven benchmarks spanning five diverse capabilities show that BIDS consistently outperforms both state-of-the-art influence-based algorithms and other non-influence-based selection frameworks. Surprisingly, training on a 15% subset selected by BIDS can even outperform full-dataset training with a much more balanced performance. Our analysis further highlights the importance of both instance-level normalization and iterative optimization of selected data for balanced learning of diverse capabilities.

The contributions of this paper include:

  1. We identify the problem of influence-based data selection algorithms in instruction tuning LLMs for learning diverse tasks, and attribute this problem to an inherent bias in cross-task influence through systematic analysis.

  2. We propose BIDS, a simple and effective influence-based selection algorithm for balanced learning of diverse capabilities.

  3. Through extensive experiments, we confirm the consistent and significant effectiveness of BIDS, and provide valuable insights on what makes a balanced set of influential data.

In this work, we introduce BIDS, an influence-based instruction tuning data selection algorithm specifically designed for balanced learning of multiple diverse capabilities. Motivated by the observation of an inherent bias in influence across various tasks, BIDS first applies column-wise normalization to the Attribution Matrix that contains pairwise data influence. Together with an iterative selection algorithm favoring underrepresented tasks, BIDS consistently outperforms various selection algorithms as well as full-dataset training with much more balanced performance. Our analysis further provides insight into the properties of an influential dataset with balanced capabilities.

Improvements for AI systems

Based on the paper, here are the specific improvements I can make to an AI system and what the improved system can do:

Improvement 1: Implement a Balanced and Influential Data Selection (BIDS) module for instruction tuning.

  • What I will do: I will add a new data selection module to the AI system's training pipeline. This module will:

  1. Normalize influence scores: Before selecting data, I will calculate the influence of each training example on a set of validation tasks. I will then normalize these scores per validation instance (column-wise) to ensure that tasks with inherently higher influence values don't dominate the selection process.

  2. Use iterative, task-balancing selection: Instead of simply picking the top-K most influential examples, I will use an iterative greedy algorithm. At each step, the system will select the training example that provides the largest marginal improvement in influence for the most underrepresented task in the currently selected set. This ensures a balanced representation across all target capabilities.

  • What the improved AI system can do: After instruction tuning, the AI system will achieve significantly more balanced performance across diverse tasks (e.g., coding, math, logical reasoning, world knowledge, instruction following). For example, instead of being very good at coding but poor at math, it will perform well on both. In our tests, this method consistently outperformed other selection methods and even full-dataset training, achieving a higher macro-average score across all benchmarks.

Improvement 2: Integrate a Balanced Influence analysis tool for training data monitoring.

Improvement 3: Implement a Balanced Capability training mode with adaptive epoch scaling.

Sources

Related papers