SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning

summary

The gist

The gist The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs) that lack strong

In short

Small Language Models struggle to handle complex codebases because they lack deep, specific knowledge. The proposed Repository-Centric Learning (RCL) shifts training focus from broad tasks to deep mastery of a single repository using a four-unit experience. This results in SWE-SPOT-4B experts that outperform larger models and commercial tools by internalizing the 'physics' of the target code.

Key concepts

Task-Centric Learning (TCL): This traditional method scales exposure by training models across many different, unrelated repositories and tasks. The paper found this fails for small models because it leads to surface-level pattern matching instead of deep understanding of a specific codebase's design and semantics.
Repository-Centric Learning (RCL): RCL is a new approach that prioritizes vertical depth—intense, repeated interaction within one single repository. It aims to teach the SLM the operational rules and design constraints of that specific software environment through focused, multi-step learning.
Repository-Centric Experience (RCX): This is a four-unit training framework used in RCL to turn static code into interactive signals. These units include analyzing design intent, modeling cross-file dependencies, learning from historical changes, and designing tests to check runtime behavior against specifications.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 19 episodes
Federated Learning — explained in 17 episodes
Retrieval-Augmented Generation (RAG) — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Interpretability — explained in 15 episodes

This episode discusses

The paper

SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning · Read on arXiv

Jinjun Peng, Magnus Saebo, Tianjun Zhong, Yi-Jie Cheng, Junfeng Yang, Baishakhi Ray

Columbia University · University of California, Los Angeles

The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs). However, they suffer from a fundamental capability gap: unlike frontier large models, they lack the inference-time strong generalization to handle complicated, unfamiliar codebases. We identify that the prevailing Task-Centric Learning (TCL) approach, which scales exposure across disparate repositories, fails to address this limitation. In response, we propose Repository-Centric Learning (RCL), a shift that prioritizes vertical, single-repository depth over horizontal, single-task breadth for SLM training. To accomplish this depth, we design a four-unit Repository-Centric Experience, transforming static codebases into interactive learning signals, to train SWE-Spot-4B, a family of highly compact models built as repo-specialized experts. Under controlled comparisons with the same teacher and data budget, RCL consistently outperforms TCL across tasks, teachers, model scales, and programming languages. On seven target repositories, SWE-Spot-4B experts match or exceed existing open-weight models up to 8 times larger (e.g., CWM by Meta, Qwen3-Coder-30B) and surpass or match efficiency-focused commercial models (e.g., GPT-4.1-mini, GPT-5-nano) across multiple SWE tasks. Further analysis reveals that RCL yields higher training and inference efficiency, deep knowledge acquisition, and can scale to repository-cluster experts. Our work emphasizes that for building efficient intelligence, repository mastery is a distinct, necessary, and practical dimension that complements general coding capability.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning".

Jane: The gist The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs) that lack strong inference-time generalization to work with…

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at the paper titled "SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning," and the title itself suggests they’re aiming to build models that are specialists in one specific repository.

Jane: That really points to an idea that these agents aren't just general coders anymore; they are becoming experts in a particular software environment.

Lu: They argue that this approach lets small language models, which usually have limited size, acquire the necessary knowledge during training through deep parametric learning instead of just trying to recover it later when they're actually being used.

Meng: I see the name "Repo-Experts," and that implies a level of depth we don't always see with smaller models when they’re just being broadly exposed to a lot of different code.

Lalam: The core idea is that for these models, getting deep mastery over one specific repository is more effective than just having shallow exposure across many unrelated repositories.

The paper's summary: Tom: So what’s the actual mechanism they propose? They suggest a new way of training where we prioritize vertical depth in a single codebase over horizontal breadth across many tasks.

Jane: Instead of just feeding the model lots of different code examples randomly, they design this four-unit RepositoryCentric Experience to turn those static codebases into signals that actively teach the model during its training.

Lu: They call this four-unit experience the RCX framework, which includes things like software design, modeling contextual implementation, learning from evolutionary replay of historical changes, and aligning semantics with runtime behavior.

Meng: That sounds like a lot of structured data they’re feeding into the model during training to ensure it learns the structure instead of just memorizing surface-level code snippets.

Lalam: They are essentially encoding the repository’s operational rules as a reusable prior, which is how they think this deep parametric knowledge acquisition actually works for these small language models.

The paper's improvements: Tom: Now let's look at what they claim is better about this Repository-Centric Learning approach when compared to the Task-Centric Learning that they’re contrasting it with.

Jane: They show that SWE-Spot, even though it’s a smaller model, it performs better than open-weight models that are up to eight times larger and even matches or exceeds efficiency-comparable commercial models like GPT-five-nano and GPT-four point one-mini <ref:2601.21649#pg3>.

Lu: That performance difference comes with a shift in how the agent behaves because by internalizing the specific structure and dynamics of the repository, SWE-SPOT achieves better training sample efficiency and substantially lower inference time cost.

Meng: Lower inference time cost is important for deployment; if it runs faster or cheaper once you put it into production, that’s a tangible benefit we care about when scaling up.

Lalam: They point out that this performance gap shows deep parametric knowledge acquisition beyond just simple context memorization or superficial pattern matching because they show "Oracle Localization" becomes redundant for the model under this approach.

Conclusion: Tom: So to wrap up, Repository-Centric Learning moves us toward models that have a distinct dimension of repository mastery that complements their general coding capability.

Jane: It suggests that for small language models, deep mastery of one specific target environment is more valuable than broad but shallow exposure across countless unrelated codebases.

Lu: The ablation studies confirm that the four units of RCX are synergistic; removing any single unit degrades performance across multiple downstream tasks, so it’s about holistic exposure rather than isolated skills.

Meng: I think prioritizing performance within a specific working scope over trying to maintain average performance across the entire world of codebases is a very practical conclusion for deployment.

Lalam: The paper "SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning" introduces this new dimension, showing that repo expertise is not just about memorizing syntax but about having robust representations that can affect each other.

More episodes

← Home