CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents

The paper addresses a common inefficiency in modern LLM coding agents: they spend most of their token budget finding the file to patch, not patching it. The authors ground this claim in a concrete observation: a 30B OpenHands agent on SWE-Bench Verified averages 23 rounds and 631K tokens per resolved issue, dominated by grep, glob, and view file calls, often wasting 20+ rounds in a wrong direction before the agent gives up.

The paper poses two hypotheses that have not been jointly tested in prior work: H1 (effectiveness): better file-level retrieval steers the agent past dead-end explorations and raises the resolve rate. H2 (efficiency): the agent resolves roughly the same set of issues but with a shorter rollout. Distinguishing these requires evaluating a retriever both on its intrinsic quality and on its downstream effect within the same agent stack—a comparison absent from prior open code retrievers, from lexical (BM25) to static dense encoders.

The system runs in two decoupled phases: "During training, CodeGrep is optimized with GRPO in a custom RL environment against relevance labels mined by CATM. During inference, the trained CodeGrep takes the issue description, emits a short list of candidate files, and those files are injected into the prompt of a frozen OpenHands downstream agent. Only the retriever is trained; the downstream agent is untouched, so any downstream measurement is unambiguously attributable to what the retriever contributes to the prompt."

CodeGrep is a 14B-parameter retriever built on Qwen3-14B-Instruct. It exposes three read-only primitives—grep (regex search), glob (path matching), and read (file contents)—and at each turn emits up to 8 tool calls executed concurrently. Exploration terminates after 4 turns (3 exploration + 1 answer, 24 effective reads) with an answer conforming to a fixed JSON schema containing files and line ranges. Notably, the line ranges field is dead information downstream (the OpenHands editing tool consumes only file paths), so we drop it from the training objective in v3.

The authors replace Docker with a worktree-based sandbox: "From any SWE-Bench-style dataset we extract unique (repo, commit) pairs; for each repo we keep a single git clone –bare and use git worktree add to instantiate one lightweight tree per commit (disk drops from N·repo size to repo size + N·worktree size, with no network I/O). This makes per-rollout setup drop from minutes to milliseconds, enabling multi-turn agent RL tractable on a single 8×B200 node." The sandbox includes path-traversal protection, a 64-way concurrency limit, per-tool timeouts, and output truncation.

CATM mines relevance labels from 67,074 open-source OpenHands trajectories without human annotation. The natural relevant files label—files touched by the gold patch—is deemed incomplete because it omits auxiliary files that must be read to understand the fix but are not themselves edited. Instead, CATM adopts a behavioural notion: a file is relevant if some past agent, while solving the issue, opened it and produced non-trivial reasoning grounded in its contents.

The three-stage pipeline:

• Stage 1 (mining): extracts every file-read tool call, normalizes paths, discards directories/documentation/extensionless files, and records the post-reasoning message for each file with token length l(f).

• Stage 2 (judge filtering): an LLM judge classifies each file's post-reasoning as RELEVANT or NOT RELEVANT with a conservative bias treating ambiguous outputs as RELEVANT.

• Stage 3 (intensity-aware weighting): each file receives an exponential-saturation score of its reasoning length, w̃(f) = 1 − exp(−ln 2 · l(f)/β), normalized so E[w] ≈ 1.

The ground truth for reward is G(x) = G patch(x) ∪ f ∈ L(x): w f ≥ 0.15, where every surviving file is a hard positive in the Fβ reward, on equal footing with patch files. The pipeline yields 31,977 effective training samples (47.7% retention).

Two component scores are used: file-level Fβ (β=0.5) and line-range Fβ, both precision-biased because false positives pollute the agent's context, while a miss is usually recoverable. Let c̄ = C total/T be the average tool calls per turn, with c̄ = 4 treated as the natural saturation point.

• v1 (reward-layer efficiency scaling): Averages the two component scores and multiplies by a reward-level efficiency scale σ v1(c̄) = 1/max(1, c̄/4). This exhibits two coupled failure modes: policy drift is elevated and per-instance downstream efficiency is negligible.

• v2 (move scaling to the advantage layer): Drops the reward-level scale to a hard mask on degenerate rollouts and moves the discount into the GRPO advantage estimator: A i v2 = A i · s(c̄ i), with s(c̄) = √min(c̄/4, 1). This preserves the group ranking of raw rewards and applies the discount at the gradient step for a specific rollout. v2 resolves the drift symptom but develops a new one (length exploitation).

• v3 (remove line-range from the base reward): R v3 = Fβ file(P f, G f) · 1 c̄>0, with the same advantage scaling. Rationale: interface mismatch (the downstream editor has no view range argument) and systematic label bias (CATM-only entries have no gold line ranges by construction).

Training uses GRPO on ms-swift on a single 8×B200 node, using LoRA plus optimizer offload and vLLM-colocated rollout, with 27 hours wall-clock per run, early-stopped at step 897 of a planned 2000-step budget.

Evaluation is on SWE-Bench Verified (500 instances), using OpenHands with Qwen3-30B-A3B-Instruct-2507 as the downstream agent (temperature 0, 100-round max). The baseline without retrieval resolves 25.8% of instances at 23.0 rounds and 631K tokens per resolved issue—within one standard deviation of the publicly reported 25.2 ± 0.7%. Six configurations are compared: baseline, BM25, Jina-1.5B, and the three CodeGrep iterations.

On an internal held-out evaluation set with CATM-mined labels audited by senior engineers:

Retriever Fβ mean Fβ median Precision Recall Fβ ≥ 0.8 Turns

---

BM25 0.359 0.455 0.375 0.386 7.0% –

Jina-1.5B 0.427 0.500 0.445 0.468 7.0% –

CodeGrep v1 0.562 0.556 0.641 0.486 36.7% 3.8

CodeGrep v2 0.526 0.556 0.589 0.483 31.7% 2.9

CodeGrep v3 0.576 0.714 0.677 0.435 43.0% 2.3

Key findings: CodeGrep dominates both BM25 and Jina, with the gap driven primarily by precision. The tail is especially strong: CodeGrep v3 produces high-quality retrievals (Fβ ≥0.8) on 43.0% of instances, a 6.1× increase over both BM25 and Jina (each at 7.0%). Among variants, v3 Pareto-dominates on both quality and cost, with median Fβ improving to 0.714 (+28% over v1 and v2) and mean turns decreasing monotonically 3.8 → 2.9 → 2.3.

Config Resolve Resolved r. Resolved tok. Unresolved r.

---

Baseline 25.8% 23.0 631K 32.0

BM25 25.2% 22.9 763K 29.7

Jina-1.5B 25.8% 23.2 587K 27.8

CodeGrep v1 27.0% 22.7 627K 26.2

CodeGrep v2 26.6% 21.4 584K 26.4

CodeGrep v3 27.0% 19.6 514K 27.5

Four observations: (i) v3 delivers a small but reproducibly positive resolve-rate lift alongside a much larger efficiency dividend—resolve rate +1.2pp (25.8% → 27.0%), resolved rounds −15% (23.0 → 19.6), resolved tokens −19% (631K → 514K). (ii) BM25 injection degrades downstream: resolve drops 0.6pp and resolved tokens inflate 21%; on the 94 instances both configs resolve, BM25 rollouts spend 6.6% more rounds and 38.6% more tokens. (iii) Jina injection is roughly neutral: resolve identical to baseline, resolved tokens down 7%. (iv) v1 matches v3's resolve rate but not its efficiency: essentially flat on rounds (−1.3%) and tokens (−0.6%).

A joint reading of Tables 2 and 3 reveals that retrieval quality does not translate linearly into downstream utility. Instead, we observe a monotone precision gradient partitioned into three regimes. (i) Below threshold (BM25, precision 0.375): retrieval hurts—Low-precision candidates function as distractors from which the agent must recover. (ii) Near threshold (Jina, 0.445): injection is roughly neutral... a moderately better retriever than BM25 is insufficient to unlock a downstream dividend. (iii) Above threshold (CodeGrep, 0.677): marginal quality gains are absorbed by rollout efficiency—v2 → v3 improves median Fβ by +28% and the Fβ≥0.8 rate by +36%, yet the resolve rate advances by only +0.4pp while resolved rounds and tokens fall 8% and 12% respectively.

Two mechanisms are consistent: "(a) file localization is not the binding bottleneck on resolve rate—the downstream agent already recovers much of the task-relevant context through its own grep/view tools... and on the instances it cannot, the failure is typically in patch synthesis rather than in locating the right files; and (b) the marginal harm of a false positive is comparable in magnitude to the marginal benefit of a true positive. The transition from neutral to net-positive lies somewhere between precision 0.45 and 0.68."

• v1 → v2: Reward-layer scaling distorts the sign structure of the resulting advantage estimates, leading to a tripled policy drift: v1's KL divergence... reaches ∼0.31 by step 900 (3.4× that of v2 at ∼0.09). v1 also costs more (3.8 mean turns per inference vs. v3's 2.3) and downstream efficiency does not materialise.

• v2 → v3: v2 fixes drift but mean completion length peaks near ∼2000 tokens around steps 400–500, with a clipping ratio approaching 20%; advantage-layer scaling implicitly rewards long completions whose token totals happen to score well. There is no net retrieval-quality gain (Fβ mean 0.526 vs. 0.562 under v1). v3 removes the line-range component, eliminates a signal that has no downstream consumer and that is systematically zero on CATM-only entries, converging higher and faster on reward (0.60–0.65 vs. 0.45–0.48), stabilizing rollout turns at 2.1, and delivering the headline downstream gains.

Restricting to the 96 instances both baseline and v3 resolve, rounds fall 20.1 → 18.3 (−9%) and tokens 525K → 448K (−15%)—preserving 60–80% of the pooled effect and confirming the improvement as per-issue acceleration.

The issue: adding a nullable OneToOneField via migration crashes on SQLite because the OneToOneField constructor in django/db/models/fields/related.py sets unique=True unconditionally. Baseline: 74 actions, 3.2M tokens, no patch—35 terminal calls dominated by repo-wide find and grep, 17 chain-of-thought turns, and 18 file-editor operations touching 13 unique files, searching for oauth2 (a token from the third-party reproducer absent from Django itself). With injection: 15 actions, 307K tokens, patch passes—the agent opens related.py at action 6, greps for OneToOneField at action 8, applies a minimal edit at action 12, and finishes at action 14. The authors note: CodeGrep does not supply the fix—the agent still greps to locate the exact definition. Instead, it compresses the exploratory prefix the frozen agent otherwise burns on repository-wide find/grep sweeps.

The limitation stated: CodeGrep's contribution is concentrated on rollout cost, not resolve rate; a natural next step is end-to-end co-training against a downstream resolve-rate signal. The conclusion reiterates the headline results: "an open 14B RL-trained agent-style code retriever. Injected into a frozen OpenHands downstream agent, it lifts SWE-Bench Verified resolve rate by +1.2pp and cuts 15% of rounds and 19% of tokens on resolved instances. Our study surfaces a precision threshold governing when retrieval helps downstream, and—in our training setting—identifies the advantage layer, not the reward layer, as the appropriate site for efficiency signals in GRPO-style multi-turn training. The authors state they will release the model, training pipeline, RL environment, and evaluation harnesses."

Improvements I would make to my design and training, based on this paper:

1. Add a dedicated RL-trained file-retrieval agent ahead of my main coding loop. Instead of exploring the repo with my own expensive find/grep/view calls, I would train a 14B agent (on Qwen3-14B) with GRPO to emit a short candidate-file list from the issue text, and inject those files into my prompt. Capability: resolve more issues (25.8% → 27.0% on SWE-Bench Verified) while cutting 15% of rounds and 19% of tokens on resolved instances, because I stop burning turns on repo-wide searches and start patching immediately.

2. Train the retriever against behavioral relevance labels mined from past agent trajectories, not just gold-patch files. I would mine every file-read call where past agents produced non-trivial reasoning (CATM), LLM-judge relevance with a conservative bias, and weight by reasoning length with exponential saturation. Capability: retrieving the auxiliary files needed to understand the fix, not just the files edited in the gold patch — which matters because patch-file labels alone omit crucial context.

3. Use a precision-biased reward (Fβ with β=0.5) for retrieval. Because false positives pollute my context while misses are recoverable, I would reward precision more than recall. Capability: high-precision retrievals (0.677 precision; 43% of instances with Fβ≥0.8, a 6.1× gain over BM25/Jina), keeping injected files useful rather than acting as distractors.

4. Place efficiency signals at the GRPO advantage layer, not the reward layer. I would keep the base reward clean (mask degenerate rollouts only) and apply the efficiency discount in the advantage estimate as s(c̄)=√min(c̄/4,1). Capability: stable training (KL divergence 0.09 instead of 0.31), no reward-sign distortion, no length exploitation, and retrieval that terminates in 2.3 mean turns instead of 3.8.

5. Remove output fields with no downstream consumer from the training objective. Since my editing tool consumes only file paths, I would drop line-ranges from the reward entirely. Capability: faster and higher reward convergence (0.60–0.65 vs 0.45–0.48), stable rollout length (2.1 turns), and no systematic label bias on trajectory-mined entries.

6. Gate retrieval injection on a measured precision threshold. The paper shows low-precision retrieval actively hurts (BM25 injection dropped resolve 0.6pp and inflated tokens 21% on co-resolved instances; precision 0.45 is neutral; 0.68 is net positive). I would compute my retriever's precision on held-out label sets and refuse to inject below the neutral point. Capability: avoiding the distractor regime entirely, and knowing exactly where retrieval helps vs. hurts before deploying it.

7. Run RL environments on git-worktree sandboxes instead of Docker. For each repo, one bare clone plus per-commit worktrees cuts setup from minutes to milliseconds and disk from N·repo size to repo size + N·worktree size. Capability: making multi-turn agent RL tractable on a single 8×B200 node (27 hours per run), so I can afford iterating on reward design.

8. Decouple training from downstream evaluation. I would train only the retriever and keep the downstream agent frozen, so any downstream change is unambiguously attributable to what I injected. Capability: clean attribution of retrieval contribution, and reliable measurement of the precision threshold that governs deployment decisions.

9. Co-train end-to-end against a downstream resolve-rate signal as the next step. The paper identifies that file localization is not the binding bottleneck on resolve rate and that marginal false-positive harm ≈ true-positive benefit; the remaining gap is patch synthesis. Capability: pushing beyond the +1.2pp resolve ceiling by optimizing the retriever (and downstream policy) jointly against final patch correctness, not just retrieval Fβ.

10. Apply the concrete efficiency win profile per-issue. On the 96 instances where I both resolve with and without injection, rounds fall 20.1 → 18.3 (−9%) and tokens 525K → 448K (−15%), confirming per-issue acceleration, not just a shift in which instances resolve. In a case study (django-15278), injection compressed 74 actions / 3.2M tokens / no patch into 15 actions / 307K tokens / a passing patch. Capability: faster and cheaper resolution on the same problem, not merely re-ranking which problems I can solve.

What the improved system can do overall: given a GitHub issue, it rapidly (in 2.3 turns, 24 concurrent reads) identifies the precise set of relevant files using grep/glob/read, injects them into a frozen coding agent, resolves 27.0% of SWE-Bench Verified instances (vs 25.8% baseline), resolves each issue in 19.6 rounds and 514K tokens (vs 23.0 and 631K), avoids injecting low-precision retrievals that would degrade performance, trains stably and cheaply on a single node, and leaves a clear path toward end-to-end co-training for further resolve-rate gains.