Learning Reusable Agent Subroutines from Experience

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Learning Reusable Agent Subroutines from Experience".

Jane: The gist The central result is that induced libraries outperform the very agent that generated their traces by tens of percentage points across various subtasks,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: They summarize how they take successful ReAct rollouts and run them through four stages to build this library of primitives. First, they filter for correctness, then extract the per-step thought strings, categorize those thoughts using LLMs, and finally synthesize the top K as these typed primitives.

Jane: So they’ve got this pipeline where one LLM call does the naming and clustering of moves, and another LLM call helps group synonyms into five to ten canonical categories if there are more than ten unique labels >

Lu: The central result they show is that induced libraries outperform the very agent that generated those traces, specifically by showing gains like plus forty-four percentage points on RuleArena NBA and plus thirty percentage points on MUSR team allocation >

Meng: Forty-four points is a lot of difference, so if this holds up under real traffic loads, it’s significant. But what about the setup? They use a single fixed configuration across five different subtasks for these comparisons >

Lalam: They even showed that this single fixed configuration beats zero-shot Chain-of-Thought on every subtask and matches or surpasses expert-authored decompositions in terms of performance >

The paper's summary: Tom: The paper highlights a few key improvements over just having an agent generate its own sequence of steps. First, they moved the primitives into reasoning space rather than environment-grounded action space, which changes how they interact with the agent loop >

Jane: That means these primitives are not fixed workflows you have to follow; instead, the agent composes them freely inside a standard ReAct loop at test time >

Lu: They also emphasize that this induction process is a single pass with no online iteration, no MCTS, and no activation steering signals, which places them in a different regime compared to other approaches >

Meng: That’s good—no complex online iteration means faster discovery. But I wonder about the cost. The paper mentions that the per-instance compute for this reasoning primitive induction is around twenty-four percent cheaper than its closest competitor, AWM, which uses about one hundred seventeen thousand tokens versus one hundred fifty-four thousand tokens >

Lalam: They also show that even when using a smaller model like Gemini Flash Lite, the pattern of the source agent outperforming the induced library still holds true across different model families >

The paper's improvements: Tom: So to wrap up, this paper on "Learning Reusable Agent Subroutines from Experience" shows that we can discover reasoning structure without needing expert authoring. They've created typed callables with natural-language docstrings that are inspectable and editable inside a ReAct loop >

Jane: It suggests that instead of baking these routines into model weights, you get an emergent property of how an agent reasons across different instances of a task family >

Lu: The main implication is that this training-free, single-pass route to discovering reasoning structure has previously required expert authoring and now it can be found automatically >

Meng: For deployment, the paper notes that you should pair this induction with human review of the synthesized library and ongoing validation against external ground truth. You can't just drop these in blind >

Lalam: It’s a way to make the agent’s reasoning vocabulary human-readable and individually auditable, which is valuable for high-stakes domains where you need to know exactly how it arrived at a decision >

Conclusion: Tom: So we've covered "Learning Reusable Agent Subroutines from Experience." Basically, they show that instead of just having an agent solve one problem, you can extract its successful reasoning moves and turn them into reusable tools for a whole family of tasks >

Jane: Right. It's about moving those transient scratchpads out of the agent's memory and making them permanent, typed callables that the agent can just plug in whenever it needs that specific type of reasoning >

Lu: The core idea is Reasoning Primitive Induction, a single-pass method where you mine successful ReAct traces, cluster those moves into a library of pseudo-tools based on their frequency >

Meng: I'm still thinking about the deployment side. This sounds like it creates an external knowledge base for the agent. What's the real friction here compared to just feeding it more context?

Lalam: It’s not just context; these are typed callables with natural-language docstrings that have zero handwritten task logic inside them, which makes them really auditable >

Tom: That’s a big deal for inspection. The results are pretty solid on the numbers. They outscore the agent they were trained on by forty-four percentage points on RuleArena NBA and thirty points on MUSR team allocation >

Jane: Forty-four is a significant gap, Tom. It suggests that this recovered structure is actually more reliable than the source policy itself, which is pretty interesting for how we view agent performance >

Lu: I think it’s because this mechanism does implicit aggregation of representative thoughts from many successful rollouts to write a corpus-level specification for each move >

Meng: Forty-four points is a lot. But what about the cost? If this induction process is more expensive than just running the agent normally, it doesn't help us deploy it at scale >

Tom: Well, they did look at that. They found that per-instance compute for this induction method is around twenty-four percent cheaper than a close competitor, AWM >

Lalam: And they showed that even when using a smaller model like Gemini Flash Lite, the pattern of the source agent outperforming the induced library still holds true across different model families >

Jane: So what does this mean for us as people who are just listening? It means we can discover expert-level reasoning structures without needing someone to manually write all those specific steps first >

Tom: Exactly. This training-free, single-pass route to discovering structure has previously required a lot of manual work from experts, and now it seems like we can automate that discovery process >

Lu: The main implication is that this automated discovery of reasoning structure means we don't need those massive expert decompositions anymore if the agent learns how to build its own better tools >

Meng: For deployment, I still think you have to pair this induction with human review of the synthesized library and ongoing validation against external ground truth. You can't just drop these in blind >

Lalam: It’s a way to make the agent’s reasoning vocabulary human-readable and individually auditable, which is valuable for high-stakes domains where you need to know exactly how it arrived at a decision >

Tom: So to wrap up, this paper on "Learning Reusable Agent Subroutines from Experience" shows we can discover reasoning structure without needing expert authoring. They've created typed callables with natural-language docstrings that are inspectable and editable inside a ReAct loop >