CB-Orchestrator: Adaptive Workflow Optimization for LLM Agents via Contextual Bandits

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "CB-Orchestrator: Adaptive Workflow Optimization for LLM Agents via Contextual Bandits".

Jane: The gist Large Language Model (LLM) agents have demonstrated remarkable capabilities in tackling complex tasks through agentic workflows,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we’re looking at CB-Orchestrator today. The authors are Jiahang Sun, Zhiwei Shang, Zhipiao Liu, Hongwei Yang, Guoqing Xie. It tackles how to make LLM agents use workflows much better without having to manually design every single thing from scratch for every new task.

Jane: Right. So the core idea is that instead of designing one perfect workflow for a specific job, you build a whole pool of good candidate workflows first, and then the system learns which one to pick based on what the user actually asks. Is that right?

Tom: Exactly. It decouples workflow generation from selection. The authors say manually designing these workflows is really labor-intensive and it doesn't give the agent enough flexibility for different kinds of queries > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1.

Lu: What I find interesting is how they build that initial pool using execution-guided evolution. It starts with an "Initial Generation" of a workflow for a seed query, then it uses "Execution-Guided Evolution" to refine it based on feedback > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Meng: From an engineering standpoint, I always worry about the cost of this feedback loop. The paper mentions that existing methods incur expensive API costs during the training phase feedback collection > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1.

Lalam: I see how they address the cost issue by creating a workflow pool rather than optimizing one single workflow repeatedly, which should make the training phase feedback collection more efficient > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1.

Jane: So they move away from optimizing a single task and towards having a diverse set of options ready to go > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. That’s the big shift we need to understand.

Tom: The second stage is where it gets clever. They formulate workflow selection as a Contextual Bandit problem > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. This lets the agent learn a policy tailored to the specific query it's seeing right now.

Title and authors: Lu: And they use this Contextual Bandit framework with a Hybrid Neural-Linear UCB selector > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. That combines neural reward prediction with a linear exploration bonus to keep things stable when the outcomes are uncertain.

Meng: I like the idea of balancing performance and sample efficiency that they mentioned > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. But what does that actually mean in terms of deployment?

Jane: It means the system learns when to trust a new workflow versus sticking with something it already knows is good, based on the context of the question > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. It’s adaptive routing for workflows.

Tom: And during inference, the agent just picks the workflow with the highest predicted reward from that learned policy > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. It’s a greedy selection based on what it has learned.

Lu: The results show this works really well across different benchmarks, even generalizing to Llama-three point one-70B backbones > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. That cross-domain transferability is pretty solid.

Meng: It’s good that it generalizes, but we need to talk about the overhead. The paper notes that this approach reduces training-phase API token overhead by no less than forty point five five percent and total end-to-end training token overhead by no less than twenty-nine point seven three percent on the MATH dataset > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. That’s a significant cost saving for us if we can actually deploy it reliably.

Jane: It sounds like they are proving that this method is not just theoretically sound, but practically more efficient than the manual design approach > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1. That’s what we need to see in production systems.

Tom: And the ablation study confirms that both the evolutionary pool construction and that hybrid UCB exploration actually contribute to those performance gains > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2. It means we can't just throw away either part of the framework without losing the benefit.

Title and authors: Lu: I also liked that they used Wilson score confidence intervals during inference to filter out unreliable workflows, requiring only the top fifty percent of workflows ranked by that metric > Improvements four.

Meng: Filtering things based on confidence is smart for stability when we’re running at scale > Improvements four. It cuts down on running potentially broken or very weak workflows during a real query.

Jane: So it’s not just about picking the best one, it’s about filtering out the ones that are statistically likely to fail even if they score high initially > Improvements four. That adds a layer of safety to the selection process.

Tom: The paper also showed that excluding pool generation cost reduces training token cost by no less than twenty-nine point seven three percent compared to baselines reporting those costs on MATH > Ablation and Cost Efficiency. That shows the value of having that initial pool construction part, even if it adds some upfront cost.

Lu: The generalizability is also strong, with scores averaging sixty-four point two four percent in Llama-three point one-70B experiments on both MATH and HumanEval benchmarks > Generalizability. It performs better than the Vanilla baseline by a large margin and is slightly better than MaAS there too.

Meng: I’m still skeptical about those benchmark scores because we haven't run them on real traffic, but the performance numbers are definitely compelling for a model like this > Generalizability. But the cost savings are what get my attention for production readiness.

Jane: So to wrap up, CB-Orchestrator is a two-stage framework that builds a diverse pool via evolutionary search and then uses contextual bandits with Hybrid Neural-Linear UCB for selection > Conclusion. It really moves us toward deploying versatile, cost-efficient LLM agents across different applications.

Tom: Yeah. It’s about making the workflow optimization process adaptive and sample efficient instead of rigid or just computationally expensive > Conclusion.

Lu: I think what this paper shows is that decoupling generation from selection is a very practical path forward for building more robust agentic systems > Conclusion.

Meng: For production, the cost efficiency metrics are what matter most right now when we’re talking about scaling these things up > Conclusion.

Lalam: CB-Orchestrator offers a concrete way to move away from manual workflow design toward automated, adaptive optimization for LLM agents > Conclusion.

The paper's summary: Tom: So we’re looking at how CB-Orchestrator works in a nutshell today. It’s this two-stage framework that stops you from having to manually design every single workflow for an AI agent > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1.

Jane: Right. So, it builds a big pool of candidate workflows first, and then it learns how to pick the best one based on what the user asks > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Tom: Exactly. Stage one is all about building that pool through execution guided evolution, which means it starts with a few initial ideas and then uses the actual feedback from running them to refine those workflows > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Jane: And stage two is where the magic happens. It treats choosing which workflow to use like a contextual bandit problem, learning a policy that adapts to every new query > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Tom: The authors use this with something called the hybrid neural linear UCB selector, which mixes predicting rewards with a little exploration bonus to keep things stable when the outcomes are uncertain > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Jane: So, what does this mean for us? It means instead of having one rigid plan, the AI agent can dynamically switch strategies depending on the question it gets > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Tom: Yeah. It lets the system learn which workflow is best for a specific task, which is way more flexible than just having one pre-made solution > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1.

Jane: And it’s not just about finding *a* good workflow. They actually found ways to filter out the bad ones during the process using something called Wilson score confidence intervals > Improvements four. That helps keep the system from picking a workflow that might look good but is actually unreliable > Improvements four.

Tom: Those filters are key for deployment, because you don't want your agent failing on a real user query because it picked a fluke > Improvement four. Plus, they showed that even when you skip the initial pool building cost, you still get major savings in training tokens compared to older methods > Ablation and Cost Efficiency.

Jane: So the paper is proving that this adaptive selection process is more sample efficient than just trying to optimize one single workflow for everything > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Tom: It’s about getting the best performance without having to run a massive search space for every new interaction, which is how you handle real-world scaling > Conclusion.

Jane: It points toward a future where AI agents aren't just following one set of instructions but are constantly optimizing their approach in real time based on the context they receive > Conclusion.

Tom: If this framework holds up across different backbones, like Llama models, it could actually make deploying complex agent systems much more practical for people building things now > Generalizability.

Jane: We'll look at those results from the Llama-three-70B experiments next, and see just how robust this works when you move to bigger models > Generalizability.

The paper's improvements: Tom: So we’re looking at what the authors suggest for improving CB-Orchestrator, moving past just making it work > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Jane: Right. So they’re talking about things like adding a way to detect when the agent is missing a whole strategy in its current workflow pool > Improvements four. It’s about checking for gaps in what it knows, and if it finds one, generating a new workflow on the fly > Improvements four.

Tom: That gap detection thing is interesting because it means the system doesn't have to wait for a massive retraining cycle just because the query is totally new > Improvement four. It lets you keep capability going during distribution shifts.

Jane: And they also mentioned using Wilson score confidence intervals during inference, not just picking the highest scoring workflow but filtering out anything that’s statistically shaky > Improvements four. They only pick the top fifty percent of workflows ranked by that metric > Improvements four.

Tom: That sounds like it adds a layer of safety to the selection process when you’re actually running stuff at scale > Improvement four. You don't want the AI picking something that looks good but is just fragile on a real user query.

Jane: So, they are pushing for this kind of online learning mode where the system actively expands its own toolbox based on what it sees > Improvements four. It moves it from a static pool to something that’s constantly evolving > Improvements four.

Tom: I like that idea of proactive expansion over just reacting when a failure happens later, which is how you build production systems > Improvement four. That proactive part is what separates this from older methods where you had to design everything upfront.

Jane: And they’re also exploring how to make the selection even more adaptive by looking at different ways to balance exploration and exploitation in that bandit problem > Contextual Bandit Selection. They are trying to find a better way for the context embedding and the reward prediction part > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Tom: Yeah, getting that exploration bonus right is crucial because it determines how quickly the system finds a good new workflow without just repeating what it already knows > Improvement four.

Jane: And they’re also focusing on making the whole process more sample efficient overall, which I think is the big win for any AI agent research > Contextual Bandit Selection. It means getting better performance with less training data and less computation > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1.

Tom: Exactly. If you can get that efficiency up, you can run these agents on much cheaper infrastructure > Ablation and Cost Efficiency. That’s where the real impact for engineers is > Conclusion.

Jane: So, the direction is clear: build a robust pool through evolution, learn to pick contextually with bandits, and add mechanisms to detect missing strategies and filter out risky options > Contextual Bandit Selection. It’s a lot of moving parts, but it’s all aimed at making these agents far more versatile > Conclusion.

Conclusion: Tom: So to wrap up, CB-Orchestrator is this two-stage framework that separates workflow building from how we select which one to use > Conclusion. It builds a diverse pool through evolutionary search and then uses contextual bandits with hybrid neural linear UCB for selection > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg1.

Jane: Right. It’s all about making the workflow optimization process adaptive instead of rigid or just computationally expensive > Conclusion. It decouples generation from selection > Preprint CB-ORCHESTRATOR: ADAPTIVE WORKFLOW OPTIMIZATION FOR LLM AGENTS VIA CONTEXTUAL BANDITS #pg2.

Tom: The real win here is that it makes agents more versatile and sample efficient for real-world use > Conclusion. It’s about having a practical path toward deploying these AI agents across different applications > Conclusion.

Jane: I think this suggests we are moving toward agent systems that can dynamically optimize their own internal logic based on the specific task at hand > Conclusion. That flexibility is what matters for real-world deployment.

Lu: I think the core idea here is decoupling those two steps, because if you can build a robust pool and then have a smart selection policy, you gain control over the system's behavior > Conclusion. It’s about giving the AI more strategic choice.

Meng: From my side, what I care about most is that it cuts down on the training token overhead by nearly thirty percent on benchmarks like MATH > Ablation and Cost Efficiency. That cost efficiency is a huge deal when you’re trying to deploy models in production > Conclusion.

Lalam: I'm confident in this direction because the results show strong cross-domain transferability, even outperforming some baselines on Llama-three-70B benchmarks > Generalizability. It seems like a really solid path forward for creating more reliable agents > Conclusion.