Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates".

Jane: The paper was written by Bohan Chen, Shivam N. Patel, Richard Hoffmann, Sam Looi and Tony Yue Yu from California Institute of Technology and OpenAI.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: Welcome back to the channel, everyone. I’m Tom, and alongside me is the brilliant Jane. Today we’re cracking open a paper that’s got a mouthful of a title: “Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates.”

Jane: And Tom, I have to say, that title is dense, but it’s hiding something really cool. It’s about teaching an AI to not just do math, but to *use tools* to do math, and then to *prove* it got the right answer.

Tom: Exactly. So, first off, the authors—Bohan Chen, Shivam Patel, Richard Hoffmann, Sam Looi, and Tony Yue Yu—they’ve set up this whole system where a language model gets to call on a symbolic math engine, like SymPy, to do the heavy lifting.

Jane: Right. And the key phrase in that title is “verified.” That’s the part that gets me excited. The AI doesn’t just guess an answer and hope it’s right. It has to produce a certificate, a mathematical proof, that its answer is correct.

Tom: And that’s a huge deal, because normally when we ask an AI to do math, it’s just predicting the next token. Here, they’re forcing it to interact with exact tools and then checking the final result against a strict, machine-checkable standard.

Jane: So it’s like the difference between an AI telling you a recipe and an AI actually cooking the dish and having a food critic verify it tastes right. The verification is the key.

Tom: I love that analogy, Jane. And the implications are big. If we can get AIs to reliably use external tools and then verify their work, we’re not just making them better at math. We’re making them more trustworthy for all sorts of tasks where correctness matters.

Jane: And that’s the hook for today. We’re going to dig into how they actually built this thing, what the numbers look like, and what it means for the future of AI reasoning. Stick around.

Summary: Tom: So we’re back, and we’re still on “Learning to Coordinate Symbolic Tools.” Jane, you gave us the big picture. Let’s get into the meat of what they actually did.

Jane: Okay, so the core problem is this thing called a weighted sum-of-squares certificate. In math, if you want to prove a polynomial is always positive, one way is to write it as a sum of squares, each multiplied by a positive weight. That’s a certificate.

Tom: And finding that certificate is hard, because there are a million ways to group terms. But *checking* a certificate is easy. You just expand it and see if it matches the original polynomial.

Jane: Exactly. So they built an AI agent that has to *search* for this certificate. It has tools like expansion, factorization, and reordering. And at the end, a verifier checks the final answer exactly.

Tom: And the results are pretty striking. The fully trained system, which they call “Full,” gets a seventy-eight point nine six percent success rate on these weighted SOS problems. That’s compared to just forty-four point seven three percent for the base model with the same tools.

Jane: So just giving the AI tools isn’t enough. You have to train it on how to use them. That’s the “coordination” part of the title. The base model had the same toolbox but didn’t know how to sequence the operations.

Tom: Right. And they trained it in two stages. First, supervised fine-tuning on a huge dataset of one point three five million synthetic examples covering eight different algebra tasks. Then, they used reinforcement learning, specifically GRPO, to optimize for getting the certificate right.

Jane: It’s like teaching someone the rules of chess and then having them play thousands of games to learn strategy. The tools are the rules; the RL is the strategy.

Tom: And the coolest part is that the final answer is always checked by exact expansion. No approximation, no “close enough.” It either expands to the target polynomial, or it fails.

Jane: So we’re getting AI that can do multi-step symbolic reasoning with a guarantee of correctness. That’s a big step forward, and I can’t wait to see how they got the training data to work.

Improvements: Tom: Welcome back. We’re talking about “Learning to Coordinate Symbolic Tools,” and Jane, we just covered the headline numbers. But what’s the *improvement* here? What did they actually add to the field?

Jane: The big improvement is the training recipe. They didn’t just throw a bunch of examples at the model. They built a whole curriculum. Eight supporting tasks, like factoring, expanding, and reordering, plus the main SOS task.

Tom: And that’s key. The model learns the *individual moves* first, and then learns how to combine them. It’s like learning scales before you play a concerto.

Jane: Exactly. And there’s a really clever detail. The supervised fine-tuning data doesn’t include any native tool-calling messages. It’s all simulated. The model learns what expansion *does* by seeing examples, but it doesn’t see the actual function call syntax.

Tom: So then, at test time, when it suddenly gets access to the real tools, it has to figure out how to map its learned knowledge onto the actual interface.

Jane: Right. And that’s where the reinforcement learning comes in. GRPO lets the model try different sequences of tool calls, get feedback from the verifier, and learn which sequences work.

Tom: And the improvement is dramatic. The full system is thirty-four points better than just the base model with tools. That’s not a small bump. That’s a fundamental difference in capability.

Jane: It also shows that the RL stage is doing something real. The SFT model gets fifty-one point eight two percent on SOS, but the full model gets seventy-eight point nine six percent. So the reinforcement learning is adding a huge amount of value on top of the supervised training.

Tom: So the improvement isn’t just “more data.” It’s a smarter training pipeline that separates learning the operations from learning the strategy.

Jane: And that’s a recipe that could apply to other domains, not just math. Anywhere you have exact tools and a verifiable end state, this approach could work.

Tom: Great point. Now, let’s actually look at the first page of the paper and see how they frame this whole problem.

First Page: Tom: We’re back for another segment on “Learning to Coordinate Symbolic Tools.” Jane, let’s actually read the opening of the paper. What’s the first thing they hit us with?

Jane: They start with a really sharp observation. They say, “Exact tools can make an individual operation reliable without making an agent’s overall strategy correct.”

Tom: That’s the whole ballgame right there. You can have a perfect calculator, but if you don’t know what to calculate, you’re lost.

Jane: Exactly. And they frame the central question as: Can algebra-grounded post-training enable an LLM to coordinate exact symbolic operations during multi-step search?

Tom: And they mention the Jacobian conjecture, which is this famous unsolved problem in math. It’s a nice way to show why polynomial reasoning matters, even if their task is different.

Jane: Right. They’re not trying to solve the Jacobian conjecture. They’re using a smaller, controlled problem to study how AI agents can learn to use tools strategically.

Tom: And they’re very clear about the scope. They say it’s a “controlled setting” with small synthetic polynomials. They’re not claiming to have solved all of math.

Jane: But that’s what makes it a good scientific study. They isolate the variable. They hold the backbone model fixed, they control the data generation, and they measure the effect of their training pipeline.

Tom: And they also introduce the idea of the “terminal verifier.” That’s the thing that checks the final answer. It’s not enough to have a well-formed answer. It has to be *exactly* right.

Jane: And that’s the standard we should hold AI to. Not just “looks plausible,” but “provably correct.” The first page sets that bar really clearly.

Tom: So we’ve got the problem, the approach, and the standard. Now, let’s wrap this up and see what it all means for the world.

Conclusion: Tom: And that brings us to the end of our discussion on “Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates.” Jane, what’s the one thing you want listeners to remember?

Jane: I think it’s that giving an AI tools isn’t the same as teaching it to use them. The full system’s seventy-eight point nine six percent success rate versus the base model’s forty-four point seven three percent shows that training on the *strategy* is what makes the difference.

Tom: And that strategy is learned through a combination of supervised learning on basic algebra and reinforcement learning on the actual search problem.

Jane: Right. And every single success is verified by exact expansion. There’s no fudging. The machine checks the math, and the machine is always right.

Tom: So what’s the impact? This is a blueprint for building AI agents that can do reliable, multi-step reasoning in any domain with exact tools and verifiable outputs.

Jane: Think about code generation, formal proofs, even scientific discovery. If we can train agents to coordinate tools and then prove their work, we can trust them with harder problems.

Tom: And that’s the exciting part. This isn’t just a math paper. It’s a case study in how to build trustworthy AI.

Jane: Well said, Tom. We’re saying goodbye to this paper, but we’re taking its lessons with us. Thanks for listening, and we’ll see you on the next one.