2608.09343-LLM-Guided Heuristic Design from Simulation Traces: A Case Study in Dynamic Production and AGV Scheduling

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "LLM-Guided Heuristic Design from Simulation Traces: A Case Study in Dynamic Production and AGV Scheduling".

Jane: The paper was written by Jinbo Li and Chuanhao Li from Department of Industrial Engineering, Tsinghua University.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper Summary: Tom: We just spent a good while reading a paper from Tsinghua's industrial engineering department, and I keep coming back to one image: an LLM agent that doesn't just read the final score from a factory simulation — it reads the event log. That's the whole premise, and it sounds simple, but almost nobody in simulation-based optimization does it. Simulation traces guide heuristic design instead of just aggregate numbers.

Jane: The loop itself is clean. Evaluate a candidate policy over several replications, take the worst-scoring one, replay it, and turn the recorded events into a queryable database. A manager agent studies that evidence and formulates bottleneck hypotheses, and several editing agents implement code-level revisions in parallel. Then repeated simulation decides what survives, and best-so-far selection keeps only improvements.

Lu: Exactly. And the case study is tough: a three-line factory with AGVs, quality inspection, rework, charging constraints, all interacting. Every candidate gets ten scoring replications plus one diagnostic replay. Starting from a rule-based policy at 62 point 49, the best run reached 78 point 61, and across five runs with the strongest model the average final score was 77 point 51.

Meng: The baselines really put that gap in perspective. A rolling MILP, 135 hand-built rule combinations, and GA, DE, and PSO all landed in the low sixties. The LLM framework sits roughly fifteen points higher, which is a lot on a zero-to-hundred scale. On 100 matched seeds, the optimized policy outscored every baseline on every seed — in the default setting and again when random faults were switched on without any re-optimization.

Jane: Wait, on every single seed?

Meng: Yes, every one of them. The paired Wilcoxon tests come out significant with Holm correction, and the paper reports zero losses across all comparisons.

Lalam: The bigger picture for me is that this opens a different feedback channel for eye-designed heuristics. The trace tells you why a policy failed — forced charging, empty travel, upstream starvation — not just that it failed. And that logic carries beyond scheduling to any discrete-event system you can query: warehouses, ports, hospital logistics.

Tom: And they don't just assert the value of the trace. They ablate the trace database and the parallel candidate generation, and both removals cost performance. The direction of the loss is consistent across two different models.

Jane: What I found striking is that the cheaper model lost more from losing the traces. That suggests concrete event-level evidence matters most exactly when the reasoning engine is weakest.

Tom: That's a nice tension to carry into the opening pages, where the paper explains why black-box scores aren't enough.

Paper Page 1: Tom: We've got the headline results in mind, so the opening page is all about the problem statement. The abstract already frames everything in one line: repeated simulation for selection, event-level traces for diagnosis.

Jane: The introduction starts with a critique of standard simulation-based optimization. A search algorithm submits a candidate, receives an estimated score, and the dynamics stay inside the simulator. Aggregate KPIs can rank alternatives, but they don't reveal which condition, priority, or threshold in an executable policy should change. That last sentence is the paper's whole motivation, and it sets up everything that follows.

Lu: Right, and then it positions itself against the LLM heuristic-design lineage — Evolution of Heuristics, FunSearch, ReEvo. Those represent candidates as executable code and use automated evaluation, and ReEvo even conditions generation on textual reflections. But none of them feeds event-level operational traces from the simulator back into candidate generation.

Meng: So the gap is precise: mean scores support candidate selection, while traces provide diagnostic evidence for targeted generation. And the paper points out that simulators can record timestamped state transitions, queue states, delays, command outcomes — everything you'd want in order to understand a failure.

Jane: They also draw a boundary around LLM use. The LLM revises between evaluation batches, and a fixed policy controls each simulation run.

Lalam: That boundary matters a lot. The search borrows the language model's creativity, but the evaluation stays rigorous and reproducible. Later they defend that choice against the alternative of letting an LLM make decisions inside a running simulation — within-run control would put inference latency and random output variation into the runtime, and the policies couldn't be versioned or audited the same way.

Tom: And the introduction previews the case study's complexity — re-entrant processing, quality-driven rework, finite buffers, charging constraints, optional faults. That's exactly the kind of interacting dynamics where a single scalar score hides everything interesting.

Lu: So the stage is set. The next pages have to show the field hasn't already built this, and then the methodology has to formalize it.

Paper Page 3: Tom: We're at the related work now, and the paper opens with production and AGV scheduling. It's a classic coupling: machine operations create transport requests, and AGV delivery times constrain the start of downstream operations, so the two decisions have to be made together.

Jane: The lineage runs from Bilge and Ulusoy's time-window formulations, through genetic and hybrid metaheuristics, to recent work combining MILP with dual-population search. In dynamic settings, researchers have handled random arrivals and machine breakdowns, real-time multi-agent negotiation, and online dispatch under battery constraints.

Lu: The case here keeps that dependency, but the policy only controls line selection, task ranking, AGV dispatch, and charging. Product routes and workstation sequencing stay under simulator control. That scoping is what makes the policy space small enough for code-level revision.

Meng: And the paper also nods to the LLM side of simulation — work that generates or adapts discrete-event models from textual specifications. So they're positioning themselves at a specific spot: assume an existing, instrumented simulator, and focus on revising executable policy code between evaluation batches.

Lalam: Then the section turns to simulation-based optimization proper, and the critique sharpens. In the standard black-box view, the simulator maps decisions to objective and constraint estimates. That supports ranking, but it leaves the entire trajectory outside candidate generation.

Tom: They're fair about mathematical programming's strengths, though. It makes structure explicit and lets solvers exploit relationships among decisions, constraints, and objectives. The point is that black-box SBO loses the operational mechanism that explains a score.

Jane: And notice how the baselines get planted here. The rolling MILP will model a tractable subset of the system. The rule-based baselines combine three decision layers. The metaheuristics search over weighted rule combinations. By the end of this page, you know exactly where existing methods stop — at the boundary between a score and the events behind it.

Lalam: One more thing, actually. The related-work discussion makes clear that simulators can already record process-level traces, so this isn't asking for new simulation capabilities. It's asking optimization methods to use what simulators already produce.

Tom: And that's a fair point — the traces are already there, sitting in databases, and most search methods just ignore them.

Lu: So the gap is well-marked. The methodology section now has to deliver the formal machinery that crosses it.

Paper Page 5: Tom: Now we're in the methodology, and the formalism is compact. The objective is the expected score over stochastic inputs, estimated by a sample mean over R replications. The evaluation function returns a pair, not a single number.

Jane: Equation three is the core: Evaluate of policy pi returns the mean score for selection, plus a queryable trace for revision. The trace may be selected from one replication, aggregated across replications, or produced by an additional replay — the framework leaves that choice open. No decomposition of the score is required, and the simulator stays whatever it is, as long as it exposes its events alongside the number.

Lu: And equation four is the revision step. The manager examines the incumbent, its stored mean score, and its trace, then produces k_t executable candidates. Each

Page 4 of the paper: Tom: Quick recap: this is the paper that lets LLM agents read a factory simulator’s event log, not just its final score, to diagnose bottlenecks and rewrite scheduling policies.

Jane: And page 7 is where the rubber hits the road — it spells out exactly what that simulator looks like and what the policy is allowed to touch. The factory has three production lines, each with two AGVs, conveying systems, buffers, a quality check station, and shared warehouses for raw material and finished goods.

Tom: The important part is the boundary. The policy controls which line gets each product, how transport tasks are ranked, how AGVs are dispatched, and when charging happens. But product routes and the internal sequencing at each workstation stay inside the simulator. That split keeps the search space manageable while still letting the LLM change the decisions that actually matter.

Lu: And the score isn’t one vague number. It’s eight separate performance metrics — on-time completion, equipment utilization, quality pass rate, cost ratio, AGV energy efficiency, that kind of thing — each normalized to a zero-to-hundred scale and combined with fixed weights. The metric groups sum to 40 percent production efficiency, 30 percent quality and cost, and 30 percent AGV efficiency.

Meng: That grouping matters because the agents don’t just see a total. They get summaries by group, which tells them whether a bad score comes from production throughput, quality issues, or dumb AGV movement.

Jane: And all the raw events — order arrivals, station states, conveyor blocking, AGV charging, faults, KPI snapshots — land in a queryable database. The manager can ask targeted questions about specific mechanisms instead of reading a wall of log entries.

Lalam: So the page gives you the full feedback interface: a structured score, coarse group summaries, and a fine-grained event database pulling together. That’s the machinery the whole diagnosis loop runs on.

Tom: The one thing that page doesn’t dig into yet is how the simulator picks which replay becomes that diagnostic trace. That’s the piece that determines whether the agent studies a typical run or a disaster. Let’s look at that next.

Page 5 of the paper: Tom: Quick recap: we’ve seen how the framework uses simulation traces, not just final scores, to guide LLM-based policy revisions, and page nine is where that machinery gets tested.

Jane: Right, this page opens the experiments section, and the first thing it does is pose three research questions: can the framework beat the baselines, can the agents actually use traces to find bottlenecks, and do the optimized policies hold up under changed conditions.

Tom: Then it lays out the default testbed. A 500-minute simulation horizon, orders arriving every ten minutes, three production lines, shared warehouses, quality checks, rework, charging, the works. And the initial policy is a simple rule-based one with three layers: line selection, task ranking, AGV assignment.

Lu: The baselines are carefully chosen, though. There’s a rolling MILP for AGV transport, a search over 135 hand-built rule combinations, and then GA, DE, and PSO searching over weighted rule combinations. Each one represents a different optimization paradigm.

Meng: The critical detail is that none of those baselines can control charging. The rule combinations don’t include it, and the MILP abstracts battery constraints away. So the LLM framework gets a broader decision space, and the paper is upfront that part of its advantage comes from that added flexibility.

Jane: And the evaluation protocol is just as important. Each candidate gets ten scoring replications plus one diagnostic replay, and the replay score is excluded from the mean. Candidates are evaluated on independent seed sets, so promotion compares fresh estimates rather than reusing the same randomness.

Lalam: That independence is a subtle but big deal. It means the search isn’t overfitting to one set of random realizations. The paper later checks the final policy on a separate set of matched seeds for an honest comparison.

Tom: So page nine sets up a fair fight: same simulator, same horizon, similar evaluation effort, but the LLM agents get trace evidence and a wider action space. The question is whether that actually translates into higher scores. Let’s look at the overall results next.

Page 6 of the paper: Quick: we’ve seen the LLM-based policies beat every baseline on the standard testbed, and page 11 now asks whether that edge survives when the factory randomly breaks.

Jane: And it breaks a lot — each production line gets its own fault generator, and faults can hit workstations, conveyors, or the AGVs themselves.

Tom: The timing is harsh too. Inter-fault times are drawn uniformly between 80 and 120 minutes, so in a 500-minute run you’re looking at roughly five failures per line. Each one takes between 20 and 60 minutes to recover.

Jane: That’s a serious stress test, but here’s the thing: the optimized policy is not re-optimized. It’s frozen from the default setting and just runs through the faults as-is.

Tom: And it still outscores every baseline on every one of the 100 matched seeds. The median gap is around fourteen points, almost exactly what we saw without faults.

Jane: That struck me as the more important result than the raw win. A policy that handles faults this well without any retraining suggests it isn’t just memorizing the normal operating pattern.

Tom: Right. The trace-guided changes — proactive charging, balanced dispatch priorities, distance-aware assignment — those seem to generalize to a factory that’s constantly failing. They’re not brittle workarounds.

Jane: The significance stars are all four, so the statistical case is airtight. But I’m more interested in why it works. My guess is that the faults basically act like extra stochastic noise, and the policy already learned to handle variability.

Tom: That’s a fair reading. The baseline rules were tuned for the smooth case, so the noise exposes their fragility. The LLM policy was diagnosed on worst-case replications, so it’s already seen failure modes.

Jane: So the fixed-policy fault test is one form of robustness. The next question is whether the framework can adapt when the whole settings change — a longer horizon, or order arrivals that are no longer regular. That’s the page we’re heading to.

Page 7 of the paper: Tom: We've seen the optimized policy hold up when faults hit without retraining, and page 13 now asks whether the whole optimization loop can adapt when the environment changes underneath it.

Jane: That's the re-optimization test. They run completely fresh optimization runs under two new settings: a six times longer simulation horizon, and order arrivals that randomly vary between five and fifteen minutes instead of coming every ten minutes.

Tom: The longer horizon is brutal for the MILP baseline. It drops to about 50, because its simplified model doesn't scale well over time. The best hand-built heuristic gets to 57, the metaheuristics sit around 63, and the proposed framework averages 74 point 16.

Jane: With variable arrivals, the baselines all cluster around 62 to 63, while the LLM framework averages 76 point 34. The matched-seed tests show the gap is significant everywhere, so the framework isn't just winning on one lucky run.

Tom: What I find interesting is that these are fresh optimizations, not transfers. The agents re-diagnose and re-tune under the new conditions. That's a different story from the fixed-policy fault test.

Jane: Right, that's the second part of RQ3: adaptation rather than just robustness. Then the page pivots to the ablation study, and the first ablation is on parallel candidate generation.

Tom: They restrict the manager to propose only one revision direction per iteration. The best outcome is still close — 78 point 56 versus 78 point 61 for Gemini — but the average falls to 73 point 65, and the minimum across runs crashes to 62 point 36.

Jane: That minimum is almost back to the starting policy. With a single candidate, one bad guess means that iteration produces zero improvement, and the search gives up after three such stalls.

Tom: So parallel candidates act as insurance against dead ends. They keep the search alive long enough to find the good revisions.

Jane: And the iteration count tells the same story: 9 point 2 average iterations drops to 5 point 8. The runs terminate early because they starve.

Tom: So we know parallel variety stabilizes the search. But that's not the core claim of this paper — the core claim is about traces themselves. The next ablation removes the trace database entirely, and that's the test we should look at next.

Page 8 of the paper: Tom: Quick recap: we’ve seen the framework win on the default setting, survive random faults, adapt to new conditions, and we’ve just learned that parallel candidates keep the search from starving.

Jane: Page 15 pulls back from the numbers and explains why this all works. The core argument is that a mean score tells you that you lost, but a trace tells you why — forced charging, empty travel, upstream starvation, downstream blocking. Those are the actual mechanisms.

Tom: And the paper frames this in a simple way: the LLM’s variation operator works on executable code, not on a fixed vector. That’s what lets it add proactive charging or restructure dispatch priorities. A genetic algorithm can only reweight the rules you already gave it.

Jane: Right, but that freedom brings a cost. Since the LLM can write arbitrary logic, you need execution checks and repair attempts. That’s why the framework runs each candidate through validity checks before spending simulation budget on it.

Tom: Then there’s the design choice they defend well: LLM revision between batches, not inside a running simulation. Within-run control would put inference latency and random output variation into the runtime, and you couldn’t version or audit the policy afterward.

Lu: Exactly. A fixed policy per run is reproducible. You can keep it, test it, and compare it. That’s what makes the whole search loop honest.

Jane: The page also sets boundaries. The framework needs an executable policy, a simulator that can run it repeatedly, process-level traces, and automatic validation. Those hold for manufacturing, warehouses, ports, hospital logistics — but the evidence here is only from the AGV case.

Tom: And then the limitations. The paper is upfront that the search doesn’t build structured memory of what worked and what didn’t. It keeps a single incumbent instead of a population of promising branches. And the diagnostic trace comes from the worst-scoring replication, which is useful for spotting failure modes but may not represent typical operation.

Jane: That last point is a nice, honest caveat. The framework deliberately studies its worst days, not its average day. That’s a feature for diagnosis, but it might bias the policy toward rare events.

Tom: So the discussion gives you the why, the boundaries, and the known gaps. The next page wraps it all into a conclusion and points out where this line of work goes next — better memory, multiple branches, and other industrial settings.

Conclusion: Tom: Wrapping up a dense read: this paper shows that giving LLM agents the event-level traces from a simulator, not just the final score, lets them diagnose bottlenecks and rewrite scheduling policies in ways that beat every conventional baseline.

Jane: And the numbers back that up pretty hard. The best run went from 62 point 49 to 78 point 61, and the final policy beat the MILP, the best heuristic, and all three metaheuristics on every single matched seed.

Tom: The fault test was the part that really sold me. No re-optimization, just a frozen policy thrown into a factory that breaks constantly, and it still wins by fourteen points.

Jane: It tells you the policy learned something general about keeping flow going, not just a trick for the training distribution.

Tom: And when they did re-optimize for a longer horizon or variable arrivals, the framework adapted again. The ablations showed parallel candidates keep the search from stalling, and dropping the trace database hurt the lighter model more — so traces matter most when reasoning is weakest.

Jane: The paper is honest about limits too: no structured memory of past revisions, a single incumbent instead of a population, and the diagnostic trace comes from the worst replication, not a typical one.

Tom: That last caveat is worth remembering. This framework studies its worst days on purpose, and that bias is probably why it generalizes.

Jane: So where does this leave the field? The big open direction is turning the search's history into reusable experience, keeping multiple policy branches alive, and testing in warehouses, ports, and hospital logistics.

Tom: And the deeper implication is that simulation traces are a general feedback channel, not a scheduling-specific trick. Any discrete-event system that logs events could plug into this same loop.

Jane: We'll keep an eye out for follow-ups that push that further. For now, that's the paper — LLM agents reading event logs to design better factory rules.

Tom: Next up on the show, we've got a paper on using language models to generate simulation models themselves from text specs, which pairs nicely with this one. See you then.