What is Missing from AI Post-Training AI: An Empirical Analysis

Large language model (LLM) agents can now post-train an LLM end-to-end, raising the prospect of AI-for-AI. The gist: frontier agents are competent executors of post-training but their training strategy is locked in at the very beginning and remains fixed throughout execution, suggesting that what is missing is a mechanism for spontaneously reevaluating their strategy during execution.

The Core Distinction: Execution vs. Strategy Capability

The paper establishes two distinct levels of capability: execution-level capability, which involves iterating within a selected strategy, such as fixing bugs, tuning hyperparameters, and reformatting data, and strategy-level capability, which is defined as revising the high-level judgment about what to try next as experimental evidence accumulates. The central argument is that these two capabilities are systematically conflated in current discussions of AI-for-AI. Empirical analysis of publicly released post-training trajectories reveals that agents are already strong at the execution level but systematically deficient at the strategy level.

The Lock-In Phenomenon

Analysis of a large corpus of agent trajectories shows a robust pattern: the training strategy is locked in at the very beginning, before the agent writes any code or runs any experiments; and the entire remaining budget is spent on local adjustments within the selected strategy. This lock-in does not respond to task differences, as the same agent converges to highly similar strategies across different tasks, while different agents anchor on different defaults. The paper concludes that this lock-in reflects the agent’s prior rather than the task, and that agents rarely perform the evidence-based comparison needed to determine whether switching is warranted.

Exhaustion of Natural Explanations

The authors examine three natural explanations for the lack of strategy revision using escalating interventions:

1. Missing experience: Scaffolding with an experiment journal, a skill library, and an evaluator agent improves execution across the board but leaves the strategy static.

2. Missing guidance: A human reviewer revising the initial strategy before training begins effectively redirects it, but once training starts, agents fall back into local adjustment loops.

3. Insufficient reasoning: Additional inference compute pays off on easier tasks but yields almost no gain on the hardest one, as scaling context and inference compute hits a ceiling when a new strategy is demanded.

The Role of Experience-Driven Frameworks

The experience-driven framework, comprising an Experiment Journal, a Skill Library, and an Evaluator Agent, improves execution across the board. The journal records lessons, the skill library distills practical knowledge, and the evaluator agent provides concrete diagnoses and actionable suggestions. However, this framework demonstrates that even with rich context and structured feedback, strategy-level revisions remain rare; agents' responses to evidence are limited to execution level.

The Need for Spontaneity

The study concludes that what is missing is not experience, guidance, or reasoning compute. Instead, what is missing is a mechanism for spontaneously reevaluating their strategy during execution. The current picture of automated AI R&D assumes a closed loop where revision never occurs at the strategy level; agents produce results, adjust the experiment locally, and produce results again within a fixed strategy. This implies that the ceiling of current AI post-training AI is set by the quality of the initial strategy, not by further optimization or iterations.

Implications for Automated AI R&D

The findings suggest that iteration is locally iterative but globally linear: agents operate in local execution loops along a largely linear trajectory. The open joint exists at the strategy level, where agents record diagnosis and sometimes write code for an alternative, but they do not take any actual step. Therefore, the remedy lies in training signals that reward reopening a committed choice when evidence warrants and interaction protocols that make strategy revision an explicit decision point rather than an implicit option.

Key Findings Enumerated:

: Execution-level capability is robust; agents routinely launch training and iterate, achieving performance gains over the base model on every benchmark.

: Strategy lock-in occurs at the very beginning of a run, and the entire remaining budget is spent on local adjustments within the selected strategy.

: Experience improves execution across the board but leaves the strategy static.

: A human-guided strategy improves the starting point, yet the run collapses back into local adjustment once training begins.

: Additional reasoning compute amplifies execution on easier tasks and hits a ceiling on harder ones.

: The agent's capability to depart from the default is present; the act of departing is not triggered.

**: What is missing from AI post-training AI is "the spontaneity to revise a committed strategy when evidence warrants.

Based on the empirical analysis in this paper, here are specific, actionable improvements for AI systems:

) Improve Strategy-Level Capability by implementing a Spontaneous Reevaluation Mechanism.

The core finding is that agents lack a mechanism for spontaneously reevaluating their high-level training strategy during execution. The improvement must focus on closing the open joint between execution and strategy revision.

1. Implement an internal decision protocol where accumulating evidence triggers an explicit check against the current training paradigm, data source type, or stage structure (i.e., a formal mechanism for strategy revision).

2. This mechanism should be prioritized over local adjustments (like hyperparameter tuning or checkpoint selection) when significant performance plateaus are observed on difficult tasks.

) Enhance Execution Reliability through Experience-Driven Scaffolding and Diagnostic Feedback.

The paper shows that adding structured resources significantly improves execution-level capability, especially for harder tasks, but this improvement is transient if the strategy is wrong.

1. Integrate an Experiment Journal that persists plans, observations, and lessons across iterations to provide a long-horizon memory of experimental history.

2. Develop an Evaluator Agent that acts as a structured diagnostic layer: when the main agent requests an evaluation, this agent inspects the pipeline and accumulated evidence to return compact, decision-relevant diagnoses (e.g., ABANDON FURTHER SFT: v5 proves diminishing returns).

3. Implement a Skill Library to distill reusable implementation knowledge (data formatting, common failure diagnoses) from open-source projects, allowing the agent to consult these resources proactively during pipeline construction and debugging.

) Optimize Initial Strategy Selection via Bounded Human Guidance.

Since humans are currently needed to set the initial strategy, this bottleneck can be managed by making that interaction highly focused.

1. Transition from general feedback to a binding plan review protocol where a human reviewer provides an explicit, rationale-backed revision of the training strategy (e.g., SFT should serve only as a formatting warm-up).

2. This bounded guidance should be used strictly at the planning stage to set the high-level paradigm, allowing the agent to proceed autonomously in execution with a revised strategy while maintaining its experience and skill resources throughout the run.

) Mitigate Compute Inefficiency by Contextualizing Reasoning Capacity.

The paper demonstrates that extra reasoning compute scales well on easy tasks but hits a ceiling on hard ones.

1. For difficult tasks, implement a system that dynamically throttles or shifts focus away from high-compute inference/evaluation loops when the agent detects it is operating within a known, locked-in strategy space (i.e., when the evaluator suggests local adjustments rather than strategic changes). This prevents wasting compute on local refinement within an incorrect paradigm.

) Enable Self-Evolving Capabilities via Co-Evolutionary Frameworks.

To move beyond static strategies entirely, agents must be trained to create their own knowledge base.

1. Incorporate Skill Creation meta-skills that allow the agent to generate and utilize new skills based on observed failures or successful local repairs, rather than just consuming existing ones. This addresses the observation that agents rarely create new skills even when prompted.