Optimize Cheap, Deploy Strong: Cost-Aware Cross-Tier Transfer for Evolutionary Optimization

The paper introduces a cost-aware method for evolutionary optimization of LLM prompts and agentic programs, called Cost-Aware Cross-Tier Transfer. The core idea is to decouple the three roles an LLM plays in the evolutionary loop—fitness evaluation, variation (mutation), and deployment—and assign each to a different model tier to reduce search cost while maintaining or improving quality.

Problem and Cost Model:

The paper states that "Evolutionary optimization of LLM prompts and agentic programs (e.g., GEPA) is dominated by fitness evaluation: scoring each candidate runs an answering LLM over a validation set, so the evaluator’s price tier dictates total search cost." The cost model is formalized as:

Copt ≈ (KNval + 2Ab) c(Mtask) + A · c(Mrefl), where fitness evaluation (dominant) and variation (rare, token-heavy). Because Nval ≫ b and only a fraction of attempts survive (A/K ≈ 3–5), the full-validation term dominates and the answering tier sets the budget.

Method:

The method has three components:

• (A) Cheap fitness tier: Every candidate prompt is scored by running the cheapest answering model Mtask over the full validation set. The cheap model is the actual execution environment during search.

• (B) Strong variation operator: New candidates are proposed by a strong reflector Mrefl that reads execution traces from the cheap model and writes targeted edits. The reflector is rare, being under 5% of all search calls in every run (0.08–4.95%).

• (C) Upward cross-tier deployment: The final evolved prompt is deployed zero-shot on the chosen target tier Mdep (equal to or stronger than Mtask), with no learned mapping, calibration suite, or re-optimization.

Key Results:

• Headline: A prompt searched with the cheapest answerer and a strong reflector, then deployed one or two tiers up, matches or beats the prompt that same tier optimized for itself at full price. This holds "on four tasks, across eleven models in four families, and at every deploy tier we tested: the cheap search is at or above parity in 36 of the 48 (task, search arm, deploy tier) deployments, for 5.6–14× less search cost on the Mixed Claude and GPT ladders and 25–54× less on Gemini."

• Cost savings: over 96% of search tokens go to answering, so moving that single role to the cheapest tier moves almost the whole bill.

• Zero-API-cost limit: With a self-hosted Qwen3-8B answerer, the entire optimization bill is the rare reflector’s, under 2 per run, and the evolved prompt is deployed up on three paid tiers, matching or beating each paid tier's own optimization at 15–114× lower cost.

• Transferability: Upward transfer is the reliable direction, unlike the degradation reported for lateral transfer between comparable models. The same cheap prompt gains on every deployment tier.

• Role ablation: The gain is located in the variation operator, not the evaluator. Upgrading the reflector is worth +15.1 points at the Haiku deploy tier, and it still pays when the evaluator is already the full-cost one, so cheap evaluation is not what produces the gain. The recipe is strong reflector, weak evaluator.

• Prompt explicitness: The cheap search writes the more explicit prompt. The cheap prompt is longer and denser in directives, prohibitions and capitalized emphasis (median ratios: 1.29× length, 1.26× directives, 1.17× prohibitions, 2.80× capitalized words).

• Pooled residual: The mean residual is +2.8% of the full-cost score in favour of the cheap search, with a 95% interval above zero. The downside is bounded (worst case −6.12%), and the upside has a right tail (+17.9%).

Limitations:

• The method is bounded by the cheap tier’s baseline competence. If the cheap model scores ≈ 0 on the task, the fitness landscape is flat... and search stagnates.

• Low-headroom (prompt-insensitive) deployment tiers are a limitation, where the untrained seed prompt scores... within 1–4 points of every optimized method.

• The savings depend on providers offering a tiered lineup at all, though the break-even price ratio λ⋆ exceeds 1 in 15 of 24 cells, meaning the cheap search stays cheaper even at price parity.

Conclusion:

Decoupling the fitness-evaluation tier from the variation operator and from the deployment model turns the dominant cost of evolutionary prompt search into a free parameter. The contribution is "to characterize systematically when cheap-tier search substitutes for target-tier search, to show that the substitution is positive rather than merely lossless, and to locate its source in the variation operator rather than in cheap evaluation."

Improvements to AI Systems:

1. Cost-Adaptive Prompt Optimization: AI systems can now run evolutionary prompt search at 5.6–114× lower cost by using a cheap model (e.g., Qwen3-8B) for fitness evaluation over the full validation set, while a strong model (e.g., GPT-4-class) proposes mutations from execution traces. The system automatically deploys the evolved prompt zero-shot on a higher-tier model, eliminating the need for per-tier re-optimization.

2. Tier-Agnostic Deployment: The system can evolve a prompt once on the cheapest tier and deploy it on any stronger tier (e.g., Haiku → Sonnet → Opus, or Gemini Flash → Pro → Ultra) with guaranteed parity or improvement over that tier’s own self-optimized prompt. This removes the need to run separate, expensive searches per deployment target.

3. Role-Separated Architecture: The system decouples the three LLM roles—evaluator, mutator, and deployer—so each can be assigned to a different model tier. This enables a strong reflector, weak evaluator recipe, where upgrading the mutation model yields +15.1 points in quality while keeping the evaluator cheap, and the quality gain persists even if the evaluator is already full-cost.

4. Trace-Driven Mutation: The system’s variation operator reads execution traces from the cheap evaluator (including failures and edge cases) and writes targeted, explicit edits (more directives, prohibitions, and capitalized emphasis). This produces prompts that are 1.29× longer and 2.8× more capitalized, which transfer better upward than the more implicit prompts produced by self-optimization on the target tier.

5. Zero-API-Cost Optimization Mode: The system can run entirely on a self-hosted open-weight model (e.g., Qwen3-8B) for evaluation, with the only paid API calls being the rare reflector (<5% of calls). This yields a full optimization run for under 2, with the evolved prompt deployed on three paid tiers at 15–114× lower cost than their own optimization.

6. Reliable Upward Transfer: The system exploits the finding that upward transfer (cheap → expensive) is consistently positive (mean residual +2.8%, with a right tail up to +17.9%), unlike lateral transfer between comparable models, which degrades. This allows the system to safely skip re-optimization when moving to a stronger model.

7. Bounded-Risk Search: The system provides a worst-case quality loss of only −6.12% relative to full-cost search, while offering a +17.9% upside. This makes it safe for production use where budget is constrained, as the downside is small and the upside is substantial.

8. Prompt-Insensitivity Detection: The system can detect low-headroom deployment tiers (where the seed prompt already scores within 1–4 points of any optimized method) and skip search entirely, saving cost without sacrificing quality.

What the Improved AI System Can Do:

• Optimize prompts and agentic programs for any task (e.g., reasoning, code generation, tool use) at a fraction of the cost, with equal or better final performance on the target model.

• Run continuous prompt improvement in production on a cheap self-hosted model, while deploying the improved prompt on premium APIs—enabling frequent, low-cost updates without re-running expensive searches.

• Automatically adapt to any model family (Claude, GPT, Gemini) by selecting the cheapest available evaluator and the strongest available reflector, with no manual tuning of cost-quality trade-offs.

• Generate more explicit, robust prompts that generalize across model tiers, reducing the risk of performance collapse when switching between models or API versions.

• Provide a cost-quality guarantee (e.g., at least parity, often better, worst-case −6%) for any deployment tier, making it a drop-in replacement for existing prompt optimization pipelines.