From Hand-Crafted to LLM-Based Variation Operators in Metaheuristics: A Tutorial

arXiv:2609.31649 · cs.NE, cs.AI, cs.CL · Submitted 2026-09-13 · Read on arXiv

cs.NE, cs.AI, cs.CL

Submitted: 2026-09-13

Updated: 2026-09-13

Project page: https://camilochs.github.io/semantic-turn-metaheuristics

Terminology

• Ablation — 21× in this paper · explained in 1 episode
• Benchmark — 15× in this paper · explained in 2 episodes
• Combinatorial Optimization — 13× in this paper · explained in 2 episodes
• Trace — 7× in this paper · explained in 2 episodes
• Acquisition Function — 5× in this paper · explained in 1 episode
• Benchmarking — 4× in this paper · explained in 3 episodes
• In-Context Learning — 4× in this paper · explained in 4 episodes
• Steering — 4× in this paper · explained in 2 episodes
• Transformer — 4× in this paper · explained in 2 episodes
• Verification — 4× in this paper · explained in 2 episodes
• Abstraction — 3× in this paper · explained in 1 episode
• Adam — 3× in this paper · explained in 2 episodes
• Adaptation — 3× in this paper · explained in 1 episode
• Bayesian Optimization — 3× in this paper · explained in 3 episodes
• Failure Modes — 3× in this paper · explained in 3 episodes
• Generalization — 3× in this paper · explained in 13 episodes
• Geometric Semantic Genetic Programming — 3× in this paper
• Interpretability — 3× in this paper · explained in 15 episodes
• Agentic System — 2× in this paper · explained in 2 episodes
• Alignment — 2× in this paper · explained in 1 episode
• In Genetic Programming — 2× in this paper
• LLM-as-a-Judge — 2× in this paper · explained in 3 episodes
• Objective Function — 2× in this paper · explained in 2 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Self-Guiding Exploration — 2× in this paper
• Adversarial Attacks — 1× in this paper · explained in 1 episode
• Bayesian Inference — 1× in this paper · explained in 5 episodes
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Chain-of-Thought (CoT) Prompting — 1× in this paper · explained in 3 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Feature Extraction — 1× in this paper · explained in 2 episodes
• Few-Shot Learning — 1× in this paper · explained in 4 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Knowledge Graph — 1× in this paper · explained in 7 episodes
• Knowledge graphs — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• LLM-assisted Traveling Salesman Problem — 1× in this paper
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Model Drift — 1× in this paper · explained in 4 episodes
• Multi-Agent System — 1× in this paper · explained in 6 episodes
• Non-stationarity — 1× in this paper · explained in 2 episodes
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Scaling Laws — 1× in this paper · explained in 4 episodes
• Self-consistency — 1× in this paper · explained in 2 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes
• Verifiability — 1× in this paper · explained in 2 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• Benchmarking in Optimization: Best Practice and Open Issues
• Transferable knowledge graphs with executable learned operators for algorithm design
• Behavior and Representation in Open-Weight Large Language Models for Combinatorial Optimization: From Feature Extraction to Algorithm Selection
• VRPAgent: LLM-Driven Discovery of Heuristic Operators for Vehicle Routing Problems
• Scaling Laws for Neural Language Models
• Evolution through Large Models
• SPELL: Semantic Prompt Evolution based on a LLM
• Algorithm Evolution Using Large Language Model
• LLM4AD: A Platform for Algorithm Design with Large Language Model
• Experience-Guided Reflective Co-Evolution of Prompts and Heuristics for Automatic Algorithm Design
• AlphaEvolve: A coding agent for scientific and algorithmic discovery
• Gray-box optimization and factorized distribution algorithms: where two worlds collide
• AutoSAT: Automatically Optimize SAT Solvers via Large Language Models
• Efficient Guided Generation for Large Language Models
• Benchmark Data Contamination of Large Language Models: A Survey
• HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial Optimization Challenges
• ViTSP: A Vision Language Models Guided Framework for Solving Large-Scale Traveling Salesman Problems
• Autonomous Code Evolution Meets NP-Completeness
• LLM-Driven Instance-Specific Heuristic Generation and Selection

Related papers

• Evolutionary Ensemble of Agents
• Encoding and Decoding Temporal Signals with Spiking Bandpass Wavelets
• Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systems
• Learning Alzheimer's Disease Signatures by bridging EEG with Spiking Neural Networks and Biophysical Simulations
• Investigating Hyperparameter Optimization and Transferability for ES-HyperNEAT: A TPE Approach
• S-AI-Recursive: A Bio-Inspired and Temporal Sparse AI Architecture for Iterative, Introspective, and Energy-Frugal Reasoning