AutoOR: Scalably Post-training LLMs to Autoformalize Operations Research Problems
Sumeet Ramesh Motwani, Chuan Du, Aleksander Petrov, Christopher Davis, Philip Torr, Antonio Papania-Davis, Weishi Yan
cs.LG, cs.AI
Submitted: 2026-08-20
Updated: 2026-08-24
Code: https://github.com/huggingface/trl
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Optimization problems are central to decision-making in manufacturing, logistics, scheduling, and other industrial settings.
Terminology
Abstract
Optimization problems are central to decision-making in manufacturing, logistics, scheduling, and other industrial settings. Translating complicated descriptions of these problems into solver-ready formulations requires specialized operations research (OR) expertise, making it hard to scale. We present AutoOR, a scalable synthetic data generation and reinforcement learning pipeline that trains LLMs to autoformulate optimization problems specified in natural language across linear, mixed-integer, and non-linear categories. AutoOR generates verified training data from standard optimization forms and uses solver execution feedback as the reward signal for RL post-training. AutoOR applied to an 8B model achieves state-of-the-art or competitive results across six established OR benchmarks, matching significantly larger frontier models. For a non-linear problem class involving physical dynamics, where frontier models score near 0%, we introduce a curriculum RL strategy that bootstraps from limited initial training data to make this class tractable for post-training. We believe that methods such as AutoOR can significantly accelerate industrial decision-making with AI.
Sources
- Autoformulation of Mathematical Optimization Models Using LLMs
- The SCIP Optimization Suite 9.0
- OptiMUS: Optimization Modeling Using MIP Solvers and large language models
- OR-LLM-Agent: Automating Modeling and Solving of Operations Research Optimization Problems with Reasoning LLM
- LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch
- MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs
- Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
- Qwen3 Technical Report
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learning
- POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
- LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languages
- OR-R1: Automating Modeling and Solving of Operations Research Optimization Problem via Test-Time Reinforcement Learning
- A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions
- The Llama 3 Herd of Models
- Step-Opt: Boosting Optimization Modeling in LLMs through Iterative Data Synthesis and Structured Validation
- OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Proximal Policy Optimization Algorithms
- Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks