MAS-OPD: On-Policy Distillation for Multi-agent Systems
cs.CL
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/huggingface/Math-Verify
Terminology
Sources
- Small Language Models are the Future of Agentic AI
- SODA: Semi On-Policy Black-Box Distillation for Large Language Models
- Improving Factuality and Reasoning in Language Models through Multiagent Debate
- AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
- The False Promise of Imitating Proprietary LLMs
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Large Language Model based Multi-Agents: A Survey of Progress and Challenges
- Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
- Measuring Coding Challenge Competence With APPS
- Reinforcement Learning via Self-Distillation
- Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
- CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- MARFT: Multi-Agent Reinforcement Fine-Tuning
- Generative Agents: Interactive Simulacra of Human Behavior
- Privileged Information Distillation for Language Models
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Self-Distillation Enables Continual Learning
- A Survey of On-Policy Distillation for Large Language Models
- Qwen3 Technical Report
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering