Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation
cs.LG, cs.CL
Submitted: 2025-10-13
Updated: 2026-09-21
Comments: ACL 2026 Main Conference
Code: https://github.com/tatsu-lab/stanford_alpaca
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
- TreeReview: A Dynamic Tree of Questions Framework for Deep and Efficient LLM-based Scientific Peer Review
- Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
- Training Verifiers to Solve Math Word Problems
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Multilingual Large Language Models and Curse of Multilinguality
- Distilling the Knowledge in a Neural Network
- Mistral 7B
- PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuning
- Evaluating Language Models as Synthetic Data Generators
- Multiple Choice Learning of Low-Rank Adapters for Language Modeling
- From System 1 to System 2: A Survey of Reasoning Large Language Models
- Task Oriented In-Domain Data Augmentation
- SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement Learning for LLM Reasoning
- Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability
- Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR
- Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
- DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
- Code Llama: Open Foundation Models for Code
- Fostering Natural Conversation in Large Language Models with NICO: a Natural Interactive COnversation dataset
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks