Instella-MoE Technical Report
cs.CL, cs.AI
Submitted: 2026-09-01
Updated: 2026-09-01
Code: https://github.com/AMD-AGI/Primus
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- DeepSeek-V3 Technical Report
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- FarSkip-Collective: Unhobbling Blocking Communication in Mixture of Experts Models
- The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains
- Better & Faster Large Language Models via Multi-token Prediction
- Skill-Targeted Adaptive Training
- Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling
- Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
- Instella: Fully Open Language Models with Stellar Performance
- Muon is Scalable for LLM Training
- Understanding R1-Zero-Like Training: A Critical Perspective
- Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
- MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
- Kimi K3: Open Frontier Intelligence
- NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model
- 2 OLMo 2 Furious
- Olmo 3
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering