Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction

arXiv:2609.25176 · eess.AS, cs.AI, cs.CL, cs.SD · Submitted 2026-09-21 · Read on arXiv

eess.AS, cs.AI, cs.CL, cs.SD

Submitted: 2026-09-21

Updated: 2026-09-24

Comments: 20 pages, technical report

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 23× in this paper · explained in 2 episodes
• GRPO — 15× in this paper · explained in 6 episodes
• Accuracy — 12× in this paper · explained in 1 episode
• Core — 12× in this paper · explained in 2 episodes
• Trace — 7× in this paper · explained in 2 episodes
• Adaptation — 6× in this paper · explained in 1 episode
• Alignment — 6× in this paper · explained in 1 episode
• Distillation — 5× in this paper · explained in 2 episodes
• Benchmarking — 4× in this paper · explained in 4 episodes
• Fine-Tuning — 4× in this paper · explained in 5 episodes
• On-Policy Distillation — 4× in this paper · explained in 3 episodes
• Big Bench Audio — 3× in this paper
• Consistency — 3× in this paper · explained in 3 episodes
• Hallucination — 3× in this paper · explained in 4 episodes
• Few-Shot Learning — 2× in this paper · explained in 4 episodes
• Full-duplex interaction — 2× in this paper · explained in 2 episodes
• Grounding — 2× in this paper · explained in 3 episodes
• Group Relative Policy Optimization — 2× in this paper
• Group Relative Policy Optimization (GRPO) — 2× in this paper · explained in 5 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Action Space — 1× in this paper · explained in 1 episode
• Chinese Do-Not-Answer — 1× in this paper
• Instruction Tuning — 1× in this paper · explained in 2 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Massive Multi-Task Audio Understanding and Reasoning — 1× in this paper
• Pruning — 1× in this paper · explained in 4 episodes
• Redundancy — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes
• Turns with Plot Advancement — 1× in this paper
• Unified Framework — 1× in this paper · explained in 5 episodes

Sources

• LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
• Qwen-Audio-3.0-ASR Technical Report
• EVA-Bench: A New End-to-end Framework for Evaluating Voice Agents
• VoiceBench: Benchmarking LLM-Based Voice Assistants
• Evaluating Hallucinations in Chinese Large Language Models
• FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
• Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interaction
• MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models
• Baichuan-Audio: A Unified Framework for End-to-End Speech Interaction
• Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities
• Full-Duplex-Bench-v3: Benchmarking Tool Use for Full-Duplex Voice Agents Under Real-World Disfluency
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• $\tau$-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains
• MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs
• DrVoice: Parallel Speech-Text Voice Conversation Model via Dual-Resolution Speech Representations
• Fun-Audio-Chat Technical Report
• CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation
• RMTBench: Benchmarking LLMs Through Multi-Turn User-Centric Role-Playing

Related papers

• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization
• Towards Audio Token Compression in Large Audio Language Models
• WaveScat: Wavelet Scattering Front-Ends with Self-Supervised Features for Speech Deepfake Detection
• ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions