Conversational Voice Aesthetic Model with Reinforcement Learning from Human Listeners

arXiv:2610.10868 · eess.AS, cs.CL, cs.LG, cs.MM, cs.SD · Submitted 2026-10-07 · Read on arXiv

eess.AS, cs.CL, cs.LG, cs.MM, cs.SD

Submitted: 2026-10-07

Updated: 2026-10-07

Terminology

• GRPO — 15× in this paper · explained in 6 episodes
• Accuracy — 13× in this paper · explained in 1 episode
• Alignment — 6× in this paper · explained in 1 episode
• Reinforcement Learning — 6× in this paper · explained in 4 episodes
• Conversational Voice Aesthetic Model — 5× in this paper
• Reinforcement Learning (RL) — 2× in this paper · explained in 25 episodes
• Two-stage training — 2× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Adversarial Training — 1× in this paper · explained in 1 episode
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Multimodal Dataset — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes

Sources

• WavReward: Spoken Dialogue Models With Generalist Reward Evaluators
• GSRM: Generative Speech Reward Model for Speech RLHF
• Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
• Qwen3-TTS Technical Report
• Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPO
• Qwen2.5-Omni Technical Report
• Qwen3 Technical Report
• Qwen3-Omni Technical Report
• NeMo: a toolkit for building AI applications using Neural Modules
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Related papers

• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization
• Towards Audio Token Compression in Large Audio Language Models
• WaveScat: Wavelet Scattering Front-Ends with Self-Supervised Features for Speech Deepfake Detection
• ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions