Beyond Speech Captions: Speech-Rewarded Style Planning for Conversational Text-to-Speech

arXiv:2610.11461 · cs.SD, cs.CL, eess.AS · Submitted 2026-10-08 · Read on arXiv

cs.SD, cs.CL, eess.AS

Submitted: 2026-10-08

Updated: 2026-10-08

Code: https://github.com/huggingface/trl

Terminology

• Acoustic Metric — 12× in this paper · explained in 1 episode
• Alignment — 12× in this paper · explained in 1 episode
• GRPO — 7× in this paper · explained in 6 episodes
• Speech-Rewarded Style Planning — 6× in this paper
• Consistency — 4× in this paper · explained in 3 episodes
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Cosine Similarity — 2× in this paper · explained in 3 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Embeddings — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Pseudo-labels — 1× in this paper · explained in 2 episodes
• Self-Supervised Learning — 1× in this paper · explained in 7 episodes

Sources

• Natural language guidance of high-fidelity text-to-speech with synthetic annotations
• ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• ISCSLP 2026 CoT-TTS Challenge: Chain-of-Thought Reasoning for Context-Aware Text-to-Speech
• CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training
• Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
• Qwen3-Omni Technical Report

Related papers

• Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
• Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
• AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
• SoundWeaver: Compositional Warm-Starting for Text-to-Audio Diffusion Serving
• WASIL: In-the-Wild Arabic Spoken Interactions with LLMs
• Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment