Learnable Classifier-Free Guidance Null Embeddings for Enhanced Controllable Speech Synthesis

arXiv:2609.25411 · eess.AS, cs.LG · Submitted 2026-09-21 · Read on arXiv

eess.AS, cs.LG

Submitted: 2026-09-21

Updated: 2026-09-21

Comments: Accepted paper at Interspeech 2026

Project page: https://airtimemedia.github.io/IS2026-LearnableCFG

License: http://creativecommons.org/licenses/by/4.0/

The gist: Classifier-free Guidance (CFG) is widely adopted in text-to-speech (TTS) systems to enhance generation quality and conditioning fidelity by interpolating between conditioned and unconditioned

Terminology

Abstract

Classifier-free Guidance (CFG) is widely adopted in text-to-speech (TTS) systems to enhance generation quality and conditioning fidelity by interpolating between conditioned and unconditioned predictions. A common unconditional technique is to use an empty representation, in the form of a fixed null vector. In this work, we propose replacing this representation with a learnable unconditional embedding, optimized to represent a meaningful unconditional state. Objective and subjective evaluations demonstrate that learnable null embeddings consistently outperform fixed null embeddings across speaker similarity, speech stability, and expressiveness, while exhibiting greater robustness to larger guidance scales. We further show that learning a distinct unconditional embedding for each of the TTS conditioning modalities allows fine-grained control over speaker and text guidance, showcasing the trade-off between similarity and quality, and stability and expressiveness in the generated speech.

Sources

Related papers