Learning task-specific subspaces via interventional post-training of speech foundation models

arXiv:2606.17967 · cs.CL · Submitted 2026-06-16 · Read on arXiv

cs.CL

Submitted: 2026-06-16

Updated: 2026-06-24

Comments: Accepted to Interspeech 2026; 6 pages (4 main body), 2 figures

DOI: 10.21437/Interspeech.2026-2542

Code: https://github.com/mjukus/interventional-post-training-speech

License: http://creativecommons.org/licenses/by/4.0/

The gist: Speech foundation models, pre-trained on large corpora of unlabelled speech data, produce general-purpose representations which are useful across tasks.

Terminology

Abstract

Speech foundation models, pre-trained on large corpora of unlabelled speech data, produce general-purpose representations which are useful across tasks. However, these representations encode information about salient speech variables in a distributed manner, while downstream speech tasks rely on only some of this variability. In this work, we propose a post-training refinement approach using interventional contrastive learning. By leveraging an interventional dataset and multi-part contrastive loss, we learn a transformation from the entangled representation space of speech foundation models into separate content and speaker subspaces. We evaluate the learnt representations on speaker verification and keyword spotting tasks, showing improved out-of-domain speaker verification performance and evidence that speaker and content information are separated across the learned subspaces.

Sources

Related papers