The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

arXiv:2609.10296 · cs.CL, cs.LG · Submitted 2026-09-09 · Read on arXiv

cs.CL, cs.LG

Submitted: 2026-09-09

Updated: 2026-10-07

Comments: 12 pages, 8 figures

License: http://creativecommons.org/licenses/by/4.0/

The gist: Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult.

Terminology

Abstract

Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.

Sources

Related papers