Seeing Speech: Learning Visible Articulatory Dynamics for Speech-Driven 3D Facial Animation

arXiv:2609.30517 · eess.AS, cs.GR, cs.LG · Submitted 2026-09-24 · Read on arXiv

eess.AS, cs.GR, cs.LG

Submitted: 2026-09-24

Updated: 2026-09-24

Terminology

• Ablation — 14× in this paper · explained in 1 episode
• Consistency — 9× in this paper · explained in 3 episodes
• Transformer — 9× in this paper · explained in 2 episodes
• Accuracy — 5× in this paper · explained in 1 episode
• Core — 5× in this paper · explained in 2 episodes
• Generalization — 5× in this paper · explained in 13 episodes
• Topology-aware Articulatory Composition — 5× in this paper
• Adam — 4× in this paper · explained in 2 episodes
• Alignment — 4× in this paper · explained in 1 episode
• Fidelity — 4× in this paper · explained in 2 episodes
• Fine-Tuning — 4× in this paper · explained in 5 episodes
• ACM Transactions on Graphics — 3× in this paper
• Benchmark — 3× in this paper · explained in 2 episodes
• Grounding — 3× in this paper · explained in 3 episodes
• Interpretability — 3× in this paper · explained in 15 episodes
• Two-stage training — 3× in this paper · explained in 2 episodes
• Ablation Studies — 2× in this paper · explained in 1 episode
• Adjacency Matrix — 2× in this paper · explained in 1 episode
• CVF International Conference on Computer Vision — 2× in this paper
• Conference on Computer Vision and Pattern Recognition — 2× in this paper
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Institutional Review Board — 2× in this paper
• Representation Learning — 2× in this paper · explained in 2 episodes
• Temporal consistency — 2× in this paper · explained in 2 episodes
• Affective Computing — 1× in this paper · explained in 1 episode
• Agnostic Learning — 1× in this paper · explained in 1 episode
• Aperture — 1× in this paper · explained in 1 episode
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• Gradient Flow — 1× in this paper · explained in 2 episodes
• Latent Space — 1× in this paper · explained in 7 episodes
• Two-Stage Training Strategy — 1× in this paper · explained in 3 episodes

Sources

• Teaching Machines to Speak Using Articulatory Control
• Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Models
• EmoFace: Emotion-Content Disentangled Speech-Driven 3D Talking Face Animation
• EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing

Related papers

• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization
• Towards Audio Token Compression in Large Audio Language Models
• WaveScat: Wavelet Scattering Front-Ends with Self-Supervised Features for Speech Deepfake Detection
• ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions