Personalizing Causal Audio-Driven Facial Motion via Dynamic Multi-modal Retrieval

arXiv:2604.23692 · cs.GR, cs.CV · Submitted 2026-04-26 · Read on arXiv

cs.GR, cs.CV

Submitted: 2026-04-26

Updated: 2026-09-18

Terminology

• Fidelity — 16× in this paper · explained in 2 episodes
• Transformer — 13× in this paper · explained in 2 episodes
• Consistency — 10× in this paper · explained in 3 episodes
• Personalization — 9× in this paper · explained in 2 episodes
• Accuracy — 7× in this paper · explained in 1 episode
• Alignment — 7× in this paper · explained in 1 episode
• Latent Space — 6× in this paper · explained in 7 episodes
• Embeddings — 5× in this paper · explained in 2 episodes
• Quantization — 5× in this paper · explained in 12 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Gaussian Splatting — 4× in this paper · explained in 4 episodes
• ACM Transactions on Graphics — 3× in this paper
• Ablation study — 3× in this paper · explained in 1 episode
• Robustness — 3× in this paper · explained in 5 episodes
• 3D Gaussian Splatting — 2× in this paper · explained in 2 episodes
• Adam — 2× in this paper · explained in 2 episodes
• CVF International Conference on Computer Vision — 2× in this paper
• Calibration — 2× in this paper · explained in 7 episodes
• Exposure Bias — 2× in this paper · explained in 2 episodes
• Tokenization — 2× in this paper · explained in 2 episodes
• 3D Gaussian Splatting (3DGS) — 1× in this paper · explained in 4 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Aperture — 1× in this paper · explained in 1 episode
• Causal Inference — 1× in this paper · explained in 4 episodes
• Core — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Exponential Moving Average (EMA) — 1× in this paper · explained in 2 episodes
• Feature Extraction — 1× in this paper · explained in 2 episodes
• Generalizability — 1× in this paper · explained in 6 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Objective Function — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Temporal consistency — 1× in this paper · explained in 2 episodes
• Two-Stage Training Strategy — 1× in this paper · explained in 3 episodes
• Two-stage training — 1× in this paper · explained in 2 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes

Sources

• Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Dataset
• GaussianSpeech: Audio-Driven Gaussian Avatars
• GSTalker: Real-time Audio-Driven Talking Face Generation via Deformable Gaussian Splatting
• Moshi: a speech-text foundation model for real-time dialogue
• Classifier-Free Diffusion Guidance
• Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction
• DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering
• GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation
• GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis
• Image and Video Tokenization with Binary Spherical Quantization

Related papers

• SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Control
• CADReasoner: Iterative Program Editing for CAD Reverse Engineering
• QuadLink: Autoregressive Quad-Dominant Mesh Generation via Point-Relation Learning
• DrawVideo: Grounded and Faithful Multi-Shot Video Generation from Storyboard Keyframe Sketches
• MotionPersona: Real-Time Locomotion Control across Personas, Bodies, and Styles
• MeshSplatBench: A Unified Benchmark for Triangle- and Mesh-Based Neural Rendering