Mechanisms of Multimodal Synchronization: Insights from Decoder-Based Video-Text-to-Speech Synthesis

arXiv:2411.17690 · cs.MM, cs.CV, cs.SD, eess.AS · Submitted 2024-11-26 · Read on arXiv

cs.MM, cs.CV, cs.SD, eess.AS

Submitted: 2024-11-26

Updated: 2026-10-08

Terminology

• Transformer — 2× in this paper · explained in 2 episodes
• Alignment — 1× in this paper · explained in 1 episode

Related papers

• ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling
• Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models
• Mitigating GenAI-powered Evidence Pollution for Out-of-Context Multimodal Misinformation Detection
• A Rate-Distortion-Classification Approach for Lossy Image Compression