The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation

arXiv:2609.02367 · cs.MM, cs.CV · Submitted 2026-09-02 · Read on arXiv

cs.MM, cs.CV

Submitted: 2026-09-02

Updated: 2026-09-18

Code: https://github.com/DAGroup-PKU/Temporal-Context-Routing

Terminology

• Accuracy — 18× in this paper · explained in 1 episode
• Alignment — 11× in this paper · explained in 1 episode
• Conference on Computer Vision and Pattern Recognition — 11× in this paper
• Transformer — 11× in this paper · explained in 2 episodes
• Ablation — 8× in this paper · explained in 1 episode
• Fidelity — 8× in this paper · explained in 2 episodes
• Temporal Context Routing — 7× in this paper
• Diffusion Model — 5× in this paper · explained in 8 episodes
• Diffusion Models — 5× in this paper · explained in 6 episodes
• Adaptation — 4× in this paper · explained in 1 episode
• Trace — 3× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Flow Matching — 2× in this paper · explained in 8 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 2 episodes
• Imaging Quality — 2× in this paper
• International Conference on Computer Vision — 2× in this paper
• Tokenization — 2× in this paper · explained in 2 episodes
• alpha — 2× in this paper · explained in 1 episode
• Ablation Studies — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• LoRA Adaptation — 1× in this paper · explained in 2 episodes
• LoRA Adapters — 1× in this paper · explained in 3 episodes

Sources

• WhisperX: Time-Accurate Speech Transcription of Long-Form Audio
• Prompt Relay: Inference-Time Temporal Control for Multi-Event Video Generation
• LTX-2: Efficient Joint Audio-Visual Foundation Model
• Native Audio-Visual Alignment for Generation
• ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Models
• Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation
• MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation
• Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation
• Improving Joint Audio-Video Generation with Cross-Modal Context Learning
• TIE: Time Interval Encoding for Video Generation over Events
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Wan: Open and Advanced Large-Scale Video Generative Models
• Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding
• Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation
• RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time
• MultiShotMaster: A Controllable Multi-Shot Video Generation Framework
• KeyVID: Keyframe-Aware Video Diffusion for Audio-Synchronized Visual Animation
• Does Hearing Help Seeing? Investigating Audio-Video Joint Denoising for Video Generation
• Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curation
• ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling

Related papers

• ControlFoley: Unified and Controllable Video-to-Audio Generation with Cross-Modal Conflict Handling
• Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models
• Mitigating GenAI-powered Evidence Pollution for Out-of-Context Multimodal Misinformation Detection
• A Rate-Distortion-Classification Approach for Lossy Image Compression