SyncRA: Learning Temporal Correspondence in Omni-Modal Models

arXiv:2609.34363 · cs.CV, cs.AI, cs.MM · Submitted 2026-09-28 · Read on arXiv

cs.CV, cs.AI, cs.MM

Submitted: 2026-09-28

Updated: 2026-09-28

Terminology

• Benchmark — 44× in this paper · explained in 2 episodes
• Accuracy — 31× in this paper · explained in 1 episode
• Fine-Tuning — 25× in this paper · explained in 5 episodes
• Alignment — 18× in this paper · explained in 1 episode
• Ablation — 7× in this paper · explained in 1 episode
• Latent Reasoning — 4× in this paper · explained in 2 episodes
• Grounding — 3× in this paper · explained in 3 episodes
• Representation Learning — 3× in this paper · explained in 2 episodes
• Trace — 3× in this paper · explained in 2 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Cosine Similarity — 2× in this paper · explained in 3 episodes
• Predictive Coding — 2× in this paper · explained in 2 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Causal Reasoning — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Contrastive Learning — 1× in this paper · explained in 5 episodes
• Linear probe — 1× in this paper · explained in 2 episodes
• Mixture-of-Experts (MoE) — 1× in this paper · explained in 7 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• Thinking mode — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• Look, Listen and Learn
• CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment
• OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains
• AVID: A Benchmark for Omni-Modal Audio-Visual Inconsistency Understanding via Agent-Driven Construction
• MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
• LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
• WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMs
• MMBench: Is Your Multi-modal Model an All-around Player?
• LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs
• Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality
• Representation Learning with Contrastive Predictive Coding
• Qwen2.5-Omni Technical Report
• Qwen3-Omni Technical Report
• Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think
• AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models
• Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models