OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning

arXiv:2610.02181 · cs.CV · Submitted 2026-10-01 · Read on arXiv

cs.CV

Submitted: 2026-10-01

Updated: 2026-10-01

Terminology

• Benchmark — 43× in this paper · explained in 2 episodes
• Alignment — 10× in this paper · explained in 1 episode
• Grounding — 10× in this paper · explained in 3 episodes
• Reinforcement Learning — 9× in this paper · explained in 4 episodes
• Accuracy — 8× in this paper · explained in 1 episode
• Consistency — 5× in this paper · explained in 3 episodes
• Foundation Model — 5× in this paper · explained in 3 episodes
• Foundation Models — 4× in this paper · explained in 7 episodes
• Generalization — 4× in this paper · explained in 13 episodes
• Hallucination — 4× in this paper · explained in 3 episodes
• Trace — 4× in this paper · explained in 2 episodes
• Agentic Framework — 3× in this paper · explained in 6 episodes
• Fine-Tuning — 3× in this paper · explained in 5 episodes
• Multi-hop Reasoning — 3× in this paper · explained in 3 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Ablation study — 2× in this paper · explained in 1 episode
• Chain-of-Thought (CoT) — 2× in this paper · explained in 4 episodes
• Computer Vision and Pattern Recognition — 2× in this paper
• Multimodal Inputs — 2× in this paper · explained in 2 episodes
• Reasoning Traces — 2× in this paper · explained in 3 episodes
• Reinforcement Learning (RL) — 2× in this paper · explained in 25 episodes
• Temporal Reasoning — 2× in this paper · explained in 2 episodes
• Action Space — 1× in this paper · explained in 1 episode
• Active Perception — 1× in this paper · explained in 2 episodes
• Adaptive Reasoning — 1× in this paper · explained in 2 episodes
• Alignment Tax — 1× in this paper · explained in 4 episodes
• Automatic Speech Recognition (ASR) — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Causal Reasoning — 1× in this paper · explained in 2 episodes
• Commentary — 1× in this paper · explained in 2 episodes
• Computational overhead — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Distribution Shift — 1× in this paper · explained in 6 episodes
• Group Sequence Policy Optimization — 1× in this paper
• Homogenization — 1× in this paper · explained in 2 episodes
• Latent Reasoning — 1× in this paper · explained in 2 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Structural Constraints — 1× in this paper · explained in 2 episodes
• Structured Reasoning — 1× in this paper · explained in 3 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes
• Test-time Scaling — 1× in this paper · explained in 2 episodes
• Tokenization — 1× in this paper · explained in 2 episodes
• Verifiable Rewards — 1× in this paper · explained in 2 episodes
• Visual Grounding — 1× in this paper · explained in 2 episodes

Sources

• Qwen2.5-VL Technical Report
• OmniVideo-100K: A Dataset for Audio-Visual Reasoning through Structured Scripts and Evidence Chains
• OmniReasoner: Thinking with Long Audio-Video via Native Tool Use
• VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
• LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
• Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence
• OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models
• VITA: Towards Open-Source Interactive Omni Multimodal LLM
• MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
• Elevating Visual Perception in Multimodal LLMs with Visual Embedding Distillation
• LLaVA-OneVision: Easy Visual Task Transfer
• VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning
• Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data
• Baichuan-Omni-1.5 Technical Report
• Qwen3-ASR Technical Report
• OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
• video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models
• LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models