RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigation

arXiv:2609.32224 · cs.AI · Submitted 2026-09-26 · Read on arXiv

cs.AI

Submitted: 2026-09-26

Updated: 2026-09-26

Code: https://github.com/rikeilong/OmniAV_Nav

Terminology

• Ablation — 6× in this paper · explained in 1 episode
• Benchmark — 6× in this paper · explained in 2 episodes
• Grounding — 6× in this paper · explained in 3 episodes
• Semantic Audio-Visual Navigation — 6× in this paper
• Embeddings — 5× in this paper · explained in 2 episodes
• Conference on Computer Vision and Pattern Recognition — 4× in this paper
• Accuracy — 3× in this paper · explained in 1 episode
• Latent Navigation Reasoning — 3× in this paper
• Action Space — 2× in this paper · explained in 1 episode
• Embodied AI — 2× in this paper · explained in 2 episodes
• Success Rate — 2× in this paper
• Transfer Learning — 2× in this paper · explained in 10 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Affordance — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• CVF International Conference on Computer Vision — 1× in this paper
• Cooperation — 1× in this paper · explained in 2 episodes
• Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Distance To Goal — 1× in this paper
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• IEEE International Conference on Robotics and Automation — 1× in this paper
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Multimodal Inputs — 1× in this paper · explained in 2 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Structured Reasoning — 1× in this paper · explained in 3 episodes
• Transferability — 1× in this paper · explained in 4 episodes
• Vision-Language Models (VLMs) — 1× in this paper · explained in 6 episodes

Sources

• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
• $A^2$Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models
• Knowledge-driven Scene Priors for Semantic Audio-Visual Embodied Navigation
• NavBench: Probing Multimodal Large Language Models for Embodied Navigation
• OctoNav: Towards Generalist Embodied Navigation
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Qwen3-VL Technical Report
• Visual Instruction Tuning

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection