Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation

arXiv:2606.00095 · cs.CV, cs.AI, cs.CL, cs.RO · Submitted 2026-05-25 · Read on arXiv

cs.CV, cs.AI, cs.CL, cs.RO

Submitted: 2026-05-25

Updated: 2026-05-25

Journal ref: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 15243-15252

Code: https://github.com/Teacher-Tom/HSGM_public

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 14× in this paper · explained in 2 episodes
• Hierarchical Semantic-Geometric Map — 12× in this paper
• Ablation — 9× in this paper · explained in 1 episode
• Success Rate — 8× in this paper
• Vision-Language Navigation — 8× in this paper
• Decoupling — 5× in this paper · explained in 2 episodes
• Grounding — 5× in this paper · explained in 3 episodes
• Action Space — 4× in this paper · explained in 1 episode
• Alignment — 4× in this paper · explained in 1 episode
• Core — 4× in this paper · explained in 2 episodes
• Ablation study — 3× in this paper · explained in 1 episode
• Adam — 3× in this paper · explained in 2 episodes
• Affordance — 3× in this paper · explained in 1 episode
• Foundation Model — 3× in this paper · explained in 3 episodes
• Instance Segmentation — 3× in this paper · explained in 3 episodes
• Ablation Studies — 2× in this paper · explained in 1 episode
• Embodied AI — 2× in this paper · explained in 2 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• Hallucination — 2× in this paper · explained in 3 episodes
• IEEE International Conference on Robotics and Automation — 2× in this paper
• Robustness — 2× in this paper · explained in 5 episodes
• Semantic Understanding — 2× in this paper · explained in 2 episodes
• Vision-Language Models (VLMs) — 2× in this paper · explained in 6 episodes
• Abstraction — 1× in this paper · explained in 1 episode
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Chain-of-Thought (CoT) Reasoning — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Motion Primitives — 1× in this paper · explained in 2 episodes
• Navigation Error — 1× in this paper
• Oracle Success Rate — 1× in this paper
• Point Clouds — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Semantic Alignment — 1× in this paper · explained in 3 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• Training-Free Framework — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• $A^2$Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Models
• Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigation
• How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
• MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chains
• Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks
• NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
• GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models
• VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning
• OpenAI GPT-5 System Card
• DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigation
• Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models