HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments with Dynamic Multi-Human Interactions

arXiv:2503.14229 · cs.AI, cs.CV, cs.RO · Submitted 2026-08-18 · Read on arXiv

Yifei Dong, Fengyi Wu, Qi He, Lingdong Kong, Heng Li, Minghan Li, Zebang Cheng, Yuxuan Zhou, Jingdong Sun, Qi Dai, Alexander G Hauptmann, Zhi-Qi Cheng

cs.AI, cs.CV, cs.RO

Submitted: 2026-08-18

Updated: 2026-08-19

Comments: Accepted to IROS 2026. 35 pages, 20 figures, website: https://uwmilab.github.io/HA-VLN-webpage/

Code: https://github.com/F1y1113/HA-VLN

Project page: https://uwmilab.github.io/HA-VLN-webpage

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 33× in this paper · explained in 2 episodes
• Vision-and-Language Navigation — 20× in this paper
• Alignment — 14× in this paper · explained in 1 episode
• Embeddings — 11× in this paper · explained in 2 episodes
• Transformer — 9× in this paper · explained in 2 episodes
• Accuracy — 8× in this paper · explained in 1 episode
• Grounding — 7× in this paper · explained in 3 episodes
• Sim-to-Real Transfer — 7× in this paper · explained in 2 episodes
• Fidelity — 6× in this paper · explained in 2 episodes
• Action Space — 5× in this paper · explained in 1 episode
• Cross-Modal Attention — 5× in this paper
• Generalization — 5× in this paper · explained in 13 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Collision Rate — 4× in this paper
• Human-in-the-Loop — 4× in this paper · explained in 3 episodes
• Robustness — 4× in this paper · explained in 5 episodes
• Success Rate — 4× in this paper
• Benchmarking — 3× in this paper · explained in 3 episodes
• Consistency — 3× in this paper · explained in 3 episodes
• Core — 3× in this paper · explained in 2 episodes
• Particle Swarm Optimization — 3× in this paper
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Ablation Studies — 2× in this paper · explained in 1 episode
• Navigation Error — 2× in this paper
• Navigation Success Rate — 2× in this paper
• Total Collision Rate — 2× in this paper
• Verification — 2× in this paper · explained in 2 episodes
• Actor — 1× in this paper · explained in 3 episodes
• Adaptability — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Cross Domain Evaluation of BEVBert — 1× in this paper
• Data Augmentation — 1× in this paper · explained in 7 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Discrete Environment — 1× in this paper
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Embodied AI — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Semantic Similarity — 1× in this paper · explained in 2 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes
• World Model — 1× in this paper · explained in 7 episodes
• Zero-Shot Transfer — 1× in this paper · explained in 4 episodes

Sources

• SHIELD: LLM-Driven Schema Induction for Predictive Analytics in EV Battery Supply Chain Disruptions
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges
• Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresight
• Language-Conditioned World Modeling for Visual Navigation
• Stay on the Path: Instruction Fidelity in Vision-and-Language Navigation
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding
• Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactions
• VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluation
• Self-Monitoring Navigation Agent via Auxiliary Progress Estimation
• Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots
• Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
• Human Motion Diffusion Model
• Towards Versatile Embodied Navigation
• GoViG: Goal-Conditioned Visual Navigation Instruction Generation via Multimodal Reasoning
• Building Generalizable Agents with a Realistic and Rich 3D Environment
• Natural Language Can Help Bridge the Sim2Real Gap
• NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
• Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection