CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation

arXiv:2609.31418 · cs.RO · Submitted 2026-09-25 · Read on arXiv

cs.RO

Submitted: 2026-09-25

Updated: 2026-09-25

Terminology

• Gaussian Splatting — 10× in this paper · explained in 4 episodes
• Grounding — 10× in this paper · explained in 3 episodes
• 3D Gaussian Splatting — 4× in this paper · explained in 2 episodes
• Consistency — 3× in this paper · explained in 3 episodes
• Fidelity — 3× in this paper · explained in 2 episodes
• Affordance — 2× in this paper · explained in 1 episode
• Calibration — 2× in this paper · explained in 7 episodes
• Verification — 2× in this paper · explained in 2 episodes
• 3D Gaussian Splatting (3DGS) — 1× in this paper · explained in 4 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Core — 1× in this paper · explained in 2 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Human-in-the-Loop — 1× in this paper · explained in 3 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Task Planning — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Agentic Scene Policies
• GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoning
• 3DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting
• Qwen3-VL Technical Report
• The Replica Dataset: A Digital Replica of Indoor Spaces
• HoloAgent-0: A Unified Embodied Agent Framework with 3D Spatial Memory

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving