EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos

arXiv:2609.39378 · cs.CV · Submitted 2026-09-30 · Read on arXiv

cs.CV

Submitted: 2026-09-30

Updated: 2026-10-02

Code: https://github.com/Ropedia/EgoTools

Terminology

• Benchmark — 122× in this paper · explained in 2 episodes
• Grounding — 58× in this paper · explained in 3 episodes
• Affordance — 30× in this paper · explained in 1 episode
• Spatial Reasoning — 16× in this paper
• Accuracy — 12× in this paper · explained in 1 episode
• Procedural Dynamics — 11× in this paper
• Fine-Tuning — 9× in this paper · explained in 5 episodes
• Ablation — 7× in this paper · explained in 1 episode
• Conference on Computer Vision and Pattern Recognition — 5× in this paper
• Adaptation — 4× in this paper · explained in 1 episode
• Benchmarking — 4× in this paper · explained in 4 episodes
• Consistency — 3× in this paper · explained in 3 episodes
• Core — 3× in this paper · explained in 2 episodes
• Physical Constraints — 3× in this paper · explained in 7 episodes
• Alignment — 2× in this paper · explained in 1 episode
• Embodied AI — 2× in this paper · explained in 2 episodes
• Verification — 2× in this paper · explained in 2 episodes
• Vision-language models — 2× in this paper · explained in 2 episodes
• Visual Grounding — 2× in this paper · explained in 2 episodes
• 3D Gaussian Splatting — 1× in this paper · explained in 2 episodes
• Actor — 1× in this paper · explained in 3 episodes
• Adaptivity — 1× in this paper · explained in 1 episode
• CVF International Conference on Computer Vision — 1× in this paper
• European Conference on Computer Vision — 1× in this paper
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Gaussian Splatting — 1× in this paper · explained in 4 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Hallucination — 1× in this paper · explained in 4 episodes
• Instruction Tuning — 1× in this paper · explained in 2 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Point Clouds — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• Temporal consistency — 1× in this paper · explained in 2 episodes

Sources

• Qwen3-VL Technical Report
• Qwen2.5-VL Technical Report
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• WearVQA: A Visual Question Answering Benchmark for Wearables in Egocentric Authentic Real-world scenarios
• EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning
• EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models
• Grounded Question-Answering in Long Egocentric Videos
• Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervision
• LifeEval: A Multimodal Benchmark for Assistive AI in Egocentric Daily Life Tasks
• Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence
• Ego4D: Around the World in 3,000 Hours of Egocentric Video
• ManipVQA: Injecting Robotic Affordance and Physically Grounded Information into Multi-Modal Large Language Models
• EgoTaskQA: Understanding Human Tasks in Egocentric Videos
• MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents
• LLaVA-OneVision: Easy Visual Task Transfer
• SAW-Bench: Learning Situated Awareness in the Real World
• Learning Precise Affordances from Egocentric Videos for Robotic Manipulation
• LangSurf: Language-Embedded Surface Gaussians for 3D Scene Understanding
• HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction
• EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models