OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents

arXiv:2609.32569 · cs.CV · Submitted 2026-09-26 · Read on arXiv

cs.CV

Submitted: 2026-09-26

Updated: 2026-09-26

Project page: https://omni-smart-home.github.io

Terminology

• Grounding — 118× in this paper · explained in 3 episodes
• Accuracy — 53× in this paper · explained in 1 episode
• Benchmark — 36× in this paper · explained in 2 episodes
• Ablation — 6× in this paper · explained in 1 episode
• Benchmarking — 4× in this paper · explained in 3 episodes
• Generative AI — 4× in this paper · explained in 2 episodes
• Reinforcement Learning — 4× in this paper · explained in 4 episodes
• Trace — 4× in this paper · explained in 2 episodes
• Verification — 4× in this paper · explained in 2 episodes
• Adam — 3× in this paper · explained in 2 episodes
• Agent Memory — 3× in this paper · explained in 1 episode
• LLM-as-a-Judge — 3× in this paper · explained in 3 episodes
• Multimodal Reasoning — 3× in this paper · explained in 5 episodes
• Agent Workflow Memory — 2× in this paper
• Verifier — 2× in this paper · explained in 2 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Agentic Search — 1× in this paper · explained in 1 episode
• Agentic System — 1× in this paper · explained in 2 episodes
• Agentic Task — 1× in this paper · explained in 1 episode
• Agentic Tasks — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Embodied AI — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Korea Advanced Institute of Science and Technology — 1× in this paper
• LLM judge — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Model-Agnostic — 1× in this paper · explained in 2 episodes
• Personalization — 1× in this paper · explained in 2 episodes
• Task Planning — 1× in this paper · explained in 2 episodes

Sources

• HumanOmni-Speaker: Identifying Who said What and When
• PersonalHomeBench: Evaluating Agents in Personalized Smart Homes
• MIST: Multimodal Interactive Speech-based Tool-calling Conversational Assistants for Smart Homes
• VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
• Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence
• RTMW: Real-Time Multi-Person 2D and 3D Whole-body Pose Estimation
• SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes
• MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
• Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI
• VLA-Pro: Cross-Task Procedural Memory Transfer for Vision-Language-Action Models
• OpenAI GPT-5 System Card
• video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Models
• Gemma 4 Technical Report
• MIRIX: Multi-Agent Memory System for LLM-Based Agents
• Qwen2.5-Omni Technical Report
• Qwen3-Omni Technical Report
• Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning
• MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models