SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenes

arXiv:2603.29798 · cs.CV · Submitted 2026-03-31 · Read on arXiv

cs.CV

Submitted: 2026-03-31

Updated: 2026-09-17

Code: https://github.com/PhyScene/PhyScene

Project page: https://sceneteract.github.io

Terminology

• Verification — 85× in this paper · explained in 2 episodes
• Trace — 45× in this paper · explained in 2 episodes
• GRPO — 28× in this paper · explained in 6 episodes
• Verifier — 25× in this paper · explained in 2 episodes
• Benchmark — 22× in this paper · explained in 2 episodes
• Affordance — 16× in this paper · explained in 1 episode
• Accuracy — 15× in this paper · explained in 1 episode
• Grounding — 13× in this paper · explained in 3 episodes
• Embodied AI — 9× in this paper · explained in 2 episodes
• Alignment — 7× in this paper · explained in 1 episode
• Benchmarking — 7× in this paper · explained in 3 episodes
• Core — 7× in this paper · explained in 2 episodes
• Gaussian Splatting — 6× in this paper · explained in 4 episodes
• Hallucination — 6× in this paper · explained in 3 episodes
• 3D Gaussian Splatting — 5× in this paper · explained in 2 episodes
• Physical Constraints — 5× in this paper · explained in 7 episodes
• CVF International Conference on Computer Vision — 4× in this paper
• False Positive — 4× in this paper
• Fine-Tuning — 4× in this paper · explained in 5 episodes
• Reasoning Traces — 4× in this paper · explained in 3 episodes
• Reinforcement Learning — 4× in this paper · explained in 4 episodes
• Adaptation — 3× in this paper · explained in 1 episode
• Fidelity — 3× in this paper · explained in 2 episodes
• Generalization — 3× in this paper · explained in 13 episodes
• Task Planning — 3× in this paper · explained in 2 episodes
• Archetype — 2× in this paper · explained in 1 episode
• Consistency — 2× in this paper · explained in 3 episodes
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Diffusion Models — 2× in this paper · explained in 6 episodes
• Failure Modes — 2× in this paper · explained in 3 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 2 episodes
• Low-Rank Adaptation (LoRA) — 2× in this paper · explained in 9 episodes
• Matthews Correlation Coefficient — 2× in this paper
• Parameter-Efficient Fine-Tuning — 2× in this paper · explained in 2 episodes
• Reward Hacking — 2× in this paper · explained in 8 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Semantic Alignment — 2× in this paper · explained in 3 episodes
• Semantic Understanding — 2× in this paper · explained in 2 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Vision-Language Models (VLMs) — 2× in this paper · explained in 6 episodes
• Accuracy Alignment — 1× in this paper · explained in 1 episode
• Alignment Scores — 1× in this paper · explained in 1 episode
• Catastrophic Forgetting — 1× in this paper · explained in 5 episodes
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• F1 Score — 1× in this paper · explained in 3 episodes
• Group Relative Policy Optimization — 1× in this paper
• Group Relative Policy Optimization (GRPO) — 1× in this paper · explained in 5 episodes
• Horizon Stability Index — 1× in this paper
• Multimodal Inputs — 1× in this paper · explained in 2 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Nash Equilibrium — 1× in this paper · explained in 2 episodes
• Segmentation Masks — 1× in this paper · explained in 2 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 8 episodes

Sources

• WorldScore: A Unified Evaluation Benchmark for World Generation
• Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
• Demystifying MMD GANs
• Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
• AI2-THOR: An Interactive 3D Environment for Visual AI
• Ministral 3
• SpatialLM: Training Large Language Models for Structured Indoor Modeling
• Human-Aware 3D Scene Generation with Spatially-constrained Diffusion Models
• LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans
• RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
• Gemma 3 Technical Report
• Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model
• Function2Scene: 3D Indoor Scene Layout from Functional Specifications
• SAGE: Scalable Agentic 3D Scene Generation for Embodied AI
• Qwen3 Technical Report
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• SceneEval: Evaluating Semantic Coherence in Text-Conditioned 3D Indoor Scene Synthesis
• Gemini: A Family of Highly Capable Multimodal Models
• Viser: Imperative, Web-based 3D Visualization in Python
• MobiAgent: A Systematic Framework for Customizable Mobile Agents

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models