Asking the World: Generalist Physical Reasoning through Agentic World Modeling and Probing

arXiv:2609.39135 · cs.CV · Submitted 2026-09-30 · Read on arXiv

cs.CV

Submitted: 2026-09-30

Updated: 2026-09-30

Code: https://github.com/NVIDIA/warp

Terminology

• Accuracy — 60× in this paper · explained in 1 episode
• Ablation — 19× in this paper · explained in 1 episode
• System Identification — 14× in this paper · explained in 3 episodes
• Benchmark — 11× in this paper · explained in 2 episodes
• World Model — 5× in this paper · explained in 7 episodes
• Asking the World — 4× in this paper
• Grounding — 4× in this paper · explained in 3 episodes
• Adaptation — 3× in this paper · explained in 1 episode
• Adaptive Reasoning — 3× in this paper · explained in 2 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• Ablation Studies — 2× in this paper · explained in 1 episode
• Alignment — 2× in this paper · explained in 1 episode
• Consistency — 2× in this paper · explained in 3 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Foundation Model — 2× in this paper · explained in 3 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Provenance — 2× in this paper · explained in 3 episodes
• Abstraction — 1× in this paper · explained in 1 episode
• Agentic Framework — 1× in this paper · explained in 6 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Causal Understanding — 1× in this paper · explained in 2 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Gaussian Splatting — 1× in this paper · explained in 4 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Latent Dynamics — 1× in this paper · explained in 2 episodes
• Multimodal Reasoning — 1× in this paper · explained in 5 episodes
• Training-Free Framework — 1× in this paper · explained in 2 episodes

Sources

• Qwen3-VL Technical Report
• VideoPhy: Evaluating Physical Commonsense for Video Generation
• SAM 3: Segment Anything with Concepts
• SAM 3D: 3Dfy Anything in Images
• Geometrically-Constrained Agent for Spatial Reasoning
• Chain-of-Frames: Advancing Video Understanding in Multimodal LLMs via Frame-Aware Reasoning
• Dojo: A Differentiable Physics Engine for Robotics
• SlotPi: Physics-informed Object-centric Reasoning Models
• QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models
• Depth Anything 3: Recovering the Visual Space from Any Views
• SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models
• Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels
• Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation
• Beyond Perceptual Shortcuts: Causal-Inspired Debiasing Optimization for Generalizable Video Reasoning in Lightweight MLLMs
• SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models
• PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning
• ReAct: Synergizing Reasoning and Acting in Language Models
• CLEVRER: CoLlision Events for Video REpresentation and Reasoning
• Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models