StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions

arXiv:2609.18430 · cs.CV · Submitted 2026-09-16 · Read on arXiv

cs.CV

Submitted: 2026-09-16

Updated: 2026-09-16

Code: https://github.com/westlakedi-awomo/StrucPhysVideo

Project page: https://westlakedi-awomo.github.io/StrucPhysVideo-Page

Terminology

• Distillation — 14× in this paper · explained in 2 episodes
• Transformer — 14× in this paper · explained in 2 episodes
• Verification — 13× in this paper · explained in 2 episodes
• World Model — 10× in this paper · explained in 7 episodes
• Benchmark — 9× in this paper · explained in 2 episodes
• Ablation — 7× in this paper · explained in 1 episode
• Consistency — 6× in this paper · explained in 3 episodes
• Diffusion Model — 5× in this paper · explained in 8 episodes
• Distribution Matching — 5× in this paper · explained in 2 episodes
• Fidelity — 4× in this paper · explained in 2 episodes
• Flow Matching — 4× in this paper · explained in 8 episodes
• Accuracy — 3× in this paper · explained in 1 episode
• Diffusion Models — 3× in this paper · explained in 6 episodes
• Agentic Retrieval — 2× in this paper · explained in 1 episode
• Alignment — 2× in this paper · explained in 1 episode
• Embodied AI — 2× in this paper · explained in 2 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Latent Space — 2× in this paper · explained in 7 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Action Space — 1× in this paper · explained in 1 episode
• Activation Checkpointing — 1× in this paper · explained in 1 episode
• Actor — 1× in this paper · explained in 3 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Core — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Exposure Bias — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• Mixture-of-Experts (MoE) — 1× in this paper · explained in 7 episodes
• Optical Flow — 1× in this paper · explained in 2 episodes
• Provenance — 1× in this paper · explained in 4 episodes
• Variational Autoencoder (VAE) — 1× in this paper · explained in 3 episodes
• alpha — 1× in this paper · explained in 1 episode

Sources

• Physics-IQ Verified
• Qwen3-VL Technical Report
• Wan: Open and Advanced Large-Scale Video Generative Models
• RoFormer: Enhanced Transformer with Rotary Position Embedding
• Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence
• Classifier-Free Diffusion Guidance
• Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation
• Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
• One-step Diffusion with Distribution Matching Distillation
• TransNet V2: An effective deep network architecture for fast shot transition detection
• Cosmos 3: Omnimodal World Models for Physical AI
• AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
• DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
• Learning Transferable Dynamics Priors from Action to World Modeling
• WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models