Rethinking Representations for World-Action Modeling

arXiv:2609.38163 · cs.CV, cs.RO · Submitted 2026-09-29 · Read on arXiv

cs.CV, cs.RO

Submitted: 2026-09-29

Updated: 2026-09-29

Code: https://github.com/hustvl/ReWAM

Terminology

• World Model — 17× in this paper · explained in 7 episodes
• Calibration — 9× in this paper · explained in 7 episodes
• Benchmark — 7× in this paper · explained in 2 episodes
• Action-Grounded Representation Shaping — 6× in this paper
• Fidelity — 6× in this paper · explained in 2 episodes
• Generalization — 6× in this paper · explained in 13 episodes
• Transformer — 6× in this paper · explained in 2 episodes
• Aggregation — 5× in this paper · explained in 1 episode
• Latent Space — 5× in this paper · explained in 7 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Foundation Model — 4× in this paper · explained in 3 episodes
• Temporal Representation Bottleneck — 4× in this paper
• Alignment — 3× in this paper · explained in 1 episode
• Embeddings — 2× in this paper · explained in 2 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• Action Flow — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Distribution Shift — 1× in this paper · explained in 5 episodes
• Flow Matching — 1× in this paper · explained in 8 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Latent Dynamics — 1× in this paper · explained in 2 episodes
• Latent Reasoning — 1× in this paper · explained in 2 episodes
• Latent Representation — 1× in this paper · explained in 2 episodes
• Latent Representations — 1× in this paper · explained in 3 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Self-Supervised Learning — 1× in this paper · explained in 7 episodes

Sources

• World Action Models are Zero-shot Policies
• Fast-WAM: Do World Action Models Need Test-time Future Imagination?
• DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks
• RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
• Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models
• What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction
• RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
• RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
• Motubrain: An Advanced World Action Model for Robot Control
• RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination
• Cosmos 3: Omnimodal World Models for Physical AI
• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control
• ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?
• Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models
• Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising
• WAM4D: Fast 4D World Action Model via Spatial Register Tokens
• DreamWAM: Beyond RGB Future Prediction for World Action Models
• EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data
• FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment
• Being-H0.7: A Latent World-Action Model from Egocentric Videos

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models