Modality-Autoregressive World-Action Models

arXiv:2609.17524 · cs.RO · Submitted 2026-09-15 · Read on arXiv

cs.RO

Submitted: 2026-09-15

Updated: 2026-10-08

Project page: https://adamhung60.github.io/ModAR

Terminology

• Transformer — 7× in this paper · explained in 2 episodes
• World Model — 5× in this paper · explained in 7 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Generalization — 2× in this paper · explained in 13 episodes
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 3 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Diffusion Model — 1× in this paper · explained in 8 episodes
• Diffusion Models — 1× in this paper · explained in 6 episodes
• Distribution Shift — 1× in this paper · explained in 7 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Flow Matching — 1× in this paper · explained in 8 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Generative Modeling — 1× in this paper · explained in 3 episodes
• Joint Attention — 1× in this paper · explained in 2 episodes
• Loss Function — 1× in this paper · explained in 2 episodes
• Representation Learning — 1× in this paper · explained in 2 episodes
• Tokenization — 1× in this paper · explained in 2 episodes

Sources

• Fast-WAM: Do World Action Models Need Test-time Future Imagination?
• Turning Video Models into Generalist Robot Policies
• WAM4D: Fast 4D World Action Model via Spatial Register Tokens
• Point Tracking Improves World Action Models
• 3PoinTr: From Human Videos to Robot Policies with 3D Point-Track Plans
• ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts
• Flex-$\pi$: A Multi-Stream World-Action Model with Compute Flexibility
• World Action Models are Zero-shot Policies
• EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data
• Cosmos 3: Omnimodal World Models for Physical AI
• Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation
• Modality Forcing for Scalable Spatial Generation
• ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning
• MV-WAM: Manifold-Aware World Action Model with Value Augmentation
• RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
• Wan: Open and Advanced Large-Scale Video Generative Models

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• ARC: A Reasoning Recipe for Robot Foundation Models
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies