EmboAlign: Aligning Video Generation with Compositional Constraints for Zero-Shot Manipulation

arXiv:2603.05757 · cs.RO · Submitted 2026-03-05 · Read on arXiv

cs.RO

Submitted: 2026-03-05

Updated: 2026-09-16

Terminology

• World Model — 10× in this paper · explained in 7 episodes
• Generative Model — 9× in this paper · explained in 2 episodes
• Generative Models — 7× in this paper · explained in 2 episodes
• Reinforcement Learning — 7× in this paper · explained in 4 episodes
• Alignment — 6× in this paper · explained in 1 episode
• Conference on Robot Learning — 6× in this paper
• Ablation — 5× in this paper · explained in 1 episode
• Failure Modes — 5× in this paper · explained in 3 episodes
• Foundation Model — 5× in this paper · explained in 3 episodes
• Hallucination — 5× in this paper · explained in 3 episodes
• Physical Constraints — 5× in this paper · explained in 7 episodes
• Grounding — 4× in this paper · explained in 3 episodes
• Foundation Models — 3× in this paper · explained in 7 episodes
• Conference on Computer Vision and Pattern Recognition — 2× in this paper
• International Conference on Learning Representations — 2× in this paper
• Latent Representation — 2× in this paper · explained in 2 episodes
• Latent Representations — 2× in this paper · explained in 3 episodes
• Latent World Models — 2× in this paper · explained in 2 episodes
• Reward Function — 2× in this paper · explained in 2 episodes
• Unified Framework — 2× in this paper · explained in 5 episodes
• Vision-Language Models (VLMs) — 2× in this paper · explained in 6 episodes
• Ablation Studies — 1× in this paper · explained in 1 episode
• Ablation study — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Core — 1× in this paper · explained in 2 episodes
• Digital Twin — 1× in this paper · explained in 4 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• International Conference on Machine Learning — 1× in this paper
• Robustness — 1× in this paper · explained in 5 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes

Sources

• Large Video Planner Enables Generalizable Robot Control
• Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations
• CreFlow: Corrective Reflow for Sparse-Reward Embodied Video Diffusion RL
• VideoAgent: Self-Improving Video Generation
• Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow
• Robot Learning from a Physical World Model
• Video as the New Language for Real-World Decision Making
• Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
• Envision: Embodied Visual Planning via Goal-Imagery Video Diffusion
• DreamGen: Unlocking Generalization in Robot Learning through Video World Models
• NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos
• Spatiotemporal Predictive Pre-training for Robotic Motor Control
• Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations
• Unified Video Action Model
• RoboDreamer: Learning Compositional World Models for Robot Imagination
• Grounding Generated Videos in Feasible Plans via World Models
• Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection
• SafeBimanual: Diffusion-based Trajectory Optimization for Safe Bimanual Manipulation
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective
• SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving