V2-STRep: VLM-Grounded Structured Task Representations for Reusable Robot Skills Acquired from Generated Videos

arXiv:2609.20582 · cs.RO · Submitted 2026-09-17 · Read on arXiv

cs.RO

Submitted: 2026-09-17

Updated: 2026-09-17

Terminology

• Grounding — 26× in this paper · explained in 3 episodes
• Alignment — 5× in this paper · explained in 1 episode
• Adaptation — 4× in this paper · explained in 1 episode
• Affordance — 2× in this paper · explained in 1 episode
• Calibration — 2× in this paper · explained in 7 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Regularization — 2× in this paper · explained in 3 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• IEEE International Conference on Robotics and Automation — 1× in this paper
• Point Clouds — 1× in this paper · explained in 2 episodes
• Temporal Reasoning — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes

Sources

• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
• ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation
• OpenVLA: An Open-Source Vision-Language-Action Model
• $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
• NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos
• Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow
• PhysV2A: Reachability-Gated and Semantic-Mask-Constrained Feasibility Completion for Video-to-Robot Manipulation
• GenVid2Robot: From Video Generation to Robot Manipulation via Rigid-Geometric Consistency
• EmboAlign: Aligning Video Generation with Compositional Constraints for Zero-Shot Manipulation
• RoboReact: Agentic Skill Distillation from Generated Egocentric Videos for Generalizable Whole-Body Manipulation
• Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
• Dreamitate: Real-World Visuomotor Policy Learning via Video Generation
• OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitation
• GeoManip: Geometric Constraints as General Interfaces for Robot Manipulation
• PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
• Wan: Open and Advanced Large-Scale Video Generative Models
• RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics
• LoFTR: Detector-Free Local Feature Matching with Transformers
• SpatialPoint: Spatial-aware Point Prediction for Embodied Localization

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving