Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning

arXiv:2609.37519 · cs.RO, cs.AI · Submitted 2026-09-29 · Read on arXiv

cs.RO, cs.AI

Submitted: 2026-09-29

Updated: 2026-09-29

Terminology

• Trace — 40× in this paper · explained in 2 episodes
• Robustness — 35× in this paper · explained in 5 episodes
• Grounding — 19× in this paper · explained in 3 episodes
• Signal Temporal Logic — 11× in this paper
• Ontology — 10× in this paper · explained in 2 episodes
• Reinforcement Learning — 10× in this paper · explained in 4 episodes
• Signal Temporal Logic (STL) — 5× in this paper · explained in 4 episodes
• Aggregation — 4× in this paper · explained in 1 episode
• Foundation Model — 4× in this paper · explained in 3 episodes
• Generative AI — 4× in this paper · explained in 2 episodes
• Parametric Signal Temporal Logic — 4× in this paper
• Reward Function — 4× in this paper · explained in 2 episodes
• Consistency — 3× in this paper · explained in 3 episodes
• Foundation Models — 3× in this paper · explained in 7 episodes
• Hysteresis — 3× in this paper · explained in 3 episodes
• Cross-Embodiment Transfer — 2× in this paper · explained in 2 episodes
• Formal Methods — 2× in this paper · explained in 3 episodes
• Meta-learning — 2× in this paper · explained in 2 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Action Space — 1× in this paper · explained in 1 episode
• Actor — 1× in this paper · explained in 3 episodes
• Alternation — 1× in this paper · explained in 1 episode
• Benchmark — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Credit assignment — 1× in this paper · explained in 3 episodes
• Deep Reinforcement Learning — 1× in this paper · explained in 2 episodes
• Embodied AI — 1× in this paper · explained in 2 episodes
• Energy Efficiency — 1× in this paper · explained in 3 episodes
• Hallucination — 1× in this paper · explained in 3 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Reward Shaping — 1× in this paper · explained in 2 episodes
• Self-Supervised Learning — 1× in this paper · explained in 7 episodes
• Verification — 1× in this paper · explained in 2 episodes
• Vision-Language Model (VLM) — 1× in this paper · explained in 3 episodes

Sources

• Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications
• Barkour: Benchmarking Animal-level Agility with Quadruped Robots
• Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation
• Model-based Reinforcement Learning from Signal Temporal Logic Specifications
• Proximal Policy Optimization Algorithms

Related papers

• FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
• MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
• RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
• HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
• APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
• Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving