STEP: State-Aware Task Estimation and Planning with Multi-Modal LLMs for Human-Robot Collaboration
cs.RO, cs.AI
Submitted: 2026-08-27
Updated: 2026-08-27
Terminology
Sources
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- OpenVLA: An Open-Source Vision-Language-Action Model
- Forecasting Future Action Sequences with Neural Memory Networks
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- Reasoning with Language Model is Planning with World Model
- Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency
- LLM-State: Open World State Representation for Long-horizon Task Planning with Large Language Model
- Making Large Language Models into World Models with Precondition and Effect Knowledge
- WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents
- Hierarchical Deep Learning for Intention Estimation of Teleoperation Manipulation in Assembly Tasks
- How Can We Know What Language Models Know?
- Generate & Rank: A Multi-task Framework for Math Word Problems
- Pretrained Transformers as Universal Computation Engines
- EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
- Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving