Scene2Demo: Self-Evolving Embodied Data Generation via Object-Action Graph
cs.RO
Submitted: 2026-02-12
Updated: 2026-10-17
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present Scene2Demo, an automated framework for offline embodied data generation.
Terminology
Abstract
We present Scene2Demo, an automated framework for offline embodied data generation. Given a single RGB scene image and a user query, Scene2Demo constructs an interactive simulated scene and generates executable task configurations, primitive action flows, multi-view execution videos, and offline robot-learning datasets. Its five role-specialized generation modules perform task understanding, symbolic decomposition, BDDL configuration generation, object-size adjustment, and action-flow planning. The resulting object-centric task specifications and action transitions are organized as path search over a scene-conditioned object-action graph. Failed or incomplete executions are refined by two feedback agents: a Robotic Safety Inspector that produces step-wise visual critiques and a Robotic Task Supervisor that revises action sequences or parameters using the rollout and previous evolution history. Across 102 automatically generated primitive scene-task pairs, Scene2Demo achieves a 71.6% execution success rate; on four representative long-horizon tasks, feedback-driven refinement improves both task success and subtask-level execution quality over primitive-only execution. Comparisons with RoboGen and GenSim2 show stronger execution performance under automated data-generation settings. Finally, behavior cloning policies achieve 96.0% and 92.0% success on two representative tasks, showing that the generated data supports downstream policy learning. Our project page is available at https://scene2demo-anon.github.io/.
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- ARC: A Reasoning Recipe for Robot Foundation Models
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies