CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Image
cs.RO, cs.CV, cs.LG
Submitted: 2026-09-22
Updated: 2026-09-22
Comments: 8 pages, 7 figures, 3 tables. Project page: https://dongwon-son.github.io/coda-project-page/
Code: https://github.com/microsoft/TRELLIS.2
Project page: https://dongwon-son.github.io/coda-project-page
License: http://creativecommons.org/licenses/by/4.0/
The gist: Robots operating safely in cluttered everyday environments often need to infer scene geometry from partial observations.
Terminology
Abstract
Robots operating safely in cluttered everyday environments often need to infer scene geometry from partial observations. Methods that detect objects in 2D and reconstruct them independently struggle in such scenes: a missed object is never reconstructed, a merged detection can fuse two objects, and separately reconstructed meshes may overlap or fail to touch their supporting surfaces. We introduce CODA (Complete Once, Decompose Afterward), a generative model that instead reconstructs the complete scene geometry from a single unsegmented RGB-D image, then separates the surface into the surrounding environment and movable objects. Still, generated scene geometry can drift from the observed partial point cloud. To reduce this drift, CODA uses two explicit 3D grounding mechanisms to keep reconstructed geometry consistent with observed surfaces while completing unseen regions. Experiments on HomebrewedDB and our custom cluttered-scene dataset show more accurate reconstructions and a higher fraction of objects remaining in place under simulated gravity than both object-first and scene-first baselines.
Sources
- GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction
- Which Reconstruction Model Should a Robot Use? Routing Image-to-3D Models for Cost-Aware Robotic Manipulation
- GraspFoM: Towards Reconstruction-Driven Robotic Grasping with 3D Foundation Priors
- DINOv3
- SAM 3: Segment Anything with Concepts
Related papers
- FMT x: An Efficient and Asymptotically Optimal Extension of the Fast Marching Tree for Dynamic Replanning
- MPCFormer: A physics-informed data-driven approach for explainable socially-aware autonomous driving
- RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies
- HRDexDB: A 4D Dexterous Grasping Dataset Across Human and Multiple Robot Embodiments
- APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
- Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving