HiPHI: A Large-Scale Benchmark for High-Precision Human Motion and Object-Interaction

arXiv:2608.16222 · cs.RO, cs.AI · Submitted 2026-08-17 · Read on arXiv

cs.RO, cs.AI

Submitted: 2026-08-17

Updated: 2026-09-08

Comments: Accepted at CoRL 2026. Project page: https://noitom-robotics.github.io/hiphi/

Code: https://github.com/NVLabs/ProtoMotions

Project page: https://noitom-robotics.github.io/hiphi

License: http://creativecommons.org/licenses/by/4.0/

The gist: Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions.

Terminology

Abstract

Humanoid intelligence requires learning over an extremely diverse space of whole-body motions and physically grounded interactions. However, existing embodied datasets remain fundamentally limited: internet-scale video data lack precise physical states and interaction grounding, while laboratory motion datasets provide high fidelity but only narrow behavioral coverage. This mismatch creates a critical bottleneck for scalable humanoid policy learning. We present HiPHI, a 600+ hour scale high-fidelity whole-body human motion dataset designed to systematically maximize coverage of the human motion and interaction manifold. HiPHI is theoretically guided by FrameNet, a linguistic framework organizing human primitives. Created using an optical motion capture pipeline, HiPHI provides sub-millimeter spatial marker tracking accuracy for full-body human motion and mesh-level object trajectories. We further introduce a benchmark suite evaluating motion-space diversity, interaction grounding, object consistency, and physical AI applications. Our analyses demonstrate that HiPHI significantly expands motion coverage compared to existing motion datasets while maintaining high-fidelity interaction quality, and establishes a scalable data foundation for training, evaluating, and generalizing humanoid policies in real-world embodied tasks, where similar extensions are also applicable to motion prior models in computer graphics. Project page: https://noitom-robotics.github.io/hiphi/

Sources

Related papers