Dataset
RH20T
Large multi-modal real manipulation dataset with multiple sensors and task types.
Dataset
HOI4D
Large egocentric human–object interaction 4D dataset for hands, manipulation, and perception.
Overview
HOI4D is a large egocentric human–object interaction 4D dataset for hands, manipulation, and embodied perception. It is more “human video + 4D annotation” than robot teleop, making it valuable for pretraining perception or studying human priors before robot transfer.
Who it is for
Egocentric perception; hand-object interaction; human-to-robot transfer researchers.
Key highlights
- Egocentric 4D HOI scale
- Hand-object focus
- Perception pretrain signal
When to use
- Egocentric pretrain
- Hand pose / interaction learning
When not to use
- Direct robot joint-action BC without retargeting
Getting started
- 1Download official split
- 2Visualize 4D annotations
- 3Define a clear transfer protocol to robots
Papers & reading
Caveats & pitfalls
- Not a robot action dataset out of the box.
Content reviewed 2026-07-23