ProjectFeatured
LeRobot
Hugging Face open-source stack for embodied AI with unified data formats and training — great for imitation learning and VLA experiments.
PythonBeginnerUpdated 2026-07-23
Learning path
A “zero to working pipeline” route for beginners. We do not rewrite full tutorials — we chain high-quality resources and mark checkpoints and pitfalls.
- 1
1. Foundations
Goal: Build a shared vocabulary: embodied AI, imitation, RL, and VLA.
Core concepts
- Embodied AI vs pure vision/language models
- Observation, action, policy, proprioception
- Sim-to-real gap
Recommended resources
★ 26.1k★ 14.3kProjectFeaturedMuJoCo
Classic physics engine widely used in robot RL and control research — lightweight and accurate.
C++ / PythonIntermediateUpdated 2026-07-22DatasetFeaturedOpen X-Embodiment
Large real manipulation corpus across 22 embodiments (1M+ trajectories) — key VLA pretrain base.
Checkpoint
Explain in your own words what a VLA is and why robot data matters.
Common pitfalls
- Do not chase the largest model first — clarify task and data.
- When jargon piles up, align on 1–2 surveys or tutorials first.
- 2
2. Simulation intro
Goal: Run an env end-to-end: random policy → a simple training loop.
Core concepts
- Physics engines and rendering
- Parallel / GPU simulation
- Task definitions and metrics
Recommended resources
★ 14.3kProjectFeaturedMuJoCo
Classic physics engine widely used in robot RL and control research — lightweight and accurate.
C++ / PythonIntermediateUpdated 2026-07-22★ 7.7kProjectFeaturedIsaac Lab
NVIDIA robot learning framework on Isaac Sim for large-scale parallel training and sim-to-real.
PythonIntermediateUpdated 2026-07-23★ 3.1kProjectFeaturedManiSkill
Large-scale manipulation benchmark and training env with GPU parallelism and diverse tasks.
★ 943ProjectGymnasium-Robotics
Farama robot RL environments compatible with the Gymnasium API.
PythonBeginnerUpdated 2026-07-20Checkpoint
Launch at least one simulator locally or in the cloud and collect a few trajectories.
Common pitfalls
- Isaac is hardware-heavy; start with MuJoCo / ManiSkill if needed.
- Make the env reproducible before algorithms.
- 3
3. Imitation learning
Goal: Train a visuomotor policy from demos; respect data quality.
Core concepts
- Behavior cloning / ACT / Diffusion Policy
- Data formats and normalization
- Success rate vs training loss
Recommended resources
★ 26.1kProjectFeaturedLeRobot
Hugging Face open-source stack for embodied AI with unified data formats and training — great for imitation learning and VLA experiments.
PythonBeginnerUpdated 2026-07-23★ 4.4kProjectFeaturedDiffusion Policy
Seminal visuomotor policy work that uses diffusion models for robot actions, with open code.
★ 2.1kProjectFeaturedACT (Action Chunking with Transformers)
Action Chunking with Transformers for low-cost bimanual imitation; popular with ALOHA-style hardware.
★ 1.5kProjectrobomimic
Imitation-learning research library from Stanford/TRI with algorithms, datasets, and eval pipelines.
DatasetFeaturedLIBERO
Simulation benchmark for lifelong learning and VLA eval with many manipulation tasks.
Checkpoint
Finish one imitation train+eval loop in sim or on a low-cost platform.
Common pitfalls
- Lower loss ≠ task success — always roll out.
- Action rate, latency, and camera extrinsics matter a lot.
- 4
4. VLA finetuning
Goal: Understand pretrained VLAs and adapt on modest data.
Core concepts
- Open X-Embodiment and pretraining
- Frozen backbone vs full finetune
- Language conditioning and cross-task transfer
Recommended resources
ModelFeaturedOpenVLA
Open Vision-Language-Action model trained on Open X-Embodiment for diverse manipulation.
7BFully openData: Open X-EmbodimentUpdated 2026-02-17ModelFeaturedOcto
Generalist robot policy Transformer with multi-modal inputs and multi-embodiment finetuning.
93MFully openData: Open X-EmbodimentUpdated 2024-07-31★ 12.9kProjectFeaturedopenpi
Open training and inference code around π0 from Physical Intelligence for generalist robot policies.
ModelFeaturedπ0 (pi0)
Physical Intelligence generalist robot foundation model with flow-matching action generation.
DatasetFeaturedBridgeData V2
Large household WidowX dataset (~60K trajectories) with rich language conditioning.
DatasetFeaturedDROID
Highly diverse real Franka manipulation data (~76K trajectories / 350h) for real-world finetune.
Checkpoint
Name a VLA’s training data, I/O interface, and finetune entrypoint.
Common pitfalls
- Limited GPU: prefer small models / LoRA / official configs.
- Check data and weight licenses separately.
- 5
5. Simple real robot
Goal: Close the loop: collect → train → deploy on low-cost or lab hardware.
Core concepts
- Teleop and data collection
- Safety limits and e-stop
- Domain randomization (optional)
Recommended resources
★ 2.3kProjectALOHA
Low-cost bimanual hardware and teleop stack paired with ACT-style algorithms.
PythonBeginnerUpdated 2024-04-19★ 2.1kProjectFeaturedACT (Action Chunking with Transformers)
Action Chunking with Transformers for low-cost bimanual imitation; popular with ALOHA-style hardware.
★ 26.1kProjectFeaturedLeRobot
Hugging Face open-source stack for embodied AI with unified data formats and training — great for imitation learning and VLA experiments.
PythonBeginnerUpdated 2026-07-23DatasetFeaturedDROID
Highly diverse real Franka manipulation data (~76K trajectories / 350h) for real-world finetune.
Checkpoint
Collect real demos and deploy a usable policy (even for a tiny task).
Common pitfalls
- Safety first: joint limits, collision checks, human-in-the-loop.
- Fix cameras and calibration before tuning algorithms.