ProjectFeatured
VIMA-Bench
Multimodal promptable manipulation benchmark accompanying VIMA models.
DatasetFeatured
LIBERO
Simulation benchmark for lifelong learning and VLA eval with many manipulation tasks.
Why: Among the most used VLA evaluation benchmarks
Overview
LIBERO is a lifelong / language-conditioned robot manipulation benchmark with procedural suites (Spatial, Object, Goal, Long). It is widely used to stress compositional generalization of IL and VLA policies.
Not a “starter toy”: set up the env carefully and track success rates per suite. Datasets and eval protocols are part of the package.
Learning Path: imitation evaluation milestone after you can train a single-task policy.
Who it is for
VLA / lifelong learning authors needing standard sim eval.
Key highlights
- Many language-conditioned manip tasks
- Lifelong learning protocols
- Community comparison magnet
When to use
- Paper eval for VLA and lifelong IL
- Controlled ablations
When not to use
- Only real-world commercial deployment validation
Getting started
- 1Install LIBERO
- 2Run a published baseline
- 3Report suite-level and per-suite scores
Install / setup
git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git
Papers & reading
How it compares
★ 854
Caveats & pitfalls
- Sim success ≠ real success; always pair with real eval for systems claims.
Content reviewed 2026-07-28