Embodied AI HubEmbodied AI Hub
ProjectFeaturedIntermediate

Diffusion Policy

Seminal visuomotor policy work that uses diffusion models for robot actions, with open code.

Official / sourcePaperBack to Project
Imitation learningOpen sourceReal robot

Why we recommend it

Diffusion Policy is a standard visuomotor imitation method with strong educational value. Recommended to learn demo-to-policy loops in sim before jumping to VLAs. Intermediate; follow the project page for code and configs.

Overview

Diffusion Policy showed that modeling action sequences with diffusion models yields strong visuomotor policies for contact-rich manipulation. The open code and follow-up ecosystem made it a default modern baseline alongside ACT.

Conceptually, it treats action generation as iterative denoising, which handles multi-modal action distributions better than plain MSE BC. Practically, you will care about observation horizons, action horizons, and inference latency — diffusion steps can be expensive on real robots unless distilled or accelerated.

Use it when demonstrations are multi-modal (multiple valid ways to solve a task) and you need smooth action chunks.

Who it is for

Visuomotor IL practitioners; teams with multi-modal demos; researchers comparing generative action models.

Key highlights

  • Strong baseline for multi-modal action distributions
  • Open training code widely forked and extended
  • Works with image observations and action chunking
  • Influential follow-ups in real-robot systems

When to use

  • Demos show multiple strategies for the same goal
  • You need smooth multi-step action chunks
  • You are comparing generative policies vs ACT / BC-RNN

When not to use

  • Hard real-time constraints with no budget for inference optimization
  • Ultra-low-data regimes where a tiny ACT model is enough

Getting started

  1. 1Clone the official repo and run the sim demo configs first.
  2. 2Match camera resolution and control frequency to your robot.
  3. 3Start with fewer denoising steps; measure task success vs latency.
  4. 4Log rollouts — loss curves alone mislead for generative policies.

Install / setup

clone

git clone https://github.com/real-stanford/diffusion_policy.git

Papers & reading

How it compares

Best when demos are multi-modal; higher latency than ACT unless distilled.

Caveats & pitfalls

  • Inference cost can dominate on edge GPUs.
  • Hyperparameters (horizon, noise schedule) strongly affect stability.

Content reviewed 2026-07-23