Diffusion-Guided Adversarial Imitation Learning

Diffusion models have emerged as a powerful tool for learning and representing distributions, yet their role in adversarial imitation learning (AIL) has remained limited to acting as policy or reward generators. We propose **Diffusion-Guided Adversarial Imitation Learning (DGAIL)**, a new framework that leverages the distribution-matching capability of diffusion models for adversarial imitation learning by training a vector-field discriminator under the guidance of a pre-trained expert score network. Our key insight is an adversarial form of the Fisher divergence underlying diffusion models, which brings the explicit score signal into the adversarial imitation framework. Theoretically, we show that the game value attained by any policy certifies a bound on its true divergence under restricted discriminator classes, and that this bound transfers to an instance-dependent imitation gap guarantee under any bounded evaluation cost. We evaluate our method on MuJoCo continuous control tasks. Experimental results demonstrate that DGAIL matches or outperforms strong AIL baselines.