Understanding and Improving the Training Dynamics of EDM2

Diffusion models have achieved remarkable success in image generation, yet their training dynamics can strongly affect both convergence speed and the persistence of high-quality generation. In this work, we investigate the optimization dynamics of EDM2 on small-scale image generation benchmarks and identify factors that influence its generation quality and training stability. We introduce several improvements, including learning-rate scheduling, activation statistics regulation, adaptive reconstruction objectives, and modifications to EDM2-specific components, which improve generation quality and extend the persistence of high-performance states. Furthermore, we analyze the optimization geometry induced by the EDM2 parameterization and find that, compared with an EDM-style control model, it exhibits lower sensitivity to relative parameter perturbations during early training. These results highlight the importance of considering optimization dynamics beyond final performance when improving diffusion models.