Structure-aware Compression Diffusion Models

The conventional reverse diffusion process focuses on the log-likelihoods based denoising, which may neglect the intrinsic laws in the natural world. In this work, we propose the Structure-aware Compression Diffusion Models (SCDM), with a novel reverse diffusion process that enforces the lifting and compression operation towards learning the low-dimensional structures of the original data. Our SCDM outperforms the conventional diffusion models on both physical datasets and image datasets. Besides, the experiments demonstrate that the low-dimensional structures learned by SCDM align well with scientific theories in several physical data generation tasks.