Sparse Data Diffusion via Sparsity Bits

Sparse data is fundamental to scientific simulations in physics and biology, from particle physics experiments to single-cell RNA sequencing, where exact zeros encode absence rather than a weak signal. However, existing generative models struggle to faithfully capture exact zeros, often blurring absence with weak signals, calling for an explicit representation of exact zeros. We introduce Sparse Data Diffusion (SDD), a novel method for generating sparse data. SDD extends continuous state-space diffusion models with an explicit representation of exact zeros by modeling sparsity through the introduction of Sparsity Bits. Empirical validation in particle physics and single-cell RNA sequencing demonstrates that SDD achieves higher quality than baselines in capturing the sparse patterns inherent to these domains, while preserving the quality of the generated data.