XGB-DDPM: Noise-Refreshed Gradient Boosting for Scalable Diffusion on the CPU

We present a method to optimize stochastic denoising objectives with Gradient-Boosted Decision Trees (GBDTs) by refreshing noise realizations across boosting rounds. Previous efforts relied on expanding the training data over many noise realizations, incurring extreme memory and runtime costs, especially for already-large datasets. We implement this method in an XGBoost fork and use it to construct XGB-DDPM: a DDPM-style tabular diffusion model combining Gaussian and multinomial diffusion for numerical and categorical features, which are denoised by XGBoost regressors and classifiers. Here, XGBoost leverages the round-wise noise refresh to estimate the diffusion loss. We show that XGB-DDPM can be trained more efficiently and scales to much larger datasets than previous GBDT-based diffusion models which expand the training data to estimate the diffusion loss. Additionally, we show that XGB-DDPM achieves similar fidelity to neural tabular diffusion models (e.g., TabDDPM, TabSyn, TabDiff, and CDTD) without increasing memorization or membership disclosure risk. XGB-DDPM is a good alternative for practitioners who want to train diffusion models on potentially large data but do not have access to GPUs.