- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors proposed a GBDT-based diffusion model for tabular data generation. Their main contribution is a round-wise noise refresh mechanism that allows XGBoost to optimize stochastic denoising objectives without explicitly expanding the training dataset. Experiments demonstrate competitive generative fidelity and improved computational scalability compared with existing GBDT-based diffusion methods
Critical strengths
-
The proposed noise-refresh mechanism addresses a computational bottleneck in GBDT-based diffusion training. The experiments demonstrate substantial reductions in training time compared with dataset duplication while maintaining competitive generation quality.
-
The paper compares against several established tabular generative models across multiple datasets, including a large-scale benchmark. The ablation studies also provide useful evidence for the effectiveness of the proposed training strategy.
Critical weaknesses
One of the main concerns is that the theoretic analysis establishes that the stochastic empirical loss is an unbiased estimator of the expected diffusion loss, conditioned on the current boosted model. However, it alone does not guarantee that the decision tree fitted using the refreshed noise provides a descent direction for the expected objective. In particular, the newly fitted tree depends nonlinearly on the sampled noise realizations, and unbiased loss or gradient estimation does not imply unbiasedness of the resulting boosting update. Then, neither expected loss reduction nor convergence of the proposed stochastic boosting procedure is established. Since round-wise noise refresh is the central methodological contribution, the absence of a theoretical characterization of its optimization behavior limits the mathematical foundation of the proposed approach.
Questions for authors
1.Can the authors establish sufficient conditions under which the proposed stochastic boosting update yields an expected decrease in the diffusion objective? Is it possible to provide a convergence or stability analysis accounting for the stochastic noise refresh and the approximation error introduced by fitting decision trees?
-
Table 4 shows increased MIA Lift following the introduction of noise refresh. Can the authors clarify this potential fidelity–privacy trade-off and provide a direct statistical comparison?
-
Clarify whether computational comparisons use equivalent hardware and training budgets, particularly when comparing CPU-based XGB-DDPM with neural diffusion models.
4 comments
Dear reviewer uP9a,
we’d be glad to hear any remaining questions you may have regarding our work. We have put substantial effort into addressing your concerns by updating the Methods section of the paper. If you feel we’ve substantially addressed your concerns, we kindly request to consider reflecting this in your score.
Thanks again for your review of our work.
Thanks for your reply, but I incline to keep my grade.
Dear reviewer uP9a,
Could you please elaborate?
We have put substantial effort into addressing your only mentioned weakness of the paper by incorporating new proofs. It seems odd that this would not influence your rating of this work.
We are glad to hear any remaining concerns or questions regarding this matter.
Thanks again for your review and consideration of our work.
Thanks for your insightful review of our work. Below we will address the mentioned Critical Weakness, and the questions.
Critical Weaknesses and Question 1
We agree with these comments. Therefore, we have included a new paragraph in the Methods section entitled “Optimization across noise-refreshed boosting rounds”. There, we show the theoretical justification for how and when the noise-refreshed boosting trajectory decreases the expected diffusion objective (and when trees in individual boosting rounds provide a descent direction).
In short, the justification follows from 3 assumptions:
We show that these 3 conditions imply that the noise-refreshed boosting procedure decreases the expected diffusion objective at each round; in particular for small enough learning rate. This directly ties in to the observation we already made regarding the learning rate, i.e., that learning rate should be set small enough so that no individual noise realization disproportionately affects the full model.
Note that, to account for the increased space this theory takes up, we moved the diffusion specifications (i.e., the math for Gaussian and multinomial diffusion) to the Appendix; this was mainly reiterating past work anyway.
Question 2
Since the noise-refresh procedure improves the estimation of the diffusion objective, it improves the accuracy of the learned generative distribution (i.e., increases fidelity). Since we model the training distribution more accurately, the generated points are prone to be more similar to the training data than before. If we accurately reproduce identifiable portions of the training data, this can yield increased privacy risk. Note that this is purely a result of more accurately modelling the distribution, but in particular, of portions of the distribution which are typical of the training data (when compared to the held-out test data).
This is the typical fidelity-privacy tradeoff which is well-known for tabular generative models. You can also observe it in the other baseline models in Table 2: high-fidelity baselines such as TabDDPM and CDTD score relatively poorly in terms of MIA Lift. Finally, we’d like to point out that our method provides a better fidelity-privacy tradeoff than other competitive baselines such as TabDDPM and CDTD in our experimental datasets (see Table 2 and Figure 4).
Question 3
Hardware was specified in Appendix B.4. Naturally, we cannot use equivalent hardware for CPU-based models, such as XGB-DDPM, and the neural diffusion models; we use CPUs for the CPU-based models and GPUs for neural diffusion models (actually all methods based on deep neural networks). See Appendix B.4 for the specific hardware used. The hardware was selected to provide a realistic expectation of how these methods may perform in somewhat restricted hardware settings; in particular, XGB-DDPM could be majorly scaled up by using larger CPU clusters.
Thanks again for your thorough review.