- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The papers proposes a corruption alignment of features, which tries to ensure that informativeness of each feature is equal at a given timestep. The authors extend results for text diffusion models to include numerical features in tabular data. This is achieved by simple binning of numerical features. Interestingly, these schedules are not learned alongside the model but analytically computed from meta data before training as a correction to a linear reference noise schedule. Results indicate some gains mostly in modeling correlations but this comes at the cost of much lower coverage, probably due to the model relying now on the cooccuring of feature values because of the noise schedule alignment.
Critical strengths
- It is known that noise schedules are important for the sample quality of diffusion models and this paper deals with noise schedule particularly designed for tabular data, for which there is little existing work.
- I like that the authors align the MLP backbones for a fairer benchmark comparison.
Critical weaknesses
- The motivation that a shared noise schedule leads to different levels of corruption among the features is weak. This is the reason why image diffusion models work so well (coarse to fine structure). It is not clear why this is a problem for tabular data and there is no evidence given that this hypothesis leads to an increased difficulty in learning the joint distribution. It could also be beneficial for the model to generate "easy-to-generate" features first, and then the more difficult features. This would require a non-aligned noise schedule.
- The code base does not use the official implementations of the baseilnes but is fully AI generated. This leads to major deviations from the actual models. For example, continuous time models, like CDTD, are still confined to a grid of only 500 timesteps, instead sampling uniformly. This is a major shortcoming, in particular since the code for these models is readily available on github. Visual results in Figures 16, 17, 20 and 21 clearly show that Vanilla, CDTD, CDCD, TabDDPM and TabSyn have not converged, since the generated distribution often stay close to Gaussian. Hence, I do not trust the main benchmark results.
- No evaluation of C2ST (detection score), despite it being the most important metric to evaluate the quality of the joint distribution and being included in all recent papers. Compared to other papers, the selection of reported metrics is very limited. There is also no metric about privacy.
Questions for authors
- Why are different levels of corruption bad for tabular data? Is there any evidence that sample quality decreases when not aligning corruption?
- From Figure 2 and Equation 4c, I see that discrete-noise schedules are always (approximately) aligned. So following that logic, I should always see that models with discrete-noise schedules outperform Gaussian-based diffusion for tabular data?
- You mention in line 204 that each feature has the same proportion of its maximum achievable entropy. How does entropy come into play here? In Definition 1 the corruption is not defined in terms of entropy. How does this compare to the entropy-based noise schedules (see https://arxiv.org/abs/2504.13612)?
- How is CDCD implemented? Since the original implementation is only for categorical data. Is the CDTD implementation reused with the CDCD time-warping?
- Why do you not rely on the official model implementations and meaningful changed the default hyperparameters of the baselines?