- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work introduces a discrete generative model with reverse transitions mixing the following actions: preserving the current state, predicting the clean token, and resampling from a categorical prior. The paper derives a continuous-time training objective for the zero-resampling setting which recovers masked diffusion as a special case. Experiments on molecular graph generation and text generation show that the proposed method achieves competitive performance with fewer sampling steps
Critical strengths
- Simple construction for controllable resampling.
This method uses the same transition rule with masking, uniform noise, or noise sampled according to category frequencies in the data. This is useful for graph generation where nonuniform priors matter.
- Strong results on both graphs and text.
The results show strong performance on both GuacaMol and GenPPL on OWT.
Critical weaknesses
- When do the theoretical results apply and what does decoupling guarantee are unclear.
Equation 14 is derived only for . The main text should make this clear. Extending this to positive resampling seems to need more analysis.
Also, changing the sampler without retraining does not guarantee good performance.
- Molecular results show improvements in some settings but not all.
On MOSES, the full step model has worse FCD than DiGress and Cometh, and lower validity than DeFoG. Best QM9 configuration uses no resampling, so it does not demonstrate a benefit from resampling itself.
Questions for authors
Please address the weakness.
3 comments
Thank you for the clarifications and additional analyses. I see that the revised paper now makes the restriction explicit, and the response clarifies that decoupling provides inference-time flexibility without guaranteeing improved sample quality. The response on dataset and budget dependent benefits also addresses my concern. Although the positive-resampling ELBO analysis remains open, these clarifications largely address my original concerns. Thus I maintain my score for weak acceptance.
We thank the reviewer for their response and are glad that their concerns have been addressed.
We thank the reviewer for their valuable comments. In the following, we address their points.
We thank the reviewer for raising this point.
The main benefit of decoupling is the flexibility to adjust sampling without retraining. In standard uniform and absorbing diffusion models, the reverse transition is fixed by the forward process through Bayes’ rule. In contrast, IDDM constructs a family of valid reverse kernels, parametrized by λ, that preserve the same prescribed marginal path. Because the training marginals are independent of λ, a single trained denoiser can be reused with different resampling schedules. This enables adaptation to the task and computational budget through inference-time tuning. Our construction applies to arbitrary categorical priors, including masking, uniform, and empirical priors.
We agree that this flexibility does not guarantee improved sample quality. However, our experiments demonstrate benefits across the evaluated priors and domains. These improvements are particularly pronounced in molecular generation at reduced sampling budgets. At 50 steps, compared with DeFoG, IDDM reduces QM9 FCD from 0.26 to 0.11. On GuacaMol, it improves validity from 91.7% to 98.1% and FCD score from 57.9 to 77.6. On MOSES, it improves validity from 83.9% to 87.1% while reducing FCD from 1.87 to 1.14. Our language experiments further show that controlled resampling improves generation quality under uniform priors and narrows the performance gap to masked diffusion language models.
Finally, we updated our manuscript to clarify that the continuous-time ELBO is derived for λ=0 under arbitrary categorical priors. We further updated our discussion to state that extending this analysis to positive resampling remains an interesting direction for future work.
We agree that the improvements depend on the dataset, number of sampling steps, and evaluation metric. While on QM9, our best FCD is achieved without resampling, our ablations on MOSES and GuacaMol demonstrate benefits from positive λ at 50 steps. On MOSES at 500 steps, resampling improves validity and SNN by trading off FCD.
Further, IDDM provides a unified framework that connects existing methods by extending ReMDM-style correction [1] to arbitrary categorical priors and recovering SID [2] as the special case λ=1. Specifically, IDDM derives a family of marginal-consistent reverse kernels parameterized by λ, allowing the resampling strength to be flexibly adjusted at inference time using a single trained model.
We again thank the reviewer for their feedback and are happy to clarify any remaining questions or suggestions.
[1] Wang, Guanghan, et al. "Remasking discrete diffusion models with inference-time scaling." NeurIPS (2025).
[2] Boget, Yoann. "Simple and critical iterative denoising: A recasting of discrete diffusion in graph generation." ICML (2025).