- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes GADD, a Gibbs-based corrector for uniform-rate discrete diffusion models that uses an existing score model without additional training. It provides convergence guarantees under stated assumptions and evaluates the method on synthetic data, text generation, and conditional music generation.
Critical strengths
-
GADD improves sampling without additional training. The reported gains in both text perplexity and runtime make it a useful option for existing models.
-
The convergence analysis gives the method a solid theoretical motivation, with polylogarithmic dependence on target accuracy under the stated assumptions. Experiments on synthetic data, text, and music also support its usefulness.
Critical weaknesses
-
The text experiments use sequences of length 128. Although the appendix includes small synthetic dimension sweeps, these do not establish how the quality and runtime gains scale to longer text. Since Gibbs correction updates individual coordinates, increasing sequence length may change the correction budget needed to maintain quality and its efficiency relative to predictors that update all tokens. Evaluating sequences of 512 or 1,024 tokens, with both generation quality and measured runtime at comparable NFE budgets, would help assess whether the reported advantages persist.
-
Text evaluation relies mainly on generative perplexity, which may overlook reduced diversity. Could the authors also report token entropy or repetition rates at matched NFE?
Questions for authors
See weaknesses.
2 comments
Thanks for the comments. The additional results on diversity and longer sequences alleviate my concerns, though the wall-clock runtime comparison at 512 tokens is still missing. I maintain my original assessment.
We thank the reviewer for carefully reading our work and providing insightful feedback, which has led to several additions to the paper.
W2: On diversity
We agree that diversity should be evaluated alongside perplexity. The default GADD setting achieves the lowest perplexity but exhibits lower lexical diversity. This tradeoff can be adjusted through the correction parameters: reducing the number of correction updates from 40 to 20 and setting the threshold to 0.2 substantially recovers diversity. Compared with Campbell, this setting achieves 12.1% lower perplexity, with approximately lower token entropy and lower Distinct-2 rate, while mean 4-gram repetition rate remains low at 0.20%. These results demonstrate an adjustable quality–diversity tradeoff, with the ablation retaining a perplexity advantage while substantially reducing the diversity loss.
Results below are mean ± standard deviation across three seeds, with 128 generated sequences per method and seed. Perplexity and diversity are evaluated on the same samples.
W1: On longer text sequences
We thank the reviewer for suggesting experiments with longer sequences. We extended our text experiments from d=128 to d=512, comparing all methods using the same checkpoint at NFE=256. Results below report mean ± standard deviation across three seeds, with 128 generated sequences per method and seed. All methods additionally use one common final denoising evaluation.
GADD retains its perplexity advantage at this longer sequence length, reducing perplexity by 46.0% relative to Campbell, the strongest baseline. The milder correction setting offers a different quality–diversity balance: it retains a 21.4% perplexity improvement, with 4.4% lower token entropy, 16.0% lower Distinct-2, and average 4-gram repetition below 1%. These results demonstrate that the perplexity benefit persists at four times the original sequence length, while also making the diversity trade-off explicit.
We hope that our response above has addressed your concern, and we would appreciate that you could raise your rating if your concern has been resolved.