- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(−1)
Read review
Paper summary
This paper studies training-free decoding in masked diffusion language models, where individually confident token predictions may be incompatible when committed together. At each decoding step, ACE tests candidate rankings based on confidence, attention, and left-to-right text position. For each ranking, it uses native attention to score a candidate's interactions with candidates ranked ahead of it, even if they are not selected. Depending on the configuration, the selection score also includes a readiness penalty and an order prior. These scores stay fixed during selection, and ACE sorts them and selects as the number of tokens that fit the budget. Experiments on LLaDA and Dream show improved accuracy-efficiency tradeoffs.
Critical strengths
The fixed-cost selector is a useful alternative to sequential compatibility checks. Its scores include interactions with predecessors that may never be selected, so it could exclude too many candidates and require more denoiser calls. I find the matched Dream comparison convincing on this point. At the default and quality settings, ACE retains accuracy and denoiser-call counts close to greedy selection despite using these conservative scores (Figure 3 and Table 17).
Critical weaknesses
I found it difficult to follow how the theory in Sections 4–5 leads to ACE. Section 5 introduces attention-based scores and states a guarantee for the attention-induced game, but the set cost covered by that guarantee is defined only in Appendix A.8, Equation (35). The main text leaves the reader to infer what the practical scores bound or consult the appendix. That definition and its connection to the scores belong in the main method description, where the reader is trying to understand what the theory establishes for the decoder.
Questions for authors
None
3 comments
After checking the response and checking the additional experiments added in responses to other reviewers, my outstanding questions have been addressed. That said, I am maintaining my score.
Dear Reviewer PJxX,
Thank you very much for your recognition and positive assessment of our manuscript. Your positive score is the greatest support and encouragement for our work. We sincerely appreciate the time and effort you dedicated to reviewing our paper.
Best regards,
The Authors
Response to Reviewer PJxX
We thank the reviewer for the positive assessment and for finding the matched Dream comparison convincing. Table and figure numbers below refer to the revised PDF.
Weakness: how the theory in Sections 4–5 leads to ACE
Section 5 now defines the set cost covered by the guarantee and states the bound itself, so the main method description says exactly what the practical scores bound.
The definition now introduces the ACE scores (Equation 7), as their ordered marginal costs:
The section's closing paragraph now states the bound, followed, as before, by the condition under which it also controls the true dependence cost:
The reader therefore sees, where the scores are defined, that the one-shot prefix certifies the attention game's cost for every coalition it can select. Appendix A.8 keeps the formal statement and proof, and the additions are marked in blue in the revised PDF. Thank you for this suggestion.
We are happy to answer any further questions.