- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper measures how confidence-weighted position selection affects masked-diffusion decoding at fixed model-call budgets. A Plackett–Luce family interpolates between uniform selection and hard confidence ranking, with a fixed-count control used to isolate ordering on 124M models. The study also varies token revision, time grids, temperature, and diversity, and reports transfer to LLaDA-8B-Base. The main positive result is a scorer-perplexity improvement from soft weighting at selected operating points; revision helps the small model but hurts LLaDA.
Critical strengths
-
The study makes several distinctions that are often obscured in decoding comparisons: model calls versus tokens committed per call, order versus revision, and scorer quality versus diversity. The fixed-count β=0 control is particularly useful. Multiple decode seeds, alternative scorers, and explicit accounting for extra revision calls make the small-model comparisons more informative than a single sampler leaderboard.
-
The paper reports important negative and conditional findings. The preferred weighting depends on temperature; revision does not transfer successfully to LLaDA; and a diversity-calibrated nucleus baseline outperforms one-shot revision on both scorer perplexity and MAUVE. These results help delimit when a decoding intervention is useful and give the work value as an empirical study.
Critical weaknesses
-
The key controlled finding has limited support in a practical generation regime. The fixed-count β=0 and hard-ranking controls are evaluated on internal 124M checkpoints whose outputs the paper itself describes as far from fluent. On LLaDA, soft order changes both position weighting and the count rule relative to the ancestral baseline; Appendix B explicitly acknowledges that neither control was run there. Thus, the public-model result does not establish that confidence weighting itself causes the reported improvement or that the small-model preference over hard ranking transfers. Since the contribution is a controlled measurement of an existing sampling family, completing that identification on a useful public model is important to its significance.
-
The quality conclusion remains tied to proxy operating points. Larger scorers preserve several rankings, which is helpful, but perplexity under a reference language model does not establish useful generation quality. Matching outputs after generation on one bigram statistic does not hold the output distribution fixed, and joint diversity matching is too small to resolve the effect. The nucleus comparison also shows that the apparent revision advantage can reverse under another reasonable diversity control. I appreciate that these limitations are disclosed, but the current evidence does not yet establish enough new knowledge about useful quality–diversity trade-offs to support acceptance. I therefore lean toward rejection.
Questions for authors
-
On LLaDA, does soft confidence weighting still improve over β=0 with exactly the same fixed-count schedule, and how does hard ranking compare under identical token sampling? These are the comparisons most likely to change my recommendation.
-
On the same public model and call budget, does the advantage remain when temperature or nucleus sampling is tuned to comparable diversity without post-selecting generated documents? A modest human or task-based evaluation of those operating points would help determine whether the scorer improvements correspond to useful quality.
-
Please reconcile Appendix D’s statement that its public-model setup isolates the order channel with Appendix B’s explicit statement that the fixed-count control was not run on LLaDA. The exact count rule for each public-model arm should be unambiguous.
Thank you again for your time and your thoughtful review. We would be grateful to hear whether our response has addressed your concerns, and we are happy to provide any further clarification.