- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies numerical nondeterminism in greedy masked diffusion decoding. It distinguishes position-selection and token-selection forks, tracks subsequent state reunions and irreversible conflicts, and proposes periodic high-precision ranking with frozen reveal orders and selective token verification.
Critical strengths
- The trajectory analysis provides a meaningful advance beyond final-output disagreement statistics. Distinguishing temporary position-order differences from irreversible token conflicts explains why the first fork alone is an inadequate measure of reproducibility.
- The experiments cover multiple execution conditions and separately report agreement, accuracy, and computational cost. The disclosure that freezing changes outputs and can reduce accuracy makes the method’s trade-offs clear.
Critical weaknesses
- The mitigation’s components are not sufficiently disentangled. Periodic refresh simultaneously introduces high-precision ranking and freezes the reveal order. Without a matched low-precision-ranking-plus-freezing baseline, the experiments cannot determine how much improvement comes from precision rather than fewer opportunities to reorder positions.
- The practical quality–cost trade-off and scope remain uncertain. The method changes the decoder: only 61 of the 495 mutually consistent LLaDA outputs match LayerCast, and Dream’s MATH500 accuracy decreases from 43.8% to 40.0%. Its cost remains above BF16, while mitigation results focus on one-token-per-step decoding. Residual disagreement also makes “guarantees agreement” an overstatement.
Questions for authors
- Does low-precision ranking with identical frozen windows achieve comparable agreement, accuracy, and latency?
- Can the mitigation retain its benefits under parallel and block-wise decoding?
- Is there a configuration that substantially reduces disagreement without the observed accuracy loss, and what additional cost does it incur?
- What causes the residual token-mismatch forks in Table 11, and how are they reconciled with §6.2’s attribution of remaining disagreement to position forks?