- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses premature token commitment in diffusion language models. It introduces SuperInfer, which replaces discrete token predictions with probability-weighted mixtures of top-K token embeddings, allowing uncertain predictions to influence subsequent denoising steps. The authors further propose SuperRL, a PPO-style framework for learning from superposition-based rollouts. Experiments across four diffusion models and several reasoning benchmarks show improvements over standard decoding, with additional gains from reinforcement learning.
Critical strengths
-
The proposed inference mechanism is simple and requires no parameter updates. Its applicability to both full-attention and block-diffusion models is a useful practical contribution.
-
The empirical results are encouraging. In particular, SDAR-4B improves from 62.4 to 67.2 on MATH500 using SuperInfer alone. The parallel decoding and collapse analyses provide suggestive, although not conclusive, evidence for the proposed mechanism.
Critical weaknesses
-
The experiments do not sufficiently isolate the benefit of superposition from delayed discrete commitment. A compute-matched delayed-unmasking baseline is missing. Similarly, SuperRL is not compared against standard discrete PPO under the same training setup, making its additional contribution difficult to assess.
-
The SuperRL objective needs further justification. Eq. (3) uses top-1 token probabilities as a surrogate for continuous superposition trajectories, while Eq. (6) appears to omit the importance weighting and clipping terms in Eq. (4). Additionally, the baseline accuracies in Figure 3 differ from those in Table 1, and the full-collapse intervention does not necessarily reproduce standard discrete decoding. These issues limit the interpretability of the reported results.
Questions for authors
-
How does SuperInfer compare against simply leaving uncertain positions masked for additional denoising steps, under matched inference cost?
-
Can the authors clarify the trajectory likelihood and gradient computation in Eqs. (3)--(6)? How much does SuperRL improve over conventional discrete PPO using the same training data?
-
How is the trigger token defined for frozen pretrained models, and what happens if it is never generated?
-
Could the authors explain the discrepancy between the baselines in Figure 3 and Table 1, particularly the large degradation under full forced collapse?