- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates trajectory selection for diffusion large language models (DLLMs) under parallel decoding. The authors propose Factorization-Aware Candidate Transition Selection (FACTS), a training-free sampler that combines a future uncertainty reduction proxy () with a positive interaction gain () to evaluate candidate transitions. Despite the rigorous theoretical derivation, the approach exhibits significant flaws regarding the cost-benefit ratio, model scale, and novelty.
Critical strengths
Solid theory: The derivation based on total conditional correlation is rigorous and impressive.
Critical weaknesses
- Lack of Cost-Benefit Trade-off and Comparison with Native DLLMs. The efficiency evaluation (Table 4) only compares the optimized FACTS with its unoptimized (naive) version, failing to report the actual wall-clock time, FLOPs, and peak memory overhead compared to native DLLM samplers (e.g., standard Confidence or Entropy). Given that FACTS requires candidate proposal, multiple prefix state generation, and sequential probing, its computational complexity scales significantly with generation length. Under such heavy computational costs, the marginal performance gains are negligible, raising serious doubts about its ROI.
- Scale of Evaluated Models is Too Small. The experiments are restricted to 7B and 8B models. Given that FACTS is a training-free inference-time optimization method, the authors should have validated its effectiveness on larger or more advanced models (such as the LLaDA 2.x series or larger parameter variants).
- Limited Novelty and Over-reliance on Existing Concepts. Discussions on future uncertainty reduction (entropy reduction/information gain) are already widespread in prior literature (e.g., Info-Gain, LookUM). The core novelty of this work seems restricted to the Positive Interaction Gain ; however, relying solely on a heuristic "sequential-versus-factorized score gap" to capture joint compatibility feels somewhat incremental and insufficient to support the novelty of the entire paper.
- Contradiction with the Core Acceleration Value Proposition of DLLMs. The primary motivation for adopting DLLMs over Autoregressive models lies in their parallel decoding capability and generation speed. However, FACTS reintroduces massive sequential computational overhead (testing multiple permutation orders and prefix state rollbacks) into the decoding loop, severely harming generation throughput. The paper fails to justify whether sacrificing the core acceleration advantage of DLLMs for marginal accuracy gains is a worthwhile trade-off.
Questions for authors
Same as Critical Weaknesses