- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes SlotWeave, a training-free dependency-aware decoding method for slot-based diffusion language models like ReFusion. It estimates token dependencies from attention weights to adaptively merge strongly coupled slots and selects non-conflicting slots through a dependency graph for parallel autoregressive infilling. Experiments on different benchmarks show improved accuracy over ReFusion, especially with small primitive slot sizes, with moderate throughput degradation.
Critical strengths
- The key insight is relatively easy to understand and flow. Overall, the proposed method address a reasonable limitation of slot-based diffusion LLM, and the experiments showcase the method's effectiveness.
Critical weaknesses
-
The method is built on top of ReFusion, which is the only one slot-based decoding dLLMs as far as I know. As a result, there is only one baseline direct baseline. There is no comparison, such as quality-speed tradeoff, against other dLLMs or autoregressive LLMs, which weakens my confidence in this work.
-
In Appendix B.4's Table 4, the accuracy of left-to-right method is higher than SlotWeave in some cases. Then, Appendix B.6 notes that the begins from 0 and ends at 0.005, which means SlotWeave is close to left-to-right method, especially in the early stage of decoding. The contribution of dependency awareness here is not isolated.
Questions for authors
-
Can you extend the evaluation with matched quality–speed comparisons against other dLLM decoding methods?
-
Can you explain why there is a parameter in the dependency estimation? What happens if we just use all the layers instead of up to ?