- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Untied Self-Conditioning, a training-free approach to improve few-step generation in continuous flow-matching language models. The key idea is to separate the use of the model's clean prediction for self-conditioning and numerical solver updates. The method introduces a projection-based conditioning transform and a history-based solver correction without additional model evaluations. Experiments on ELF and LangFlow demonstrate substantial improvements in generative perplexity, particularly at low sampling steps.
Critical strengths
-
Practical training-free improvement: The method improves generation quality without retraining or additional model evaluations. The large improvements at low NFE are particularly promising for efficient diffusion language model inference.
-
Strong experimental evidence: The paper evaluates multiple model sizes and sampling budgets. The ablation studies demonstrate the importance of separating solver corrections from self-conditioning and provide useful evidence supporting the proposed design.
Critical weaknesses
-
Limited generalizability of the proposed correction: While the paper provides a theoretical motivation for separating conditioning and solver updates, the implemented correction relies on prediction histories, offline trajectory statistics, and configuration-dependent coefficients. Although the authors provide supporting diagnostics, it remains unclear how well the method generalizes to different architectures, sampling schedules, or checkpoints without additional calibration.
-
Limited evaluation of broader generation quality: The improvements are primarily demonstrated through generative perplexity on unconditional text generation. While the authors also provide LLM-judge evaluations and diversity measurements, it remains unclear whether these improvements extend to broader language-generation capabilities. Additionally, Appendix C.3 reports higher word 4-gram repetition rates across the evaluated LangFlow configurations, suggesting a potential trade-off that warrants further investigation.
Questions for authors
-
How sensitive are the improvements to the offline trajectory statistics and configuration-specific coefficients? Can the authors demonstrate whether the same calibration transfers across checkpoints or sampling schedules?
-
The paper provides strong improvements in generative perplexity and complementary LLM-judge evaluations. Can the authors further clarify how these gains translate to broader text-generation quality or downstream capabilities (Math, Coding etc), where supported by the evaluated models?
-
Appendix C.3 reports consistently higher word 4-gram repetition rates for Untied Self-Conditioning. Can the authors explain this increase and whether it indicates a trade-off between generative perplexity and repetition?
-
While the projection-overlap ablations support the proposed conditioning transform, could the authors clarify under what assumptions geometric overlap between input projections is a reliable indicator of redundant conditioning information?
Dear Reviewers,
The author-reviewer discussion phase is ending soon. If you have not done so yet, please read the author rebuttals and respond.
Please recall that, as a reviewer, you have agreed to engage in discussion with the authors to clarify and improve the paper. Failure to do so may result in penalties on your own submissions, as per the official ICLR policy. Given the high volume of submissions, your participation is crucial.
Thank you for your time and help.
Best regards,
Your AC