- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Branching Latent Diffusion, a framework for continuous diffusion language model that compresses block-level latents instead of token-level latents. BLD uses 16x token-to-latent compression ratio and leverages a branching token realization decoder. BLD further introduces a grouped latent flow that divides the K latents into groups with size G. The groups are generated autoregressively while the latents inside each group is denoised by flow model.
Experiments on unconditional text generation show that BLD reduces generation compute and much higher throughput compared with the baselines. Ablation study also investigates the effect of the group size, the design of the decoder, and the effect between reconstruction and diffusion generation.
Critical strengths
-
The paper is well motivated to compress the representation of the long sequence into groups to achieve speedup for the diffusion. The experiment also verifies the efficacy of such a design in terms of remarkably reducing the inference latency.
-
The ablation study is comprehensive and the in-depth analyses, e.g., the PCA experiments and interpolation probes in appendix, offer much more valuable insights.
Critical weaknesses
-
The evaluation scope is limited. The current evaluation is performed on unconditional generation using OpenWebText. It would make the paper stronger if experiments on conditional tasks, e.g. summarization or translation, could be evaluated, like in ELF. This might be critical since each branch decodes tokens independently without observing the tokens realized in adjacent branches, so the behavior on unconditional task may be different from conditional tasks.
-
It is not clearly discussed in the paper how the grouped BLD can leverage KV cache across groups, which would be important given that previous works like Block Diffusion have already implemented KV cache, which is quite standard to control the inference complexity when the model operate on long sequences.
Questions for authors
-
Could the authors provide more justifications on conditional tasks, or why they are not evaluated?
-
Could the authors make more discussions about how KV cache could be integrated into the framework, which is quite important when scaling the model to long sequence generation?
-
Why was G=64 (G=K) excluded from Table 3? How does it perform compared with the reference appeared in Table 1?
I am willing to raise the score if the concerns are resolved sufficiently.
6 comments
Dear Reviewer,
With the discussion period coming to a close, we hope that our responses and new experimental results have helped address your concerns. We would be glad to discuss any remaining questions if time permits.
Dear Reviewer v27A,
Apologies for the additional follow-up. As the discussion period is coming to a close, we wanted to kindly ask whether our new experiments on conditional generation, KV caching, and fully parallel generation have sufficiently addressed your concerns.
We would be happy to clarify any remaining questions. If you feel that your concerns have been resolved, we would sincerely appreciate your reconsideration of our work in your final evaluation.
Thank you again for your time and consideration.
Thank you for the positive assessment of our motivation, architecture, and ablation analyses. We have added conditional generation experiments, clarified KV caching across groups, and extended the controlled group-size comparison.
W1 / Q1: Conditional generation. Our primary focus has been on unconditional generation, as we aim to develop a general framework for language generation under strong latent compression. Unconditional generation involves greater uncertainty over content and global structure, whereas conditional tasks such as summarization provide source information that reduces this uncertainty but introduces the challenge of satisfying conditioning constraints.
To evaluate BLD in this setting, we added XSum summarization experiments using ELF's setup and evaluation script on the full test set of 11,334 articles (General Response R4; Appendix D.4). We evaluate both fine-tuning from OpenWebText and training from scratch, reporting ROUGE-1/2/L scores (higher is better), as shown below.
BLD trained on XSum achieves ROUGE-1/2/L of 33.9/11.7/27.8 with approximately 105M parameters used at generation time, comparable in size to ELF-B (105M), while matching its ROUGE-L and outperforming the reported MDLM and AR baselines across all three metrics. These results demonstrate that BLD can support conditional generation despite decoding token branches independently, as the branches share the complete latent context.
W2 / Q2: KV caching across groups. KV caching can be applied to BLD in a similar way to Block Diffusion. Since generation is causal across groups and previously generated latents remain fixed, their keys and values can be cached and reused by subsequent groups. We have implemented this optimization and evaluated its effect on computation and runtime.
The benefit increases with the number of groups, as caching avoids repeatedly processing previously generated latents, suggesting that it may be particularly useful for longer sequences. We have added the implementation details and measurements to Appendix C.5.
Q3: Fully parallel generation (G=64). Due to limited time and computational resources, we had not completed the controlled G=64 experiment at the time of the original submission. We have now extended Table 4 (original Table 3) to include G=64, trained with the same architecture, encoder, decoder, training budget, and sampler as G=8 (General Response R3).
The complete controlled group-size comparison is shown below (Table 4 in the revised manuscript).
The original G=K reference in Table 1 was separately developed and tuned to provide a stronger fully parallel baseline. It achieves Gen-PPL 24.6, entropy 5.08, and MAUVE 0.77 at 4.8 TFLOPs. The new controlled G=64 model achieves slightly lower generation quality (25.1/5.06/0.75), but requires only 0.93 TFLOPs. Compared with G=8, it is approximately 6× faster (0.30 vs. 1.82 s), but has lower MAUVE (0.75 vs. 0.82). These results clarify the trade-off between fully parallel and grouped latent generation.