- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces the Predictive State Flows (PSF) which decouples predictive information from the token generation. It uses rectified flow to generate the predictive states representing the context before each block, then generates the blocks in parallel with the autoregressive decoding within each. As a result, it reduces the number of FLOPs while achieving competitive perplexity.
Critical strengths
The paper proposes an interesting combination of diffusion and autoregressive decoding that allows for multiple blocks to be generated in parallel. The experiments include useful ablations on block length, landing targets, and realization codes to inform how the different components influence generation.
Critical weaknesses
1- While the paper claims that predictive states are easier for the diffusion prior to model than content states, the experiments do not show this hypothesis properly. The content state baseline reuses a model trained for predictive states, and the performance advantage remains even when the prior is bypassed (Table 3). This suggests that the improvement could be due to either the prior or the decoder. The comparison with ELF-B is also not balanced: PSF uses more parameters and training compute, its perplexity advantage is relatively small, and its throughput advantage depends on the batch size (Table 1). More controlled experiments are needed to support the main claims.
2- Also, novelty and positioning of the paper is unclear (described in Q3)
Questions for authors
1- The role of "realization codes" is unclear and contradictory, as removing them improves perplexity but hurts MAUVE, diversity, and topic consistency (Table 1, Figure 11). Could the authors please elaborate on this trade-off and provide a more balanced evaluation about whether predictive states cause better generation on average?
2- PSF demonstrates weaker topic consistency and coherence than several baselines despite its perplexity improvements (Figure 11, Table 18). Could the authors test stronger cross-block conditioning and demonstrate whether it improves coherence without sacrificing the efficiency gains of a block decoder? It is more of a scope question.
3- The novelty of the method compared to earlier work is unclear. Zhu et al. (2025) already use diffusion-generated segment representations with autoregressive decoding. Diffusion guided language modeling has also been explored by Lovelace et al. (2024) and latent planning approaches have been proposed for text generation by Wang et al. (2022). Could the authors clarify the differences between their approach and these earlier ones or offer a comparison?
Zhu et al. "Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models." ACL, 2025. Lovelace et al. "Diffusion Guided Language Modeling." Findings of ACL, 2024. Wang et al. "Language Modeling via Stochastic Processes." ICLR, 2022.
5 comments
Dear Reviewer,
As the discussion period approaches its end, we hope our responses and additional experimental results have addressed your concerns. We welcome any further discussion, as time allows.
Thanks
Thank you for the response. The added controls and clarifications help, but I still find the novelty over prior latent diffusion and planning methods limited. Inconsistencies in the reported results and training descriptions remain, and the evidence for comparable coherence is still not strong enough. In my assessment, these concerns leave the contribution below the standard I expect at ICLR. I therefore maintain 2 (weak rejection).
Thank you for your careful review.
W1: The content-state control and prior versus decoder. To address the concern about the asymmetric content-state control in Table 3, we trained both models from scratch at K=8 with their own encoders and decoders, using identical architectures and training budgets (General Response, R1; new Appendix F.1). At matched entropy, the predictive model achieves substantially lower perplexity (26.0 versus 41.7), and the 32B judge prefers its whole documents by 32:10. The gap also persists under true-plan decoding, confirming a decoder-side contribution. To evaluate the prior separately, we measured its performance without the decoder on held-out text. The predictive prior achieves much lower normalized denoising error at a 50% noise level (14% versus 41%) and a smaller Fréchet distance between generated and real states (11.6 versus 40.3). These results support both advantages: predictive states are easier for the prior to model, and the decoder generates better text from them. The revision now reports these effects separately.
W1 (continued): Balanced comparison with ELF-B. We trained PSF-S at ELF-B's model size (104M versus 105M generation parameters) using fewer training tokens (31B versus 46B; General Response, R3; new Appendix F.3). At entropy 5.16, PSF-S achieves 22.8 perplexity versus 24.0 for 32-step ELF-B, and 0.86 versus 0.73 MAUVE@512, with 46× fewer inference FLOPs. Thus, the advantage holds at matched model size with a smaller training budget. PSF-S is now included in Table 1. For our 197M model, the perplexity advantage reaches 21% on length-controlled 512-token windows (Table 15), with MAUVE@512 of 0.93 versus 0.73. The 23× reduction in FLOPs per document holds across batch sizes, while the throughput advantage becomes apparent at larger batches (62.9 versus 49.8 documents/s at batch 256). The revision distinguishes these two efficiency measures.
W2 / Q3: Novelty relative to Zhu et al. (2025), Lovelace et al. (2024), and Wang et al. (2022). We now discuss these works in Related Work and provide a detailed comparison in Appendix F.4 (Table 22). Our core contribution is not the combination of diffusion latents and autoregressive decoding, but the choice of what the diffusion prior should generate. Unlike content latents that encode the text they represent, PSF uses predictive states computed strictly from preceding context, leaving token realization to the decoder. Our symmetric control shows why this distinction matters: content states have nearly three times the prior denoising error and 60% higher perplexity under identical training. PSF realizes this principle through a rectified-flow prior over predictive states and parallel block decoding with one-token landing targets. The three cited works use different latent targets and decoding schemes, as detailed in Table 22.
Q1: Realization codes: contradiction or trade-off? This reflects a trade-off between two operating points of the same model. Realization codes are 64-dimensional, KL-limited side inputs that carry a small amount of information about each block. Generating these codes helps coordinate wording across blocks (6.9% topic-switch rate, 0.93 MAUVE), whereas setting them to zero lowers perplexity by 6–9% but reduces MAUVE by 0.05 and raises the topic-switch rate to 12–13%. Regarding whether predictive states improve generation quality overall, our symmetric control shows that they outperform content states in perplexity, MAUVE, and judged coherence simultaneously.
Q2: Stronger cross-block conditioning. We evaluated two variants that condition blocks on realized text (new Appendix F.2): a refinement round that re-encodes the generated document (+0.22 TFLOPs), and sequential block decoding, where each block reads the realized text of preceding blocks (8× serial depth). Neither improves judged whole-document coherence over the single-pass model (12:14 and 16:23 order-consistent pairs). In these tests, stronger cross-block conditioning did not improve coherence, despite its additional cost. For topic consistency, the lexical coupling in Appendix D.5 improves lexical topic persistence to the level of real text without additional FLOPs, at the perplexity cost reported there.