acceptodds
Under review as a conference paper at ICLR 2027

Reading Latent States: Adaptive Halting in Continuous Diffusion Language Models

Abstract

Continuous diffusion language models think in vectors, not words, refining a latent state step by step and decoding text only at the end. Yet each trajectory is run to its final call, and two things are unknown. (1) Whether a latent state can be read at all: it is not text, so there is no candidate to score and no confidence to threshold. (2) Whether the full budget is even necessary. To address both, we split the job between a teacher and a student. Offline, a frozen teacher readout marks where the answer enters and whether the state is good enough to stop. At decision time a student reader sees only the latent prefix and a budget signal and emits HALT or CONTINUE—never text or future states. We apply this to CoLA and ELF latent trajectories. The property under test, trajectory parseability, must hold on three counts: predictability from a prefix, information beyond what the computation clock alone gives, and a usable decision. On WMT14 the policy uses 8.8 of 64 solver calls—86% less computation—and returns BLEU 26.2572 against 26.1466 for full inference. On XSum, 9.5 of 64 calls (85% less) at ROUGE-L 27.9232 against 27.811. Unconditional generation is harder: 24.6 of 32 calls, 23% less, at a small perplexity cost. Trajectories make up their minds early: in CoLA the output identity is in place after the first of four latent blocks for 63.6% of saved trajectories, and WMT14 quality peaks at call 19 of 64. Stopping early is thus less a compromise than a correction, because quality is not monotone in computation. Results are replayed from saved trajectories, not a measured speedup.

Primary area:
generative models
Keywords:
Continuous diffusion language models, Latent space interpretability, Adaptive halting, Early exit, Inference efficiency, Diffusion language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FaCzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether the intermediate latent states of continuous diffusion language models (such as CoLA and ELF) can be "interpreted" prior to decoding. It proposes a teacher-student protocol wherein an offline "teacher" observes the full trajectory to annotate "when the answer appears" and "when it is sufficiently good," while the "student" decides whether to HALT or CONTINUE based solely on the latent prefix and a budget signal. Results demonstrate that these latent states indeed encode sample-specific information beyond mere "computational progress"; on WMT14 and XSum, quality comparable to full inference can be achieved using only about one-seventh of the solver calls.

Critical strengths
  1. It shifts the focus of the "early stopping" problem from observing output confidence to directly reading latent states, and defines the "interpretability of the latent space" itself as a falsifiable property—termed "trajectory parseability"—rather than simply assuming it holds true.

  2. the design of the "causal boundary" for the teacher-student division of labor is elegant: the teacher operates offline, utilizing future information, reference answers, and scoring mechanisms to generate supervision signals, whereas the student, during decision-making, is restricted to the prefix and the computational budget. This approach leverages rich offline annotations while ensuring that information constraints required for deployment are strictly respected.

Critical weaknesses
  1. the reported savings were calculated based on replaying recorded trajectories; the overhead of the reader itself, the integration of the online sampler, and real-world latency were not measured. Consequently, the claim of "86% computational savings" falls short of a truly deployable acceleration solution.

  2. compared to baselines with longer budgets (fixed16/fixed32), the policy quality actually declined slightly. Furthermore, the early-stopping experiments on CoLA relied on single-seed runs and post-hoc calibration—yielding merely exploratory results—meaning the claim of "early stopping without quality loss" lacks robust support outside the ELF framework.

Questions for authors

How significant is the inference overhead of the Reader at each step compared to a single solver call? If the Reader is integrated into an online sampler and its overhead is factored in, what is the remaining expected end-to-end speedup?

The strategy lags behind fixed16/fixed32 by 0.19 BLEU and 0.20 ROUGE-L; is this due to inaccurate decisions by the Reader, or a mismatch between the position-wise supervision and the "first-stop" decision objective? Have there been any attempts to directly optimize the stopping utility (e.g., via RL or sequential decision-making loss)?

Reviewer_R717OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether intermediate latent states in continuous diffusion language models (CoLA and ELF) can be read and used to determine when to stop early. The authors propose a teacher–student protocol: complete generation trajectories are read out offline to construct labels based on when outputs emerge and whether further computation offers additional quality gains. The student observes only the current and preceding latent states, together with computational progress, predicts these labels, and converts its predictions into stopping decisions. Offline replay results show that, on translation and summarization tasks, the student policy uses approximately one-seventh of the solver calls while achieving quality comparable to full inference.

Critical strengths

The research question has practical value. Intermediate states in continuous diffusion models are vectors, so stopping rules that rely on decoded text or token confidence cannot be directly applied. Investigating whether latent states can be used directly to determine when to stop is a practically meaningful question. Latent states may contain information about computational progress. Therefore, accurate student predictions do not necessarily indicate that the student uses the current sample’s content; it may simply have learned at which step stopping is usually appropriate. The authors use matched shuffled controls to distinguish these possibilities, which is a worthwhile approach.

Critical weaknesses

The abstract emphasizes the substantial reduction in solver calls relative to 64 steps. However, simply truncating at 16 steps already achieves most of these savings and obtains higher quality than full inference on WMT14 (26.45 vs. 26.15). Therefore, the comparison against 64 steps does not isolate the contribution of the learned policy. The student policy does outperform non-adaptive allocation at the same budget (+0.46 BLEU / +0.05 ROUGE-L), but the paper does not provide a complete quality–step curve, making it difficult to assess its overall advantage over different fixed-budget alternatives. This is the main basis for my rating.

The writing is difficult to follow. Parts of the main text are verbose, while important information is placed in the appendix, increasing the effort required to understand the paper.

Questions for authors

The student uses approximately 8.8 steps on average and achieves BLEU 26.26, while stopping at a fixed 16 steps achieves 26.45. My question is: what is the minimum number of steps a fixed stopping policy needs to match the student’s quality? If approximately 10 fixed steps can already achieve 26.26, the additional savings from learning the policy may be relatively limited. If substantially more steps are required, this would provide stronger evidence of its value. Could the authors provide the complete results for fixed k = 1–16? This is important evidence for assessing the magnitude of the paper’s contribution.

Table 6 shows a clear reversal on LAMBADA across all three seeds: the aligned student achieves an AUPRC of approximately 0.21–0.22, whereas the shuffled student achieves approximately 0.84–0.95. This is not a small fluctuation. I would appreciate a more detailed explanation rather than simply characterizing the result as task-dependent.

The argument for ELF has two parts. First, the aligned reader, which observes the current sample’s vectors, has lower prediction loss than the shuffled reader, which observes another sample’s vectors. This supports the presence of sample-specific predictive information. Second, the aligned reader achieves favorable quality and step counts when used for early stopping. However, a direct comparison connecting these two findings is missing: the paper does not report the stopping performance of the shuffled reader. If the shuffled reader achieves similar quality and step counts, the predictive advantage of using the current sample’s information has not translated into better allocation. Conversely, worse stopping performance from the shuffled reader would provide more direct evidence connecting the first finding to the second.Please therefore replay the shuffled reader on the same test trajectories using exactly the same stopping rules as the aligned reader, and report its average step count and BLEU / ROUGE-L alongside the aligned reader’s results in Table 8, such as 8.78 steps and BLEU 26.26 on WMT14. This would allow readers to assess the decision-level benefit of using the current sample’s information.

Reviewer_NJfhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper trains a small reader to predict when a frozen continuous diffusion language model can stop, using latent states and supervision from complete trajectories. CoLA experiments suggest that the states carry useful information beyond the step counter, while ELF experiments show gains over the reported nonadaptive rules at the same average call budget. The stopping decisions are evaluated on saved trajectories, and actual runtime savings are not measured.

Critical strengths

The paper addresses a useful question about whether latent states reveal how much computation a particular example needs. The shuffled-state controls help distinguish this information from simply knowing the step counter. The translation result is promising. About nine calls achieve quality close to the 16-call prefix and improve by 0.46 BLEU over the reported equal-budget rule.

Critical weaknesses
  1. The benefit of learned stopping is difficult to separate from redundancy in the sampling schedule. Quality is already near its peak after eight calls, and a fixed 16-call prefix outperforms the full 64-call run. The reader may offer further gains, but a stronger short-schedule baseline is needed to establish how much it adds beyond a better allocation of sampling steps.

  2. The equal-budget comparison does not fully isolate the value of sample-specific decisions. The baseline matches the average number of calls, but not the distribution of stopping times. Randomly reassigning the policy’s stopping times across examples would test whether choosing the right time for each example matters. The comparison also excludes feature extraction and the three-reader ensemble, so equal call counts do not imply equal total cost.

The results are promising. These controls would clarify the reader’s contribution and could change my assessment.

Questions for authors
  1. Does the reader still help with a better short sampling schedule? A validation-selected schedule would be an immediate control. More ambitiously, adapting the ideas in Entropic Time Schedulers for Generative Diffusion Models (Stancevic et al., 2025) or Denoising Growth Complexity (Wainwright, 2026), then training the reader on that schedule, would help separate schedule redundancy from the benefit of sample-specific stopping.

  2. How does quality change if you randomly reassign the policy’s stopping times among examples of comparable length?

  3. Does the advantage remain after counting feature extraction and all three readers?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.