acceptodds
Withdrawn from ICLR 2027

From Fixed to Flexible: Via Latent Length Awareness in dLLMs for Variable-Length Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The fixed-length generation paradigm of Diffusion Large Language Models (dLLMs) inherently limits their capability in tasks requiring variable-length outputs. Existing training-free approaches address this limitation through dynamic length adjustment, but often disrupt the model's learned distribution. In this work, we reveal that dLLMs possess latent length awareness: the remaining generation length can be encoded in the hidden state immediately following the given context. This latent information can be extracted from a single token, termed the Navigator Token, using a lightweight regression probe. Based on this observation, we propose NaVi, a training-free strategy that enables variable-length generation through predictive and minimally disruptive length recalibration. Specifically, NaVi initializes the generation budget via Navigator Token-based length prediction, and during decoding performs schedule-aware threshold adaptation and dynamic length recalibration with semantic prior injection to preserve distributional consistency. Experiments across diverse models and benchmarks demonstrate that NaVi improves generation quality and efficiency without task-specific tuning. Code and models will be made publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, variable‑length generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZpJDOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses the important problem of variable-length generation in diffusion large language models (dLLMs). The authors propose NaVi, a method that uses a Navigator Token and a lightweight regression probe to predict the remaining generation length. The framework combines initial length prediction, dynamic length recalibration, semantic momentum injection, and schedule-aware threshold decoding.

The paper presents positive accuracy and efficiency results across several models and benchmarks. The research problem is interesting and well motivated. However, my primary concern is that the experimental evidence is not sufficiently decisive to support the central claims, particularly the existence of transferable latent length awareness and the proposed distribution-preservation mechanism. I also identified issues involving validation methodology, statistical interpretation, algorithmic correctness, and reproducibility.

Critical strengths
  1. The paper studies a relevant and well-motivated research problem. Fixed-length generation is a practical limitation of diffusion language models, and developing efficient variable-length inference strategies is valuable.

  2. The Navigator Token idea is interesting. Predicting the remaining generation length from a single hidden-state representation could be useful if its generalization properties are established convincingly.

  3. The authors evaluate the approach across multiple diffusion language models and several benchmarks involving mathematical reasoning, code generation, and instruction following. The reported improvements in accuracy and efficiency are encouraging.

  4. The paper includes useful ablation studies, including experiments on semantic momentum injection, repetition behavior, threshold sensitivity, and probe configuration.

  5. The method does not require retraining the underlying diffusion language model, although it does require offline supervised fitting of a regression probe. This makes the proposed approach potentially practical for existing models.

Critical weaknesses

1. Insufficiently decisive evidence for latent length awareness (Major).

My main concern is the evaluation of the paper's central contribution.

Appendix E.2 explains that 700 successful MATH trajectories are converted into 3,500 prefix/target examples using five truncation ratios. These examples are then randomly shuffled and divided into training and validation sets.

Under this stated protocol, different prefixes of the same original response can appear in both sets. These observations are not independent. Consequently, the reported validation R-squared may partly reflect trajectory-specific information rather than reliable length prediction on unseen problems.

Moreover, the same validation set is used to choose the transformer layer and ridge regularization coefficient, and the resulting best validation R-squared is reported. Hyperparameter selection using validation data is standard, but that selected score is not independent test evidence.

This is a major problem because transferable latent length awareness is one of the paper's central claims.

The authors should perform trajectory- and problem-disjoint splitting, select hyperparameters using training/validation data, and evaluate the final probe on an untouched test set. Initial-prompt prediction should also be reported separately from intermediate-prefix prediction.

The fitting procedure additionally selects correctly answered generations, which may not represent the general online inference distribution. The authors should examine how the probe behaves on unsuccessful and unusually long trajectories.

To be clear, this concern is not an allegation of benchmark-test contamination. The paper states that the fitting data originate from the MATH training split.

2. Statistically problematic interpretation of R-squared and inverse-log predictions (Major).

Table 5 compares R-squared values before and after applying a logarithmic transformation to the target remaining length.

However, R-squared calculated on the original length scale and R-squared calculated on the logarithmic scale are not directly comparable. A higher value after transformation does not necessarily demonstrate better prediction of the actual number of remaining tokens.

Both approaches should be evaluated on the same original token-length scale, with MAE, RMSE, and calibration statistics.

There is another important statistical issue. Under ideal squared-error regression in logarithmic space, exponentiating the conditional log expectation produces a geometric-mean-type estimate rather than the conditional arithmetic mean.

For example, consider two equally probable remaining lengths of 9 and 99. The inverse-log estimator predicts approximately 30.6 tokens, while the conditional arithmetic mean is 54 tokens.

This does not imply that logarithmic regression is inherently invalid. However, it can lead to budgets below the conditional mean. Since under-allocation can trigger additional recalibration and expansion, the paper should justify its prediction target and evaluate under-allocation explicitly.

3. The distribution-preservation claim is insufficiently supported (Major).

The paper claims that semantic momentum injection mitigates the disturbance caused by dynamic length expansion.

However, the paper does not establish mathematically that adding the scaled Navigator hidden-state vector to mask embeddings necessarily brings the resulting output distribution closer to the chosen reference distribution.

The KL experiment in Appendix D.13 is particularly concerning.

The authors state that the expanded and upfront-allocation reference runs share the same verified prefix and total generation length. Under a deterministic, stateless forward-pass formulation, a no-momentum expanded run and an upfront-allocation run with identical token IDs, embeddings, positions, attention masks, and execution state should produce identical logits, apart from numerical differences.

Nevertheless, Table 15 reports substantial nonzero KL divergence, including 7.59 for LLaDA-8B-Instruct at k=128.

This does not necessarily mean that the experiment is incorrect. It could indicate differences in model cache, decoding state, attention structure, positional treatment, or another input variable. However, these differences must be explicitly described and controlled.

Without this clarification, the experiment does not convincingly isolate the effect of expansion history or demonstrate distribution preservation.

I acknowledge that the repetition analysis and random-bias ablation provide useful empirical evidence. However, they are insufficient to establish the stronger claim that the original generation distribution is preserved.

4. Insufficient attribution of the reported improvements (Major).

NaVi simultaneously changes the initial generation budget, length recalibration, mask embeddings, and token acceptance thresholds.

The experiments do not sufficiently isolate the effect of the Navigator-based length predictor from the effects of these other decoding changes.

In particular, a fixed-length baseline using exactly NaVi's token acceptance and threshold policy is needed. An initial-prediction-only variant without recalibration or injection would also be highly informative.

A simple heuristic length controller would help distinguish the contribution of learned representations from the more general benefit of dynamically changing the generation canvas.

Therefore, although the benchmark results are positive, it remains unclear which proposed component is responsible for the headline improvements.

5. Algorithmic inconsistencies and missing termination guarantees (Major).

Algorithm 1 appears to initialize the fitting dataset using the same symbol as the input trajectory collection. Taken literally, this overwrites the input collection with an empty set before the subsequent iteration. The two datasets should be represented separately.

To the best of my understanding, Algorithm 2 also contains an inconsistent SPI calculation. The algorithm computes gamma_k but subsequently uses S_k in the threshold expression.

Moreover, the threshold is computed before possible length recalibration, even though recalibration changes the allocated budget and therefore the SPI denominator.

More importantly, the inner decoding loop does not provide an explicit progress or termination guarantee. Tokens are accepted only when their confidence exceeds the threshold. If no remaining masked position satisfies this requirement and the input state remains unchanged, the loop can continue indefinitely.

A forced-acceptance fallback, iteration limit, or other guaranteed-progress mechanism should be specified.

The outer stopping condition also needs clarification under repeated expansion.

6. Additional mathematical and reproducibility issues.

Equation (7) produces real-valued length predictions, whereas the number of allocated masks must be a nonnegative integer. The paper should specify rounding, clipping, maximum lengths, block alignment, and boundary handling.

The paper should also distinguish distribution shift from violation of the mathematical Markov property. A variable-length state process can still be Markovian.

There are numerical inconsistencies affecting reproducibility:

  • Table 1 reports 101,692 seconds for the LLaDA-8B-Instruct/GSM8K fixed-length configuration achieving 77.9% accuracy, whereas Table 8 reports 10,169 seconds.
  • Table 17 identifies layer 23 as optimal for LLaDA-8B-Instruct, whereas Appendix D.11 describes layer 22 as the chosen optimum.
  • Table 4 reports Avg@30 with nonzero standard deviations, while Appendix E.4 describes greedy decoding without random sampling.

Finally, AIME2024 and AIME2025 each contain only 30 problems. The accuracy improvements in Table 3 correspond to approximately two and one additional correct problems, respectively. These results require careful problem-level uncertainty analysis.

Overall assessment.

I find the research problem interesting and the empirical improvements encouraging. However, the central claims are stronger than the evidence currently supports.

The non-independent probe evaluation, the insufficiently controlled distribution-shift experiment, and the unresolved algorithmic details substantially weaken my confidence in the claimed contribution.

These are central scientific issues rather than minor presentation problems. Therefore, I lean toward rejection in the current form.

Questions for authors

1. Independent evaluation of latent length awareness

Can the authors rerun the probe study with all five prefixes of each generated response, and all responses associated with the same underlying problem, assigned to a single partition?

Please select the transformer layer and ridge regularization coefficient using training/validation data, then report performance on an untouched, problem-disjoint test set.

Please report initial-prompt prediction separately from intermediate-prefix prediction, including length error in tokens, and compare against a simple predictor using prompt length, prefix length, and decoding stage.

If held-out performance drops materially, which claims about latent length awareness would the authors revise?

2. Interpretation and controls of the KL experiment

For Table 15, what are the exact token IDs, position IDs, attention masks, embedding modifications, and model cache or execution states used for the no-momentum expansion and upfront-allocation reference?

How is an identical verified prefix enforced when the reference trajectory is rerun?

Under otherwise identical inputs and a fresh deterministic forward pass, why is the KL divergence nonzero?

Please explain which variables differ between conditions or provide a revised controlled diagnostic that genuinely isolates the effect of expansion history.

3. Matched baselines and component attribution

Can the authors report a fixed-length baseline using precisely NaVi's token acceptance and threshold policy?

Please also evaluate a variant in which the Navigator predicts only the initial budget, without dynamic recalibration or semantic momentum injection.

A simple heuristic length controller and a norm-matched randomized or shuffled Navigator-vector control would further clarify which improvements arise specifically from the learned representation.

Please include all Navigator forward-pass costs in online runtime and report offline probe fitting and trajectory collection costs separately.

4. Algorithmic correctness and termination

Can the authors correct Algorithm 1's dataset notation and reconcile gamma_k and S_k in Algorithm 2?

Is SPI and the decoding threshold recalculated after a length adjustment?

What guarantees progress when no remaining masked position exceeds the confidence threshold?

Please specify the forced-acceptance fallback, iteration limit, outer-loop stopping condition, rounding and clipping of predicted lengths, and handling of incomplete decoding blocks.

5. Statistical evaluation of the length predictor

Can the authors compare the original and log-transformed regression approaches using evaluation metrics calculated on the same original token-length scale?

Please report MAE, RMSE, under-allocation frequency, and prediction calibration across different length ranges.

How does the inverse-log transformation affect length underestimation, especially for long responses?

6. Reproducibility and uncertainty

Please reconcile the runtime discrepancy between Tables 1 and 8 and the optimal-layer discrepancy between Table 17 and Appendix D.11.

Table 4 reports Avg@30 with nonzero standard deviations, whereas Appendix E.4 describes greedy decoding without random sampling. What exactly varies across the 30 evaluations, and what statistical unit is used?

For the AIME benchmarks, can the authors report paired problem-level uncertainty estimates and clarify the robustness of the reported one- or two-problem improvements?

Reviewer_GFipOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

dLLMs must fix the generation length before decoding. This paper observes that appending one [MASK] after the prompt and the decoded prefix gives a position whose mid-to-deep hidden state supports linear ridge regression of log remaining length, with held-out R2R^2 of 0.68-0.72 across four models. NaVi predicts the initial budget from it, re-predicts and adds or truncates masks once the remaining budget falls to one block, injects that hidden state scaled by the expansion intensity into newly added mask embeddings, and maps the ratio of predicted length to remaining budget onto an unmasking threshold in [0.90,0.99][0.90,0.99]. Across four models and seven benchmarks, accuracy is comparable or better with much lower runtime.

Critical strengths

I enjoy reading Section 2.

The Navigator Token is the first. Appending a single mask is enough to read the remaining length off linearly, the construction is simple, inference overhead is close to zero, and it avoids the main cost of repeatedly inserting and truncating masks. That is a nice finding.

The second is the analysis of how abrupt mask insertion disturbs the decoding distribution: KL shift in deep-layer attention, two named failure modes with real examples, repetitive-loop counts, and a KL diagnostic that holds the verified prefix and the total length fixed and varies only whether that length was reached through expansion. Training-free papers rarely pin down a failure mechanism this concretely. The ablations are also more thorough than usual.

Critical weaknesses
  1. The central claim is never isolated. The appended [MASK] sits after the entire context, so the probe may just be reading the prompt, and no prompt-only baseline is tried. arXiv 2603.06123 already proposes that the prompt representation alone can estimate output length; it is not cited.

  2. The probe is fitted and measured on MATH only, then frozen and used on five other benchmarks with no direct accuracy numbers there. Its Min-Max bounds also come from MATH's length distribution, so another domain is extrapolation. Tables 9 and 10 vary the fitting domain, not the evaluation domain.

  3. How hnavh_{\mathrm{nav}} is obtained online is not stated. The probe sequence appends one mask while the decoding sequence carries hundreds after that position, and under bidirectional attention the two differ. Either an extra forward per block, whose cost belongs in the runtime numbers, or a probe applied to a different distribution than it was fitted on.

  4. The origin of the length signal is not investigated. LLaDA-family SFT pads short responses with |EOS| and keeps those tokens in the loss, so the probe may be reading how many EOS tokens remain.

Questions for authors
  1. What R2R^2 do the last prompt token, prefix mean pooling, an arbitrary pre-existing mask position, and surface-feature regressions reach, and what is the Navigator Token's net gain over them? How does this relate to arXiv 2603.06123?

  2. On GSM8K, HumanEval, MBPP, IFEval and AIME, what are the frozen probe's R2R^2 and token-level error? What happens when response lengths fall outside MATH's Min-Max range?

  3. Where does hnav(ℓ∗,k)h^{(\ell^*,k)}_{\mathrm{nav}} come from at inference? If it needs an extra forward per block, is that cost in the runtime numbers; if it is read from the decoding pass, what is the probe's R2R^2 under that input distribution?

  4. What happens with a norm-matched random vector, or a Navigator state from a different prompt? How many injections does a generation receive on average, and is the bias refreshed or removed once a position enters the current window?

  5. With a "best fixed length plus threshold parallel acceptance" baseline, how much runtime reduction comes from length adaptation itself? Were DAEDAL and ρ\rho-EOS run with threshold acceptance?

Reviewer_7tF4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a practical limitation of diffusion language models: they typically require the output length to be set before generation. A short budget can cut off an answer, while a long budget wastes computation and may hurt output quality. The authors find that the hidden state of a single mask token placed after the current text can help predict how many tokens remain. Based on this observation, they propose NaVi, which uses a lightweight regression probe to set and adjust the generation budget. It also adapts the decoding confidence threshold and injects information from that hidden state into added mask tokens to help generation continue smoothly. Experiments across four models and several reasoning, coding, and instruction-following benchmarks report improved efficiency while generally maintaining or improving task performance.

Critical strengths
  1. The method tackles a practical problem with a simple idea: use one mask token’s hidden state to estimate how much output space is needed, without changing the language model’s weights.

  2. Results across four models and several tasks show promising speed gains while generally preserving accuracy. The reduction in repetitive generation is also useful in practice.

Critical weaknesses
  1. The probe evaluation may overestimate performance on unseen problems. Appendix E.2 randomly splits prefixes after extracting several from each response, so the same problem can appear in both training and validation. Splitting by original problem is needed to establish how well length prediction generalizes.

  2. The evidence for distribution preservation needs clarification. Appendix D.13 states that the expanded and reference runs share the same prefix and total length. If their remaining inputs and model states also match, their predictions should be identical without momentum. The unexplained nonzero KL makes it difficult to assess the claimed mechanism.

Questions for authors
  1. Have you evaluated NaVi on distilled models that generate in fewer steps? When decoding is already fast, the extra forward passes for length prediction may offset the savings. It would be useful to understand whether NaVi still improves efficiency in this setting.
  2. Could you report probe performance with training and validation split by original problem, rather than by prefix? This would better test generalization to unseen problems.
  3. How much does the Navigator Token help compared with simpler predictors, such as prefix length or the hidden state of the last text token? This would clarify whether the added mask token is necessary.
  4. Does SPI outperform well-tuned fixed confidence thresholds? Table 8 only compares against a threshold of 0.9, so it is unclear whether the gains require dynamic adjustment.
  5. How much does accurate length prediction matter for the final results? Could a simple strategy that starts with a fixed budget and expands whenever space runs low achieve similar performance?
  6. The probe is fitted only on successful responses. How reliable are its predictions when the model makes mistakes or starts repeating itself? Could these predictions cause NaVi to keep expanding the budget for an already failing response?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.