acceptodds
Under review as a conference paper at ICLR 2027

When to Commit? Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models

Abstract

Discrete diffusion language models (dLLMs) enable parallel token updates with bidirectional attention, yet typical generation adopts blockwise semi-autoregressive decoding, where fixed-size or heuristic blocks are committed to the prefix. This creates a future-dependent commitment problem: tokens may be frozen before enough future context is available, causing early local errors to propagate across later blocks. Motivated by this, we propose Variable-Size Self-Contained Blocks (VSB), a commitment criterion that replaces fixed boundaries with an operational self-containedness test. VSB follows a propose-then-test design, where the bidirectional dLLM first proposes a window hypothesis, then compares the predictive distributions under two views of that hypothesis: a No-Future view truncated at a candidate boundary and a Future-Aware view over the full window. Low NF-FA divergence indicates that the candidate block remains consistent even when the hypothesised future is hidden, making it a self-contained block to commit. VSB applies to pretrained dLLMs and can be paired with optional No-Future training for stronger alignment. Across reasoning, code, and general knowledge benchmarks, VSB improves over fixed-size and heuristic block baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Discrete Diffusion Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_oxmKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Variable-Size Self-Contained Blocks (VSB) for discrete diffusion language models. It compares No-Future (NF) and Future-Aware (FA) predictions under the same decoding hypothesis and uses a length-aware objective to select commitment boundaries. Coarse-to-fine search reduces scoring cost, while optional blockwise No-Future training aligns training with inference. Experiments cover three backbones and seven benchmarks.

Critical strengths
  1. Comparing predictions with and without the hypothesised future directly addresses the future-dependent commitment problem. The theoretical bounds link NF-FA divergence to predictive stability.

  2. The reported gains span several models and tasks. Comparisons with fixed-size decoding at matched average block lengths or similar NFE budgets strengthen the evaluation. The runtime analysis also makes the cost of boundary scoring explicit.

Critical weaknesses
  1. The endpoint of the budgeted window needs clarification. Under Eqs. (5)-(7), NF and FA appear to coincide at b=q, giving S(p,q)=0. Eq. (9) would then favor the full window, leaving it unclear how the stated rule selects shorter self-contained blocks.

  2. Although the paper compares against adaptive baselines, it remains unclear how much NF-FA divergence itself adds over token confidence for boundary selection.

Questions for authors
  1. At b=q, do NF and FA use the same conditioning context, including the masked positions beyond q? A short example showing how the length-aware objective selects b<q would help clarify the boundary rule.

  2. Could the authors compare NF-FA divergence with token confidence for boundary selection while matching average block lengths and keeping the backbone and other decoding settings fixed? Please report accuracy/pass@1 and NFE or runtime.

4 comments
AuthorsOpenReview ↗

Response to Reviewer oxmK (1/1)

We thank the reviewer for the concrete questions. We provide detailed response to your questions below.

Re W1: Is the objective degenerate at the endpoint b=qb=q?

Thank you for your careful reading and we apologies for the confusion. We clarify that NF and FA use different conditioning context, therefore it does not degenerate at b=qb=q as evident in our VSB average commit blocksize analysis (Figure 6). Specifically, FA conditions on the masked continuation Mq+1:L\mathbf{M}_{q+1:L}, which Sec. 3.1 includes in the future ("with tokens beyond qq as masked Mq+1:L\mathbf{M}_{q+1:L}, which we omit in the following for clarity"(Line 144)). To avoid repetition, we kindly invite the reviewer to visit our detailed response to W1 from Reviewer 5zYj.

Re Q1: Same conditioning context at b=qb=q? A short example

Following from our response to W1, VSB uses different conditioning at b=qb=q, where FA conditions on [x^1:q;Mq+1:L][\hat{x}_{1:q};\mathbf{M}_{q+1:L}] and NF on x^1:q\hat{x}_{1:q}. Here we show an example of the smooth length-aware objective (Eq.9) at different candidate boundaries of a block:

candidate boundaryS(p,b)S(p,b)Eq. (9) score
mid-sentence, 16 tokens0.3110.34
end of the first reasoning step (line break after "1. Calculate the total distance Jenna drives."), 32 tokens0.0053.42
full window, 64 tokens0.0681.99

VSB thus commits the completed step. Appendix J shows the same rule over a full decoded solution.

Re W2: What does NF-FA add over token confidence?

Thank you for the insightful question, we highlight that our propsoed NF-FA measures future dependence, the property that the commitment phenomenon we study requires, whereas confidence measures how peaked a prediction is now.

Confidence thus cannot separate a token that is confidently predicted yet still contingent on the continuation (e.g., a line break that may become a comma, a closing bracket, the last digit of a partial sum) from one that is settled. NF-FA compares the two conditionals directly, which is why premature commitment is defined by it and why our stability bound is stated in it.

  1. What it detects: The two scores are nearly uncorrelated (only 2% of NF-FA's top-10% tokens are in confidence's top 10%, against 10% by chance). The tokens NF-FA flags while confidence rates them safe are mostly line breaks and delimiters, which are the positions where a unit either ends or continues (where our Table 9 locates premature commitment).
  2. Why it needs no calibration: NF-FA is a divergence with a natural zero, reached when the block is self-contained, and Eq. (9) trades it against length in the same units, so one default setting serves every task. A confidence score has no such zero, such that we had to tune its weight to reproduce VSB's block length, and that weight does not transfer across tasks (its blocks come out 10-12 tokens shorter on MATH500/MBPP).
  3. What it guarantees: Lemma 3.3 to Corollary 3.5 bound the chance that a committed prediction is unstable in terms of NF-FA, and confidence does not provide this. The bound's premise holds for most commits: 75% of VSB's commits pass the self-containedness test, against 42% of the calibrated confidence rule's (and 44% of random cuts of the same length), and the region VSB defers has 1.77x the flip rate of the region it commits (Table 11).
  4. Accuracy: With backbone, drafting and search fixed, NF-FA is at or above the calibrated confidence rule on both math tasks and within noise on code (Table in Q2 below).

Re Q2: How does NF-FA compare with confidence at matched block length?

Experimenting with a fixed backbone (LLaDA-8B) and length-aware objective, with only the boundary signal changes our NF-FA objective is ahead of confidence on both math tasks, within noise on code. We highlight that NF-FA also carries the stability guarantee of Lemma 3.3 to Corollary 3.5 that confidence lacks.

  1. The confidence rule replaces S(p,b)S(p,b) by the mean (1−confidence)(1-\text{confidence}) over the same positions, with its weight chosen so that its mean block length approximates VSB's.
  2. The backbone, prompts and decoding settings are identical. Only the boundary rule differs, so pass counts are an outcome of the rule, not a setting.

LLaDA-8B, accuracy (fwd per problem). Potential performance deviation can be due to hardware differences.

boundary signalGSM8K (1319 samples)MATH500 (500 samples)HumanEval (164 samples)MBPP (500 samples)
NF-FA (VSB)82.87 (616)42.40 (872)45.73 (893)40.60 (285)
token confidence81.58 (509)41.20 (1033)46.34 (737)41.00 (405)

These results combined therefore supports NF-FA for boundary selection. We will revise these content in a future update.

Dear Reviewer oxmK,

We sincerely apologise for this gentle reminder and greatly appreciate your time and thought-provoking review. As the discussion deadline approaches, we would be grateful if you could share your feedback on our responses.

We have provided detailed responses to address all your questions and will carefully incorporate all feedback into the final version of our paper. We would greatly appreciate your feedback on whether our responses have adequately addressed your concerns.

Thank you again for your time and consideration!

Authors

I thank the authors for the detailed response and additional experiments, which address most of my concerns. Based on my overall assessment of the paper's contributions, I will maintain my original score.

Dear Reviewer oxmK,

We sincerely thank you for the time and effort you have devoted to providing constructive and thoughtful feedback. We also appreciate your positive recognition that we have addressed most of your concerns.

We fully respect your assessment and would be grateful if you could share any remaining concerns that we might not have adequately addressed. We would sincerely appreciate the opportunity to provide additional clarifications during the discussion period, with the hope of fully addressing your concerns and potentially improving your assessment.

Thank you again for your valuable feedback and consideration!

Authors 10359

Reviewer_5zYjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the future-dependent commitment problem in blockwise semi-autoregressive decoding of discrete diffusion language models (dLLMs), where fixed-size or heuristic boundaries may prematurely freeze tokens that depend on future context. To address this issue, the authors propose Variable-Size Self-Contained Blocks (VSB), an inference-time framework for adaptive block commitment. Specifically, VSB constructs a model-generated hypothesis and compares token predictive distributions under No-Future (NF) and Future-Aware (FA) views. Their KL divergence serves as a self-containedness measure, while a length-aware objective and coarse-to-fine search select variable-size commitment boundaries. The authors also introduce optional No-Future training alignment to reduce the training-inference mismatch. Experiments on three LLaDA backbones across seven benchmarks demonstrate overall performance improvements over fixed-size and heuristic blockwise decoding methods. Additional ablations further examine boundary selection, block length, and inference cost.

Critical strengths

1.Intuitive and effective adaptive commitment strategy. The paper introduces a reasonable inference-time approach that determines when to commit tokens based on their dependence on hypothesized future context. Using NF-FA predictive divergence to guide variable-size block selection is an intuitive and meaningful idea, with notable performance improvements on several benchmarks. Although the method remains constrained by the predefined decoding window and cannot overcome the limited future context available in blockwise decoding, it provides a promising direction for improving commitment decisions. 2.Clear presentation and well-designed figures. The paper is generally well written and logically organized, with visually appealing figures and a clean, professional layout. The illustrations and tables effectively support the explanation of the motivation, methodology, and experimental findings, making the paper easy to follow.

Critical weaknesses

Potential inconsistency in the boundary selection objective. According to Eqs. (7)–(10), the candidate boundary set includes (b=q). However, when (b=q), the NF and FA views become identical, yielding zero KL divergence. Since KL divergence is non-negative and the length reward increases with block size, the proposed objective appears to always favor the full-window boundary (b=q). This seems inconsistent with the claimed variable-size block selection. Clarifying this formulation and its relationship to the actual implementation would strengthen the theoretical justification and credibility of the proposed method.

Questions for authors

1.Boundary selection: Given that (S(p,q)=0), how does the proposed objective avoid always selecting the full-window boundary (b=q)? 2.Future dependence: Since the decoding hypothesis is generated through bidirectional denoising, tokens within a candidate block may already incorporate information from future tokens. How does NF-FA divergence distinguish genuine future independence from information already incorporated into the hypothesis?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 5zYj (1/1)

We thank the reviewer for the precise reading of the boundary objective. We carefully addres your questions below, with equation and section numbers refering to the manuscript.

Re W1: Do NF and FA become identical at b=qb=q?

Thank you for your careful reading, and we apologise for the confusion. In short, The two views are not going to be identical, and they differ by the masked continuation, which Sec. 3.1 includes in the future ("with tokens beyond qq as masked Mq+1:L\mathbf{M}_{q+1:L}, which we omit in the following for clarity" (Line 144)). We agree that Eq. (6) as wrtten made this easy to miss. Thus, expanding it, we have:

PiNF(b)=pθ(Xi∣maski(x^1:b))P_i^{NF}(b)=p_{\theta}(X_i | mask_i(\hat{x}_{1:b}))

PiNF(b)=pθ(Xi∣maski([x^1:q;Mq+1:L]))P_i^{NF}(b)=p_{\theta}(X_i | mask_i([\hat{x}_{1:q}; \mathbf{M}_{q+1:L}]))

S(p,b)=1b−p∑i=p+1bKL(PiNF(b)∣∣PiFA)S(p,b)=\frac{1}{b-p}\sum_{i=p+1}^{b}KL(P_i^{NF}(b) || P_i^{FA})

Intuitively, for each position ii in the candidate block, the model predicts token ii through: 1. seeing only the text up to bb (NF) and 2. seeing the whole drafted window plus the masked slots after it (FA).

At b=qb=q the two views therefore differ by the continuation. And the divergence is zero only if the window is self-contained as a unit. Therefore, a window that ends inside an equation or a line of code can thus have nonzero divergence. Proposition B.1 uses the same quantity with a tolerance ε\varepsilon.

The objective is therefore well defined at b=qb=q, and the variable-size behaviour follows from it (Q1). We will clarify this in a future revision.

Re Q1: Why does the objective not always select b=qb=q?

  1. Let m(b)=(b−p)S(p,b)m(b)=(b-p)S(p,b) be the block's future-dependence mass. With SL-A(b)=log⁡(b−p)−m(b)S_{\text{L-A}}(b)=\log(b-p)- m(b), Eq. (9) picks b<qb<q iff λ[m(q)−m(b)]>log⁡q−p+1b−p+1\lambda[m(q)-m(b)]>\log\frac{q-p+1}{b-p+1}. This shows that extending the block from bb to qq earns a length reward (the right-hand side) but also adds the divergence accumulated between bb and qq (the left-hand side). The shorter block wins whenever the added future dependence outweighs the added length.
  2. S(p,b)S(p,b) rises along the window (e.g., median 0.02 at b−p=8b-p=8, 0.13 at 32 and 0.28 at 64, post hoc over 33 completed windows), so the divergence mass grows fastest near its end.
  3. Empirically, we observed that VSB picks b<qb<q in 80% of windows, and shown in the mean commit block size in Figure 6.

Appendix J shows full case studies.

Re Q2: Does future information already in the hypothesis confound SS?

Thank you for the insightful question. In short, since both views see the same draft, so SS measures only what remains future-dependent and will NOT cou, which is exactly the commitment question. By holding the block fixed, SS isolates the dependence that commitment would freeze:

  1. Both views see the same block tokens x^p+1:b\hat{x}_{p+1:b}. Whatever the drafting pass already resolved into those tokens is common to both views.
  2. S(p,b)S(p,b) measures what remains: how much the block's predictions still change with the context after bb, which a committed block must do without.
  3. VSB asks "can this block be frozen now?", not "how was the draft produced?" (Appendix A).
  4. Empirically, what SS measures tracks instability: the region VSB defers has 1.77x the flip rate of the region it commits (Table 11).

Dear Reviewer 5zYj,

We sincerely apologise for this gentle reminder and greatly appreciate your time and thought-provoking review. As the discussion deadline approaches, we would be grateful if you could share your feedback on our responses.

We have provided detailed responses to address all your questions and will carefully incorporate all feedback into the final version of our paper. We would greatly appreciate your feedback on whether our responses have adequately addressed your concerns.

Thank you again for your time and consideration!

Authors

Thank you for the detailed rebuttal and additional experiments. The clarifications regarding the NF-FA conditioning and the boundary selection objective help address my earlier concerns. The additional experiments on prediction stability and hypothesis sensitivity also strengthen the experimental evidence. I appreciate that the paper is well organized and presents a meaningful approach to adaptive block commitment.

However, I still have some concerns about the relatively high inference cost compared to the performance gains. Overall, I consider this paper borderline, but the additional evidence and clarifications are sufficient for me to raise my rating.

Dear Reviewer 5zYj,

Thank you very much for your positive feedback and for raising your score. We sincerely appreciate the time and the effort you have dedicated to our work. Your thoughtful comments mean a lot to us for improving the quality of our manuscript! We are glad that you find our paper is well organised and our study is meaningful.

We will incorporate all the feedback received during the discussion into the final version of our paper and continue to investigate into cost optimisation as future work.

Authors 10359

Reviewer_GJdEOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper addresses the issue of premature token commitment in semi-autoregressive block decoding for diffusion language models (dLLMs), where tokens in previously decoded blocks cannot be revised as generation proceeds. The authors propose Variable-Size Self-Contained Blocks (VSB), which dynamically selects block boundaries based on a self-containedness criterion. Specifically, self-containedness is measured by the divergence between token prediction distributions with and without access to hypothesized future tokens. The method formulates boundary selection as an optimization problem that balances block length against future dependence. The authors evaluate VSB across mathematical reasoning, code generation, and general knowledge benchmarks using several LLaDA models. They also introduce an optional training objective to align the model with the proposed inference procedure.

Critical strengths
  1. Clear mathematical formulation and Clear presentation. Section 3 provides a well-defined formulation of self-containedness, supported by theoretical results that clarify the relationship between NF-FA divergence and predictive stability under the stated assumptions. The paper is generally well-written and organized, with clear explanations of the proposed method and accessible presentations of the experimental results.

  2. Applicability to pretrained dLLMs. The proposed method can be applied to pretrained dLLMs without additional training, and its applicability is demonstrated across multiple LLaDA model variants. The optional training objective also provides a complementary extension to the inference-time method.

Critical weaknesses
  1. Insufficient motivation for self-containedness. The paper does not provide sufficient empirical evidence that future-dependent commitment is a significant bottleneck in blockwise dLLM decoding. While the examples in the Introduction illustrate how prematurely committed tokens may lead to errors due to their dependence on future context, the paper does not quantify how frequently such failures occur in practice or how much they contribute to downstream performance degradation. Furthermore, existing re-masking-based decoding methods offer an alternative approach by allowing previously committed tokens to be revised. Although these methods are acknowledged in the Introduction, they are not included in the empirical comparisons. Without such evidence or comparisons, the necessity of explicitly identifying self-contained blocks as a solution remains insufficiently justified.

  2. Modest empirical gains relative to additional inference cost. The empirical improvements over existing adaptive decoding baselines appear relatively modest, raising concerns about whether they justify the additional forward passes required for NF-FA divergence estimation. While Section 5.1 briefly highlights the performance gains, Table 1 shows limited improvements over competitive baselines. In particular, the differences from DAEDAL on mathematical reasoning benchmarks are marginal and may fall within experimental variability. Moreover, Section 5.3 primarily investigates the efficiency of the proposed coarse-to-fine boundary search, rather than providing a comprehensive accuracy–compute trade-off against competing methods. A comparison under matched total inference budgets (e.g., total forward passes or denoising steps, alongside wall-clock latency) would be important to establish the practical benefits of VSB.

Questions for authors
  1. Can the authors provide evidence that future-dependent commitment is a major source of generation errors, and that VSB specifically mitigates these errors rather than merely selecting shorter or more confident blocks?

  2. Since VSB evaluates self-containedness using a single model-generated hypothesis, how sensitive is the proposed criterion to the accuracy of this hypothesis? In particular, can a block with low NF-FA divergence still contain incorrect predictions that are consistently supported by an inaccurate hypothesized continuation? An analysis of the relationship between hypothesis quality, NF-FA divergence, and actual commitment errors would help clarify when the proposed criterion is effective.

4 comments
AuthorsOpenReview ↗

Response to Reviewer GJdE (1/2)

We thank the reviewer for the constructive feedbacks. Our answers rest on three points: 1. Premature commitment is measurable and tied to errors (W1a). 2. NF-FA measures the future dependence that confidence does not (Q1). 3. VSB turns this into more stable boundaries, at or above re-masking at a similar budget, and ahead of fixed-size decoding (W1b, W2, Q1).

Re W1(a): Is premature commitment a significant problem?

Yes: it is frequent, concentrated at unit boundaries, and tied to wrong answers. We measured it on 500 GSM8K test problems decoded with fixed-size blocks. A committed token counts as premature if the model itself revises it once the full output is visible (each position re-masked, the rest of the output given).

  1. 38% of generations contain at least one such token.
  2. These tokens are 2.1x as frequent in wrong answers as in correct ones.
  3. 57% of wrong answers contain one, against 35% of correct ones. Generations that contain one are 13 points less accurate (75.4% vs. 88.3%).
  4. Table 9 shows where future dependence concentrates: delimiter/closure (76%), arithmetic (11%) and code syntax (12%).

This is the problem our VSB targets.

Re W1(b): Why VSB rather than re-masking?

Thank you for the insightful question, we clarify that the two act at different stages, and VSB leads re-masking at close budgets. All decoders here already use LLaDA's in-block re-masking. The methods the reviewer refers to also re-mask tokens after drafting. Our VSB decides where a block ends, so the two are orthogonal (Table 7, Appendix D). For a direct comparison, we gave fixed-size decoding training-free post-draft re-masking, following re-masking samplers such as ReMDM:

  1. After each block is drafted, its 16 lowest-confidence tokens are re-masked and re-denoised before commitment (4 rounds on GSM8K, 2 on HumanEval).
  2. Its total pass count is comparable to VSB's, spent on revision instead of boundary scoring.

LLaDA-8B, accuracy (fwd per problem).

decoderGSM8KHumanEval
fixed-size + re-masking82.25 (568)43.90 (959)
VSB82.87 (616)45.73 (893)

Re W2: Are the gains worth the inference cost?

Thank you for this thoughtful question, while we acknowledge and have acknowledged (e.g., Sec 5 Line 368 and Limitations) that cost is important, we clarify that VSB is a commitment-quality method and we evaluate it as one. Our primary target is understanding stable commitment at the standard decoding schedules, and we have fully disclosed and are transparent about the cost (e.g., Sec. 5.3 and Table 12 report NFE and wall-clock, and we states in Appendix A plainly that VSB is not a speedup method).

  1. Most of the cost is not scoring: Relative to fixed-size decoding at the same per-window schedule, VSB's pass ratio is (W/bˉ)(1+K/D)=1.8×1.2(W/\bar b)(1+K/D)=1.8\times1.2 (WW window, bˉ\bar b mean commit, KK scoring and DD denoising forwards per window). Boundary scoring is the 1.2x factor, 16-19% of passes. The 1.8x factor is re-drafting the deferred part of each window, which any decoder that commits fewer than WW tokens per block pays. The scoring forwards within one search level are independent and batch into a single pass, which cuts scoring time by about 5x with identical decisions. Appendix A also describes a future-aware activation-caching variant for further acceleration.
  2. Budget-controlled comparisons: Table 13 of the paper already matches the fixed block size to VSB's average block length under the same budget: VSB leads on 5 of 6 settings (GSM8K 81.05 vs. 75.13 and HumanEval 43.29 vs. 35.98 at the 32-token budget).
  3. What the passes buy. The region VSB defers has 1.77x the flip rate of the region it commits (Table 11), which is the instability Corollary 3.5 bounds. VSB is also at or above DAEDAL (Table 1), with the largest margins on code (+2.44 HumanEval, +2.00 MBPP).
AuthorsOpenReview ↗

Response to Reviewer GJdE (2/2)

Re Q1: Does VSB mitigate premature commitment, or merely pick shorter or more confident blocks?

We highlight that VSB identifies and defers future-dependent tokens, which provides an orthogonal perspective to block length or confidence. W1(a) measures how often these errors occur. The four analyses below show that VSB separates stable from unstable regions, selects more self-contained boundaries, and pulls ahead under aggressive drafting.

  1. Mitigation: Within a window, the region VSB defers has 1.77x the flip rate and 1.81x the low-margin positions of the region it commits (Table 11). A fixed-size decoder commits both.

  2. Shorter blocks: Our block budget-controlled analysis in Appendix H.3 Table 13 also shows that VSB's gains are not simply from using smaller blocks, but to choosing better boundaries. Such that matching the fixed-size LLaDA baseline to VSB’s observed average block length does not improve the baseline’s performance.

  3. More confident blocks: NF-FA and confidence flag different tokens. Their scores are nearly uncorrelated, and NF-FA singles out confidently predicted line breaks and delimiters (63% of its distinct flags vs. 27% of tokens). Re-scored on the final outputs, confidence's blocks are no more self-contained than random cuts of the same length: 58% vs. 56% exceed S=0.05S=0.05, against 25% for VSB, which minimises SS.

  4. Under aggressive drafting: We also ran an entropy-gated schedule that commits several tokens per pass, so the whole decode uses about a quarter of the passes in typical decoding (e.g., fixed-size 61 vs. 284 vs. 197 for VSB on GSM8K). We note that accuracy is lower for every decoder under this setting, and premature commitment is most frequent, which is the regime that tests boundary placement. Nonethless, VSB is still ahead of fixed-size decoding:

entropy-gated draftingGSM8KMATH500HumanEvalMBPP
LLaDA-8B: fixed-size / VSB77.79 / 78.0936.00 / 38.2029.27 / 30.4836.80 / 37.40
LLaDA-1.5: fixed-size / VSB76.57 / 79.0835.00 / 36.0030.49 / 28.6634.40 / 35.20

Re Q2: How sensitive is VSB to the quality of the hypothesis?

VSB commits less when the hypothesis is less complete, and its accuracy is consistent. We varied hypothesis quality directly by scoring with snapshots taken at different points of the window's denoising (LLaDA-8B, GSM8K, 286 problem subset, T=64T=64):

hypothesis (window decoded)25%50%75%100%
SS at selected boundary (median)0.00890.00810.00630.0064
mean commit (tokens)18.726.632.034.5
commits past the final-hypothesis boundary19.6%27.1%27.8%-
accuracy81.983.982.982.5
  1. A less complete hypothesis raises the divergence, and VSB responds by committing shorter blocks. Fewer of its commits then extend past the boundary the complete hypothesis would choose.
  2. Accuracy is stable across stages: VSB offsets a less complete hypothesis by committing less.
  3. Can a low-divergence block still be wrong? Yes, by design: NF-FA certifies the property an irreversible commitment needs, stability with respect to what follows, not factual correctness (Appendix A). A block can therefore have low divergence while its content is wrong. NF-FA measures stability under future conditioning, which determines whether a block can be committed without substantial future dependence, while correctness remains the responsibility of the underlying model. The snapshot study shows the practical effect: even at 25% completion, where 19.6% of commits extend past the boundary the complete hypothesis would choose, accuracy is consistent, because VSB commits less when the hypothesis is less reliable.

Dear Reviewer GJdE,

We sincerely apologise for this gentle reminder and greatly appreciate your time and thought-provoking review. As the discussion deadline approaches, we would be grateful if you could share your feedback on our responses.

We have provided detailed responses to address all your questions and will carefully incorporate all feedback into the final version of our paper. We would greatly appreciate your feedback on whether our responses have adequately addressed your concerns.

Thank you again for your time and consideration!

Authors

Thank you for the detailed response and additional analyses. I appreciate the authors' efforts to address my concerns. However, my main concerns remain.

Regarding W1, the proposed measurement of premature commitment is not entirely convincing. Re-masking a single token after the full sequence has been generated only shows that its prediction changes when conditioned on subsequent tokens. This does not necessarily imply that the original commitment was erroneous, especially since the subsequent tokens were themselves generated conditioned on that commitment. A more direct assessment would examine how alternative commitments at the same decoding point affect the final generation and its correctness. Moreover, because most identified cases involve delimiter/closure tokens, their contribution to downstream errors remains unclear.

Regarding W2, My concern was not the lack of computational cost analysis, but whether the performance gains justify the additional inference cost. While the authors clarify the sources of computational overhead and provide additional comparisons, the improvements remain relatively modest, leaving the practical benefit of VSB insufficiently convincing.

Therefore, I will maintain my original assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.