- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Variable-Size Self-Contained Blocks (VSB) for discrete diffusion language models. It compares No-Future (NF) and Future-Aware (FA) predictions under the same decoding hypothesis and uses a length-aware objective to select commitment boundaries. Coarse-to-fine search reduces scoring cost, while optional blockwise No-Future training aligns training with inference. Experiments cover three backbones and seven benchmarks.
Critical strengths
-
Comparing predictions with and without the hypothesised future directly addresses the future-dependent commitment problem. The theoretical bounds link NF-FA divergence to predictive stability.
-
The reported gains span several models and tasks. Comparisons with fixed-size decoding at matched average block lengths or similar NFE budgets strengthen the evaluation. The runtime analysis also makes the cost of boundary scoring explicit.
Critical weaknesses
-
The endpoint of the budgeted window needs clarification. Under Eqs. (5)-(7), NF and FA appear to coincide at b=q, giving S(p,q)=0. Eq. (9) would then favor the full window, leaving it unclear how the stated rule selects shorter self-contained blocks.
-
Although the paper compares against adaptive baselines, it remains unclear how much NF-FA divergence itself adds over token confidence for boundary selection.
Questions for authors
-
At b=q, do NF and FA use the same conditioning context, including the masked positions beyond q? A short example showing how the length-aware objective selects b<q would help clarify the boundary rule.
-
Could the authors compare NF-FA divergence with token confidence for boundary selection while matching average block lengths and keeping the backbone and other decoding settings fixed? Please report accuracy/pass@1 and NFE or runtime.
4 comments
Dear Reviewer oxmK,
We sincerely apologise for this gentle reminder and greatly appreciate your time and thought-provoking review. As the discussion deadline approaches, we would be grateful if you could share your feedback on our responses.
We have provided detailed responses to address all your questions and will carefully incorporate all feedback into the final version of our paper. We would greatly appreciate your feedback on whether our responses have adequately addressed your concerns.
Thank you again for your time and consideration!
Authors
I thank the authors for the detailed response and additional experiments, which address most of my concerns. Based on my overall assessment of the paper's contributions, I will maintain my original score.
Dear Reviewer oxmK,
We sincerely thank you for the time and effort you have devoted to providing constructive and thoughtful feedback. We also appreciate your positive recognition that we have addressed most of your concerns.
We fully respect your assessment and would be grateful if you could share any remaining concerns that we might not have adequately addressed. We would sincerely appreciate the opportunity to provide additional clarifications during the discussion period, with the hope of fully addressing your concerns and potentially improving your assessment.
Thank you again for your valuable feedback and consideration!
Authors 10359
Response to Reviewer oxmK (1/1)
We thank the reviewer for the concrete questions. We provide detailed response to your questions below.
Thank you for your careful reading and we apologies for the confusion. We clarify that NF and FA use different conditioning context, therefore it does not degenerate at b=q as evident in our VSB average commit blocksize analysis (Figure 6). Specifically, FA conditions on the masked continuation Mq+1:L, which Sec. 3.1 includes in the future ("with tokens beyond q as masked Mq+1:L, which we omit in the following for clarity"(Line 144)). To avoid repetition, we kindly invite the reviewer to visit our detailed response to W1 from Reviewer 5zYj.
Following from our response to W1, VSB uses different conditioning at b=q, where FA conditions on [x^1:q;Mq+1:L] and NF on x^1:q. Here we show an example of the smooth length-aware objective (Eq.9) at different candidate boundaries of a block:
VSB thus commits the completed step. Appendix J shows the same rule over a full decoded solution.
Thank you for the insightful question, we highlight that our propsoed NF-FA measures future dependence, the property that the commitment phenomenon we study requires, whereas confidence measures how peaked a prediction is now.
Confidence thus cannot separate a token that is confidently predicted yet still contingent on the continuation (e.g., a line break that may become a comma, a closing bracket, the last digit of a partial sum) from one that is settled. NF-FA compares the two conditionals directly, which is why premature commitment is defined by it and why our stability bound is stated in it.
Experimenting with a fixed backbone (LLaDA-8B) and length-aware objective, with only the boundary signal changes our NF-FA objective is ahead of confidence on both math tasks, within noise on code. We highlight that NF-FA also carries the stability guarantee of Lemma 3.3 to Corollary 3.5 that confidence lacks.
LLaDA-8B, accuracy (fwd per problem). Potential performance deviation can be due to hardware differences.
These results combined therefore supports NF-FA for boundary selection. We will revise these content in a future update.