acceptodds
Under review as a conference paper at ICLR 2027

HiBlock: Learning Hierarchical Block-Size Policy for Cross-Domain Generalisation in Diffusion Large Language Models

Abstract

Recent advances in diffusion large language models (dLLMs) enable efficient block-based generation in single-domain settings, where blocks have fixed sizes and tokens within each block are denoised jointly. However, directly extending fixed-size block decoding to multi-domain scenarios is challenging because different domains naturally favour different block size strategies and no single size performs best across all domains. Recent adaptive block-size methods attempt to address this by adding heuristic rules that extend block boundaries to certain predefined tokens or newline characters. However, these methods rely on external rules and prompt-specific cues, thereby not enabling dLLMs to explicitly learn adaptive block-size selection ability for cross-domain generalisation. To address this, we propose HiBlock, a Hierarchical Block-size policy learning framework via Group Relative Policy Optimization (GRPO). HiBlock comprises a high-level policy that selects a block-size action and a low-level policy that generates the response under the selected block size, transforming block size from a predefined hyperparameter into a learnable action. During post-training, HiBlock explores diverse block-size actions across GRPO rollouts, while during inference, the learnt policy selects a block size for each input task before response generation. Experiments on 13 benchmarks show its effectiveness in dLLM cross-domain generalisation. Our code is available at https://anonymous.4open.science/r/HiBlock-2026.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Cross-Domain Generalisation, Diffusion Large Language Models, Block Size

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_exLGOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HiBlock, a hierarchical policy for learning input-dependent block sizes in diffusion LLMs. The method treats block size as a high-level action and jointly optimizes block-size selection and block-based response generation with a GRPO-style objective. During training, different block sizes are explored across rollouts, while at inference the learned policy selects one block size before generation. Experiments cover 13 benchmarks, 6 dLLM backbones, and several RL algorithms.

Critical strengths
  1. The paper addresses a practical limitation of fixed-block dLLMs with a simple and useful idea: making block size an input-dependent learned action rather than a fixed hyperparameter.

  2. The empirical study is fairly extensive. The improvements are consistent across multiple benchmarks, model backbones, RL algorithms, and the additional multi-seed experiment.

Critical weaknesses
  1. The checkpoint-selection protocol is unclear and may affect the cross-domain claim. The paper states that the best checkpoint accuracy is reported, with checkpoints evaluated every 100 training steps, but I could not find how the best checkpoint is selected. If target test performance is used to select a checkpoint separately for each benchmark, this would introduce target-test model selection and weaken the claimed zero-shot cross-domain generalization.

  2. The current ablations do not fully isolate the benefit of the learned block-size policy. HiBlock trains the response policy using rollouts from all candidate block sizes, while the main fixed-size baseline is trained only with block size 32. Thus, part of the gain may come from multi-block-size training itself rather than from the learned high-level selector. A stronger control would use the same multi-block-size training as HiBlock but remove the learned selector at inference. I would also like to see a fixed-block baseline whose block size is selected on source-domain validation rather than always using 32. This is particularly important because closely related work such as Block-R1 has already studied block-size conflict in multi-domain dLLM RL; the main novelty here appears to be the learned online selector rather than the conflict itself.

Questions for authors
  1. How exactly is the "best checkpoint" selected for each reported target benchmark? Is any target test data used for checkpoint selection? Could you report results using a single checkpoint selected only from source-domain validation?

  2. What happens if the low-level response policy is trained with exactly the same balanced multi-block-size exploration as HiBlock, but inference uses a fixed, random, or source-validation-selected block size instead of the learned high-level policy? This would help separate the effect of multi-block training from the effect of the learned selector.

  3. During training, block-size actions are assigned by balanced exploration rather than sampled from the old high-level policy. What is the justification for using the current/old policy likelihood ratio for these action positions in Eq. 14? What objective should this update be interpreted as optimizing?

8 comments
AuthorsOpenReview ↗

Response to Reviewer exLG [1/2]

We sincerely thank you for your constructive feedback. We appreciate your recognition of HiBlock's motivation and extensive empirical evaluation. We summarise our response and highlight the key information as follows:

Re W1 & Q1: Checkpoint Selection

Thank you for the question. We totally agree with your opinion.

Therefore, we would like to double confirm that our checkpoint selections:

  • Are following existing work and kept strictly fair and consistent in evaluation.
  • Only use held-out source-domain validation data;
  • No test data is used since that's simply data leakage.

We would like to provide more details:

  1. Reporting the best checkpoint results based on validation set is a common practice in existing dLLM research. We quoted as below,

    • "Following d1, we report results from the best-performing checkpoint.", wd1 (ICLR 2026, arXiv:2507.08838v1, Section 4, Page 6)
    • "We report the best checkpoint performance for each domain following most prior methods.", Block-R1 (NeurIPS 2026, arXiv:2605.11726, Appendix D.5)
  2. For all our experiments, 10% of training data is reserved for validation. This is used to evaluate checkpoints every 100 training steps.

    • Source domain validation (Countdown, GSM8K, MATH500, KodCode, Sudoku, and Knights-and-Knaves). Checkpoints are selected using only the held-out validation sets of the training domains. In leave-one-domain-out experiments, the target domain is excluded from both training and checkpoint selection.
    • Related domain validation (HumanEval and MBPP). For these unseen code benchmarks, checkpoints are selected using KodCode's held-out validation set, without accessing their test performance.
    • Multi domain validation (HellaSwag, MMLU, ARC-E, MMLU-Pro, and ARC-C). We train on all six source domains and select a single checkpoint based on their average held-out validation performance, without using any target test data.

We would like to reassure that (1) only validation data is used for model selection; (2) test data is absolutely prohibited in training or validation.

Re W2.1 & Q2: Fixed, Random, or Source-Validation-Selected Block Size

Thank you for this insightful suggestion. We totally agree with you on isolating the contribution of learnt block-size selection.

Following your suggestion, we conduct additional experiments with the same HiBlock checkpoint while changing the block size policy as follows:

  1. Learnt: Block sizes selected by the learnt HiBlock policy.
  2. Random: Randomly shuffle HiBlock's selected block sizes across inputs within each benchmark, preserving the original block-size distribution.
  3. Fixed-32: Use a fixed block size of 32 by default following most prior dLLMs.
  4. Valid: Use a fixed block size selected using source-domain validation data.
Inference StrategyCountdownGSM8KMATH500SudokuAvg.
Learnt (HiBlock)59.7681.5834.4025.8050.39
Random28.9162.2427.406.4231.24
Fixed-3236.7267.8528.606.6934.97
Valid42.5875.5130.2012.1740.12
  • HiBlock outperforms all alternatives across all benchmarks.
  • Our main contribution is from the learnt block size policy.

Re W2.2: Comparison with Block-R1

We would like to respectfully clarify that the targeted problem and the technical contributions are fundamentally different between Block-R1 and our HiBlock.

  1. Our HiBlock:
    • actively trains a block size selection policy;
    • conduct dynamic size inference for each prompt.
  2. Block-R1 requires an additional privileged dataset (Block-R1-41K) with:
    • predefined block-size labels assigned to each training sample;
    • multi-size training based on these labels, rather than directly learning from the original training data (discussed in Appendix B.3).
AuthorsOpenReview ↗

Response to Reviewer exLG [2/2]

Re Q3: Current/Old Policy Likelihood Ratio for Block-Size Actions in Eq. 14

Thank you for raising this important theoretical question. We clarify our design motivation, experimental evidence, and optimisation objective as follows:

  1. We initially designed purely on-policy rollouts for both the high and low level policies.
  2. Our further analysis reveals that the initial block size policy exploration was insufficient, as shown below:
Block Size48163264128
Without Balance52.4%8.5%18.6%5.2%12.5%2.8%

The policy strongly favours block size 4 (52.4%), while block size 128 is rarely explored (2.8%). This limits the reward signals for other block sizes.

  1. Thus, we introduce balanced exploration of block size actions to ensure sufficient exploration of all block sizes. This design is reasonable because:

    • We can obtain the policy likelihood of each assigned block size action from the dLLM.
    • The action itself does not need to be differentiable. Its likelihood provides the gradient for policy optimisation.
    • Balanced sampling ensures that each block size is explored within a GRPO group.
  2. We follow your suggestion to provide an additional ablation study to verify the effectiveness of balanced exploration:

Training StrategyCountdownGSM8KMATH500SudokuAvg.
With Balance (HiBlock)59.7681.5834.4025.8050.39
Without Balance39.7270.3329.008.6736.93

Our proposed balanced exploration improves significantly.

To further interpret this optimisation objective, we kindly provide that:

  1. We use balanced exploration to ensure sufficient coverage of candidate block sizes.
  2. The current/old policy likelihood ratio in Eq. 14 provides a proximal reference for advantage weighted policy updates.
  3. This ratio is not an importance sampling correction for the balanced exploration distribution.
  4. Thus, Eq. 14 defines a GRPO style clipped surrogate over balanced exploration trajectories, rather than the standard on-policy GRPO objective.

We sincerely thank you for this insightful question. We have clarified the objective in Section 3.3 Line 302 to 305 and have added this important ablation study in Section 4.3, Line 429 to 439. Your feedback has greatly helped us improve the manuscript.


We wholeheartedly thank you again for your constructive feedback, which has helped us significantly improve the quality of paper. If you have any further questions, we would be happy to discuss them with you.

I have one remaining question regarding Q3. While the balanced exploration ablation demonstrates its empirical benefit, it does not directly address the mismatch between the balanced block-size sampling distribution and the policy likelihood ratio in Eq. 14. Could the authors further clarify the theoretical justification for this surrogate objective? In particular, have you considered comparing it with an alternative objective that explicitly accounts for the behavior sampling distribution?

Reviewer_Z6c9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes HiBlock, a Hierarchical Block-size policy learning framework via Group Relative Policy Optimization (GRPO). It comprises a high-level policy that selects a block-size action and a low-level policy that generates the response under the selected block size, transforming block size from a predefined hyperparameter into a learnable action.

Critical strengths
  1. The paper is overall well-written and easy to follow.
  2. Previous method view block-size in block diffusion llms as a hyperparameter. This paper found the evidence that dllm has different preference (in terms of performance) on different tasks (e.g. math, coding), and proposes a method to view the block-size in block diffusion llms as a policy, and learn the policy via RL, which is interesting. Experiments on diffenrent benchmarks show its effectiveness.
Critical weaknesses
  1. How the block-size action z is implemented is under-specified.
Questions for authors

I have no other questions, please see the weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Z6c9

We sincerely appreciate your positive assessment and constructive feedback. We clarify the implementation of the block-size action below.

Re W1: How Block Size Action Is Implemented

We sincerely apologise for the lack of clarity here.

  • In one word for implementation, block size action is just an integer number output/decoded by the dLLM itself.

To be more specific:

  1. Our original design rationale is that we used zz to denote the token sequence corresponding to the numerical block size cc.
  2. This is reasonable because the dLLM predicts and optimises actions at the token level.

Following your suggestion,

  1. We realised that introducing both zz and cc could cause unnecessary confusion.
  2. Thus, we have simplified the notation by directly using c∈Cc\in\mathcal{C} as the block size action in the updated manuscript. The dLLM still uses its original numerical tokens, and the training trajectory is now written as yg=Tok⁡(cg)∥ogy_g=\operatorname{Tok}(c_g)\Vert o_g (Eq. 11), where Tok⁡\operatorname{Tok} denotes tokenisation.
  3. This revision does not change the underlying method or the implementation.

We greatly thank your question and have revised the corresponding parts in Sections 3.2-3.4 and Algorithm 1 accordingly.


We sincerely thank you again for your constructive feedback. Your insightful questions have greatly helped us improve the clarity and overall quality of our paper. We truly appreciate your valuable suggestions and welcome any further questions or feedback.

I thank the authors for the clarification. The block-size action is now clear to me, and I will keep my score.

One remaining question: the vanilla multi-domain RL baseline falls below the base model on several benchmarks (e.g., GSM8K 76.19 → 57.24), and HiBlock with α = 0 already reaches 49.25 vs. 31.32. An ablation that trains with the same balanced block-size exploration but decodes with a fixed block size at inference would help clarify whether the gains come from learned block-size selection or from more diverse rollouts during training.

We sincerely thank you for your follow-up question and are glad that our clarification has addressed your previous concern.

Re: Contribution of Learned Block-Size Selection

We totally agree with you that it is important to seperate the benefits of balanced exploration during training from those of learnt block-size selection at inference.

Following your suggestion, we have conducted exactly this ablation using the same trained HiBlock checkpoint, keeping the training unchanged and only changing the block-size selection strategy at inference:

  1. Learnt: Use the learnt HiBlock block-size policy.
  2. Random: Shuffle the selected block sizes across inputs.
  3. Fixed-32: Use a fixed block size of 32.
  4. Valid: Use a fixed block size selected using source-domain validation data.
Inference StrategyCountdownGSM8KMATH500SudokuAvg.
Learnt (HiBlock)59.7681.5834.4025.8050.39
Random28.9162.2427.406.4231.24
Fixed-3236.7267.8528.606.6934.97
Valid42.5875.5130.2012.1740.12

Under the same checkpoint and balanced training, HiBlock consistently outperforms both fixed-size alternatives across all four benchmarks. This directly shows that the improvements are not solely due to diverse rollouts during training, and learnt block-size selection provides additional gains at inference.


We wholeheartedly thank you again for your thoughtful feedback. We would be very grateful if you could consider championing our paper in your final assessment. If there is anything else we could clarify or improve to earn your further support, we would be more than happy to address it!

Dear Reviewer Z6c9,

Thank you very much for your thoughtful review! We have carefully addressed your questions in our rebuttal and have taken all your suggestions into account in revising our paper.

With the discussion period coming to an end, we would sincerely appreciate it if you could take a moment to look over our responses and share any further thoughts. We are eager to clarify any remaining questions or concerns and would be truly grateful if you could consider championing our paper during the final discussion. Your support would be greatly appreciated!

Best regards,
Authors 23988

Reviewer_5SbNOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes HiBlock, a framework that learns input-dependent block-size selection for diffusion language models through a GRPO-style objective. A high-level policy selects a block size before response generation, and a low-level policy generates the response using the selected size. Both policies share the same model and are jointly optimized using task rewards and a block-size reward. The paper evaluates the approach across multiple benchmarks, model backbones, and RL algorithms.

Critical strengths
  1. Well-motivated and practically relevant problem. Block size controls an important aspect of diffusion-based generation, and the observation that different tasks favor different generation granularities provides a reasonable motivation for learning input-dependent selection.

  2. Promising empirical results. HiBlock substantially improves upon the fixed-size multi-domain RL baseline on several benchmarks. The evaluations across multiple backbones and RL algorithms, together with the multi-seed results in Appendix G, provide useful evidence for the effectiveness and robustness of the overall framework.

  3. An appealing integration of block-size selection into post-training. Learning the selection policy within the same model, without requiring separate predictors or pre-annotated block-size labels, is a useful design choice.

Critical weaknesses
  1. Incomplete attribution of the gains. The “w/o High-Level Policy” ablation uses a fixed block size, changing both training-time exploration and inference-time selection. Evaluating the same HiBlock checkpoint with learned, random, and validation-selected fixed block sizes would better isolate the selector’s contribution.

  2. Insufficient implementation clarity. The manuscript should clarify how block-action likelihoods are estimated, how masking implements the conditioning in Eq. (10), and how Eq. (14) is instantiated with StableDRL. The interpretation of the clipped objective under balanced, policy-independent action exploration also needs further explanation.

  3. Indirect evidence for instance-level adaptation. Figure 3 shows similar aggregate block-size distributions, which does not establish that HiBlock selects suitable sizes for individual inputs. Shuffling selected block sizes across inputs within each benchmark would provide a more direct test.

Questions for authors
  1. How much of the gain remains when the same HiBlock checkpoint uses random or validation-selected fixed block sizes at inference? Does preserving multi-block-size training without explicitly optimizing action tokens recover a substantial portion of the improvement?

  2. How exactly are block-action likelihoods computed during training and inference, and what masking or conditioning scheme implements the factorization in Eq. (10)? How is Eq. (14) instantiated with StableDRL?

  3. Appendix C.4 specifies a fixed total budget of 128 denoising steps per response. Under this setting, how are steps allocated across blocks, and how does block size affect actual forward-pass counts and latency? This would clarify the rationale for using block size as an efficiency reward.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 5SbN [1/2]

We greatly appreciate your recognition of HiBlock's motivation, empirical effectiveness, and robustness. Your constructive suggestions are very helpful in strengthening our work and we address your concerns as follows.

Re W1 & Q1: Learned/Random/Validation-selected Block Size

Thank you for this insightful question. We totally agree with you on further ablation with block size selection.

Following your suggestion, we conducted additional experiments on HiBlock checkpoint with:

  1. Learnt block size selection trained with the checkpoint;
  2. Randomly shuffle block size selection directly applied with the checkpoint;
  3. Valid fixed size based on the validation set;
  4. Fixed-32, the default fixed size.
Size SelectionCountdownGSM8KMATH500SudokuAvg.
Learnt59.7681.5834.4025.8050.39
Random28.9162.2427.406.4231.24
Valid42.5875.5130.2012.1740.12
Fixed 3236.7267.8528.606.6934.97

HiBlock outperforms these alternatives across all four benchmarks, demonstrating the performance gain coming from the learned block size selection.

Re W2.1 & Q2.1: Clarity of Block Action Likelihood

Sincere apologies for the lack of clarity in the block action likelihood. We clarify below:

  1. The block size action is an integer c∈Cc\in\mathcal{C}, represented by ordinary numerical tokens from the original dLLM tokeniser, without introducing a separate predictor.
  2. The action likelihood is calculated from the dLLM's denoising probabilities of the corresponding numerical tokens. During training, the current/old likelihood ratio is used in Eq. 14. During inference, these probabilities guide the constrained decoding of cc in Eq. 15.

We have clarified these details in Sections 3.2 to 3.4 of the revised manuscript.

Re W2.2 & Q2.2: Clarity of Masking/Conditioning of Eq 10&14

Sincere apologies for the lack of clarity in the detailed implementation of Eqs. 10 and 14.

For decoding and masking:

  1. During block size generation, the first token is masked for the dLLM to decode.
  2. After decoding the block size, dLLM will keep the block size as context and use it as the block size for the block-based generation of the whole response.

For condition generation:

  1. Decoding the block size cc is conditioned on input xx.
  2. Decoding the whole response oo is conditioned on both input xx and block size cc.

During dLLM training:

  1. Both action and response tokens are included in the completion and contribute to the masked likelihood optimisation.
  2. Eq. 14 applies StableDRL's clipped objective to both action and response token positions.

In the original notation, zz denoted the numerical tokens of cc. We now use cc consistently for the block size.

We have clarified and updated these details in Sections 3.2 to 3.3 of the revised manuscript.

Re W2.3: Further Explanation of Clipped Objective

We sincerely apologise for the lack of clarity in the interpretation of Eq. 14. We clarify as follows:

  1. Block-size actions are assigned through balanced exploration rather than sampled from the old policy. However, their likelihoods can still be computed under both the current and old policies.
  2. The current/old likelihood ratio is used for proximal clipping, rather than importance sampling correction for balanced exploration. The group-relative advantages guide the likelihood updates using reward feedback from the explored actions.
  3. Thus, Eq. 14 defines a GRPO-style clipped, advantage weighted surrogate under balanced exploration, rather than the standard on-policy GRPO objective.

We thank you for this insightful question and have clarified this interpretation in Section 3.3, Line 302 to 305 of the revised manuscript.

Re W3: Instance-level Block Size, Fig 3 & Shuffling Size Experiment

We sincerely thank you for this insightful suggestion.

  1. We would like to clarify that Fig 3 is a population level evidence of the match of block size selection with oracle. Yet Fig 3 is still a collection of individual evidence.
  2. We totally agree with you and following your suggestion, we have followed your suggestion and conducted additional instance level block size randomly shuffle experiment to validate the individual level effectiveness as:
Size SelectionCountdownGSM8KMATH500SudokuAvg.
Learnt59.7681.5834.4025.8050.39
Shuffle28.9162.2427.406.4231.24

It can be seen that our Learnt block size has a higher individual level effectiveness than the shuffling size.

AuthorsOpenReview ↗

Response to Reviewer 5SbN [2/2]

Re Q3.1: Diffusion Steps across Blocks

Thank you for the question. We provide the details below:

  1. We follow existing work (d1, wd1, and Block-R1):
    • to have a maximum of 128 diffusion steps for each response for a default generation length of 256;
    • for each diffusion step, two tokens will be decoded.
  2. Thus, for decoding every block, there will be {block size}/2 diffusion steps.
  3. This diffusion step budget is a fair and consistent experimental setup across all methods.

We have updated these clarification details in Section 4 (Implementation), Lines 381–384 of the revised manuscript.

Re Q3.2: Block Size vs Forward Pass Latency

Thank you for the insightful question. We provide the rationale below:

  1. One full sequence forward pass is conducted at the beginning of each block. This is because we support the block-based inference in existing dLLMs (AdaBlock-dLLM, Fast-dLLM, and dInfer) to adopt block-level KV caching.
  2. Thus, the number of blocks decides the number of forward passes and thus affects the latency.

The trade-off between efficiency and performance in block size is widely discussed, as in b1 (ICML 2026) and Block-R1 (NeurIPS 2026):

  1. Smaller blocks may improve performance;
  2. More small blocks require more full sequence forward passes and frequent inter block transitions, resulting in lower decoding speed.

Thus, our block size reward encourages larger block sizes for greater decoding parallelism. Our Table 3 reports the end-to-end inference throughput:

  • HiBlock: 24.78 tokens/s.
  • HiBlock without block-size reward: 20.75 tokens/s.
  • Fixed-size baseline: 26.51 tokens/s.

We sincerely appreciate your valuable feedback and positive assessment. Your suggestions have been greatly helpful in strengthening our experimental analysis and improving the clarity of our manuscript. We would be grateful for any further questions or feedback you may have.

Dear Reviewer 5SbN,

Thank you so much for your thoughtful review and encouraging feedback! We have provided thorough responses to your questions and have incorporated all the feedback received during the discussion into the final version of our paper.

As the discussion deadline is approaching, we would be very grateful if you could kindly review our responses and share your thoughts. We are eager to address any further questions or concerns and sincerely hope you might consider championing our paper during the final discussion. We would greatly appreciate your support!

Best regards,

Authors 23988

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.